本文為作者原創(chuàng )，轉載請注明出處：https://www.cnblogs.com/leisure_chn/p/10584910.html
I幀：I幀(Intra-coded picture, 幀內編碼幀，常稱(chēng)為關(guān)鍵幀)包含一幅完整的圖像信息，屬于幀內編碼圖像，不含運動(dòng)矢量，在解碼時(shí)不需要參考其他幀圖像。因此在I幀圖像處可以切換頻道，而不會(huì )導致圖像丟失或無(wú)法解碼。I幀圖像用于阻止誤差的累積和擴散。在閉合式GOP中，每個(gè)GOP的第一個(gè)幀一定是I幀，且當前GOP的數據不會(huì )參考前后GOP的數據。
P幀：P幀(Predictive-coded picture, 預測編碼圖像幀)是幀間編碼幀，利用之前的I幀或P幀進(jìn)行預測編碼。
B幀：B幀(Bi-directionally predicted picture, 雙向預測編碼圖像幀)是幀間編碼幀，利用之前和(或)之后的I幀或P幀進(jìn)行雙向預測編碼。B幀不可以作為參考幀。
B幀具有更高的壓縮率，但需要更多的緩沖時(shí)間以及更高的CPU占用率，因此B幀適合本地存儲以及視頻點(diǎn)播，而不適用對實(shí)時(shí)性要求較高的直播系統。
DTS(Decoding Time Stamp, 解碼時(shí)間戳)，表示壓縮幀的解碼時(shí)間。
PTS(Presentation Time Stamp, 顯示時(shí)間戳)，表示將壓縮幀解碼后得到的原始幀的顯示時(shí)間。
音頻中DTS和PTS是相同的。視頻中由于B幀需要雙向預測，B幀依賴(lài)于其前和其后的幀，因此含B幀的視頻解碼順序與顯示順序不同，即DTS與PTS不同。當然，不含B幀的視頻，其DTS和PTS是相同的。下圖以一個(gè)開(kāi)放式GOP示意圖為例，說(shuō)明視頻流的解碼順序和顯示順序
在FFmpeg中，時(shí)間基(time_base)是時(shí)間戳(timestamp)的單位，時(shí)間戳值乘以時(shí)間基，可以得到實(shí)際的時(shí)刻值(以秒等為單位)。例如，如果一個(gè)視頻幀的dts是40，pts是160，其time_base是1/1000秒，那么可以計算出此視頻幀的解碼時(shí)刻是40毫秒(40/1000)，顯示時(shí)刻是160毫秒(160/1000)。FFmpeg中時(shí)間戳(pts/dts)的類(lèi)型是int64_t類(lèi)型，把一個(gè)time_base看作一個(gè)時(shí)鐘脈沖，則可把dts/pts看作時(shí)鐘脈沖的計數。
不同的封裝格式具有不同的時(shí)間基。在FFmpeg處理音視頻過(guò)程中的不同階段，也會(huì )采用不同的時(shí)間基。
FFmepg中有三種時(shí)間基，命令行中tbr、tbn和tbc的打印值就是這三種時(shí)間基的倒數：
tbn：對應容器中的時(shí)間基。值是AVStream.time_base的倒數
tbc：對應編解碼器中的時(shí)間基。值是AVCodecContext.time_base的倒數
tbr：從視頻流中猜算得到，可能是幀率或場(chǎng)率(幀率的2倍)
測試文件下載(右鍵另存為)：tnmil3.flv
使用ffprobe探測媒體文件格式，如下：
12345678think@opensuse> ffprobe tnmil3.flv ffprobe version 4.1 Copyright (c) 2007-2018 the FFmpeg developersInput #0, flv, from 'tnmil3.flv': Metadata: encoder : Lavf58.20.100 Duration: 00:00:03.60, start: 0.017000, bitrate: 513 kb/s Stream #0:0: Video: h264 (High), yuv420p(progressive), 784x480, 25 fps, 25 tbr, 1k tbn, 50 tbc Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s
關(guān)于tbr、tbn和tbc的說(shuō)明，原文如下，來(lái)自FFmpeg郵件列表：
There are three different time bases for time stamps in FFmpeg. The
values printed are actually reciprocals of these, i.e. 1/tbr, 1/tbn and
1/tbc.
tbn is the time base in AVStream that has come from the container, I
think. It is used for all AVStream time stamps.
tbc is the time base in AVCodecContext for the codec used for a
particular stream. It is used for all AVCodecContext and related time
stamps.
tbr is guessed from the video stream and is the value users want to see
when they look for the video frame rate, except sometimes it is twice
what one would expect because of field rate versus frame rate.
除以上三種時(shí)間基外，FFmpeg還有一個(gè)內部時(shí)間基AV_TIME_BASE(以及分數形式的AV_TIME_BASE_Q)
12345// Internal time base represented as integer#define AV_TIME_BASE 1000000// Internal time base represented as fractional value#define AV_TIME_BASE_Q (AVRational){1, AV_TIME_BASE}AV_TIME_BASE及AV_TIME_BASE_Q用于FFmpeg內部函數處理，使用此時(shí)間基計算得到時(shí)間值表示的是微秒。
av_q2d()將時(shí)間從AVRational形式轉換為double形式。AVRational是分數類(lèi)型，double是雙精度浮點(diǎn)數類(lèi)型，轉換的結果單位是秒。轉換前后的值基于同一時(shí)間基，僅僅是數值的表現形式不同而已。
av_q2d()實(shí)現如下：
123456789/** * Convert an AVRational to a `double`. * @param a AVRational to convert * @return `a` in floating-point form * @see av_d2q() */static inline double av_q2d(AVRational a){ return a.num / (double) a.den;}av_q2d()使用方法如下：
1234AVStream stream;AVPacket packet;packet播放時(shí)刻值：timestamp(單位秒) = packet.pts × av_q2d(stream.time_base);packet播放時(shí)長(cháng)值：duration(單位秒) = packet.duration × av_q2d(stream.time_base);
av_rescale_q()用于不同時(shí)間基的轉換，用于將時(shí)間值從一種時(shí)間基轉換為另一種時(shí)間基。
12345678910/** * Rescale a 64-bit integer by 2 rational numbers. * * The operation is mathematically equivalent to `a × bq / cq`. * * This function is equivalent to av_rescale_q_rnd() with #AV_ROUND_NEAR_INF. * * @see av_rescale(), av_rescale_rnd(), av_rescale_q_rnd() */int64_t av_rescale_q(int64_t a, AVRational bq, AVRational cq) av_const;
av_packet_rescale_ts()用于將AVPacket中各種時(shí)間值從一種時(shí)間基轉換為另一種時(shí)間基。
123456789101112/** * Convert valid timing fields (timestamps / durations) in a packet from one * timebase to another. Timestamps with unknown values (AV_NOPTS_VALUE) will be * ignored. * * @param pkt packet on which the conversion will be performed * @param tb_src source timebase, in which the timing fields in pkt are * expressed * @param tb_dst destination timebase, to which the timing fields will be * converted */void av_packet_rescale_ts(AVPacket *pkt, AVRational tb_src, AVRational tb_dst);
容器中的時(shí)間基(AVStream.time_base，3.2節中的tbn)定義如下：
1234567891011121314151617typedef struct AVStream { ...... /** * This is the fundamental unit of time (in seconds) in terms * of which frame timestamps are represented. * * decoding: set by libavformat * encoding: May be set by the caller before avformat_write_header() to * provide a hint to the muxer about the desired timebase. In * avformat_write_header(), the muxer will overwrite this field * with the timebase that will actually be used for the timestamps * written into the file (which may or may not be related to the * user-provided one, depending on the format). */ AVRational time_base; ......}AVStream.time_base是AVPacket中pts和dts的時(shí)間單位，輸入流與輸出流中time_base按如下方式確定：
對于輸入流：打開(kāi)輸入文件后，調用avformat_find_stream_info()可獲取到每個(gè)流中的time_base
對于輸出流：打開(kāi)輸出文件后，調用avformat_write_header()可根據輸出文件封裝格式確定每個(gè)流的time_base并寫(xiě)入輸出文件中
不同封裝格式具有不同的時(shí)間基，在轉封裝(將一種封裝格式轉換為另一種封裝格式)過(guò)程中，時(shí)間基轉換相關(guān)代碼如下：
1234av_read_frame(ifmt_ctx, &pkt);pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX);pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX);pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);
下面的代碼具有和上面代碼相同的效果：
1234// 從輸入文件中讀取packetav_read_frame(ifmt_ctx, &pkt);// 將packet中的各時(shí)間值從輸入流封裝格式時(shí)間基轉換到輸出流封裝格式時(shí)間基av_packet_rescale_ts(&pkt, in_stream->time_base, out_stream->time_base);
這里流里的時(shí)間基in_stream->time_base和out_stream->time_base，是容器中的時(shí)間基，就是3.2節中的tbn。
例如，flv封裝格式的time_base為{1,1000}，ts封裝格式的time_base為{1,90000}
我們編寫(xiě)程序將flv封裝格式轉換為ts封裝格式，抓取原文件(flv)的前四幀顯示時(shí)間戳：
12345678910111213141516think@opensuse> ffprobe -show_frames -select_streams v tnmil3.flv | grep pkt_pts ffprobe version 4.1 Copyright (c) 2007-2018 the FFmpeg developersInput #0, flv, from 'tnmil3.flv': Metadata: encoder : Lavf58.20.100 Duration: 00:00:03.60, start: 0.017000, bitrate: 513 kb/s Stream #0:0: Video: h264 (High), yuv420p(progressive), 784x480, 25 fps, 25 tbr, 1k tbn, 50 tbc Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/spkt_pts=80pkt_pts_time=0.080000pkt_pts=120pkt_pts_time=0.120000pkt_pts=160pkt_pts_time=0.160000pkt_pts=200pkt_pts_time=0.200000
再抓取轉換的文件(ts)的前四幀顯示時(shí)間戳：
123456789101112131415161718think@opensuse> ffprobe -show_frames -select_streams v tnmil3.ts | grep pkt_pts ffprobe version 4.1 Copyright (c) 2007-2018 the FFmpeg developersInput #0, mpegts, from 'tnmil3.ts': Duration: 00:00:03.58, start: 0.017000, bitrate: 619 kb/s Program 1 Metadata: service_name : Service01 service_provider: FFmpeg Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 784x480, 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 127 kb/spkt_pts=7200pkt_pts_time=0.080000pkt_pts=10800pkt_pts_time=0.120000pkt_pts=14400pkt_pts_time=0.160000pkt_pts=18000pkt_pts_time=0.200000
可以發(fā)現，對于同一個(gè)視頻幀，它們時(shí)間基(tbn)不同因此時(shí)間戳(pkt_pts)也不同，但是計算出來(lái)的時(shí)刻值(pkt_pts_time)是相同的。
看第一幀的時(shí)間戳，計算關(guān)系：80×{1,1000} == 7200×{1,90000} == 0.080000
編解碼器中的時(shí)間基(AVCodecContext.time_base，3.2節中的tbc)定義如下：
1234567891011121314151617181920212223242526typedef struct AVCodecContext { ...... /** * This is the fundamental unit of time (in seconds) in terms * of which frame timestamps are represented. For fixed-fps content, * timebase should be 1/framerate and timestamp increments should be * identically 1. * This often, but not always is the inverse of the frame rate or field rate * for video. 1/time_base is not the average frame rate if the frame rate is not * constant. * * Like containers, elementary streams also can store timestamps, 1/time_base * is the unit in which these timestamps are specified. * As example of such codec time base see ISO/IEC 14496-2:2001(E) * vop_time_increment_resolution and fixed_vop_rate * (fixed_vop_rate == 0 implies that it is different from the framerate) * * - encoding: MUST be set by user. * - decoding: the use of this field for decoding is deprecated. * Use framerate instead. */ AVRational time_base; ......}上述注釋指出，AVCodecContext.time_base是幀率(視頻幀)的倒數，每幀時(shí)間戳遞增1，那么tbc就等于幀率。編碼過(guò)程中，應由用戶(hù)設置好此參數。解碼過(guò)程中，此參數已過(guò)時(shí)，建議直接使用幀率倒數用作時(shí)間基。
這里有一個(gè)問(wèn)題：按照此處注釋說(shuō)明，幀率為25的視頻流，tbc理應為25，但實(shí)際值卻為50，不知作何解釋?zhuān)渴欠駎bc已經(jīng)過(guò)時(shí)，不具參考意義？
根據注釋中的建議，實(shí)際使用時(shí)，在視頻解碼過(guò)程中，我們不使用AVCodecContext.time_base，而用幀率倒數作時(shí)間基，在視頻編碼過(guò)程中，我們將AVCodecContext.time_base設置為幀率的倒數。
視頻按幀播放，所以解碼后的原始視頻幀時(shí)間基為 1/framerate。
視頻解碼過(guò)程中的時(shí)間基轉換處理：
12345678910111213141516AVFormatContext *ifmt_ctx;AVStream *in_stream;AVCodecContext *dec_ctx;AVPacket packet;AVFrame *frame;// 從輸入文件中讀取編碼幀av_read_frame(ifmt_ctx, &packet);// 時(shí)間基轉換int raw_video_time_base = av_inv_q(dec_ctx->framerate);av_packet_rescale_ts(packet, in_stream->time_base, raw_video_time_base);// 解碼avcodec_send_packet(dec_ctx, packet)avcodec_receive_frame(dec_ctx, frame);
視頻編碼過(guò)程中的時(shí)間基轉換處理：
123456789101112131415161718AVFormatContext *ofmt_ctx;AVStream *out_stream;AVCodecContext *dec_ctx;AVCodecContext *enc_ctx;AVPacket packet;AVFrame *frame;// 編碼avcodec_send_frame(enc_ctx, frame);avcodec_receive_packet(enc_ctx, packet);// 時(shí)間基轉換packet.stream_index = out_stream_idx;enc_ctx->time_base = av_inv_q(dec_ctx->framerate);av_packet_rescale_ts(&opacket, enc_ctx->time_base, out_stream->time_base);// 將編碼幀寫(xiě)入輸出媒體文件av_interleaved_write_frame(o_fmt_ctx, &packet);
音頻按采樣點(diǎn)播放，所以解碼后的原始音頻幀時(shí)間基為 1/sample_rate
音頻解碼過(guò)程中的時(shí)間基轉換處理：
12345678910111213141516AVFormatContext *ifmt_ctx;AVStream *in_stream;AVCodecContext *dec_ctx;AVPacket packet;AVFrame *frame;// 從輸入文件中讀取編碼幀av_read_frame(ifmt_ctx, &packet);// 時(shí)間基轉換int raw_audio_time_base = av_inv_q(dec_ctx->sample_rate);av_packet_rescale_ts(packet, in_stream->time_base, raw_audio_time_base);// 解碼avcodec_send_packet(dec_ctx, packet)avcodec_receive_frame(dec_ctx, frame);
音頻編碼過(guò)程中的時(shí)間基轉換處理：
123456789101112131415161718AVFormatContext *ofmt_ctx;AVStream *out_stream;AVCodecContext *dec_ctx;AVCodecContext *enc_ctx;AVPacket packet;AVFrame *frame;// 編碼avcodec_send_frame(enc_ctx, frame);avcodec_receive_packet(enc_ctx, packet);// 時(shí)間基轉換packet.stream_index = out_stream_idx;enc_ctx->time_base = av_inv_q(dec_ctx->sample_rate);av_packet_rescale_ts(&opacket, enc_ctx->time_base, out_stream->time_base);// 將編碼幀寫(xiě)入輸出媒體文件av_interleaved_write_frame(o_fmt_ctx, &packet);
[1]. What does the output of ffmpeg mean? tbr tbn tbc etc?
[2]. 視頻編解碼基礎概念, https://www.cnblogs.com/leisure_chn/p/10285829.html
[3]. 對ffmpeg的時(shí)間戳的理解筆記, https://blog.csdn.net/topsluo/article/details/76239136
[4]. ffmpeg中的時(shí)間戳與時(shí)間基, http://www.imooc.com/article/91381
[5]. ffmpeg編解碼中涉及到的pts詳解, http://www.52ffmpeg.com/article/353.html
[6]. 音視頻錄入的pts和dts問(wèn)題, https://blog.csdn.net/zhouyongku/article/details/38510747
2019-03-16 V1.0 初稿
2019-03-23 V1.1 增加3.7節
聯(lián)系客服
