When we get a packet from av_read_frame(), it will contain the PTS and DTS values for the information inside that packet. Generally the PTS and DTS will only differ when the stream we are playing has B frames in it. So, in this case, our stream might look like this: The decoding timestamp tells us when we need to decode something, and the presentation time stamp tells us when we need to display something. This is why we have a decoding timestamp and a presentation timestamp on each frame. Because of this, the frames might be stored like this: I P B B. Now, we need to know the information in P before we can display either B frame. So let's say we had a movie, and the frames were displayed like: I B B P. B frames are the same as P frames, but depend upon information found in frames that are displayed both before and after them! This explains why we might not have a finished frame after we call avcodec_decode_video2. P frames depend upon previous I and P frames and are like diffs or deltas.

The two other kinds of frames are called 'I' frames and 'P' frames ('I' for 'intra' and 'P' for 'predicted'). Some formats, like MPEG, use what they call 'B' frames (B stands for 'bidirectional'). To understand these two values, you need to know about the way movies are stored. Instead, packets from the stream might have what is called a decoding time stamp (DTS) and a presentation time stamp (PTS). However, if we simply synced the video by just counting frames and multiplying by frame rate, there is a chance that it will go out of sync with the audio. Audio streams have a sample rate, and the video streams have a frames per second value. So what do we do?įortunately, both the audio and video streams have the information about how fast and when you are supposed to play them inside of them. It plays the video, yeah, and it plays the audio, yeah, but it's not quite yet what we would call a movie. So this whole time, we've had an essentially useless movie player. While this code still works, it doesn't look good, and there are many more improvements that this tutorial could use. Today, it is a totally different program, and improvements in the ffmpeg libraries (and in ffplay.c itself) have caused some strategies to change.

When I first made this tutorial, all of my syncing code was pulled from ffplay.c.

Text version Tutorial 05: Synching Video Code: tutorial05.c CAVEAT

Ffmpeg extract frames with timestamp