[MPlayer-dev-eng] [PATCH] mencoder, B-frames, and video/audio delay

[Corey Hickey]

B-frames causing a-v desync is a well-known and long-standing problem
with encoding. I'd really, really like to get to the bottom of the issue
and either fix it or find and document the best workaround. The attached
patch might be the solution...



The first video source I've been testing with is the Serenity trailer.
The video has distinctive sounds at several scene transitions, which
makes it easy to spot at a glance whether the audio and video are out of
sync. Correct a-v sync when encoding this source can be forced by using
-mc 0 -noskip (more about that later). It's available at:
http://movies.apple.com/movies/universal/serenity/serenity_m480.mov

My general options for encoding it are:
mencoder serenity_m420p.mov -oac pcm -vf scale=480:208 -ovc x264 \
-x264encopts bitrate=800:bframes=16 -frames 1000 -o wayoff.avi

* According to the formula from Michael[1], the video delay caused by
B-frames is equal to (1+b_frames)/fps. Using 16 B-frames is overkill for
a normal encode, but it conveniently makes the video delay an
easy-to-spot 17/24 seconds (the video fps for this file is 24, not
24000/1001).
[1]http://www1.mplayerhq.hu/pipermail/mplayer-dev-eng/2005-December/039425.html

* I'm using -oac pcm because -oac copy doesn't work with AAC audio.

* The scaling parameters and -frames are only there to speed up encoding.



The second video source is a 60-second clip cut from near the beginning
of The Fellowship of the Ring extended version DVD. It's my standard
test clip for pretty much everything, and the big difference in this
situation is that it does not have correct a-v sync when -mc 0 and
-noskip are used, regardless of whether the encode has B-frames. When
mplayer and mencoder first start reading the file, the audio is delayed
about 0.5 seconds; both programs compensate within several seconds when
allowed to do so. You can download it here:

http://fatooh.org/files/lotr_test.vob

My encoding options for this file:
mencoder /lotr_test.vob -frames 1439 \
-vf crop=712:354:4:62,scale=688:288 -aid 128 -oac copy -ovc x264 \
-x264encopts bitrate=800:bframes=16 -ofps 24000/1001 -o test.avi




As I understand it, the reason for the delayed video is that the encoder
has to buffer enough input frames before it can start encoding B-frames
(naturally), and doesn't start sending any frames to the muxer until
after the audio encoder has already been doing so for a while. This
isn't a problem by itself, but mencoder sees the video stream is delayed
and encodes a bunch of empty video frames (several "1 duplicate
frame(s)!" messages) to compensate. Thus, mencoder thinks it is in sync
and, up to the end of the encode, mencoder has muxed audio and video
streams in equal duration; however, the empty frames in the video stream
displace real frames, shifting all subsequent frames later in time.
During playback, this shift is apparent as a video delay. Once the
encode finishes, the encoder emits all the video frames it has buffered,
making the encoded video stream trail past the end of the audio stream.

The attached patch disables the portion of mencoder.c that decides
whether to adjust the A-V sync based upon the relative positions of the
muxer stream timers. I don't know if that's the correct fix. If mencoder
can still skip/duplicate frames when the input streams are out of sync,
then it should be ok, right? Perhaps I'm missing something, but my
patched mencoder can encode not only the Serenity trailer (which works
without a-v adjustment) but the DVD clip (which begins out of sync but
is adjusted quickly), with 16 B-frames and perfect a-v sync. If somebody
more knowledgeable than I gives me the go-ahead, then I will commit this
patch.

Please test it.

-Corey