[MPlayer-dev-eng] Lots of stuff for NUT

[Rich Felker]

On Fri, Dec 30, 2005 at 09:10:02PM +0200, Oded Shimon wrote:
> 3. strict interleaving method
> 
> We still have the deprecated DTS method in the spec. My vote is for MN 
> rule, it has only one flaw IMO. you can do stupid things like audio 
> preload if you set decode_delay higher than it should be.
> Here is a text Rich and I prepared for MN rule a while ago..
> -
> all timestamps must satisfy a monotonicity condition: if I and J
> are two frames, not necessarily from the same stream, and I
> precedes J in the bitstream, then the _decode_ timestamp of I must
> be less than or equal to the _presentation_ timestamp of J,
> compared in a common time base.
> -

All good interleaving rules must satisfy the "MN condition", but the
MN condition is hardly a rule by itself because there are many ways to
satisfy it, many of them not-so-good like the audio preload hack you
mention above. Note that strict monotone dts ordering also has this
issue. The problem is that, the way dts is defined, it depends on
decode_delay, i.e. multiple valid choices for decode_delay will give
different dts values. If you are unable to know the optimal
decode_delay (e.g. remuxing h264 with pyramid crap) and just choose a
big number to be safe, it will be like having large 'video preload',
the opposite of audio preload, which makes demuxing large-buffer
intensive.

Actually I would like to revisit what uau said about dts the other day
on irc. His proposal was the same as my old 'mts' (muxing timestamp)
one, which I had largely abandoned and was not too eagar to embrace
again. It provides packets in a slightly less optimal order for
demuxing than the dts method does if correct decode_delay is used, but
it's totally independent of decode_delay as long as a valid delay is
chosen.

The formula is simple:

mts = min (this frame's pts, all later frames' pts)

Of course you don't have to look forward at the entire rest of the
file, just decode_delay frames forward. This _seems_ to require
buffering in the muxer (which I strongly oppose) however it actually
does not as long as the encoder can output frames immediately without
delay, or provide a flag indicating whether the frame has earlier-pts
frames that depend on it. Even if the encoder does not provide this or
there is no encoder (remuxing), a framer can provide this information.

> 2. syncpoints must be max(dts) (min(pts)?)
> 
> At the moment there's absolutely no rule about syncpoint pts, it can 
> practically be random numbers and it's still legal. We say it must be 
> either max dts across all streams, or min pts across all streams...
> I think i preffer max dts. dts is "almost" monotone, making the syncpoint 
> fit nicely between the frames...
> Also, given subtitles, min(pts) is completely bogus. even with EOS, think 
> during a subtitle.
> Both methods basically make the syncpoints themselves also follow the MN 
> rule. Which brings us to next thing...

If we use mts rule, the obvious choice is that syncpoint timestamp is
minimum of all subsequent frame pts values, which is the mts of the
frame attached to the syncpoint.

Rich