[MPlayer-dev-eng] Nut and B frames

[D Richard Felker III]

On Tue, Apr 27, 2004 at 05:33:08PM +0200, Michael Niedermayer wrote:
> what we do need is some definition of dts (decoding time stamps) which is the 

Maybe just order is enough for this. I was discussing it with Ivan.
The issues is whether the muxer/codec relation is push or pull.

If the player design is pull (the only correct design IMO), then
there's no problem. The player asks the codec for the next frame, and
the codec fails to output a frame the first time, so it asks again and
gets the B frame, and everything is fine.

On the other hand, if the player is push-based (get a frame from the
demuxer, push it to the codec at the right time based on pts) then it
will have trouble if it doesn't know that B frames are present. IMO
this is an idiotic design, but maybe we should try to accomodate such
bad players.

There's also the problem of a simple remuxer, which won't be decoding.
If you just pull frames from the demuxer in order and use their
timestamps for remuxing (without knowing about out-of-order frames),
you'll mux a broken file...

> time at which a packet needs to be passed into the decoder, for them we could 
> enforce monotonicity
> 
> and its easy too
> 1. add a pts_reorder_style flag into each stream_header
> 
> pts_reorder_style=0 
> 	rpts=pts
> 
> pts_reorder_style=1
> 	if(max_pts < pts){
> 		rpts= max_pts;
> 		max_pts=pts;
> 	}else
> 		rpts= pts;
> 
> example:
> type&pts: I0 P3 B1 B2 P4 P6 B5
> max_pts :  -  0  3  3  3  4  6
> rpts    :  -  0  1  2  3  4  5

IMO this is overly complicated and probably won't account for future
codecs with more obscure frame ordering. Suppose for example we have a
codec that interpolates "B" frames between one past I/P frame and two
future P frames. Then we have:

type&pts: I0 P3 P6 B1 B2 P9 B4 B5 ...
max_pts :  -  0  3  6  6  9  9  9
rpts    :  -  0  3  1  2  6  4  5

My idea for helping dumb players was to include a "delay" field in the
stream header, telling the player how many frames it needs to demux
ahead to be sure it knows the next display pts. For normal files this
would be 0; for files with B frames, 1. In principle it could be
larger for codecs with nastier misordering.

I'm still not sure how to write down the rule for the muxer, tho...
The basic principle is that timestamps should be monotone except that
reference frames needed should be stored in decode order. But the
rule gets complicated when you have multiple streams with out-of-order
frames... :)

Rich