[MPlayer-dev-eng] Nut and B frames

[Michael Niedermayer]

Hi

On Wednesday 28 April 2004 01:46, D Richard Felker III wrote:
> On Wed, Apr 28, 2004 at 12:47:43AM +0200, Michael Niedermayer wrote:
> > > Maybe your complaint is that the codec won't know _which_ "pending"
> > > frame to output for one of the t=. points.
> >
> > yes
> >
> > > But IMO this isn't
> > > possible. The codec always has to know which frame it's going to
> > > output next, independent of the container...
> >
> > well yes, but theres the codec less streamcopy issue, in which case we
> > need the pts, which are no longer stored in such a way that the demuxer
> > can figure them out without the codec
>
> Ah, streamcopy to a different format that wants decode ordering with
> the pts attached... I see. Hmm, this is a problem. How about this for
> the nut B frame muxing rules:
>
> Apply your pts-to-dts algorithm to all frames. The monotonicity rule
> applies to all frames with pts==dts (which necessarily exist within
> every group of <delay>+1 frames by the definition of delay). The
> remaining frames can be muxed anywhere as long as they remain in
> decode order.
>
> What I was missing before was a clear definition of _which_ frames are
> subject to the monotonicity/interleaving rule. But then I realized
> delay provides a very simple rule for it. So I'm happy with choice #1
> now, as long as you are!
i am, i will update the spec, with what we have decided, tell me if i forget 
something ...

>
> So here's the basic process for a program writing nut files:
>
> while (!eof) {
>   if (video.pts < audio.pts || video.out_of_order) {
>     write_frame(video)
>     video = get_next_video_frame()
>   } else {
>     write_frame(audio)
>     audio = get_next_audio_frame()
>   }
> }
>
> Naturally it's more complicated with more streams, but it's nice to
> see that proper interleaving is easy in the simplest case.
>
> BTW I still think we should have 0-byte packets for (hypothetical)
> codecs that store several pictures in one coded frame (think frequency
> transformations in time axis). A good codec like this would provide
> its own dummy packets to indicate "show the next picture", but if it's
> designed for use in its own container or with only a limited set of
> fixed framerates, it might not...so it can't hurt to have our own
> packets for the same purpose. Unfortunately this would allow people to
> abuse nut by storing "DIVX packed" B frames, though...
hmm, i agree that we should have 1 frame at the muxer level for each displayed 
frame if possible, now with current temporal transforms thats not a big issue 
by using 0 byte frames, but i can imagine that some advanced codecs may not 
have frames at all, think of 3d fractals for example, or some 3d+time scene 
description where the decoder renderes the scene with opengl or similar, 
anyone wanna write an encoder for these? :)

[...]
-- 
Michael
level[i]= get_vlc(); i+=get_vlc();		(violates patent EP0266049)
median(mv[y-1][x], mv[y][x-1], mv[y+1][x+1]);	(violates patent #5,905,535)
buf[i]= qp - buf[i-1];				(violates patent #?)
for more examples, see http://mplayerhq.hu/~michael/patent.html
stop it, see http://petition.eurolinux.org & http://petition.ffii.org/eubsa/en