[MPlayer-dev-eng] Lots of stuff for NUT

[Rich Felker]

On Thu, Jan 05, 2006 at 01:58:29AM +0100, Michael Niedermayer wrote:
> > > and if its not omitable someone will fork and it would be omited again
> > > trying to enforce philosophical rules does not work, people will do
> > > what they think is best, if they dont need some stuff in the files for
> > > their purposes but that stuff means overhead they will find a way
> > > to remove it, this is one reason why open source is better then closed 
> > > source its easy to change stuff against the developers will
> > 
> > Open well-designed file formats don't get forked, especially when the
> > forked version would be inherently incompatible with the standard
> > demuxer. Hacks and forks only happen on top of broken formats that
> > were originally closed, like avi and quicktime.
> 
> ogg -> ogm is not a closed format,

OK, you got me there. However despite having a published spec it was
designed to be very limited (only Xiph's codecs). Notice that as soon
as something better came (mkv) everyone dropped the hacked ogm crap. I
don't think they would have hacked ogg to make ogm at all if it hadn't
been for the fact that vorbis (an excellent codec that everyone wants
to use for movie files) seemed to be tied to this incredibly crappy
container format.

> avis design wasnt broken, it just wasnt
> designed for variable framerate or codecs with dts!=pts

Yes it was. 24 bytes of overhead per frame, no seeking without index,
noninterleaved files allowed (contradicting the name even!), 2gb
filesize limit, ...

> quicktime didnt support dts!=pts either until the extensions by .mp4
> (ok it did support it by mpeg-ps in mov or so maybe but thats sick)
> quicktime is what happens when you try to design a format which can handle
> all non existing / theoretical things you get a bloated format where neither
> muxer nor demuxer supports even a fraction of the features and new stuff
> like b frames still needed changes ...

Yep. That's exactly what I _don't_ want nut to be!

> i strongly belive somehing similar will happen with nut if we overdo the
> futureproofness, noone can predict what weird things codecs might do in
> the future, so being prepared to adapt the format is better then trying
> to support everything imagnable IMHO at least

Yes, that's why you've successfully convinced me to forget about uau's
nagging concerns, drop SOR, etc. However The topic at hand (seeking)
isn't about future pathologies, it's about something I want NUT to be
able to do right now.

> about inherently incompatible, people always find a way to break the rules
> if theres a reason to break them, here its not even hard
> the compression methods oded and me came up with all lead to the possibility
> to set all ptrs and pts equal to significantly reduce their overhead, i
> can already almost see the windows kids writing nut compressers which remove
> the pts&ptrs and encoding guides recommanding to run these over the files
> as noone has ever noticed any disadvantages except maybe the reference 
> (de)muxer failing ...

I agree it's possible but I don't think any of this will happen as
long as the spec is clear about what's forbidden and why it's
forbidden. We've already planned to include a "nutlint" program to
verify conformance of NUT files.

In addition, I believe it's good design to make sure that illegal data
actually has a semantic meaning, and moreover one that's contrary to
what people tempted to make illegal data would intend. For instance,
rather than requiring framecode 'N' to be flagged illegal, we could
just make it so that it's not coded at all in the file, i.e. a max of
255 framecodes instead of 256 where the entry after 'M' is the data
for 'O'.

> well, lets see, what i meant but didnt clearly say is somethnig like:
> the only realistic case i can imagine ATM where you cant seek
> during muxing is when you send the data away immedeatly like realtime 
> streaming, broadcasting and so on,

Not necessarily. You might just be wanting to save to a remote
machine, e.g. something like:

encoder - | ssh myhost "cat > file.nut"

Being able to do things like this for formats that don't have an
inherent need for random-access writes is fundamental to the unix
philosophy. If NUT is less functional when written this way, I
consider it a serious design flaw and a windows-ism.

> well the user might want to seek to the
> future here but he wont be able to no matter what :)
> and seeking into the past, well the player had seen this part so it has
> complete knowledge about it, furthermore the file on the server can be
> updated

The people running the server probably don't have a method for you to
seek back at all, or even download the complete stream. It's much more
likely that you will be using something like mplayer -streamdump, in
which case you'll want the dumped file to be fully valid and seekable.

> > > 99% of the cases and a near point in the remaining that seems fine
> > > why should the user have to accept 2x overhead for something she doesnt
> > > need?
> > 
> > It's not 2x, and as Oded has shown the overhead is trivial. For low
> > bitrates where overhead matters it's several orders of magnitude
> > smaller than the frame headers, and for high bitrate it's something
> > like 25% IIRC.
> 
> the case with 10 streams IIRC had 2x addtional overhead but i
> might be wrong, still 25% is alot too

The 25% increase is for high bitrate files, and it's like 100k for 700
megs. For modem-bitrate the increase is nearly immeasurable since
there will be very few syncpoints per unit time. Please look at the
numbers, but unless I'm quite mistaken they're so small that even the
lamest windows warez kids won't try to make broken files to reduce
it..

Rich