[MPlayer-dev-eng] About the future - libao2

[Arpi]

Hi,

> > not so simple :(
> > codecs are very different. few codec can return exactly teh requested amount
> > of bytes/samples, but most codec is unpredictable. especially windows ones.
> > 
> > and, get_space() should return free buffer space only, which depends on
> > audio playback, it can't be constant.
> > if audio buffer is full, it should return 0 to avoid blocking.
> Ok I understand why the buffer management is built the way it is.
> What is the future of making mplayer into a multi threaded application?

NO! MPlayer has its advantages and very high stability because its singlethread design.
I won't change this one.

> > > This approach will also make the sync calculation more simple since
> > > mplayer.c doesn't have to take the buffer usage into consideration,
> > > since libao2 is keeping track of it.
> > i don't understand this.
> Never mind it is just an obscure detail, and can not be made in this
> way anyway because of what you are saying above anyway.
:)

> > > > - use the general, extendable control() interface for driver-specific things.
> > > >   it's like ioctl()'s.
> > > Part of my idea is to make mplayer even more transparent to the
> > > details of the HW, and I guess that is sort of the point with having
> > > libao2 at all. I had a look at the control() function an as I can see
> > > it the only thing it is used for is volume control and status
> > > information, and I guess that is ok, but I don't think it should be
> > > extended beyond that. 
> > control() is not yet used. there are many things to do with it, but
> > currently it isn't used.
> >
> > > I have however a suggestion of how HW specific stuff could be
> > > accessed: modules in libao2 that can be configured registers one or
> > > several calback functions and strings. This information is passed on
> > > to the GUI which displays a list (when pressing a button or whatever)
> > > of the strings. When the user selects from the list, the callback is
> > > executed and a window is opened (, state is toggled or whatever).
> > > 
> > > I just got an idea: This type of interface could be extended to any
> > > type of event, so that any type of configurable module could register
> > > it self to be triggered to any event. Example: The fast forward and
> > > backward audio click removal function could register it self to be
> > > executed wenever -> or <- is pressed, or the EQ to the sound control
> > > on the remote. Just an idea :).
> > ehh.
> 
> What plans are there for extended features (I am talking of sending
> control commands to libao2 and i guess all other modules as well). I
> like generic approaches, as the one described above but I can agree
> that it might be a bit too generic :) for extended discussion see below.
> 
> > > > - thinking about: some codecs (mp3lib, ffmpeg?) can do some work about
> > > >   effects, resampling (1:2, 1:4 for mp3), mono<->stereo.
> > > I don't think we want to use them, reason: the resampling is made by
> > > zero padding the data before the IDCT. This is not as efficient as FIR
> > > resampling since it requires a lot of multiplications and additions on
> > > 0. The polyphase implementation avoids this. We also don't want to
> > > upsample before the data is filtered (EQs, effects, etc), to reduce
> > > number of calculations required.
> > hmm. i mean natural side-effects of codecs.
> > mpeg audio decoder has dct->pcm converterts build for 1:1 1:2 1:4
> > resampling. and if you select mono, it will do less calculations as
> > it mix channels in dct space, and so on.
> I will look into this further.
> 
> > A'rpi / Astral & ESP-team
> 
> I think we are starting to sort out the limitations that define the
> interface. I will start writing some kind of framework (a header file
> + documentation based on this discussion). I will focus on the basic
> functionality to start with and I will not make changes not previously
> agreed on. I will also try to define a roadmap of the development of
> the audio features (see end of mail).
> 
> There are a few issues regarding the interface still to sort out:
> 
> 1. Initialisation of libao2.
> As I see it there are three ways to go: 
> 
> 1.1 Keep the control of the initialisation (interpretation of cmdline
> switches and settings saved in file etc) inside mplayer and inform
> libao2 of what mplayer is expecting of it. What I mean is that init
> takes two structs one telling libao2 what it will get and the other
> what the user wants, all switches and config file parameters are
> decoded by mplayer.
what is the first struct for?
btw struct for cmdline/cfgfile options is ok, i already have this idea for
libvo2 to avoid lots of globals

> 1.2 Tell libao2 what it will get and let it autoinitialize based on
> some kind of rule structure (typical M$ structure that will piss the
> users off). Will however make mlpayer easier to configure for suckers,
> but harder to debug when it doesn't work. A god thing however is that
> the current interface doesn't need any changes.
I don't like intelligent programs :)

> 1.3 Let libao2 keep the control of the initialisation, meaning that
> cmdline switches are passed on and libao (sort of like gcc is passing
> on switches to as using the -W switch) is keeping and maintaining it's
> own config file. This approach has the advantage that developers
> doesn't need to make changes to the mplayer source when they want to
> add sound features, which makes the source easier to maintain. It will
> also be easier to write new sound stuff.
what about subconfig? it's a new feature of config parser, used mainly for
mencoder. this way modules/libs can have their own set of options, while
the main parser does all checks and processing.
I mean, you can define and fill a struct in libao3 core, make it global.
cfg parser will have a subconfig line pointing to this struct, so it will
parse libao options using this set. this way libao shouldn't care of cfg
parsing, file paths etc while can be extendable without any change to
mplayer.

> I vote for option nr 3, are there other better ways to go? Perhaps
> combinations of the above.
> 
> 
> 2. Runtime interface.
> How to send commands to libao2 during execution (pause reset etc).
> 
> 1.1 Keep current interface and disallow it to expand. All extended
> stuff needs to be done off line (when mplayer is not running). Good
> don't need to change anything. If something needs to be changed it
> will be made through the control() function.
i vote to this one. it's the main design goal for libvo2 too.
standard functions have their function in libao drivers, and all
extras or primitive set/get/query thing have control().

> 1.2 Make some kind of pluggable command interface so that the new
> features can hook on to events. Not fun to implement and might cause
> some runtime overhead.
no...

> 1.3 Try to guess what the user want and take action accordingly. Will
:)))))))))))

> not work well and might require ESP hardware :). Just kidding what I
> am talking about are functions that analyses the data stream and take
> action according to some state space model. One example would be
> removing clicks in the audio data when -> or <- is pressed. But such
> algorithms are never completely reliable so it would probably end up
> removing explosion sounds (or something) as well as clicks :(.
seeking stuff always call audio_reset. so you can handle ckicks in it.
not requires any extra stuff, current API is enough for this.
other things can be controlled using control()
forget events, this is not an OOP program :)

> 1.4 Make some external interface running in parallel (like analysing
> the keyboard pressing using a separate process and give the info to
> libao2 through some back door). Very ugly.
forget it.

> I like 1.2 but it is too generic and I don't feel like implementing it
> but I don't like 1.1 cause it is a bit to static, and inflexible...
> Give me input please, anyone with better ideas? Perhaps extend
> control() to accept generic arguments like ioctl does. Will still
> require changes in mplayer for new functionality :(
as everything. mpalyer.c is the boss, it must be modified for new stuff
related to playback control. pure audio effect options can be implemented
without any change outside, using subconfig.

> Ideas for a roadmap:
> 
> 1. Refine the libao2 interface to make it easier to implement new
> features.
> 
> 2. Define and implement an extendible buffer management feature with
> little or no overhead inside libao2 + perhaps a pluggable module
> interface for new audio modules. I will spy a bit on others first
> before I suggest a design.
detail this one

> 3. Implement resampling.
> 
> 4. Type conversion (change number of bits/sample).
> 
> 5. Click removal when fast forward/backward.
> 
> 6. Synthetic 4 channel stereo.
> 
> 7. AC3 output from any type of input data.
How?

> 8. Functionality for manipulating AC3 and DTS streams (volume and
> add/remove copy protection, etc).
> 
> 9. Stereo perspective panning and calibration for 3 or more speakers. *
> 
> 10. Equaliser (boring to implement perhaps steal one and adapt it). *
:)

> 11. Reverb and other delay effects (makes the room sound larger). *
> 
> 12. Add / remove water marking of the audio streams.
???

> Feel free to shuffle and come up with your own ideas.

what about xmms effect plugin support?

> The options marked with * require advanced user interface features
> that will be hard to implement using only control(). 
they need gui?


A'rpi / Astral & ESP-team

--
mailto:arpi at thot.banki.hu
http://esp-team.scene.hu