[MPlayer-dev-eng] About the future - libao2

[Anders Johansson]

Hi,

> not so simple :(
> codecs are very different. few codec can return exactly teh requested amount
> of bytes/samples, but most codec is unpredictable. especially windows ones.
> 
> and, get_space() should return free buffer space only, which depends on
> audio playback, it can't be constant.
> if audio buffer is full, it should return 0 to avoid blocking.
Ok I understand why the buffer management is built the way it is.
What is the future of making mplayer into a multi threaded application?

> > This approach will also make the sync calculation more simple since
> > mplayer.c doesn't have to take the buffer usage into consideration,
> > since libao2 is keeping track of it.
> i don't understand this.
Never mind it is just an obscure detail, and can not be made in this
way anyway because of what you are saying above anyway.

> > > - use the general, extendable control() interface for driver-specific things.
> > >   it's like ioctl()'s.
> > Part of my idea is to make mplayer even more transparent to the
> > details of the HW, and I guess that is sort of the point with having
> > libao2 at all. I had a look at the control() function an as I can see
> > it the only thing it is used for is volume control and status
> > information, and I guess that is ok, but I don't think it should be
> > extended beyond that. 
> control() is not yet used. there are many things to do with it, but
> currently it isn't used.
>
> > I have however a suggestion of how HW specific stuff could be
> > accessed: modules in libao2 that can be configured registers one or
> > several calback functions and strings. This information is passed on
> > to the GUI which displays a list (when pressing a button or whatever)
> > of the strings. When the user selects from the list, the callback is
> > executed and a window is opened (, state is toggled or whatever).
> > 
> > I just got an idea: This type of interface could be extended to any
> > type of event, so that any type of configurable module could register
> > it self to be triggered to any event. Example: The fast forward and
> > backward audio click removal function could register it self to be
> > executed wenever -> or <- is pressed, or the EQ to the sound control
> > on the remote. Just an idea :).
> ehh.

What plans are there for extended features (I am talking of sending
control commands to libao2 and i guess all other modules as well). I
like generic approaches, as the one described above but I can agree
that it might be a bit too generic :) for extended discussion see below.

> > > - thinking about: some codecs (mp3lib, ffmpeg?) can do some work about
> > >   effects, resampling (1:2, 1:4 for mp3), mono<->stereo.
> > I don't think we want to use them, reason: the resampling is made by
> > zero padding the data before the IDCT. This is not as efficient as FIR
> > resampling since it requires a lot of multiplications and additions on
> > 0. The polyphase implementation avoids this. We also don't want to
> > upsample before the data is filtered (EQs, effects, etc), to reduce
> > number of calculations required.
> hmm. i mean natural side-effects of codecs.
> mpeg audio decoder has dct->pcm converterts build for 1:1 1:2 1:4
> resampling. and if you select mono, it will do less calculations as
> it mix channels in dct space, and so on.
I will look into this further.

> A'rpi / Astral & ESP-team

I think we are starting to sort out the limitations that define the
interface. I will start writing some kind of framework (a header file
+ documentation based on this discussion). I will focus on the basic
functionality to start with and I will not make changes not previously
agreed on. I will also try to define a roadmap of the development of
the audio features (see end of mail).

There are a few issues regarding the interface still to sort out:

1. Initialisation of libao2.
As I see it there are three ways to go: 

1.1 Keep the control of the initialisation (interpretation of cmdline
switches and settings saved in file etc) inside mplayer and inform
libao2 of what mplayer is expecting of it. What I mean is that init
takes two structs one telling libao2 what it will get and the other
what the user wants, all switches and config file parameters are
decoded by mplayer.

1.2 Tell libao2 what it will get and let it autoinitialize based on
some kind of rule structure (typical M$ structure that will piss the
users off). Will however make mlpayer easier to configure for suckers,
but harder to debug when it doesn't work. A god thing however is that
the current interface doesn't need any changes.

1.3 Let libao2 keep the control of the initialisation, meaning that
cmdline switches are passed on and libao (sort of like gcc is passing
on switches to as using the -W switch) is keeping and maintaining it's
own config file. This approach has the advantage that developers
doesn't need to make changes to the mplayer source when they want to
add sound features, which makes the source easier to maintain. It will
also be easier to write new sound stuff.

I vote for option nr 3, are there other better ways to go? Perhaps
combinations of the above.


2. Runtime interface.
How to send commands to libao2 during execution (pause reset etc).

1.1 Keep current interface and disallow it to expand. All extended
stuff needs to be done off line (when mplayer is not running). Good
don't need to change anything. If something needs to be changed it
will be made through the control() function.

1.2 Make some kind of pluggable command interface so that the new
features can hook on to events. Not fun to implement and might cause
some runtime overhead.

1.3 Try to guess what the user want and take action accordingly. Will
not work well and might require ESP hardware :). Just kidding what I
am talking about are functions that analyses the data stream and take
action according to some state space model. One example would be
removing clicks in the audio data when -> or <- is pressed. But such
algorithms are never completely reliable so it would probably end up
removing explosion sounds (or something) as well as clicks :(.

1.4 Make some external interface running in parallel (like analysing
the keyboard pressing using a separate process and give the info to
libao2 through some back door). Very ugly.

I like 1.2 but it is too generic and I don't feel like implementing it
but I don't like 1.1 cause it is a bit to static, and inflexible...
Give me input please, anyone with better ideas? Perhaps extend
control() to accept generic arguments like ioctl does. Will still
require changes in mplayer for new functionality :(




Ideas for a roadmap:

1. Refine the libao2 interface to make it easier to implement new
features.

2. Define and implement an extendible buffer management feature with
little or no overhead inside libao2 + perhaps a pluggable module
interface for new audio modules. I will spy a bit on others first
before I suggest a design.

3. Implement resampling.

4. Type conversion (change number of bits/sample).

5. Click removal when fast forward/backward.

6. Synthetic 4 channel stereo.

7. AC3 output from any type of input data.

8. Functionality for manipulating AC3 and DTS streams (volume and
add/remove copy protection, etc).

9. Stereo perspective panning and calibration for 3 or more speakers. *

10. Equaliser (boring to implement perhaps steal one and adapt it). *

11. Reverb and other delay effects (makes the room sound larger). *

12. Add / remove water marking of the audio streams.

Feel free to shuffle and come up with your own ideas.

The options marked with * require advanced user interface features
that will be hard to implement using only control().