Introduction of developers sponsored by UHU-Linux

Árpád Gereöffy's discourse: introduction of video player software MPlayer

I'd like to introduce MPlayer, which is a video player software for Linux but nowdays it also runs on several UNIX-like platforms, including BSD's, Solaris and MacOS 10.

The primary goal of the project is being able to play the well known video formats like MPEG, QuickTime, Windows Media, RealVideo which were spread by the evolution of Internet. Nowdays this goal is overperformed, MPlayer not only needs less system resources for playing than the original players of these formats, but it also has got better stability, fault tolerance and also, these formats can be converted into open standard formats.

When I started this project there were no usable video player for Linux, and of course there was no player which was able to play every formats. Since that time several similar programs were created with similar abilities.

The development of MPlayer was started two years ago, I started it alone. After some months - thanks to the fact that I started it as open source project and the source was freely available through the Internet - quite a few people joined the project. In fact MPlayer became world wide project in only half a year, what is more, almost 30 developers are working for mplayer in their full spare time nowdays and hundreds of people are sending patches and bug fixes.

On the chart the number of downloads is marked with the red line. As you can see, it has reached quite a high level within half a year. The peeks mark the release of stable versions.

The yellow curve indicates the number of added source lines. I know this is a very subjective measure (microsoft method), but it indicates the activity of development quite well after all. This was topped about a year ago. Since then we didn't implement lots of new features, instead, we mostly try to fix bugs and speed up existing features. Besides, there are some new features even now.

After the time of releasing version 0.90pre (which was at the end of April), in May, the number of downloads increased so rapidly that according to the statistics of one of the most popular Linux portal sites http://freshmeat.net, MPlayer became the leader. It overtook projects like the Linux kernel or our biggest competitor xine, but other projects like GCC and several other projects also fell down on the top list.

The usefulness of this statistic can be questioned because usually programs used by many people or programs which target wide user-base can gain better scores. So if a project which is being worked on a lot or a project which is developed by more developers may be downloaded fewer times if it targets a smaller amount of users, so these projects won't become the part of the top list.

There were several video players even two years ago. I marked players with bold style which are being developed even now. I can mention xine and Avifile besides MPlayer. These players play today's mediaformats more or less.

Other players are specialized for one format. For example ogle is for playing DVD, and probably it's the best in this category. XMovie is about playing quicktime stuff. Open Media System should be mentioned here although it hasn't been developed for more than a year. About two years ago it was started to clamp the world of Linux players and to create a solid standard and codec interface. This was needed because it would have prevented everybody to start developing from zero. Unfortunately the project has fallen flat: at the beginning plans were overcomplicated so only todo lists left with no people to code them. So at the end the project died, and several separate, smaller programs were developed instead.

Evolution of mplayer wasn't always easy. At the beginning - when several people joined the development - the project was placed to the server of SourceForge. But in 2001 VA-Linux had got several problems and the firm seemed to become bankcrupt, they started to reorganize themselves, and money was substracted from SourceForge. This was noticable in their services' degradation too.

The first and biggest problem of us was the symptom that CVS service became unreliable. Anonymous CVS users' downloads sometimes caused stale lock files, and because of this even developers couldn't access the source.

They didn't have enough system resource even for serving mailing lists, so mail delivery became more and more slower. During a development process like ours it's very bad to wait one and a half hour for someone to get our mail, then wait again to get the answer ... Of course it could be said that we should have been use iRC or other service instead. So this was our problem with mailing lists. They tried to solve the problem with removing field Reply-To from mailing lists, so answers did not go to their server but directly to the sender. But this wasn't a solution really. Furthermore there was no archive of mailing lists, so it was impossible to find something, for example new developers didn't know what older conversations were about. Then they ceased FTP which was quite important for us since users uploaded several unplayable files which were then checked by us to address the problem. Lately they started to place more and more advertisements onto the site. There were also banners before this but mainly open source and freely available programs were advertised while nowdays we can even see large advertisements from Microsoft.

Then UHU-Linux started to support us. As a first step of their support, they helped us with a domain registration (mplayerhq.hu) and with a dedicated server for MPlayer. We moved to the new server in one month and since then we have been using this server. This machine runs several services like CVS, web and mailing lists. This was not only important because we could avoid problems of SourceForge but we could solve problems which we wouldn't have been able to solve on SourceForge, like file renaming in CVS which could be only done in shell and this was not possible at SourceForge. Several tasks can be automatized as well like daily CVS builds, or the automatized process of making modifications in the documentations be seen on the web immediately with a mail sent to mirror sites to ask them to fresh that page.

Then support from UHU continued, developers were given several hardware to develop and test support for these devices. At the UHU discourse it was mentioned that UHU-Linux also had this problem: they could not test with all kinds of hardware. We had this problem too, but fortunately we don't have to test with 25 kinds of hardware types, we must pay attention only to some certain hardware components like video cards. To be exact: drivers for Linux in the world of multimedia are quite poor. The fact that we have visible picture with Xserver is quite far from the goal to be able to play video on this. We must access some features of today's video cards like frame syncronization, buffering, scaling, colorpsace convertions and other internal functions which usually can't be accessed by X drivers. We have started to develop our own video drivers. This will be discussed later as well.

I should say about UHU (though it was mentioned in the UHU discourse) that there was some kind cooperation between UHU and us: they helped us with hardware devices, we tested both UHU-Linux and our own developments on them. And we can also help UHU with answers on questions like what driver should be used for a certain card, and what type of settings should be payed attention. Furthermore UHU Linux sponsors GUI development as well, after my discourse, Ponekker Zoltán will speak about this briefly, and he will also show what it is exactly.

You may ask what the final goal is. We develop the software, write more and more code, but when will we stop and say that it's ready? Several answers exist for this. For example: we will stop after releasing version 1.0. But it's a known fact that there is no finished software, only a software which development was stopped. So we probably won't stop, newer and newer ideas will present themselves. User friendly player - good question, we're moving towards this but this is not our primary goal. MPlayer Desktop Environment - maybe this sounds funny but we discovered quite large amount of problems during the development like problems with CVS, C compilers or whatever.

We thought about writing our own stuffs several times, like our own C-compiler, own X-terminal etc. And if enough ideas will be accomplished then we can imagine some kind of futuristic dream like this. Probably - quoting Gabucino - world domination. Of course I don't mean it really, furthermore, we haven't got goals like competing with other players. We also don't want to gain users from other players either, mainly because we don't need more users.

Users cause much more problems than advantages. The truth is that MPlayer development is not started as click-n-play software though the GUI helped a lot in this area. If you really want to use it in the right way, configure it in a way that even the last bit of the hardware is maximized. For this, you should read large amount of the documentation, and you should do settings and command lines according it. Usually this is forgotten. MPlayer is downloaded by a user, he types 'mplayer' and press enter, and then presently an email is sent that 'it does not work, help me'.

The problem is that 99% of them are of this kind, and large amount of them will really write this mail. Well, about 0.1% of users who really help us, who really write usable bugreports. Who upload files which should be playable by our software but it does not, while other players can play them. So we can check what is the problem, check if we can fix it. These small amount users write about newer hardwares we don't support yet. In this case we check for some modification, while others may even send patches.

We prefer cooperating with other players instead of dominating their users. We have different type of users, different goals; MPlayer's goal differs from xine or Avifile. Open source projects can coexist because they have got different goals. If they have the same goal, then there wouldn't be such a large amount of softwares. We work together with several projects on components which are needed by everyone and can be found in every program.

For example we developed the DLL loader with the cooperation of project Avifile. This is used for loading and using codecs which were written for windows on x86 architectures running Linux, BSD, Solaris. The solution is a bit tricky, some parts of wine and wabi windows emulators are used. The method is loading a DLL as any regular dynamic library. Once it's loaded it almost works since a codec does not require too much windows services eg it won't open a window, it won't call windows specific functions, a codec tipically converts data from one data-set to another so it has got minimal windows dependency.

With the cooperation of VideoLan, we also worked on the code of dvdread and dvdcss, it's needed for reading DVDs. Mainly they developed the code but we sent several patches as well. We borrowed the SVQ1 video codec from xine, this is a Quicktime format for the Sorenson 1 codec. They developed the codec but with the help of an ex-mplayer developer and they borrowed code as well. We're working with FFMpeg on libavcodec which is an open source codec collection with the ability to decode the wide-spread MPEG4, MPEG1/2, MJPEG, H.263 video, and recently also windows media video and audio.

The OMS project (which development was stopped as I've mentioned) gives the MPEG2 and AC3 codecs which are required for DVD playing. These were optimized and further developed by us. Recently we're using liba52 instead of libac3, this is a newer version. MPlayer's libvo is based on libvo of mpeg2dec project, though nowdays they haven't got much more relationship other then their names because MPlayer's libvo was almost totally rewritten but sometimes you can discover the legacy.

The most important difference between MPlayer and other players is that MPlayer uniquely runs within one thread, within one process while other players run within multiple threads. So one thread for audio decoding, one thread for video decoding, one thread for visualization, one for controlling etc. So they separated each function while we implemented the whole thing within one thread. Lots of people are disputing whether it's good or not.

Here, I'd like to mention some argument. At first, scheduler of even a very good operating system won't ever know that we need audio, video, whatever in our processing. This information is known by the kernel of the player (but not the OS) and this determines exactly the amount of free space in buffers, the fact that we should decode video or audio now, or process some command received from the user. So the best is to do the timing in the program itself, and not leaving it for the operating system. Another problem with 2.2 kernels (it was used when I started development) is that switching between threads or even between processes is a slow procedure. 2.4 kernels don't give full solution for this issue either. This does not means that we switch only a few times, but you should think about playing a 30FPS video and sound when we would switch several hundred times per seconds. Since the scheduler of the Linux works at 100Hz (at least before kernel 2.4) it's very few for video playback when we should switch at 33 and 40 milliseconds. For this purpose 10 millisecond resolution was not precision enough.

In kernel 2.5 faster thread switching has been implemented, and resolution has been changed to 1000Hz, but it does not work to rewrite MPlayer for being multithreaded player just because of it. On the other hand, while we saved time would have spent for task switchings there's another detrimental issue with threading. These codecs are optimized for good usage of CPU CACHE. I don't know who knows programming on Intel platform in the audience, but here mainly with newer processors we can issue CACHE controller commands to controll what CPU should load into the CACHE, CPU is optimized to know what exactly resides in the first and second level CACHE, and also code can be optimized not only for register usage but also for keeping in mind the actual content of the CACHE. The problem is that CACHE is erased on task switching so CPU must start afresh to load data into the CACHE with the data of the other task, so this is not too good because we must often switch. For example while we decode a video frame we should switch two or free times during the processing, the CACHE would have been erased each times. This is very time-intensive method and memory is not fast enough. Further advantage of single thread application is to get cleaner and simplier code, so we needn't keep multiple threads in our mind while programming, we needn't manage threads, and we needn't do mutexes and locking to control a thread no to interrupt the other, etc.

So we came through these problems, and because of the simplicity of the code, there are fewer bugs, and stability is also better. At least we did not notice problems like other players have. Debugging is also easier: the program runs withing once thread, so if it stops working we can check out the point where it stops and why. So we don't have to take care of multiple threads when we don't know exactly which thread caused the problem.

But there is a disadvantage of one-thread design: we can't exploit full power of SMP systems. But if we check the problem deeply, we discover that MPlayer is rarely run on servers. We're talking about a video player while multiple CPUs in desktop machines are rare. But even if you have got more CPU, for video playback one CPU is enough, it's not used at full, so we even don't need another CPU. Probably you can use other CPU(s) to run other programs or run MPlayer in multiple instances at the same time.

Currently, MPlayer supports most video formats, in a nutshell: standard MPEG 1 (Video CD), MPEG 2 like DVD and DVB, but you can play MPEG4 as well, including several MPEG4 codecs and variants like DivX versions and Microsoft "MPEG4". At least MPlayer can play all the MPEG4 formats we have a sample file for.

The notion of AVI file format is a loose notion because it only determines the format of the file, but you can use almost all of the codecs inside. But because we support windows DLLs, we can play almost every file. There is only some DLL we have problem with, eg because they have been written for 16 bit windows, so it won't work with out emulator.

Recently the formats windows media audio/video (known as ASF format before) became known worldwide. Of these formats, MPlayer can play versions 7 and 8 without any problem. Now we have native codecs for video 7 and audio 7,8, so you needn't have got DLL for these. The work on 8 is in progress, but now it's only question of time to be solved. Recently, version 9 was announced (I think it's in beta stage for windows). It's a big challange for us, since Microsoft modified the format of codec, so the new codec is not compatible with older DLLs which means we should write new interface for this purpose.

RealAudio, RealVideo: in theory we support all versions, new versions of these formats can be played by MPlayer with the help of native codecs given with their own player. It's important to note here, that these were released for several platforms not only x86, so you should use the correct plugin of them with MPlayer.

The Quicktime format was created on Macintosh, and we can play it as well. Here, we have some problems with codecs because they use some extreme and closed codecs. Some older formats have already been reverse engineered enough to play them, but e.g. with Sorenson 3 and QDM audio we have problems. Nowdays we're working to be able to use windows Quicktime plugins with MPlayer, so after this we can play these formats at last on x86 platform.

Vivo 1, 2 - probably you remember this format when it was well-known. Recently they are extinct, thanks mostly to ASF.

Sony digital video: for format DV there's about four decoders: two open source and two DLLs, so you can choose one; their speed and abilites vary.

Furthermore, there are a lot of old file formats: file formats of computer games, and file formats of rarely used video editors. We can usually play these formats too, though this support was not developed by us but users made them and sent patches.

MPlayer has got one additional advantage: the support of output devices. Most of the players require X11 and they can exploit only few features of X. We support almost all feature of X11, including plain XImage, XSHM and XVideo, where the latter was introduced in XFree version 4.x. If you have proper hardware acceleration for OpenGL, it can be used for video playback as well, and you can use DGA1 and DGA2 for fullscreen playback as well. Furthermore, if there is no X11 installed on a machine or you don't want to use it (it's a common problem on embedded systems), then you can use frame buffer of Linux, SVGAlib, or even the VESA BIOS of video cards, where you don't need any Linux driver for it: so if even there is no Linux driver, you will be able to use MPlayer, it will switch on graphical mode with calling BIOS inside DOS emulation mode, it will query parameters, and it will write directly into the RAM of the card. So almost all cards can be used.

We have developed some drivers as well, this will be discussed later. For example mga_vid, which is a special driver for Matrox video cards. There are some intermediate libraries named wrappers which controll the bridging between different APIs and program. For example SDL, GGI, DirectFB and our own developed one: Vidix. But AAlib is a similar one too, it's created for text modes. Our program supports even hardware decoder cards. Also you can save a video into image files frame by frame.

I've mentioned video drivers before. mga_vid supports Matrox G200 and newer cards. We've started to develop our drivers because we have a big problem here; namely the fact that quality of drivers evolved a lot, but it's far from the goal even now that all videoplayback features can be exploited fully. Matrox video cards can switch between four video buffers automatically during the time when electron beam travels aback (so during the time of vertical retrace), so you won't notice flashes or stripes. Also you can place these buffers into the video RAM and the card will switch between them automatically.

This can't be solved with X drivers, because X places frames into the system memory and a separated process copies into the video memory, and even there they can handle only two buffers. So there are some problems like this, which can be avoided, but these solutions have got negative drawbacks for efficiency and image quality. The VIDIX driver is a unified system which joins the interface of our own developed drivers. It was created when we develope it for drivers of ATI cards but when others remains with their own interfaces.

There is a driver in Linux kernel named TDFXFB which can exploit the full abilities of older 3Dfx cards. Here UHU helped a lot as well, because without them we wouldn't have been able to test and develop for such a wide range of cards. Also many people requested the support of nVidia cards. I don't know who was present in the lunch break at Free SW forum, but this very problem was discussed: hardware vendors don't release any documentation about their chips. This is also our problem with nVidia: the vendor denies the request to release information, while Matrox, ATI or 3DFX released all of the needed information at least for developers. With nVidia we can use only binary drivers which should be reverse enginered and we must try to figure out its internals by us. This is a hard and slow process. We're working on it for several months, but it takes at least another months to get something usable code, and it takes another mounths again to get the optimal solution. Anyhow support of nVidia is planned. I hope some day vendors will develop Vidix drivers in the future or at least they will release documentation to support the development. Unfortunately they're not interested in video playback for Linux nowdays too much. If a hardware vendor takes care of Linux, this usually only means some server hardware support.

Version numbering of MPlayer is a quite an interesting issue. Version 0.01 was released two years ago for want of two days, but since then interestingly numbered versions were released. This is a long story. But it's important that releasing version 0.90 is delayed since April. The first pre beta version was released in April, and we've released 9 pre version since then. We would have released 10 yesterday but it was delayed again, so I don't know exactly when stable 0.90 will be released. Probably within some months or weeks.

The problem is that we don't want to or even can't give correct deadlines. So I can't tell we can release 0.90 at 8 of january: everybody must stabilize everything for that day. This does not work in this way since developers work as their hobby, and this is a fully non-profit oriented project. So we can't say that you will do this for tomorrow and it must be finished, because developers have got their own works as well like attending to school or working for a company. So we can't rely on this. On the other hand, significant amount of development (probably more than fifty percent) is not our work but patches and bug fixes sent by users, which is impossible to rely ahead on. However a bugfix like this can cause a whole avalanche: we will modify lots amount of code which process was started by that inital modification: maybe it flashed the need of a major modification. So even only a new patch like that can delay releasing for months, we delay releasing 0.90 since April and even now we can't forecast a release date. But it's feasible to release in this year. The question of version 1.0 is another hard one. About a year ago I wrote a list of things we want to include in 1.0; what we needs to relase it. That list has been already ticked, but the list is growing since then because I delete a line then add two lines, so it won't ever end. So it will be released likely during the next year. In reality we try to follow a policy not to say too severe date but not too short as well (so we try to create a flexible date) but we haven't got too much success to observe it now.

And some words about our plans in the long run, what there will be after 1.0 if finally it is released. We have plans for a video editor. We have already got the large amount of code needed for this goal (like: codecs, mencoder, demuxers) so for the need to read, write and decode files. Practically an interface should be created and existing code should be modified to be able to be able to work with process multiple video files at the same time. Now it's implemented in a way to be able to read only one file simultaneously. It can be modified with relative smaller work. Furthermore I can mention the need of our own streaming server MPlayer to be able to be used for video conferencing and video broadcasting. For this goal we also have almost everyting though we should implement networking part.

Web browser plugin - this is requested by many users. There is an existing project without reference to us trying to reach this goal, but probably the full solution is to implement it inside code of MPlayer, solve this problem with external controlling code is very hard. Code modularization - this has already been begun nowdays, we're working hard on this code to become a bit more modularized which is quote monolithic at the moment. The program is separated into several libraries internally, and these start to become separated. So if a program needs this, it will use a certain part of MPlayer. It's also important to have the capability to use a single part together so some parts will be portable between players. These were our plans in the long run.

Questions - answers

Q: Doesn't this plan for modularization means that data exchanging between more separated modules will slow down the playback?

A: The question is right, it may slow down. Our plans for modularization is a very slow process just because of it: we must think on each step to find the solution which does not cause slowdown and does not casuse any disadvantage from the point of view of code. The modularization must mean standarizing the behaviour of modules, to force a common interface for all modules. I'm sure we will remove some features in seek of this goal. Our work will be complicated by situations like when a format specific data can't be interpreted by a certain decoder while another one needs it. This is because this process is slow. Nowdays we're working on new configuration parsing code, this not even in CVS because it's quite beta stage development, our work should be tried to be made easier a bit by exactly this new code. So when a certain module starts it can register itself into a common configure code to sign what type of function it can operate with what type of protocol. So finally, we won't force a common interface to all modules too hard, we will leave some freedom an external program to be able to examine a module and its capabilities.

Media files have two main categories: there're interleaved formats where auido and video stream are multiplexed together, ant there're formats when audio and video exists separated in the file. These two categories should be manipulated fully differently so player should know what type of file it try to play at the moment. And there're lots of similar small issues. The fact that MPlayer has reached its current speed and stability is caused by our regard on each of that issues. Other players tried to standarize into a common interface all of the things from the begining.

This is a very interesting point, the problem is similar to X and other drivers, when we try to use a common interface for everything. The example is very good, that's why the Matrox driver was introduced into the Vidix interface, but we still use the kernel driver because the latter one can use more function than we can reach through the standarized common interface of Vidix. So we invented an interface which is not good enough to utilize the possibilities of a new card through our existing common interface. Maybe this is the problem with X as well. The other problem that XVideo was created for digitalization and grabbing and playbacking was included only into the next version which feature was somewhat hack. It lacks of vital features like frame synchronization which is quite problematic in the case of tv-outut because it should be displayed exactly on 25Hz otherwise the picture on your TV won't be stable. This isn't visible on monitors. Maybe this problem was not even thought when they released it, and later they don't put it without rewrite or change the whole thing again.

Q: Have you ever heard about the trend to write a program with using the least possible interfaces? What do you think about this? This is the opposite of the direction you would go, the need of modularization. This trend however says that instead of modularization you should reuse code at source-code level. Because if you don't write too much interfaces then the code is smaller and fewer problem will be caused because of the usage in once. It's like you package something into several layers.

A: We also try to avoid this, but you shouldn't compare our plans to GTK where the whole thing is overcomplicated a bit, and several layers are placed on each other. We try to create only one layer, and that one also only means some optional functions. Codecs were modificated with modularity keeping in mind in this spring. This layer has got a common structure and a codec can set the status for a functionality if it implements that or not. And then the player should check what function from the implemented ones is the most optimal for the current situation. So unoptimal solutions won't be implemented.

This English translation of the originally hungarian text was done by LGB, additional translation fixes were done by Gabucino.