Linux DevCenter    
 Published on Linux DevCenter (http://www.linuxdevcenter.com/)
 See this if you're having trouble printing code examples


Linux for Video Production

by Jono Bacon
08/18/2005

The terms "Linux" and "multimedia" have traditionally sat rather uncomfortably together in a sentence. As open source communities have produced a prominent and capable offering, the focus has largely been on the day-to-day meat and potatoes of the consumer and corporate desktop. Although the efforts towards these word processors, desktop environments, spreadsheets, and business applications have moved steadily, the multimedia side of Linux has traditionally been less focused and more sporadic. This multimedia support has manifested itself in a range of specific projects to play and encode specific formats. The challenge facing both users and developers has been that of consistency. Developers would struggle to create applications, as the nitty-gritty details of managing codecs and multimedia algorithms stood in the way of creating a simple and effective tool. This inconsistency has resulted in users suffering in many aspects of common multimedia creation.

One of the most challenging subsets of multimedia to support has been video editing and production. These challenges have not only included creating usable and effective applications, but also creating consistent support for video cameras, capture devices, hardware accelerators, and more. The current solutions available for video production are rather limited. Cinelerra, possibly the most functional application, suffers from a user interface that acts as a barrier to its expansive feature set. Another prominent contender, Kino, targets an entirely different type of video engineer and is too simple for many video editing requirements. Although both applications have decent support for some mainstream codecs, the applications themselves manage support for devices and codecs, meaning that support for new hardware and codecs takes time to implement.

Looking Forward with PiTiVi

At the GNOME User And Developers European Conference (GUADEC) in Stuttgart, Germany, PiTiVi was one of the most promising applications demonstrated. Currently being developed by French-born Edward Hervey, PiTiVi offers a refreshing outlook for video production on Linux.

Related Reading

Linux Desktop Hacks
Tips & Tools for Customizing and Optimizing your OS
By Nicholas Petreley, Jono Bacon

Hervey's goals for PiTiVi have been clear from the beginning. His intention was to create a simple, intuitive, and powerful editor that can be useful for both home and professional needs. Back in 2004, PiTiVi was Hervey's end of studies project, and he spent his time proving that the freedesktop.org-hosted GStreamer framework could handle audio and video editing. Since March 2005, Hervey took PiTiVi from the ground up and identified where he could go with the existing codebase. He decided to convert the application to use Python. PiTiVi currently uses the new GStreamer 0.9 release, which includes many improvements for supporting complex end-user multimedia applications--such as PiTiVi.

Hervey acknowledges that his three main goals set at the beginning of the project came from analyzing the (dis)advantages of existing editors. PiTiVi incorporates a strong "application-level" plugin system that allows developers to focus on general audio/video editing features in PiTiVi's core, and have some specific uses handled by the plugins. This framework will support editing of a portion of the video by another application (such as CinePaint or Gimp), creating a credit scroller, or even creating a slideshow maker to sync photos and video with a soundtrack. The plugin system is also useful in creating a template system to speed up work in repetitive environments (such as a news desk).

Aside from creating a technically capable and extensible application, Hervey is keen to ensure that the application is suitable in industry. "From the few talks I was able to have with some professional video producers (from the BBC and other editing companies), the design of PiTiVi seems to suit their needs," Hervey says. "The design is only the tip of the iceberg. [The] actual problem with professional video editing is the workflow and how existing tools are used too (Avid comes to mind here)." To solve the workflow issues between different tools, Hervey is working to integrate the Advanced Authoring Format in PiTiVi. This open source format is an accepted industry standard, with implementations in Avid, Adobe Premiere, and Final Cut Pro. "AAF seems like the most promising way for PiTiVi to gradually make its way into the professional editing world," says Hervey. "I am looking forward to meet with more people to get their feedback about how to best support their needs. PiTiVi has a clear separation between the core and the interface, so it is possible to make a more professional-compatible user interface for it."

Although he wrote the first incarnation of PiTiVi in C, Hervey decided to rewrite the application in the popular high-level Python language. The main catalyst for the change was to encourage more people to participate in PiTiVi development; Python considerably lowers the bar for people to get involved. "I'd really like to see all those people who are very knowledgeable at video editing be able to chip in to PiTiVi development, since they have a better idea of what they want," Hervey says. He continues, "Python is a very flexible and extensible language, and the plugin system for PiTiVi would be very hard to put into place if it were created in another language; here, the plugins really extend the application."

Augmenting the increased development rate and accessibility of using Python is the use of other technologies to satisfy the core PiTiVi requirements. One such example is the use of the popular HAL and DBUS tools. These technologies make handling hardware such as video cameras and capture cards far easier in applications such as PiTiVi. With this support, PiTiVi will be able to detect and use any video sources attached to the system and incorporate them into PiTiVi seamlessly.

Making Multimedia Work on Linux

The GStreamer technology at the heart of PiTiVi has been in development for six years. It provides a means of stringing together different components to satisfy various multimedia needs. As an example, an application can read in a file, send it through a decoder plugin, run it through an effects plugin, and then display it to the screen or create an encoded file. GStreamer provides a comprehensive means of hooking together these different plugins in an infinite number of combinations. The plugin architecture opens up the development process to allow anyone to make GStreamer plugins for a range of different needs.

Ronald Bultje has worked heavily with GStreamer since 2001, focusing his efforts on video handling. Before working with GStreamer, Bultje worked on Video4Linux, and has more recently poured his efforts into improving the integration of multimedia into the GNOME desktop. "Historically, Xine, MPlayer, and the like have had the most horrific user interfaces ever imagined. I still can't get my head to accept the fact that I used any of them more than once. You need hacks to get them to integrate into something such as Firefox, and I see nothing that integrated with, for example, OpenOffice.org." He continues, "We want to move forward to create better integration. If I plug in a USB soundcard, I want it to pop up a dialog asking me to switch to that for all applications. We already have an implementation that changes on the fly, so pressing 'yes' would make all applications switch to the new device instantly. I want a decent desktop-integrated video recorder to be started when I plug in my analog or digital camera. I want an audio and video editor to be there; one that is simple to use (think iMovie), but the back end could be so powerful that more complex editors could be created."

Commercial Viability

As GStreamer has developed into a strong technology, Barcelona-based Fluendo has become synonymous with GStreamer. Besides employing and contracting several of the core GStreamer developers--including both Hervey and Bultje--Fluendo sponsors core GStreamer development and provides a range of commercial services and software products. In addition to their Flumotion streaming server product, Fluendo have also worked alongside Nokia to integrate GStreamer technology into the up and coming Nokia 770 Internet Tablet.

Fluendo sponsors PiTiVi development, and Hervey spends each day at Fluendo working on his application. Fluendo obviously see some commercial viability in funding the development: "PiTiVi is a new way of using the GStreamer framework," says Hervey. "[Unlike] other GStreamer-based applications such as players (Rhythmbox, Totem, etc.) or the Flumotion streaming server, a non-linear editor is very demanding; you must be able to seek quickly in any part of the timeline, be able to do very complex combinations of filters and effects, allow the import and export of different media formats, and so forth."

It is evident that PiTiVi is an application that can test the viability of GStreamer for such demanding requirements. "PiTiVi is very interesting for GStreamer development, in the sense that it allows us to stress-test the framework, hence improving it and all the other applications based on it," says Hervey. "[This] results in Fluendo having an even better framework for the services and products it is developing. Also, having such a demanding application allows Fluendo to sell more of the plugins it will shortly be marketing for Real and Window Media codecs."

Hervey is referring to the work going on inside Fluendo to sell the critically important and proprietary codecs that are the backbone of video production. These patent-encumbered and legally restricted codecs have been a challenge in open source adoption, as distributors cannot legally distribute many of the codecs with their software. Fluendo will provide MPEG2, MPEG4, H.264, MP3, AC3, Windows Media, and Real codecs available for use with GStreamer, and as a result, available for all GStreamer-compatible applications such as PiTiVi.

A Bright Future

It is evident that GStreamer is proving to be a solid option for supporting multimedia in Linux and other systems. Although the combination of focused development, funding, and applications such as PiTiVi are clear benefits, the real challenge comes in persuading the greater open source community to take up the technology. One of the criticisms that often levelled towards open source developers is that they often solve a problem by simply developing another framework that will ultimately die from lack of adoption. There have indeed been similar projects, such as the KDE-related aRts project, but Bultje is confident that GStreamer offers a superior option. "[Seriously], you can't compare those. We're technologically way more advanced. The most important difference? We have the community support."

Community support, or more specifically, developer support, really is key to the success of GStreamer. Although the GStreamer offering has not yet reached version 1.0, it is clear that GStreamer offers possibly the most compelling solution, with support both from a commercial entity and in the community. It is promising to see that support is flowing in from a range of developers. This has naturally included the GNOME developers behind Totem, Rhythmbox, Jamboree, Muine, Sound Juicer etc., but there has also been support with KDE with Amarok, JuK, KISS, and commercial support with Flumotion and the Nokia 770.

The result of this increasing GStreamer adoption is that media creation is becoming a reality on open source systems. With such attention to detail going on within the GStreamer community, the process of building solid applications for creating and consuming media is a case of developing the application and not having to worry about the mechanics of dealing with media at such a low level. This will result in more expansive applications created more quickly and efficiently--GStreamer manages the heavy lifting and applications can simply use it.

It is an exciting time for multimedia on Linux, and it will be interesting to see how the story unfolds. We could be tentatively close to breaking the lock that Windows and Mac machines have held on the media creation market.

Jono Bacon is an award-winning leading community manager, author and consultant, who has authored four books and acted as a consultant to a range of technology companies. Bacon's weblog (http://www.jonobacon.org/) is one of the widest read Open Source weblogs.


Return to the Linux DevCenter.

Copyright © 2009 O'Reilly Media, Inc.