At last month's PyCon in Washington D.C. several developers of the Twisted network application framework gave presentations. (Papers are available on the Python wiki.) In some ways, it was Twisted's big debut. Many attendees have made positive remarked the presentations. Once obscured, Twisted is stepping into the light.
The conference also gave Twisted's developers a chance to meet in one place. The Twisted project was originally conceived on the OpenProjects (now Freenode) IRC channel, #python. Its core developers are spread all over the world. While inspired by extreme programming--which among other things, promotes programming in pairs--they rarely have an opportunity to do work that closely.
I spoke with Chris Armstrong, a.k.a. radix, about both Twisted and the conference. Armstrong lead a productive pre-conference programming sprint at PyCon. 14 key Twisted developers were joined by 2 other developers to discuss and refine some important Twisted modules. He told me that "it went great! We're incredibly productive when together. It's not easy to do much pairing over the internet--you don't get the same feedback."
I heard about Twisted a couple of years ago but never took a close look at it. Helping design a distributed middleware application recently, I decided to take a closer look at Twisted. It wasn't easy for me to understand it. There was so much to Twisted I didn't think I would need. I felt overwhelmed. I asked Twisted developer Chris Armstrong, "what's the best way for someone to approach learning Twisted?"
Also in Python News:
"I try to stress that they look at the parts they need for their application," he told me. "Usually it's okay after that--depending on what parts they need. Woven is still a moving target and sometimes people have problems with that." Woven is the module used to create web interfaces for networked applications. Armstrong tells me its getting better. Some much needed documentation is on the way. "Start with what you need. If you are writing a web application, don't look at how HTTP is implemented, for example."
That said, I would have found Twisted easier if I had understood asynchronous programming better. Rather than using multiple processes or threads to handle multiple clients, Twisted uses an asynchronous, event-driven model. I didn't begin to understand that until I started to think of Twisted as being like a GUI application framework like Qt or Gtk. GUI frameworks are also event driven application frameworks.
What might also have helped was to spend a little time playing with asyncore.py, Python's own built in asynchronous socket library. Asyncore comes from Sam Rushing's Medusa, the framework used by Zope. Sam has a low level tutorial on Medusa that would have helped me immensely for general concepts. Twisted abstracts all of this nicely. You don't have to understand how Twisted does things, but I feel more comfortable understanding what's being abstracted.
Twisted originated as the underlying framework for an open source, massively multi-player game called Twisted Reality. Armstrong tells me when he got involved with the project in January of 2001, "I was in High School and was bored. I was just getting into Python and Twisted looked cool. I was interested in massively multi-player games. Twisted started as a multi-player interactive fiction game, twisted reality. Everything came out of that, the web server, the protocol support." Eventually the split off Twisted Reality as a separate project. But there is certainly an element of fun remaining in Twisted, including module names that are puns on peanut butter and jelly sandwiches.
Armstrong told me development of Twisted Reality continues. "When I first got into the project, Twisted Reality development was going strong, but since has faded off. We recently started a rewrite. It's going slow, but hopefully that will change soon."
Many of the Twisted developers can still be found online daily at the Freenode IRC #twisted and #python channels. Many users are there as well. Twisted community support is strong, and it's easy to get assistance either through IRC or the mailing list. For businesses whose needs might exceed what the community can easily provide, a couple of companies provide commerical support.
Twisted is the basis for several other open source projects that exhibit the same friendliness and support. If you are looking for a hot project to get involved with, Twisted is a great place to start. Armstrong tells me working with Twisted has been a great place to learn programming. "All my programming knoledege--I wouldn't say it's self taught. Glyph (Lefkowitz) and Dash (Allen Short) have been great mentors for me over the last few years. I have learned a ton from Glyph and a lot of it though the Twisted project."
Twisted developers recently released version 1.0.4. This point release includes several improvements as well as bug fixes, including optimizations of Twisted's GUI integration tools. It also introduces a new version of Jelly, Twisted.Spread's serialization module allowing you to securely and safely pass objects over the network in a language neutral way. NewJelly will eventually replace the existing Jelly, but because it isn't backwards compatible, it's available as a separate module. Twisted.Lore now provides support for generating presentation slides.
Take a close look at Twisted before embarking on your next network application project. Even if you have no need for the higher level tools, you can take advantage of Twisted's handling of network transports and protocols as well as it's asynchronous model to quickly develop fast, robust, secure servers and clients.
Stephen Figgins administrates Linux servers for Sunflower Broadband, a cable company.
Read more Python News columns.
Return to Python DevCenter.