Heard about this from Tarek, and you can find more of them on Planet Python (where I found Tarek’s post).
- What’s the coolest Python application, framework or library you have discovered in 2009?
- Probably Tornado. Tornado is an interesting application, because it blurs the line a bit between a framework like Django and a traditional web server. If you can picture it, it’s a barebones, lightweight, almost overly simplified Django, with a production-ready web server instead of Django’s built-in dev server. In reality, Tornado is (or feels) more integrated than that, but that leads to some interesting issues on its own.
Still, it’s been a heckuva lot of fun to play with. One thing that always concerned me about Django was the ORM. It’s fine for my little hobby website, or a simple wiki for my wife and I to use, and even some slightly more complex applications, but if you have a database-driven site that serves “lots and lots” of users and it needs to manage complex relationships and never slow down…. I don’t trust the ORM to do that. What’s more, I’m actually pretty skilled in data modeling, database administration, etc., and I understand abstraction. I don’t really require Django’s models (though, again, I love Django for doing low-traffic sites very quickly).
Playing with web frameworks is a lot of fun, and if you’ve played with a few, you’ll like the “clean slate” that Tornado provides you to mix-n-match your favorite features of the frameworks you’ve used. I’ve done some hacking around Tornado to provide some generic facilities I’m likely to use in just about every project I use Tornado for. The sort of pseudo-framework is available as Tornado-project-stub on github.
- What new programming technique did you learn in 2009 ?
- Thread and process pool management. Whereas in previous roles I focused on optimization at the system and network level by testing/deploying new tools, poking at new paradigms, or just reworking/overhauling things that were modeled or configured suboptimally, my new role is something you really would call “scalability engineering”. I believe everything I’m involved in at the moment involves the words “distributed”, “asynchronous”, “multithreaded”, “multiprocess”, and other terms that imply “fast” (for varying definitions of fast, depending on the project).
Though I’ve had to understand threading and multiprocessing (and microthreads and other stuff too) in the past, and I’ve even written simple threaded/multiprocessing stuff in the past, I’m now knee deep in it, and am getting into more complex scenarios where I really *need* a pool manager, and would really *like* to pass objects around on the network. Happily, I’m finding that Python has facilities for all of this built-in.
Aside from that, I have to say that while most of what I’m doing now doesn’t involve techniques I’ve never heard of, I’m really reveling in the opportunity to put them into practice and actually use them. Also, since I now code full-time, I find the ability to code doesn’t ever escape my brain. I can code fast enough now that I can implement something two or three different ways to compare the solutions in no time!
- What’s the name of the open source project you contributed the most in 2009 ? What did you do ?
- Actually, it’s not released yet, but I’ve almost completed a rewrite of Golconde, a queue-based replication engine. I was able to make the queuing protocol, the message processor (the thing that takes queued messages and turns them into database operations), and the database backend swappable. Golconde was written to support STOMP queuing and PostgreSQL specifically. I’ve already used STOMP and AMQP interchangeably in the rewrite, and I’m now on to swapping out the database and message processor components, documenting how users can create their own plugins for these bits along the way.
Golconde is also threaded. My rewrite is currently threaded as well, but I’m going to change out the threads in favor of processes, because the facilities that can help the project moving forward (in the short term, even) come more from multiprocessing than from threading. One thing I’ve already accomplished is refactoring so that there need be only a single thread class, which makes using worker pools more convenient and natural. It’s coming together!
- What was the Python blog or website you read the most in 2009 ?
- I read Planet Python every day, and keep up with Python Reddit as well. Besides aggregators, Corey Goldberg and Jesse Noller seem to overlap my areas of interest a lot, so I find myself actually showing up at their blogs pretty often. Neither of them blog enough, imo 😉
- What are the three top things you want to learn in 2010?
- Nose – because I want to become so good at testing that it makes me more productive, not less. Right now I’m just clumsy at testing, and I come across situations that I just plain don’t know how to write a test for. I need to know more about writing mock objects and how to write tests for threaded/multiprocessing applications. I know enough to think that Nose is probably the way to go (I’ve used both unittest and doctest before, so I’m not totally ‘green’ to the notion of testing in general), but I haven’t been able to work it into my development process yet.
- Erlang – There doesn’t seem to be a language that makes concurrency quite as brainless as Erlang. That said, learning the language and OTP platform is *not* brainless.
- Sphinx – I hear all the cool kids are using it. Some people whose opinions I trust love it, but I have some reservations based on my experience with it. The one thing that gives me hope is that Django’s documentation site, which I like the interface and features of, uses it.