When I first got into Python, I read lots of blog posts that mentioned that Python was “the batteries included language”, but those same posts were short on any explanation of what that really meant. A few years and lots of projects later, I think I’m now qualified to at least give a beginner a basic understanding of what people mean when they say that.
What it means
Python has what’s called the “Standard Library”, which is a collection of modules to make some set of tasks within a particular problem domain simpler on you. The standard library modules are all part of the standard Python installation — you don’t have to add it to your installation. Here is a short list of things I’ve used Python for, and the standard library modules I used to get the work done:
- Wrote a simple filesystem backup routine that works on my Linux and Solaris servers, as well as my Mac laptop. For this I used the os, stat, bz2, gzip, time, datetime, and tar modules. Oh – and optparse!
- The first iteration of the loghetti Apache log file analysis tool used one external module, which I’m replacing now with optparse. The other modules used are urlparse, cgi, datetime, operator, re, sys, and mmap.
- I’ve written a couple of simple web API clients using nothing more than urllib/urllib2, ElementTree, and various bits of the xml package (xml.minidom comes to mind).
- I wrote a MySQL backup script using the sys, os, time, shutil, glob, tarfile, and optparse modules.
There are built-in modules for XML/HTML parsing, url parsing, network communications, threading and multiprocessing, image and audio manipulation, and lots of other tasks you’re likely to come across. The story of Python cannot be told from the standard library alone, but you can do an awful lot of work with what’s provided.
Why you care
You care because you want to write the code that solves the problem at hand without worrying about a whole lot of low-level details like socket communications and memory management. I mean, it’s nice to know that Python exposes those low-level details to you should you ever get a wild hair, but if you frequently had occasion to really need that, you’d probably code in C.
Perhaps ironically, you also care about the batteries included because of what it means for those batteries that *aren’t* included. Chances are the Python-driven applications and external modules you wind up using make very heavy use of the standard library modules, making the code for those add-ons simpler to read and understand. It has the potential to make them more reliable and consistent as well.
I’d rather see a threaded application using Python’s threading module rather than trying to code the threading implementation by hand. I’d rather see a Python web server using some descendant of Python’s socket module than coding socket operations by hand. Having things like socket and threading in the standard library means that tools across various problem domains that happen to require certain common functionality work in a standard way where that functionality is concerned.
The same holds true for your own code. If you need to write a Jabber messaging server in Python, and six months later you need to write a queue-based networked job dispatching server, you’re going to be using similar modules, and you might even be able to reuse some of your old Jabber server code. Reuse for the win!
Remember LEGO? I was obsessed with LEGO. I remember becoming so intimately familiar with every type of block, window, platform, person, light post, wheel, and car base that envisioning my own custom-made Grand Galaxy Cruiser was easy, and actually building it (from pieces of probably 10 different LEGO sets) was only slightly more difficult. The standard library, in essence, is your LEGO set. Really, instead of “batteries included” Python’s mantra could well be “There’s a block for that” 🙂
It’s not all beer and skittles
Even in LegoLand there are pieces that don’t fit together the way you’d like. I wanted to build a house once with car windshields in place of windows. Turns out it’s not so simple.
Python 3 improves things a lot in terms of how the standard library is organized, but if you have an external module your code depends on, it might not be ready for Python 3, which means (like me and many others), you’re stuck in Python 2-land for a while. It’s not a big deal really, but I do find that often times I need more than one standard module to do what should really be handled in one.
One example of this is the urllib and urllib2 modules which have an overlapping set of features and problems they address. As a result, you’ll often see these two modules used together. In my own code I’ve had to use both of these modules in addition to the cgi module, *and* the urlparse module. In Python 3, I’d only need urllib. Yay!
XML is another place where I’ve needed multiple modules, and again there has been some consolidation in Python 3.
In the end, this doesn’t get in the way of getting work done very much. It’s just a pattern you start to notice as you work through more of the standard library and use it for your own projects.
Missing Batteries
Python will sometimes surprise you with what’s included. There’s a json module, for example, and the sqlite3 module, both of which are nice to have. But while there’s a whole section of the library devoted to protocols, neither LDAP nor SNMP are represented, and I need both of them 🙁
It might also be nice to have more native file format support. YAML, for one, would be useful.
I also wrote a while back about how nice it would be to have a bash-like ‘select’ feature in Python. There’s already a cmd module which is a pretty good tool for creating interactive command line programs, but without a select-like feature, it’s a little limited.
At the end of the day…
No language provides every single tool that every single developer could possibly need, nor do they implement the tools they do have in the perfect way for all developers who will ever come along. I’ve written lots of code in Perl, PHP, and enough in Java, C++, and C to know that Python does a fantastic job at making my life easier as a developer, and I’m really encouraged by what I’m seeing in Python 3.