I’ve been developing a quick and dirty data migration routine to get terabytes of data stored in AWS S3 as parquet files into our ClickHouse Cloud cluster. I’m really happy that I took some time to read up on the clickhouse local command, which is included in any installation of ClickHouse. Not only was this…
Category: Uncategorized
Python Selenium Webdriver Notes
I used Python Selenium Webdriver for a project wherein a client needs a program that will log into around 25 different web sites, and download a total of 750-1000 different documents. Automating operations across so many different sites has been a huge learning opportunity for me. It’s a lot of fun! I had a passing familiarity…
Sending Alerts With Graphite Graphs From Nagios
Disclaimer The way I’m doing this relies on a feature I wrote for Graphite that was only recently merged to trunk, so at time of writing that feature isn’t in a stable release. Hopefully it’ll be in 0.9.10. Until then, you can at least test this setup using Graphite’s trunk version. Oh yeah, the new…
Book Review: Python Standard Library by Example
Quick Facts: Author: Doug Hellmann Pages: 1344 Publisher: Addison-Wesley (Developer’s Library) ETA: June 5, 2011 Amazon link: http://www.amazon.com/Python-Standard-Library-Example-Developers/dp/0321767349/ref=sr_1_1?ie=UTF8&qid=1307109464&sr=1-1-spell What this book says it does: From the book’s description: This book is a collection of essays and example programs demonstrating how to use more than 100 modules from Python standard library. It goes beyond the documentation…
Lessons Learned Porting Dateutil to Python 3
The dateutil module is a very popular third-party (pure) Python module that makes it easier (and in some cases, possible) to perform more advanced manipulations on dates and date ranges than simply using some combination of Python’s ‘included batteries’ like the datetime, time and calendar modules. Dateutil does fuzzy date matching, Easter calculations in the…
PyCon 2011 Predictions
PyCon 2011 is right around the corner. Are you going? You should. The talks are sick. You can still register — it’s too late to be an early bird, but registration is still open! Well, I am, and I’m here to get the rumor mill started by sharing some predictions for this year’s PyCon. Packaging…
The Makings of a Great Python Cookbook Recipe
I’ve seen some comments on Twitter, Buzz, Reddit, and elsewhere, and we’ve gotten some suggestions for recipes already via email (thanks!), and both Dave and I thought it’d be good to present a simple-to-follow ‘meta-recipe’; a recipe for making a great recipe that has a good shot at making it into the cookbook. So let’s…
Good Things Come in Threes: Python Cookbook, Third Edition
It became official earlier today that David Beazley and myself will be co-editing/co-curating the next edition (the Third Edition) of the Python Cookbook. That’s really exciting. Here’s why: It’s Python 3, Cover to Cover Go big or go home. The third edition will be a Python 3 Cookbook. This by itself makes this a rather…
Nose and Coverage.py Reporting in Hudson
I like Hudson. Sure, it’s written in Java, but let’s be honest, it kinda rocks. If you’re a Java developer, it’s admittedly worlds better because it integrates with seemingly every Java development tool out there, but we can do some cool things in Python too, and I thought I’d share a really simple setup to…
Python Packaging, Distribution, and Deployment: Volume 1
This is just Volume 1. I’ll cover as much as I can and just stop when it gets so long most people will stop reading đŸ™‚ I’ve been getting to know the Python packaging and distribution landscape way better than I ever wanted to over the last couple of weeks. After 2 or 3 weeks…