Archive for the ‘Other Cool Blogs’ Category

Programmers that… can’t program.

Monday, March 15th, 2010

So, I happened across this post about hiring programmers, which references two other posts about hiring programmers. There seems to be a demand for blog posts about hiring programmers, but that’s not why I’m writing this. I’m writing because there was this sort of nagging irony that I couldn’t help but stumble upon.

In a blog post, Joel Spolsky talks about the mathematical inaccuracies associated with claims of “only hiring the top 1%”. It seemed pretty obvious to me that whether or not you’re hiring the top 1% of all programmers is pretty much unknowable, and when managers say they hire “the top 1%”, I assume they’re talking about the top 1% of their applicants. Note too that I always thought it was idiotic to point this out, because, well, isn’t that what you’re SUPPOSED to do? You’re not very well going to aim for the middle & hope for the best are you?

Apparently I’ve been giving too much credit to management. There I go giving people with ties on the benefit of the doubt again.

Then, in another blog post, Jeff Atwood talks about how it’s very difficult to even get interviews with programmers who can actually program. The problem is real.

The original blog post that pointed me at the two others is one by Roberto Alsina where he talks about his own methods for weeding out the non-programmers. He’s clearly seen the issue as well.

But if you open all three of these posts in separate tabs and read them, you’re likely to come away with the same basic problem I did:

  • Who the hell are these managers who can’t figure out a dead simple statistics problem?
  • How can a person fairly inept at simple math be qualified to make a hiring decision for anything but a summer intern?

That sorta blew my mind a little. But it blew my mind a lot when Atwood started describing the problems that interviewees *couldn’t* perform in an interview! One task described by Imran was called a ‘FizzBuzz’ question. Here’s one such question:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

Here’s the part that blew my mind: He says, and I quote:

Most good programmers should be able to write out on paper a program which does this in a under a couple of minutes.

Want to know something scary ? – the majority of comp sci graduates can’t. I’ve also seen self-proclaimed senior programmers take more than 10-15 minutes to write a solution.

That’s amazing to me. I decided to quickly pop open a Python prompt and see if I could do it:

>>> for i in range(1,101):
...     if (i % 3 == 0) and (i % 5 == 0):
...             print i,'FizzBuzz'
...     elif i % 3 == 0:
...             print i, 'Fizz'
...     elif i % 5 == 0:
...             print i, 'Buzz'
...     else:
...             print i

Note that I’ve taken the liberty of printing out the numbers in addition to the required words. I’m playing the role of interviewer and interviewee here, and wanted to be able to easily verify that things were correct, since there was no time for unit testing 🙂

Turns out it worked on the first try! That was pasted directly from my terminal screen. I didn’t time myself, but it took far less than 5 minutes. This leads to my other question, of course, which is “if you’re going to complain about CS degree holders not writing good code, maybe it’s time to open the doors to non-CS degree holders?”

New Job, Car, Baby, and Other News

Thursday, October 22nd, 2009

New Baby!

I know this is my geek blog, but geeks have kids too, so first I want to announce the birth of our second daughter, Sadie, who was born on September 15th. She’s now over a month old. This is the first time I’ve stayed up late enough to blog about her. Everyone is healthy, if slightly sleep-deprived 🙂

New Job!

The day before Sadie’s birth, I got a call with an offer for a job. A *full-time* job, as a Senior Operations Developer for After learning about the cool and very geeky things going on at MyYearbook during the interview process, I couldn’t turn it down. I started on October 5, and it’s been a blast digging into all of the cool stuff going on there. While I’m certainly doing my fair share of PHP code review, maintenance, and general coding, I’m also getting plenty of hours in working out the Python side of my brain. I’m finding that while it’s easier switching gears than I had anticipated, I do make some really funny minor syntax errors, like using dot notation to access object attributes in PHP ;-P

What I find super exciting is something that might turn some peoples’ stomachs: at the end of my first week, I sat back and looked at my monitors to find roughly 15 tabs in Firefox open to pages explaining various tools I’d never gotten to use, protocols I’ve never heard of, etc. I had my laptop and desktop both configured with 2 virtual machines for testing and playing with new stuff. I had something north of 25 terminal windows open, and 8 files open in Komodo Edit.


The projects I’m working on run the gamut from code cleanups that nobody else has had time to do (a good tool for getting my brain wrapped around various parts of the code base), to working on scalability solutions and new offerings involving my background in coding *and* system administration. It’s like someone cherry-picked a Bay Area startup and dropped it randomly 30 minutes from my house.

My own business is officially “not taking new clients”. I have some regular clients that I still do work for, so my “regulars” are still being served, but they’ve all been put on notice that I’m unavailable until the new year.

New Car!

I’m less excited about the new car, really. I used to drive a Jeep Liberty, and I loved it. However, in early September, before Sadie’s arrival, it became clear to me that putting two car seats in that beast wasn’t going to happen. The Jeep is great for drivers, and it has some cargo space. It’s not a great vehicle for passengers, though.

At the same time, I was running a business (this was before the job offer came along), and I was finding myself slightly uncomfortable delivering rather serious business proposals in a well-used 2003 Jeep. So, I needed something that could fit my young family (my oldest is 2 yrs), and that was presentable to clients. So, I got a Lexus ES350.

I like most things about the car, except for the audio system. It seems schizophrenic to me to have like 6 sound ‘zones’ to isolate the audio to certain sets of speakers, but then controls like bass and treble only go from 0 to 5. Huh? And the sound always sounds like it’s lying on the floor for some reason. It’s not at all immersive. The sound system on my Jeep completely kicked ass. I miss it. A lot.

Other News

I’ve submitted an article to Python Magazine about my (relatively) recent work with Django and my (temporarily stalled) overhaul of, and my experiences with various learning resources related to Django. If you’re looking to get into Django, it’s probably a good read.

I’ve been getting into some areas of Python that were previously dark, dusty corners, so hopefully I’ll be writing more about Python here, because writing about something helps me to solidify things in my own brain. Short of that, it serves as a future reference point in case it didn’t get solidified enough 🙂

My sister launched The Dance Jones, a blog where she talks about fitness, balance, dance, and stuff I should probably pay much more attention to (I’m close to declaring war on my gut). Also, if you ever wanted to know how to shoulder shimmy (and who hasn’t wanted to do that?), you should check it out 🙂

Two extremely handy geek URLs

Wednesday, March 25th, 2009

I know, I know. I haven’t been posting nearly enough. But I did come across two URLs that are too handy not to pass on:

  1. Command-line-fu: this is a repository of handy one-liners submitted by pretty much anyone. You can log in with OpenID or register on the site itself. I expect this, or something like it, will become a great resource. You can browse the sweet one-liner goodness, or “grep the archive”. Nice.
  2. Down for Everyone or Just me? is a site that’s handy to know about if you’re, say, holed up in a hotel room, forgot to set up port forwarding on your FIOS router, and so don’t have a remote shell to test from, and you can’t reach a site. Pop the url into this site, and it’ll test access for you. Of course, it’s limited — it’ll change url’s with “:22” to “%3A22”, so you’re not going to get it to be a generic service tester, but still… handy!


What Ordinary Users Think About IE: Debunked

Wednesday, December 17th, 2008

Point all of your chain-mail-forwarding family and friends at this post. It’s a collection of things people have said to me, or that I’ve overheard, that reveal little tidbits about what people are thinking when they use IE.

I have to use IE – it’s my internet!

IE is not your internet. IE is what’s known as a web browser. There are lots of different web browsers. IE just happens to be the one that comes with Windows. It doesn’t make it a good browser or anything. It’s just there in the event that you have no other browser. If the only browser on your system is IE, the first thing you should do is use it to download Firefox by clicking here.

If IE is so horrible, how come everyone uses it?

They don’t, actually. There was a time not too long ago where over 90% of internet users used IE. However, with the constant flood of security issues (IE usage really should be considered dangerous at this point), IE’s horrible support of web standards (which makes it hard for web developers to create cool sites for you to use), and its inability to keep up with really cool features in modern browsers, its share of the internet usage market has been declining steadily over the last couple of years. In fact, this source puts IE usage at around 45% currently, so not even a majority of people use IE anymore, if statistics are to be believed. Accurate statistics for browser use are difficult to nail down, and are probably more useful to discern a trend, not hard numbers. Still, the usage trend for IE is moving downward, steadily, and not particularly slowly. If you’re still using IE, you’re almost a dinosaur. Just about the entire tech-savvy world has migrated over to Firefox, with small contingents choosing Safari (Mac only) and Chrome (Windows only). Very small camps also use Opera and Konqueror.

This is also not to be trusted, but it’s my opinion based on observation of the IT field over the past 10 years: of the 40% of people still using IE, probably half of them are forced to use it in their offices because they don’t have the proper permissions on their office computers to install anything else. The other half probably just don’t realize they have any choice in the matter. You do. There are other browsers. I’ve named a few in this post. Go get one, or three, of them.

Will all of the sites I use still work?

It has always been exceedingly rare that a web site actually *requires* IE in order to work properly. Your online banking, email, video, pictures, shopping, etc., will all still work. The only time you might need IE around is to use the Microsoft Update website. In all likelihood, you’ll be much happier with your internet experience using something like Firefox than you ever were with IE. Think about it this way: I’m a complete geek. I use the internet for things ordinary users didn’t even know you could do. I bank, shop, communicate, manage projects, calendars and email, registered and run my business completely online. It’s difficult to think of a task that can be done on the internet that I don’t use the internet for, and I haven’t used IE in probably 8 years, and have not had any issues. If you find a web site that absolutely, positively CANNOT be used UNLESS you’re viewing it with IE, please post it in the comments, and I’ll create a “hall of shame” page to list them all, along with alternative sites you can access WITHOUT IE, which probably provide a better service anyway 🙂

I’m not technical enough to install another browser.

Who told you that?! That’s silly. You installed Elf Bowling didn’t you? C’mon, I know you did. Or what about that crazy toolbar that’s now fuddling up your IE window? Or those icons blinking down near the clock that you forgot the purpose of. At some point, you have installed something on your computer, and it was, in all likelihood, harder to do than installing Firefox would be. It’s simple. You go here, click on the huge Firefox logo, and it presents you with super-duper easy instructions (with pictures!) and a download. It takes less than 3 minutes to install, and you DO NOT have to know what you’re doing in any way or be geeky in any way to install it. If you can tell whether you’re computer is turned on or not, you’re overqualified to be a professional Firefox installer.

I Like IE. I have no problems with IE.

Whether you realize it or not, you have problems with IE, believe me. I had a cousin who said he had no problems with IE too. Then he came to my house one day, knocked on my door, and when I opened it, he handed me a hard drive from his computer. He said that all of his pictures of his first-born child were on there, and his computer had contracted a virus, and he couldn’t even boot from the hard drive. So it was up to me to recover the only pics he had of his only son being born. True story. Turns out, I tracked down the virus on the hard drive, and it was contracted by IE. Also, it wasn’t the only virus he had. If you think you’re safe because you have antivirus software, you’re sadly mistaken. He had it installed too, but it hadn’t been updated in 6 months, so any viruses released since the last update weren’t recognized by the antivirus software, and were allowed to roam freely onto his hard drive.

There has never, in the history of browsers, been a worse track record with regards to security than IE. Never. I promise – but you’re free to Google around for yourself. Half of the reason antivirus software even exists is purely to protect IE users (though email viruses are a problem independent of what browser you use, admittedly).

The other reason you might say you like IE is because you’ve never used anything else. As an alternative, I strongly suggest giving Firefox a shot.

Why do you care what browser I use?

I’m a technology guy. I’m one of those people that would work with technology even if he wasn’t being paid. Some people care about cooking, or quilting, or stained glass, or candlemaking, or knitting, or sewing, or horticulture, or wine. Heck, my mom cares about every single one of those things! Me, I care about technology, and I care about the internet. I want the internet to be a better place. Browsers play a non-trivial role in making the internet a better place. Also, one reason I care about technology is that it helps people do things they might otherwise be unable to do. Browsers enable users to do great things, and it allows us developers to make great things available to you. But when countless hours are spent trying to make things work with IE, it just slows everything down, and you don’t get cool stuff on the internet nearly as fast as you could.

So, it’s less about me caring what browser you use. In fact, I don’t really care if you use Firefox or not, it just happens to be the best browser out there currently. If you want to try something completely different, I encourage that too. It’s more about me caring about technology, the internet, and your browsing experience.

How Are You Staffing Your Startup?

Monday, December 15th, 2008

I have, in the past, worked for startups of varying forms. I worked for a spinoff that ultimately failed but had the most awesome product I’ve ever seen (neural networks were involved, need I say more?), I helped a buddy very early on with his startup, which did great until angel investors crept in, destroyed his vision, and failed completely to understand the Long Tail vision my buddy was trying to achieve, and I worked for a web 2.0 startup which was pretty successful, and was subsequently purchased… by another startup!

Working in academia for 6 years also exposed me to people who are firing up businesses, or projects that accidentally become businesses, and some of those go nowhere, while others seem to be on the verge of NYSE listing now, while a year ago they were housed in the smallest office I’ve ever seen, using lawn furniture for their workstations.

Of course, I’ve also consulted for, and been interviewed by, a host of other startups – recently, even.

First, the bad news

The bad news is that most or all of these startups are headed by developers, and they have applied *only* dev-centric thinking to their startup. They’ve thought about how to solve all of the app-level issues, mapped out use cases, drawn up interfaces, hacked together prototypes, and done all kinds of app-level work. Then, they’ve hired more developers. Then more after that.

Some seem to have given almost zero consideration to the fact that their application might become successful, and its availability might become quite critical. They haven’t given much thought to things like backups or disaster recovery. They have no plan for how to deploy their application such that when it comes time to scale, it has some hope of doing so without large amounts of downtime, or huge retooling efforts.

They’ve also given very little thought to how to enable their workforce to communicate, access their applications and data remotely without huge security compromises, and generally provide the back end system services necessary to run a business effectively (though, admittedly, most startups don’t require much in the way of things like NFS, or even internally-hosted email in the very beginning).

In short, they’ve either assumed that systems folks’ jobs are so easy that it can be handled by the developers, or they think that scalability lives entirely in their code, or they’re just not thinking about system-level aspects of their application at all. And don’t even get me started about the databases I’ve seen.

I know of more than one startup, right now, months late in going live. None of them, at the time I spoke to them, had a systems person on staff, or a deployment plan that addressed things that a systems person would address. What they had was a lot of developers and a deadline. Epic fail.Yes, even if you use agile development methodologies.

The Good

The good news is that, while some companies hire no systems folks at all and flounder around system-related issues forever, others hire at least one or two good systems folks, and make them a part of a collaborative effort to solve systems problems in interesting ways, utilizing the knowledge and experience of both systems and development personnel to create truly unique solutions. When sysadmins and developers collaborate to solve these issues, I have learned that they can create things that will blow your mind (in a good way).

In fact, Tom Limoncelli wrote recently that systems administration needs more PhDs. Well, I suppose that would help, but I think we’d get really far, really fast, if we could just break down some of the walls between sysadmins and developers, give them a common goal, and let them hash it out. Sysadmins have an understanding of the system and network-related issues that developers aren’t likely to have. Developers, in most cases, can probably write better code, much more quickly, than a sysadmin. Developers and sysadmins working together, sharing their knowledge and communicating with each other, can solve systems problems in new, unique, creative, and very effective ways.

The End

In the end, issues facing startups now blur the line between development and system administration a bit more than in the past. There are problems that need solving for which there is no RPM or Deb package. These problems require some knowledge of how related or analogous problems have been solved in the past. A knowledge of the systems and development approaches that have worked, and why. Enough experience to have seen where and when things go bad, and why. It also requires creative and critical thinking. I think that good, senior systems and development people have these skills, and much more.

For whatever reason, it seems that the only time these two camps ever meet is on opposite sides of a deployment or application support problem. Perhaps this happens with enough frequency for people to think that the two camps can’t, or won’t, work together. They can, and they will. People with a love for technology will come together if the common goal involves furthering technology and its use, regardless of their background. Sure, it takes proper management like any other project, but it can be done.

If you’ve had experiences, good or bad, with dev/sysadmin collaboration, I’d love to hear your stories!

WordPress 2.7 – Ahhhhhh!

Thursday, December 11th, 2008

I guess WordPress doesn’t consider the changes they’ve made in 2.7 (released today) to be big enough to warrant a change to the major version number (which would make it 3.0). However, there are a few features now built-in that I’ve been dreaming about for so long that simply incrementing the second number seems to sell this version short. At least they named it after one of my favorite jazz musicians. This release is called “Coltrane”. Nice.

My top two feature requests: Check!

First and foremost, the number one thing on my list of desired features is now a reality: I can make bulk changes to the categories of my posts. So, when I add a category to WordPress, and then realize that lots of my old posts really belong there, I don’t have to go searching around and changing them by hand. I still might take a stab at doing back-end automation here, by scripting a tool that’ll search the content of all of my posts, and if the content has, say, 2 out of 3 terms in my search criteria, it’ll add the post to the category, using whatever database trickery is necessary. However, this solves almost all of my needs (save my need to hack things, sometimes for its own sake).

The other feature I’ve been wanting for a long time is also now a reality: replying to comments without having to go to the post page to do it. You can now moderate and reply to comments right in the dashboard.

This, for me, is huge. I’ve been waiting for these two particular features since about 2005.

More Baked-in Goodness

Some other niceties are now built-in that used to be addon modules in WordPress, which is great, because I’m always worried about third-party modules breaking and being abandoned as new WP releases come out. The nicest for me, as someone who maintains their own wp install, is the automated WP upgrade. Used to be an addon, now built in.

Another nice feature, if you *are* someone who doesn’t mind third party modules, is that now you can browse available modules, and install them, without leaving the wp interface.

Yes, another complete redesign

The admin interface has been completely overhauled, again. The last time they did this, a buddy and I discussed it, and although he felt one or two things were nicer, I felt that they had not addressed the biggest problems with the interface. Well, they fixed it by doing something I didn’t actually expect: they admitted defeat.

Instead of overhauling the interface, they’ve empowered the user to do it for themselves. Want the editor to fit the width of the browser window? No problem. Never use all of those features in the editing interface? Get rid of them. Only just noticed all those news items in the dashboard? Make them more prominent. You can do all of this by dragging and dropping things around, or collapsing them to ‘icon-only’ view.

I am writing this in 2.7, and in the editor interface, I definitely feel like more of what I need is readily available instead of buried somewhere in the countless blocks and sections and whatnot – which reminds me that there’s also a new (and quite nice) menu interface – also a part of the interface you can customize.

Check out the video and notes on the WordPress site. The tour video does a great job of giving a quick rundown of the new features I’ve mentioned here, and lots and lots of features I *didn’t* cover.

Boto 1.5b glitch, and workaround

Thursday, November 20th, 2008

Boto is a Python library for interacting with Amazon’s web services. I’ve used it in the past, and am currently using it for an ‘s3get’ implementation based on a simple example I found buried in a post on Patrick Altman’s blog.

While testing my code, I noticed I was getting import errors from boto/, because I didn’t have a module on my system named ‘hashlib’. Then I found an svn trunk commit that clued me in to the fact that I wasn’t supposed to have hashlib, because I was running a pre-2.5 version of Python. They had put in a fix for pre-2.5 users, but somehow it wasn’t being obeyed.

Then I noticed that the import errors weren’t from, where the fix was committed, but from, which was explicitly importing the module itself. Closer inspection revealed that it was also importing, which itself imports hashlib. I commented out the explicit import in (and, later, in boto/s3/, and stopped getting import errors.

If you’re still having issues with Patrick’s code, it’s probably because you need to change this line:

if name == 'main':

To this:

if __name__ == '__main__':

I’m a Top 25 Geek Blogger… for some value of “Top”

Monday, November 10th, 2008

I’m not someone who wakes up every day and looks at how my blog is ranked by all of the various services. I check out my WordPress stats, but that’s really about it. However, someone went and did some of the work for me, and they’ve decided that, of the blogs that they read or that were suggested to them, this blog ranks #20 in a listing of 25.

I’m really flattered, but wonder if it’s an indicator that this is a quality blog, or that they should aim higher in their blog reading ;-P  Either way, listing 25 bloggers in a flattering way is a fantastic marketing technique, because most of us are probably egomaniacal enough to say “Hey! Look!” and link back to the list on *your* blog, resulting in lots of traffic. Kudos, and thanks Mobile Maven!

Useful stuff – 2008 – first half

Friday, July 11th, 2008

Having a Google account is sometimes useful in ways you hadn’t planned for. For example, at a few different employers I’ve been at, I’ve had to prepare for reviews by providing a list of accomplishments to my supervisor. One decent tool for generating this list is email, though it can take some time. Another useful tool is the Web History feature of your Google account.

Though this isn’t necessarily indicative of everything I’ve accomplished in the first half of 2008 per se, it’s definitely indicative of the types of things I’ve generally been into so far this year, and it’s interesting to look back. What does your Web History say?

  • Gearman – this is used by some rather large web sites, notably Digg. It reminds me a little of having Torque and Maui, but geared toward more general-purpose applications. In fact, it was never clear to me that PBS/Maui couldn’t actually do this, but I didn’t get far enough into Gearman to really say that authoritatively.
  • How SimpleDB Differs from a Relational Database – Links off to some very useful takes on the “cloud” databases, which are truly fascinating creatures, but have a vastly different data management philosophy from the relational model we’re all used to.
  • Reblog – I found this in the footer of someone’s blog post. It’s kinda neat, but to be honest, I think you can do similar stuff using the Flock browser.
  • Google Finance APIs and Tools – did I ever mention that I had a Series 7 & 63 license two months after my 20th birthday? I love anything that I can think for very long periods of time about, where there’s lots and lots and LOTS of data to play with, where you can make correlations and answer questions nobody even thought to ask. Of course, soon after finding this page I found the actual Google Finance page, which answers an awful lot of potential questions. The stock screener is actually what I was looking to write myself, but with the data freely available, I’m sure it won’t be long before I find something else fun to do with it. I’m not a fan of Google’s “Feeds” model, but I’ve dealt with it before, and will do it again if it means getting at this data.
  • Bitpusher – it was recommended to me as an alternative to traditional dedicated server hosting. Worth a look.
  • S3 Firefox Organizer – This is a firefox plugin that provides an interface that looks a lot like an FTP GUI or something, but allows you to move files to and from “buckets” in Amazon’s S3 service.
  • Boto – A python library for writing programs that interact with the various Amazon Web Services. It’s not particularly well-documented, and it has a few quirks, but it is useful.
  • OmniGraffle – A Visio replacement for Apple OS X. I like it a lot better than Visio, actually. It has tons of contributed templates. You shouldn’t have any trouble making the switch. A little pricey, but I plunked down the cash, and have not been disappointed.
  • The Python Queue Module according to Doug – Doug Hellmann’s Python Module of the Week (PyMOTW) should be published in dead tree form some day. I happen to have some code that could make better use of queuing if it were a) written in Python, and b) used the Queue module. I was a little put off by the fact that every single tutorial I found on this module assumed you wanted to use threading, which I actually don’t, because I’m not smart enough…. though the last person I told that to said something to the effect of “the fact that you believe that means you’re smart enough”. Heh.
  • MySQL GROUP modifiers – turns out this isn’t what I needed for the problem I was trying to solve, but the “WITH ROLLUP” feature was new to me at the time I found it, and it’s kinda cool.
  • WordPress “Subscribe to Comments” plugin – Baron suggested that it would be good to have this, and I had honestly not even thought about it. But looking around, this is the only plugin of its kind that I found, and it’s only tested up to WP 2.3x, and I’m on 2.5x. This is precisely why I hate plugins (as an end user, anyway. Loghetti supports plugins) 😉
  • Lifeblogging – I had occasion to go back and flip through some of the volumes of journals I’ve kept since age 12, wondering if it might be time to digitize those in some form. I might digitize them, but they will *not* be public I don’t think. Way too embarrassing.
  • ldapmodrdn – for a buddy who hasn’t yet found all of the openldap command line tools. You can’t use ‘ldapmodify’ (to my knowledge) to *rename* an entry.
  • Django graphs – I haven’t yet tried this, because I’m still trying to learn Django in what little spare time I have, but it looks like there’s at least some effort towards this out there in the community. I have yet to see a newspaper that doesn’t have graphs *somewhere* (finance, sports, weather…), so I’m surprised Django doesn’t have something like this built-in.
  • URL Decode UDF for MySQL – I’ve used this. It works really well.
  • Erlang – hey, I’m game for anything. If I weren’t, I’d still be writing all of my code in Perl.
  • The difference between %iowait in sar and %util in iostat – I use both tools, and wanted the clarification because I was writing some graphing code in Python (using Timeplot, which rocks, by the way), and stumbled upon the question. Google to the rescue!
  • OSCON ’08 – I’m going. Are you going? I’m also going to the Oregon Brewers Festival on the last day of OSCON, as I did in ’06. Wonderful!
  • Explosion at one of my hosting providers – didn’t affect me, but… wow!
  • hypertable – *sigh* someday…when there’s time…
  • Small-scale hydro power – Yeah, I’m kind of a DIYer at heart. I do some woodworking, all my own plumbing, painting, flooring, I brew my own beer, I cook, I collect rain in big barrels, power sprinklers using pool runoff to give my lawn a jumpstart in spring… that kind of stuff. One day I noticed water coming out of a downspout fast enough to leap over one of my rain barrels and thought there must be some way to harness that power. Sadly, there really isn’t, so I did some research. It’s non-trivial.
  • You bet your garden – I also do my own gardening and related experiments.
  • RightScale Demo – WATCH YOUR VOLUME – a screencast showing off RightScale’s features. Impressive considering the work it would take me, a lone admin, to set something like this up. The learning curve involved in effectively/efficiently managing/scaling/monitoring/troubleshooting EC2 is non-trivial.
  • Homebrew Kegerator – Maybe if this startup is bought out I can actually afford this thing to put my homebrewed beer in. The 30-year-old spare fridge in the basement is getting a little… gamey.
  • The pound proxy daemon – I use this. It works well enough, but I’ve crashed it under load, too. I’ve also had at least one hosting provider misconfigure it on my behalf, and I had to go and tell them how to fix it :-/
  • Droid Sans Mono – a fantastic coding font. Installing this font is in my post-install routine for all of my desktops.
  • Generator tricks for systems programmers – David Beazley has made available a lot of Python source code and presentation slides from what I imagine was a great talk (if you’re a systems guy, which I am).
  • The Wide Finder Saga – I found this just as I was writing Loghetti. There are still some things in Mr. Lundh’s code that I haven’t implemented, but it was a fantastic lesson.
  • Using gnu sort for IP addresses – I’ve used sort in a lot of different ways over the years… but not for IP addresses. This is a nice hack for pulling this off with sort, but it doesn’t scale very well when you have millions of them, due to the sort utility’s ‘divide and conquer’ method of sorting.
  • Writing an Hadoop/MapReduce Program in Python – this got me over the hump.
  • Notes on using EC2/S3 – This got me over some other small humps
  • BeautifulSoup – found while searching for the canonical way to screen scrape with Python. I’d done it a million times in Perl, and you can do it with httplib and regex and stuff in Python if you want, but this way is at least a million times nicer.

Well, that’s a decent enough summary I guess. As you can see, I’ve been doing a good bit of Python scripting. Most of my code these days is written in Python instead of Perl, in part because I was given the choice, and in part because Python fits my brain and makes me want to write more code, to push myself more. I’ve also been dealing with things involving “cloud” computing and “scalability” — like Hadoop, and EC2/S3. I haven’t done as much testing of the Google utility computing services, but I’ve used their various APIs for some things.

So what’s in your history?

Cloud computing hype overload

Monday, July 7th, 2008

I’ve been working with what I used to call “utility computing” tools for about 6-9 months. However, for about the past 2 months, I’ve been seeing the term “cloud computing” all over the place, and there is so much buzz surrounding it that it’s reaching that magical point best described using Alan Greenspan’s words: “Irrational Exuberance”.

When Alan Greenspan used those words to describe the attitudes of investors toward the markets, what he was basically saying was that there were people who didn’t really know what they were doing, putting more money than they ought, into things they knew relatively little about. Further, he was saying that the decisions people were making with regards to where to put their money were a) bad, or at least b) not based on sound reasoning, or the ‘facts on the ground’.

This, I think, is where we are at with “cloud computing”. The blog post that put me over the edge is this one, for the record. I read Sean’s writings often enough, but this one strikes me as being a little off, a little sensationalistic, not based in reality, and a little misleading.

Maybe he just didn’t put enough qualifiers in there. His post might make more sense if he limited its scope and provided more facts, but I guess it’s just an opinion piece so he decided not to go that route, and that’s his prerogative I guess.

By limiting the scope, I mean he should’ve realized that there are millions of web sites currently scaling quite nicely without the use of cloud computing. In addition, some of the new ones that are having issues are also not using cloud computing, and when they hit bumps in the road, they make it through, and the great thing is that they also share their stories, and those stories indicate that a cloud (or, the current cloud offerings) wouldn’t have helped much (there’s lots of other evidence of that too). What would’ve helped is if they had paid more attention to:

  • monitoring
  • initial infrastructure design
  • their own app code and app design
These aren’t issues that cloud computing takes away. What’s more, cloud computing is something of a moving target, many of the solutions aren’t as mature as you’d want them to be if you’re betting the house on them (EC2 only recently got “elastic IPs” and persistent storage is still not there, AppEngine only supports Python and has some rather severe limitations on functionality of your app), and they introduce a potentially large learning curve both in terms of how the individual services work, as well as how the heck to make your app fit into the cloud solution of your choosing. Think SimpleDB scales? Well, it does, but it’s also not a relational database, and doesn’t guarantee…. much of anything, including data integrity. You can’t interface with it using the drivers, interfaces, and language you’re used to using, either, because it’s not just a mysql wrapper or something – it’s a new beast entirely. Enjoy!
This is not to mention, of course, that some people have absolutely no choice but to scale without the help of the cloud, because corporate policy, common sense, or other forces mean that they can’t have their data passing through non-corporate-owned machines and/or networks. Also, Sean omits any mention of the cost factor, which is often a huge driver in getting startups to use these services, but may not really make the move “worth it” in some cases.
Anyway, in short, all I’m really saying is that it’s disingenuous to say that the future of web computing is “the cloud” because “only the cloud can scale”. That’s just silly. Non-cloud infrastructures can scale fine depending on the balance between the demands of the application and the funds available. The future of web computing will probably involve shared, utility computing architectures, but the future doesn’t depend on cloud computing.