Archive for July, 2008

Is it time to switch VM apps on OS X?

Tuesday, July 29th, 2008

Thought I’d throw this out to the lazyweb for advice. I’ve been running OS X as my primary desktop OS since I got my macbook pro around May 2006 or so. I installed Parallels as soon as I found it, and it was great. I still have it installed. However, I’ve found two issues with Parallels in the past two years:

1. There’s no kind of easy upgrade path for existing parallels users that I can find. Their web site is a mess, (like most software vendors, oddly) and there’s no upgrade license that I could find.

2. It’s non-trivial to install of versions of linux that are released after the version of parallels you’re trying to install to, for whatever reason.

I remember having issues with installing an Ubuntu version, and a CentOS version, both newer than the parallels I had installed. Then I upgraded parallels, and they installed fine.

So today I tried to do the impossible: install CentOS 5.2 on a build of Parallels from March 2007. I get a kernel panic, which I remembered a hack for (add “linux kernel agp=off” to the boot: line), but even that didn’t have any effect. Guess I just pushed the limit too far this time.

So, since there’s no upgrade license to make it cheaper for me to stay with Parallels, I have no particular reason to stay loyal to them. However, I cannot find a single mention of Linux on VMWare Fusion’s web site (?!), so….

What’s everyone doing about virtualization on the OS X (Leopard) Desktop?

Microsoft Makes Progress, but Still Misses the Point

Monday, July 28th, 2008

I’m still digesting certain parts of OSCON 2008, but I think I’ve finally settled on a conclusion for my thoughts on Sam Ramji’s presentation of Microsoft’s contributions to open source projects, and their progress toward rethinking how they do business to be more friendly toward competing platforms and technologies, so that other platforms can integrate with Microsoft products more easily.

I think the talk should’ve been entitled “What Microsoft Has Done for You Lately”. To Microsoft’s credit, they have, in fact, done things that would’ve been unimaginable just a few years ago. However, I think they sort of still miss the point — or they didn’t and just don’t know how to do anything about it.

One must admit that Microsoft has made strides in making various technologies run in a Windows environment. PHP can be run on IIS, and it’s a supported language in their development tools. They’ve opened up the SMB/CIFS spec so that the Samba team can create more robust tools to integrate Windows and UNIX environments, and they’ve become a major sponsor of the Apache Software Foundation. Things like this are nice, but I can’t really accept them as any kind of long-term, reliable solution. Sure, they’ve opened up the SMB spec, but what happens when Microsoft decides to change the spec, or they decide to deprecate SMB altogether?

The answer is that we’re left in the dust, and that leads to my point. The major difference between opening up a spec and opening up the source code is that if you open the spec and then decide to go in another direction, we’re left in the lurch. If you open the source code and decide to change directions, the community building tools around that functionality can decide for themselves to either change directions or fork and exec, so to speak. The real point here is that Microsoft is under no obligation to involve any of the communities with the Open Source world in discussions about their direction with regards to any of the technologies they use. They’re a closed-source, proprietary company, with a fiduciary responsibility to look out for their investors and their bottom line. If they see an opportunity to make more money because more customers want to see them do something that requires something other than SMB, they’re free to do that, and if I were a shareholder, I think I’d want them to do it, because I’d want my stock price to go higher.

I have yet to see a corporate entity involve the community in their direction with regards to technology. I have seen lots of hand-waving, a lot of lip service, and a lot of people speaking at conferences, mostly trying to hide really big elephants behind a well-worded (but otherwise empty) speech, at least vetted, if not written by, marketing and PR folks. Talks like this are pretty transparent.

I think the issue is a hard one. How do you involve a community mostly unconcerned about financial matters (or, at least, mostly not involved with them directly on a regular basis), in a decision-making process that necessarily involves coming to a solution that is “profitable”? Well, actually, I think the issue starts at a higher level than that. The real trouble is a situation which was beautifully illustrated in a talk by Robert “r0ml” Lefkowitz at this year’s OSCON: the corporations are viewing these interactions as taking place between “thinkers” and “doers”, instead of two “thinker” parties which happen to have different philosophies about how things should be done. Given the numerous cultural differences between the groups, one could forgive the corporate types for making the error (by the way, if anyone finds the “Praxis/Techne” talk by r0ml online in video format – post the link! There *was* a video camera there!)

Of course, specific to Microsoft, there are other issues besides simple matters of tech-level software interaction. There’s the issue of software patents, and Microsoft’s “promise” not to sue. There’s also the issue of standards, and MIcrosoft’s attempts to either own them, or destroy those which it can’t own. These are issues that are deep, cultural issues at Microsoft that no single person giving a talk at OSCON is in a position to solve (or to convince me they’ve solved). If Microsoft really does want to interact with the community and interoperate with our software, they should probably be prepared to spend years and dollars just to earn some level of trust. I sincerely hope that they are sincerely trying to open up, but I can’t help but have my doubts.

The promise of Drizzle

Sunday, July 27th, 2008

I got to actually speak to Brian Aker for maybe a total of 5 minutes after his micro-presentation about Drizzle, which took place at the Sun booth at OSCON 2008. I was a bit nervous to ask what questions I had out loud, because the things I had wondered about were things I really didn’t see too much discussion about out in the intarweb. I’m happy to report that, if Brian Aker is to be considered any kind of authority (hint: he is), my ideas are not completely ridiculous, so maybe I’ll start talking a bit more about them.

UPDATE: lest anyone get the wrong idea, Brian Aker did, in fact, state that views are not on the short list of priority items for Drizzle, but he did say that views are one of the features he finds most useful, and that they’d probably be higher on any future priority list than, say, stored procedures. So, take my notes below about views with a grain of salt. It’s not necessarily “coming”.

My three ideas were these:

  1. Materialized views: my experience with views in MySQL is that they just plain old don’t scale well compared to other database systems I’ve used. I used Sybase in 2000 and views scaled better for me then than MySQL views do now, and I’m using them in mostly the same way (which is to say that I’m not using them to do evil things – I’m using them in the way most of the database community agrees they should be used). In the past, I thought materialized views were “nice to have”, but now that I’m working with much larger data sets, without a need for my reporting to be 100% real-time, materialized views would be great. To be honest, I win either way with Drizzle in all likelihood, as Brian Aker has proclaimed that views in Drizzle will not look like views in MySQL. He confirmed that materialized views would be a great thing to have a closer look at, and I was happy with that reply.
  2. “Query Fragments”: I didn’t know they were called query fragments. What I explained to Brian Aker was that I wanted to harvest subsets of cached result sets. So, for example, if I do a date range query (I do that a lot), and the result set is cached, and then my app does another query which is identical save for that the date range is a subset of the one in the cached result set, I’d like to grab that data from the cache. My actual question to Brian was “can this be built in such a way that this would be a reliable, trustworthy result coming from some middle tier component?” And that’s when he told me about query fragments. He said that this idea was not at all crazy, and was also worthy of further discussion.
  3. Vertical data-padding. Well… that’s what I’m calling it. Here’s the logic: I have lots of temporal data. Tons of it. My queries are largely things like “for this user, show me all of the foo’s bar’d — group by day — where day is between these two days”. MySQL is good at these kinds of queries (assuming you index well and, basically, that you’ve read “High Performance MySQL” a couple of times… per edition), but there’s something missing. When I get the result set back, any of the days for which no foo’s were bar’d, I don’t get a record. This is perfectly within the realm of reasonable behavior, but I’ve never known MySQL to let things like being reasonable get in the way of helping their users, so my suggestion was that MySQL, in order to do the comparisons for the “WHERE” clause, *MUST* know what dates fall within the dates in, say, a “BETWEEN” statement. It would then seem logical to have some way to tell MySQL “If one of those dates has no value, return a NULL” (or 0, or an empty string, or something). I don’t know the real name for this proposed feature, but I call it “vertical data-padding” because you’re padding columnar data, which, in my mind, is visualized vertically. Just like when you do a “GROUP CONCAT” or something, I would refer to that as horizontal data padding. I explained to Brian that one way I’ve seen this handled is to have a lookup table of static dates that gets joined to the main data table. You do a left outer join with the date lookup table on the left, and you get back a row for every date whether there’s data in the right-hand table or not. This works when ‘n’ is small (like everything else), but it’s hurrendous when you have, say, 10 million rows to deal with. Then you’re in what I call the “No Bueno Zone”. Brian seemed interested in the problem, and I’ll be discussing that with him further when his life settles down a bit (he’s been at OSCON, and he’s still settling in at Sun).

I want to thank Brian Aker for his enthusiastic attitude toward helping others, and for all of the work he pours into all of this stuff. I also want to say that, for me, Drizzle is really exciting, not so much because the feature set is more or less cherry picked to map onto what I do for a living, but also because it represents an opportunity to get ideas in the door before a lot of legacy cruft makes it impossible to implement these somewhat idealistic features without rocking the boat for the millions of users already launching it into production.

OSCON Update: Coverage gets harder as conference moves faster

Thursday, July 24th, 2008

As expected, the conference really started to deliver starting Wednesday with the opening of the Exhibit Hall, announcements of all kinds from all companies, and parties at every turn. Wednesday also marks the first day of sessions that are only 45 minutes long instead of the 3-hour sessions on Monday and Tuesday. I’ve attended too many talks, and have been moving too quickly to cover it all “on the fly” as I had been doing earlier, but look for a new BoF board picture today, and very heavy wrapup coverage on Saturday night when I get back home to the east coast.

I attended talks about Python, Ubuntu, PHP, Virtualization, Memcached, and a great scalability talk given by a Facebook engineer. Most of the talks I attended were wonderful. Thanks to the presenters!

One thing I will say now as well — I was right again about how to pack. Yesterday alone I collected 5 t-shirts, in addition to a nice pair of MySQL boxer shorts!

OSCON Day 2: Launching a Startup in 3 Hours

Tuesday, July 22nd, 2008

Launching a Startup in 3 Hours was a great talk given by Andrew Hyde (of and Gavin Doughtie (of Google). Both of the speakers are heavily involved in the recent trend of doing “Startup Weekends”, and is an organization that hosts startup weekends all around the US (and I think internationally as well – Andrew mentioned one in Germany if I heard correctly).

The first half of the talk was about the general concept of a startup weekend, the problems it avoids (“we’ve been working for 9 months and haven’t launched anything”), the problems it brings up (“If you’re not using Java, you’re an idiot, so count me out!!”), and lots of details about how to organize, how to assign roles, and some common tools they use (like Basecamp and whatever your IM of choice is). There was also talk of legal issues, how (basically) to think about forming the company with the people involved, and decisions that need to be made at a business level aside from just the coding.

The second half of the talk wasn’t a talk at all. Instead, people who had ideas stood up, presented their idea in a couple of sentences, and once the ideas were out there, we were told to break into groups and get to work! So people would get up and move over to the person whose idea they liked, and they’d start brainstorming. I decided to head out after about 30 minutes of observing and talking with people about ideas, but when I left, there were probably 6-8 groups of people engrossed in conversations, and the energy level was very high. Overall, it was a really exciting experience!

OSCON Evening 1 Begins, and More Portland Tips

Monday, July 21st, 2008

The evening plans didn’t wait for talks to be done. The IRC channel (#oscon on was alive with talk of prospects for dinner and drinks after the conference. I myself was torn between a group going out for Lebanese and another going to Henry’s, but opted to go with my buddies from home to Henry’s.

It was worth it. If you haven’t been, Henry’s Tavern boasts 100 beers and hard ciders on tap (oddly, the beer list is the only menu *not* online – guess it changes too frequently). There are a ton of local beers that you can’t even get on the east coast just waiting for you to try, but there are also some rare treats, like the Belgian Lambic beers, which you don’t often see on tap. The food is a little pricey, but is really good, and the staff is very friendly. IMG_4491.JPGA couple of us were in a rush to get back by 7 for the BoF sessions, and when we asked the waittress how easy it was to catch a cab, she immediately informed us that she would have the hostess call one for us. About 2 minutes later we were in a cab on our way back (we wouldn’t have made it back in time if we had to walk back to catch the light rail).

I was not one of those rushing to a BoF, so I did a little poking around the area near the convention center. It was getting dark, and I didn’t want to stray too far, but I did find a couple of points of interest. First, there’s a bank right across the street from the convention center. I’d be willing to bet that the ATM there is less than the $3 the ATM inside the center charges.
Beyond that is a paintball place. It was closed by the time I found it, and I don’t know if they run every day, or anything else, but interested parties might find it open during the lunch breaks or something if you wanted to check it out. The paintball place is located behind a building that is directly across the street from the conv. center. If you see the bank, it’s on the other side of the side street the bank sits on.

Tonight appears to be low-key from what I can tell. There’s currently no chatter on irc, the hotel bar had a few people chatting, and I might go down to catch the rush of people as they return from dinner and BoF sessions. Stay tuned tomorrow for more!

OSCON Day 1 Comes to a Close

Monday, July 21st, 2008

I think I have pictures of most of the basic parts of the conference at my OSCON Flickr set, and I thoroughly enjoyed day 1 of the conference. Of course, while *day* 1 is over, *night* 1 has yet to even begin. There are lots of BoF sessions, and maybe even more smaller meetups going on, as smaller groups take to discussing things over dinner and a beer or three.

I have to say, that I occasionally pop into irc channels for conferences I’m not even at and follow up on that because I’m involved a bit in conference planning as part of my work with Python Magazine (I’m helping to organize the PyWorks conference in November). This conference seems to have a pretty happy audience, if IRC chatter is any indication (and it usually is). Sure, there are a couple of weak spots in the wireless network, there are some fuzzy projectors, and there was a little confusion regarding breakfast this morning, but the important bits have been well-covered by the OSCON organizers and the “boots on the ground” here on site. Kudos to them all.

This afternoon I hopped to a couple of different talks: one on Memcached and MySQL, and the other on A/B testing. Both contained good content. Of course, I’m a systems guy primarily, so I sort of wanted more of an overview of memcached from the point of view of an admin who is deploying it rather than a developer implementing their code around it. I still got plenty of value out of that talk, and this *is* really more of an open source *developer’s* conference, so the expectations of 99% of the people in the room were met, I’m sure.

A/B testing is just not an exciting topic, and I would imagine that peoples’ bosses made them go to that talk whether they liked it or not. Not to say the talk wasn’t good – the parts I saw (I came in after the break) were good, and I learned from it, and that was the goal. If you’re a QA/QC person, I’m sure the talk was riveting, and there were a lot of good ideas and things I’d never considered flying by in the slides.

Overall, Day 1 is a win. I’ll cover more about this evening’s events in the pre-breakfast hours tomorrow. Stay tuned!

OSCON Day 1: The BoF Board, for your perusal

Monday, July 21st, 2008

I’ve posted a picture of the BoF board for day 1. Click on it to see bigger sizes. The full size image (maybe smaller) is perfectly suitable for reading at your leisure. I’ll update this if/when I see significant changes to it:


Day 1 progresses… “Python in 3 Hours”

Monday, July 21st, 2008

The morning session I attended was “Python in 3 hours”, which provided excellent coverage of Python for folks who may never have seen it before. Steve Holden, being more qualified than most to present this material, did a stellar job of not only presenting the material, but also addressing questions coming from the attendees.

Steve’s talk is fantastic, but I really wish there was an introductory talk for folks who were coming in with the question “Why Python?”. Steve’s talk assumes interest in the language, and that’s completely fair, but I brought a buddy who I think was looking more for the “why” than the “how”. Of course I try to evangelise the language where I can, but to be honest, I’m not as good a salesperson for Python as I wish I was. I use Python for all of my sysadmin tasks these days and love it, but I already hated the other languages I was using, so I guess I just wasn’t a very hard sell. 🙂

Day 1 of OSCON Begins, and More Tips for Conference-goers

Monday, July 21st, 2008

I got an early start. Too early. But I’m from the west coast, so my body thinks I slept in. I wandered around a bit, took a few pics which you can see at my Flickr OSCON set, and I discovered a couple of things that might be of interest:

  • The starbucks in the conference center charges over $2 for a small cup of joe. There’s a starbucks right across the street (you can see it from the breakfast area – seriously, it’s 5 seconds away), and they charge less than $2 for a medium (grande). That’s less than I pay at home.
  • The ATM outside the starbucks charges $3 for cash. I’ll report back when I find a cheaper one, but most places seem to take plastic here.
  • Every computer involved in this conference, from registration to the video screens that dot the common areas, are running Windows XP. Just sayin’.
  • The light rail system is free to go just about anywhere except for the airport, so there’s no excuse not to get out and see Portland and take in the food and beer and stuff.
  • For beer-lovers, not only is there the Oregon Brewers Festival starting at the tail end of this conference, but there’s apparently another festival that we missed *last* weekend!! Keep that in mind when you’re planning to come to OSCON next year.