Teaching a Course on Profiling and Debugging in Linux

Posted on by bkjones

Dear Lazyweb,

So, I’ve been in Chicago for a week teaching a beginner and an intermediate course on using and administering Linux machines. This week, I’ll teach an intermediate and an advanced course on Linux, and the advanced course will cover profiling and debugging. The main tools I’m covering will be valgrind and oprofile, though I’ll be going over lots of other stuff, like iostat, vmstat, strace, what’s under /proc, and some more basic stuff like sending signals and the like.

So what makes me a bit nervous is, being that the advanced students are mostly CS-degree-holding system developers, they’ll probably be expecting me to know very low-level details of how things are implemented at  the system/kernel level. I’d love to know more about that myself, and actively try to increase my knowledge in that area! Alas, most of my experience with low-level tools like this is in the context of trying to understand how things like MySQL do their jobs.

They may also turn up their nose at the admin-centric coverage that I believe is actually very important in order to get a complete view of the system and to reduce duplication of effort. Of course, I’ll use a bit of time at the beginning of day 1 to properly set the expectation, and we’ll see how they respond. As they say in the hospitality industry, presentation is everything.

The portion of the course that covers valgrind and oprofile won’t be until Thursday, or perhaps even Friday, so I figured I’d take this opportunity to ping the lazyweb and find out a couple of things:

  • What tools do you use in conjunction with valgrind and/or oprofile?
  • What kinds of problems are you solving with these and similar tools?
  • What most annoys you about these and similar tools?
  • Do you use these tools for development, administration, or both?
  • If you have cool links, share!
  • If you’ve been able to make effective use of oprofile inside of a vmware instance, share (because my thinking is that this probably *should* be nearly impossible unless vmware actually simulates the hardware counters oprofile needs access to!)
  • This one is just for me, not the course: are there any demos/tutorials on using valgrind with Python? I’ve seen the standard suppression file, but it still seems like profiling a Python script would be difficult being that you are actually going to be profiling the interpreter (or so it seems).

Thanks!