Archive for June, 2009

Linux/Unix File Copy Trick

Wednesday, June 17th, 2009

I have a need for this hack every now and then, and I *always* forget it for some reason, so I’m putting it here for safe keeping. I’ll start a “hacks” category, too, so I can locate these quickly, and so you can too 🙂 So, here’s the hack:

[jonesy@cranford testing]$ ls
bar  foo
[jonesy@cranford testing]$ ls foo
1  2  3
[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6
[jonesy@cranford testing]$ yes n | cp -i bar/* foo 2>/dev/null
[jonesy@cranford testing]$ ls foo
1  2  3  4  5  6
[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6

I’m piping a constant “no” to the “cp -i” command, which asks you if you want to overwrite (or not) any files that already exist in the target directory. You don’t have to send STDERR to /dev/null — there’s just some messy output if you don’t. The end result is that the copy command will only copy files into the destination directory if they don’t already exist. It’ll skip files that exist in both directories (Well, technically the *copy* command won’t, but the operation will when you do it this way).

Of course, I could just forcibly overwrite the directory contents, but I don’t know if the files that exist in both directories are identical. Or I could move one sideways and the other into place, but the same issue exists.

My first screencast: The Linux Boot Process

Friday, June 12th, 2009

So, I’ve taught the Linux Boot Process as part of a couple of different training courses now, and I thought I’d share it with the world in the form of a screencast (it’s hosted at my co.’s site). This is also a test to help me figure out how to “do screencasts”, generally.

The material in the screencast is slightly adjusted, because different training clients want to see different things, and some just can’t afford to spend a lot of time on the boot process in their training classes.

I welcome any feedback anyone might have on it.

A Missing Battery in Python: a bash “select” equivalent

Wednesday, June 3rd, 2009

So, someone showed me a script they were writing in bash. They were doing a lot of manual menu creation, then using “read” to collect input, and manually mapping the input to longer strings, and then doing string manipulation stuff. Trying to help simplify and make things a little more sane and readable, I discovered that I was going to break some of my own rules with regards to bash scripting:

  1. It was likely going to involve arrays, which is usually the time I migrate to Perl or Python.
  2. It was going to be longer than 60 lines, which is usually when I migrate to Perl or Python.
  3. The script didn’t make calls to any standard system commands, which means bash was being used for its generic scripting structures like case statements and if-test checks, which is something I try to foresee when I write scripts. If I’m not greatly aided by standard CLI utilities, I try not to do it in bash, because in the end, it’s a shell, not a scripting language.
  4. There was no requirement to use bash, and Python and Perl are both available.
  5. XML was involved.

The first order of business, though, was to replace all of the menu-related stuff with “select” loops instead of doing all of the manual stuff. It made things a lot simpler. Once that was out of the way, I inspected more of the code, and at that time realized “oh wait a minute….”, and became slightly ill as the above menu items flashed into my brain.

So I set about the task of fooling around with the task in Python. My first discovery there was that there isn’t any “select”-like functionality built into Python. Then I tried the curses module, but that makes doing “select”-like menus even messier, because not only does it *not* store the *string* associated with an option, it doesn’t even store the option string directly — it stores the ASCII code. This makes curses a lot more robust than using bash, but between messing with screen, window, and box semantics, and then still having to do some kind of option-to-value mapping by hand, it’s not much of a win in the simplicity department.

There are a couple of third-party modules that deal specifically with text-based, menu-driven, non-curses apps, so solutions do exist (this one looks good at first glance), but it requires altering a stock, rpm-based Python installation, which isn’t so bad except that it also means maintaining that alteration going forward — something I’d like to avoid.

What ‘select’ Actually Does in Bash

Here’s a bit from the bash reference manual, which is way more concise than I’m ever likely to be:

The select construct allows the easy generation of menus. It has almost the same syntax as the for command:

          select name [in words ...]; do commands; done

The list of words following in is expanded, generating a list of items. The set of expanded words is printed on the standard error output stream, each preceded by a number. If the ‘in words’ is omitted, the positional parameters are printed, as if ‘in “$@”’ had been specified. The PS3 prompt is then displayed and a line is read from the standard input. If the line consists of a number corresponding to one of the displayed words, then the value of name is set to that word. If the line is empty, the words and prompt are displayed again. If EOF is read, the select command completes. Any other value read causes name to be set to null. The line read is saved in the variable REPLY.

The commands are executed after each selection until a break command is executed, at which point the select command completes.

Here is an example that allows the user to pick a filename from the current directory, and displays the name and index of the file selected.

          select fname in *;
          	echo you picked $fname \($REPLY\)

I’ll figure out a solution that isn’t too ugly and post it here. In the meantime, if you’ve already done all of that, or I missed something in the docs for Python or curses, post links or code in the comments. Thanks!

PS – Before I get flamed: I chose Python because it’s my go-to language for all of my scripting tasks. I don’t recall Perl having an easy way to do this either without also installing an external module, and I figure I’ll learn something new about my favorite language while trying to solve this problem.

[UPDATE] – As soon as I launched the Python interpreter and started thinking along the lines of Python dictionaries, things fell into place:

>>> d = {1:”foo”, 2:”bar”, 3:”baz”}
>>> d
{1: ‘foo’, 2: ‘bar’, 3: ‘baz’}
>>> for k,v in d.iteritems():
…     print k,v

1 foo
2 bar
3 baz
>>> d.get(1)

That works, is still much cleaner than bash “read”, and gets me the option-to-value mapping, without loading *any* modules.

[UPDATE 2: It was pointed out that I failed to explain what ‘select’ actually does, so I’ve added a description from the bash documentation, which I think explains it pretty clearly. ]