rss feed 2008 2007 2006 2005 2004 2002 livejournal

Took my last antibiotic pill last night. That's round 6 for this sinus infection. The fact I had a sore throat before I'd even run out of pills isn't filling me with confidence, but oh well. Hopefully it's an unrelated (viral) cold.

Woke up tired. Took a nap for 4 hours. Woke up tired again, but this time covered in cats. So there's that.

This episode of Bleach is trying to be comedy, and none of the cultural references translate unless you're japanese. I can tell they're parodying specific things, just not what. The entire episode is designed to be an aside and not affect the main plot at all. (I think it's a special for their 50th episode.)

Hmmm, episode 52 introduced new credits again. Ah, "if you continue you may die before you reach $PLOTPOINT". It's Ichigo being addressed. Hands up everybody who remembers his limit break.

Ooh, did red haired guy pick up Ichigo's limit break? Nope, apparently not... Ah, I see, he's channeling Ichigo. Well, that works. I'm starting to think a flashback montage right before an attack is a bad sign in this context.

Do Japanese people really throw random english phrases in conversation whenever they want to emphaisze something? I suppose it's a bit like english speakers with Latin and french...

So where did I leave off yesterday? I found out that the reason the "FORK=1 ./smoketest-all.sh" script was failing was that formatting a dozen virtual partitions (to create the "hdb" images to mount on /home inside each qemu instance) was causing so much I/O contention that the timeout was elapsing before each test could actually start. At a guess, mke2fs has some fsync() calls in there, and having an 8-way server with 32 gigs of ram doesn't help if flushing to disk is the bottleneck.

The easy way to deal with this is to add an option to _not_ create the hdb images, because in this case they're not needed... Ok, done.

Ok, Obama's speech at the Radio and TV correspondents dinner was totally, totally upstaged by John Hodgman's follow-up.

There really, really, really should be an easier way to do this:

function genocide()
{
  local KIDS=""

  while [ $# -ne 0 ]
  do
    KIDS="$KIDS $(pgrep -P$1)"
    shift
  done

  KIDS="$(echo -n $KIDS)"
  if [ ! -z "$KIDS" ]
  then
    genocide $KIDS
    kill $KIDS
  fi
}

I suppose I should probably call it something other than "genocide", even though that's what it does to a process tree.

The xfce terminal program lets you change font size (either with the preferences toolbar button, or the edit->preferences menu entry), and when you do this it affects every open terminal. Both KDE and Gnome's terminal programs let you change this per-window.

Having a dozen different implementations of each major piece of functionality isn't much use when they're not designed to mix and match. I'm using kmail with xfce but when I click on a link it pops up abiword, not a browser. (The way you change kde file type associations is through the web browser, konqueror. No, it doesn't make any sense, but it's what you do. Unfortunately, if you install Konqueror on a non-kde desktop, it crashes when you try to bring up its preferences dialog.)

Oh, and the first time I tried to change the font size, all my open terminal windows (with multiple tabs) closed instantly:

[145829.419872] xfce4-terminal[3417] general protection ip:7f7d8a9d4e9d sp:7fff93cb7700 error:0 in libvte.so.9.5.0[7f7d8a9b6000+11f000]

Wheee...

I learned a new thing about how shell scripts work! (Translation, I spent a couple hours trying to understand a really strange bug that bit me.)

In shell script, a for loop works like:

for i in one two three
do
  echo $i;
done

The problem with that is if you want multiple related arguments (for i in "one 111" "two 222" "three 333"), you have to split them yourself, which is fiddly. (You can do it with awk, with cut, and a couple other fiddly ways that tend to be brittle.

The dirty trick to get around it is to do this:

echo -e "one 111\ntwo 222\nthree 333" | while read one two
do
  echo $one $two
done

The problem is when you try to set a variable in the loop for use after the loop, or return from a shell function inside the loop. The problem is the second half of the pipe runs in a subshell (it's a separate process), and the entire body of the loop is considered the second half of the pipe. So you're setting variables in the subshell or returning from the subshell, and that has no affect on the parent process.

That's subtle and fiddly and evil. I understand why it's doing it, but it took me by surprise anyway.

Watching mythbusters season 1. Highly entertaining. (One of the episodes you can get on dvd through netflix mentioned the rocket car, but the dvds didn't actually contain that episode. Crazy, eh? It's such a chopped down subset, with the individual episodes edited differently than they actually aired. You have to go to bittorrent to get the real ones.)

When it gets to be 5:30 am, and meanings get reversed and then unreversed in the same bisect, it's hard to work out which of good or bad your next bisect should be. Screw up once and your result is useless.

Resuming the next morning isn't a whole lot of fun either. Where did I leave off? (And now you know why I keep this blog!)

For bonus points, the squashfs bug occurs intermittently. I've run a kernel with it twice and had it run to completion the first time and die with the squashfs bug the next. That's going to be fun to track down if 2.6.30 still has it once I've got the boot bug out of the way.

Tracked the bug down and posted about it on the list. It turns out that there's a new config symbol I need to switch on (which is described in the commit message). So all that work was to change a single config symbol. (Not even a code patch.)

Par for the course, pretty much. If it seems like I'm doing a huge amount of work for very little progress... It's pretty much all like this. (Pretty masochistic hobby if you stop and think about it. Oh well.)

Several people pointed out to me in email that I can go "git archive HEAD" to export the current version. The reason that didn't work when I tried it? Because "head" doesn't work, it must be all upper case, even though "master" is lower case. (Yeah, that's git.)

I also read the man page for git bisect. (git help bisect == man git-bisect, therefore if the git man pages aren't installed, which they aren't by default, git has no help.) Anyway, it turns out there is a way to specify a skip result from the run script, which is to exit with the magic error code 125. No rhyme or reason behind it, because this is git, but it's nice that there is a way. (I'm still not using it, since it's trivial to do by shell script.)

The part that isn't trivial is that my script has so far done bad, good, bad, skip, skip, skip, skip, skip, skip, skip... It may go through all 1398 remaining revisions individually? Now it's done it twelve times. It takes several minutes per run, I'm watching Bleach while it does this...

Bleach continues to be weird, not that I expected different. Up to disk 12 now. Episode 46 was a giant gratuitous flashback for secondary characters, and now 47 is apparently internal politics on the part of the antagonists, and now we're back to protagonist boy Ichigo dueling with the antropomorphic personification of his sword. (Yes, really. While werecat woman watches.) Episode 48, more internal politics with long dramatic swaths of recycled footage and the same three songs, and now the catgirl is wondering why Ichigo isn't levelling up during training (because she hasn't pushed him to limit break yet, you've got to beat him silly before he levels and the current session hasn't even drawn blood yet). Well of course they shortened the deadline, that's plot 101. And now episode 49, and it's flashback land again. "Don't worry, it's just a reconnaisance mission". Oh yeah, to quote ghostbusters "this chick is toast"...

And the video froze again... And the disk icon won't let me eject it because "mount volume" is where eject normally goes. (Why would that be related? I want to eject the unmounted volume, why does the gui not give me that option? Make it orthogonal, guys. Of course the eject button on the actual drive still doesn't work.)

I still need to buy a mac. It might also be buggy crap, but then if I complained about it there would be someone to complain to who wouldn't imply its lack of quality was somehow be _my_ fault. Also, the mac has had an impressive improvement trajectory since the introduction of MacOS X, while Linux has pretty much been treading water on the UI front.

Back to Bleach. Ah, whatserface didn't die, just the red shirts. Ok, hang on, she's posessed? And slaughtering everybody. It's kind of odd the huge power disparity amoung the soul reapers, the nameless ones are one hit kills. ("If we help him, what will become of his pride?" Sigh, they've gone all insanely japanese again. And he's dead because of it. Oh _now_ you're willing to kill him. Sigh.)

Ok, this bisecting isn't getting anywhere, it's gone through well over a dozen iterations of skip and every one it looks at still build breaks with:

/home/landley/firmware/thingy/build/temp-powerpc/alt-linux/arch/powerpc/include/asm/highmem.h: In function 'kmap_atomic_prot':
/home/landley/firmware/thingy/build/temp-powerpc/alt-linux/arch/powerpc/include/asm/highmem.h:98: error: implicit declaration of function 'debug_kmap_atomic'

What I probably need to do is grab the current revision it's looking at and the last known "bad" one (which reproduced the error I'm looking for, meaning _this_ error was fixed), find the fix, make a patch, and bisect between the last known "good" and "bad" applying the patch to each iteration that's horked by this bug.

Anyway, back to _this_ bisection. I've got the last commit bisect found that manifests the problem I'm actually interested in, and I can grab any of the commits it's been stuck on which show the problem I'm _not_ interested in (but which git bisect skip is too stupid to get past), and then bisect those two find the patch that _fixed_ it, then apply that patch to re-test broken versions in the original bisect and hopefully actually make _progress_.

This is another fun case of "good" and "bad" having to be redefined, since each is showing a different bug:

git bisect good 18b41f1cd537168a886c43237297692ba8d0a143 # shows build break
git bisect bad a0e0404fb06164100991cacf8e055f6b30f87cc9 # shows hang

So here, "good" means "the bug I'm trying to find the fix for in the short term", and "bad" means "the bug I'm ultimately trying to find a fix for". Wheee... I hate git.

And hey, one of the intermediate commits shows the squashfs bug, but not the build break. (It's e0cf8f045b2023b0b3f919ee93eb94345f648434 and yes, that's how git expects you to refer to it. You can use any arbitrary consecutive subset of those digits, but assuring you're using enough to avoid collisions with any of the tens of thousands of other hash codes the tree is made up from is your problem.)

Anyway, that commit shows _NEITHER_ of the bugs I'm looking for (it instead shows a third bug unrelated to either do; did I mention that the general testing level of non-x86 kernels is crap?), and that's progress! The last "good" showed the build break, so this is after that was fixed. The last "bad" showed the hang, and this is before that was introduced. So depending on what I answer here, I could track down either the fix for the build break or the introduction fo the hang, but since what I want is the hang and this proves that it wasn't introduced in the range of commits covered by the build break, I can answer "good" here and change what I'm looking for (mid-bisect) to track down that hang!

Did I mention git's user interface sucks rocks and it's essentially unusable if you don't know how it was implemented?

Late. Tired. Cranky. Feeling kind of nauseous actually (15 penguin mints in one night may be a bit much, even of the chocolate ones). Finish up in the morning...

Bisected a powerpc problem and reported it upstream. Found out I'd bisected it wrong, and that the list I was supposed to report it to is A) not on kernel.org but on ozlabs instead, B) unlisted in their index.

Wrote a script to automate the bisection. Somebody pointed me that git's had yet another random feature bolted to the side of it in a hap-hazard fashion that I might have been able to use instead. Decided I didn't care. (And I decided that _before_ I found out that there's no way to indicate a "maybe" response to git bisect run, which makes it kind of useless because I've hit two unrelated bugs already in the bisect. One build breaks the kernel, the other one screws up squashfs some time after the root filesystem is mounted. The one I'm looking for is distinguishable in the kernel boot messages, before init runs.)

Ok, current Ubuntu's uselessness factor continues to rise. Attempting to suspend my laptop, I get the pop-up "Request to do policy action: multiple applications have stopped the policy action from taking place. vlc: Playing some media. vlc: Playing some media."

First, no running software should be able to the system from suspending. Second, there are no copies of vlc running anymore, they exited before I tried to suspend but this status _leaked_.

Right, BIG HAMMER TIME. Ooh, it' s been a while, what do I need to do... switch to a text console, become root, "echo -n mem > /sys/power/state"...

Back from suspend. (It worked.)

Ok, here's my objection. It's not that the system was just complaining about nothing, it's that a sus

The status leaking might be because doing something simple like extracting a linux tarball paralyzes the rest of the system for a minute or more at a time. (For example, in the above sentence, vi hung after the "m" in system and nothing else I typed displayed until the period after "time". I could have typed _way_ more in that gap, but I stopped because cursoring back up to fix typos three sentences back is annoying.)

This means that a bisect script which uses "git archive" to feed a tarball into my build system can't just run in the background and be ignored, it makes the rest of the system INSANELY unresponsive. Firefox also hangs for upwards of a minute at a time (not only can I not scroll the page, but if I move another window over it it won't _redraw_). And it hangs like this about every 3 minutes. On a dual core 1.7 ghz laptop with 2 gigabytes of ram.

Yes, the "Linux on the Desktop" people have been trying for 15 years now, and this is what you've come up with. Moore's Law says they've now got 2^12th times as much memory, processor power, and disk space as they started out with and this is still the best they can do. (How many years did they waste insisting that the display driver (X11) needs to run as a normal process so normal system load can freeze your mouse pointer, making it hard to even queue up actions for the system to do when it stops playing with itself?)

An invocation I've found useful:

git archive $(git show | head -n 1 | awk '{print $2}') --prefix=linux/ | bzip2 > ~/firmware/thingy/packages/alt-linux-0.tar.bz2

It would of course be nice if "git archive" could just spit out the current version without having to be told what version that _is_, but this is git we're talking about. There's probably a way 300 lines into a seemingly unrelated man page, but you'll never figure it out yourself.

In that way it's very much in the spirit of vi or emacs. You don't learn how they work by playing around with it. You learn by grabbing a spellbook and reading through hundreds of pages of incomprehensible jargon in ancient dead languages to become worthy of discovering the mystic incantion you will NEVER figure out otherwise. And in reality, you learn about 5 functions it does and ignore the other million settings on the Sonic Screwdriver unless you're the guy who _built_ it.

And this is why I vastly prefer Mercurial.

The reason powerpc is giving me so much trouble is that too many different things have changes since the last time I had it working. The qemu version is different (largely because of laptop hard drive failure and subsequent reinstalls), the linux kernel version I'm building (and thus trying to boot) is different, the compiler I'm building it with is different (4.2.1 instead of 4.1.2), and my build scripts have changed. When I run across any particular symptom, _and_ of that could be the cause. (Or it could be one of the other things that changed like the busybox tools developing a subtle sed bug, been bitten by that before.

Dug up the mailing list thread from last time a random openbios change broke powerpc, and it says svn 6657 used to work. That translates to git 2d18e637e5ec, so let's fire that up and build it verbatim... And it works. Yay.

But in this case "works" means it boots the _old_ binaries. The ones I'm currently building don't give me any output at all, which implies the serial console is horked. Hmmm...

Now I'm iterating through different versions of my build scripts (much faster) to see where I introduced whichever tool broke it. (That way I'm only testing along one axis.) The 0.9.6 release worked. The last version right before I switched to gcc 4.2.1 worked, so it wasn't the infrastructure changes since then. Commit 752 (first "expected to work" version after the compiler switchover fallout had shaken out) worked. So it's not the compiler. Now we go more slowly. Next commit (753) updated busybox version... and that worked, so it wasn't busybox screwing up the build tools. The actual kernel switchover to 2.6.30 was 754... Which doesn't build because the powerpc config needs the gratuitous new symbol to avoid build breaks. Ok, tweak that, and...

invalid/unsupported opcode: 00 - 12 - 00 (00009024) 0000938c 0

Now it's failing with a _reason_, which is an improvement. But it's definitely the kernel update that's doing it.

Of course now we're back to "two changes which can't be separated" territory; is the new kernel horked or does the config need to be changed more? (These are sort of aspects of the same question, but still...)

Now I need to bisect through kernel versions, of course. (Bisecting through things that take several minutes to build is way slower than it seems, because I switch to other windows and get distracted and don't come back promptly. It can take me an hour to go through three versions if other things are interesting enough that I forget what I was doing...)

Ok. Dug up the release binaries of system-image-powerpc, which was known working with some previous version of qemu as of 3 months ago (albeit with some patching). The two new problems (the -hda argument actually setting /dev/hdc, and init segfaulting with the kernel trying to access memory it shouldn't) are both in the old binary, which means qemu is what changed.

Bisecting through qemu revisions that need to be patched to have any chance of booting anything sucks. Especially when different ranges of the history need different patches...

Ok, 179a2c1971 can't find _any_ hard drive with this bios, 2fbc409571 can. Looks like 513f789f6b18 might be the relevant change between those... And it is. That's where qemu switched from using hardwired layout info to querying openbios (and hence the device tree) for it. The new layout has hda/b and hdc/d swapped (the primary and secondary IDE controllers, apparently), and the panic due to accessing some invalid resource seems similar. (It's not the kernel, I'm testing a binary that used to work.)

Went to the doctor and got another cat scan (we have four at home, but the one they have is specially trained) which showed that I still have a sinus infection but A) not as bad, B) in a totally different place. I'm callin this progress! Got another 10 days of antibiotics. Feeling better already, although that's probably the placebo effect.

Mark had a series of videos called "Chi comes home" (and a second season called "Chi's new address") which are hard to describe. "Kitty porn" comes pretty close. Kitten porn in this case. Or perhaps "Kawaii diabetic attack". I DARE you to get through even the opening credits with a straight face, let alone a full episode. (There are 104 of those in the first season. That's episode 2. Yes, it's a series of 3 minute cartoon videos about a kitten, and it is horrible yet awesome. I have not yet shown Fade because she might go into shock.)

The first episode isn't quite representative of the series, since it's the kitten wandering away from its mother and hence three minutes of way too cute _angst_ instead of pure uncut refined cuteness like the others. Although if you want angst done inexplictably well, there's a blog telling the story of a homeless family of sims. It's a father and daughter pair of sims set up on an empty lot with no money and nothing but 2 park benches as "home", allowing Sims 3's "free will" option to dictate their behavior as much as it can (everything but aspirations and opportunity pop-ups), and writing up their story with screenshots and prose. It is FAR more emotionally involving than it has any right to be, although you really have to start at the beginning to appreciate it.

There's a weird sort of yin/yang in the above two paragraphs, now that I come to think of it.

Got the perl removal patches resubmitted for 2.6.31. (It's technically still the merge window, let's see if they make it in this time.)

Even though the changes from 2.6.29 were one minor cosmetic tweak and rediffing so the offsets matched up, it took hours to do this because I'd totally forgotten how (and I had to write up new descriptions). My own cannonicalize.sh script seems to have gotten lost sometime in the past 3 months, so I had to write a new one. (It puts a patch in the cannonical format for linux-kernel. Yes, it has my name and email hardwired into it, it's my script.)

Now at Starbucks. Still bisecting the qemu powerpc stuff. Building qemu is slow...

Hey, my daily google alert thingy just pointed me at my Firmware Linux page on the impactlinux website. Um, yay?

So bisect narrowed down 9d479c119b42b8a548f8d79a8e5a1c1ce2932d91 which was the openbios update that made the sucker start booting again, but which was _after_ whichever change swapped hda and hdc. (And presumably whatever caused the powerpc image to start panicing with a null pointer dereference right after userspace got control even when you work around that.)

It's possible the panic is a kernel change and not a qemu change, but lemme try 0.10.1 with the other openbios and make sure I can reproduce the problemt here before trying to bisect the kernel build.

Last night my laptop did one of those "everything freezes but the mouse pointer" things so I had to power cycle my laptop and lost the debugging context for powerpc. (Ubuntu 9.04 is like Windows 95 when it comes to reliability. Oh well.)

This morning (up before noon!) I'm rebuilding powerpc with gcc 4.1.2, and if the behavior differs I'll build the various packages with each toolchain and see _which_ package is having a behavior difference. (It's also possible that it stopped working when I reinstalled Linux at the start of the month, which means the qemu it's using now and the one it used to have are very different. This is really to see which of those two ratholes to go down.)

And it's the qemu one. Ok, fire up git... And I have to go --disable-werror now because the Werror disease has spread to qemu. (Different gcc versions produce different warnings. Differently patched distros produce different warnings, such as not checking the return value of asprintf(). Apparently that's a warning now. So is ignoring the return value of chdir("/"). Maybe that's an Ubuntu thing, I dunno, but in general -Werror is _useless_ these days.

Repairing my desktop again while it builds. I don't know why some of the things I add to the xfce taskbar survive reboots, but others don't. This is the... 5th time I've added the CPU governor plugin, I think? (The battery icon doesn't show you current CPU speed, or let you change it, which is one of the main ways to control battery life. You have to go dig up a separate control for that.)

I suppose I don't really mind having a separate little fiddly plugin for everything. What is it about volume controls that Linux desktop developers have such trouble with? The KDE one was good: that had a little speaker in the tool bar that gave you a slider pop-up when you clicked on it, and went away when you clicked on anything else. The Gnome one had a slider pop-up that never went away unless you explicitly clicked on the speaker icon a second time to dismiss it, and displayed above everything else. That was annoying.

The Xfce one doesn't have a pop-up, instead it creates a new window every time you click on it. Did I mention that creating a new window takes about 5 seconds, during which you have no acknowledgement that anything is happening until the new window suddenly appears? Did I mention that you can have a half-dozen of these old volume windows lying around if they fall to the back of the z-order instead of getting closed? Did I mention they take up almost 1/4 of the screen for what is essentially a single control slider? Yeah, I know. Oh well.

I want the xfce general desktop with the kde volume control. Even if I could mix and match them (they don't seem to have a common docking API), the KDE thing would suck in 100 megabytes of shared libraries to twiddle the volume. The "let 1000 flowers bloom" approach only works if there's some way to integrate the result, and after 20 years of X-based GUI development we've got nothing in that department.

Ok, so the first problem is that -hda is showing up as /dev/hdc. That's odd. Is this because I'm using the -git version of qemu? I think I was using the 0.10.1 release before, lemme try building that...

Great, 0.10.1 dies in the bios (I traced this down once; doing so sucked), and in current -git the bios is fine but the emulated hardware has at least two obvious things wrong with it. Maybe I can git bisect a sweet spot where everything worked...

Meanwhile, upgraded the kernel to 2.6.30. The perl removal patches are unchanged except for applying at an offset now. I am officially nervous about this, but can respin and resubmit them Monday morning.

So, smoketest-all.sh is reporting that armv5l, armv6l, i586, i686, mips, mipsel, and x86_64 are passing. This leaves m68k, powerpc, powerpc-440fp, sh4, and sparc reporting failure.

Sparc's been broken forever, and m68k isn't supported by qemu yet. I checked sh4 by hand and the problem is it's so slow the new timeout logic is triggering (which is realy a minor qemu problem, very slow emulator for that target). Run by hand with no timeout, it works. It just takes over a minute to boot up and compile "hello world". (It seems to be some kind of timeout in the emulated hardware, actually. Once hello world is built, it runs essentially instantly.)

The powerpc target seems to be building fine, it just won't run. Or more accurately, the kernel boots but it can't find the hard drive to mount the root filesystem. This sounds familiar. But how did chaning the _compiler_ break this? (Build break due to pointless extra error checking rejecting a construct that used to compile just fine, I can at least understand. But this is a change in _behavior_.)

Hey, the sound just died on my DVD, 17 minutes into playing it. Backing up a bit didn't help, fiddling with the volume control didn't help, killing and relaunching VLC didn't help... Sigh. Time Invoke Knowledge a non-technical end user wouldn't have: dmesg has no clues, check and see if the youtube videos I predownloaded have audio... And the flash plugin rectangles are grey and unresponsive... but the netscape plugin isn't dead. Ok, killall npviewer.bin, relaunch vlc... yup, and the sound is back.

When a single failed program can stop any other program from producing sound (but the volume and mute controls insiste everything is fine), it's not really a "mixer", is it?

Hey, after half an episode of running around Ichigo got stabbed and is near death. Finally. That means he gets to use his limit break, which is to level up. This is why he constantly takes on purples. (And now a dream sequence training montage on the importance of levelling your equipment and not just your character. Yes, I'm up to disk 10.)

Beautiful, now most of an hour later the VIDEO died. (Frozen and not repainting, when I drag the window off the edge of the screen and back the obscured portion comes back solid black) But the sound is continuing fine! I'm actually giggling from the MST3K-style horribleness of this crap. I mean honestly, it's staging its failures to cycle through every aspect of its functionality. Luckily a simple restart of VLC fixed that one...

Slept horribly last night. My sinus infection is making me tired all the time (it's like swelling in my head is cutting off circulation or something), but when I lie down it hurts so much it wakes me up. I've been through a half-dozen courses of antibiotics on this thing already over the past couple years, but it looks like I need to go back and be insistent about it because this is ridiculous.

Right before I left home, I fought with my laptop for 20 minutes to get it to suspend so I could head out. It kept timing out the freezer after 20 seconds and booting back up to the desktop, first complaining that the nspluginviewer.bin wouldn't freeze. (Well, actually some white text flashed by on the display to quickly to read it, and then gave me the desktop back with no explanation. As with any user interface issue on Linux you have to know to fire up a shell prompt and run dmesg.)

So I figured out what dmesg was trying to say (the multi-page stack dumps didn't actually help here, they were just a distraction from the list of processes it couldn't suspend at the end) and killed all instances of nspluginviewer. Next it complaining cifsd wouldn't freeze, so I tracked down a dead samba mount that hadn't prevented it from freezing when I was at Mark's place earlier today, but was suddenly a problem now. (At a guess, the samba thread only blocks uninterruptibly waiting for a response from the server when it doesn't hear back from it, so when I was behind Mark's firewall it happily suspended because the server said it was ok, but when I wasn't it wouldn't. I gloss over 5 minutes of trying to get it to umount when one of my open terminal tabs had a shell prompt open with the current directory under that mount point. Well, actually the terminal tab had a root prompt at /, but if you exited _that_ the parent process was a shell prompt with its current directory under the /mnt in question. Linux: smell the usability.)

I'm gradually wandering back into BusyBox development, although this currently consists of posting on the list a lot and getting into arguments with the other developers. (Yeah, par for the course I know.)

Some prominent TODO items I have, off the top of my head:

• Busybox:
• Check toybox commands, check what can be ported to busybox. (Patch definitely. Netcat?)
• Test suite: Check toybox tests, see what an be merged. Covert rest of busybox tests from subdir to command.tests format.
• Update/break up toybox structural documentation, post each piece to busybox with rationale. Convince people it's a good idea, then do huge amounts of coverting work to move all the busybox applets over. (Figure out how to do this in stages; it's a huge amount of work.)
• Port busybox to the iPhone.
• FWL
• Get all architectures working again. (Yes, that includes sparc. And see if the m68k stuff that's gotten posted to the list actuallly works.)
• Get nightly builds happening and uploading for uClibc, busybox, and kernel tests. Figure out where to upload stuff. (To impactlinux.com of course, but should I also upload toolchains to my dir on kernel.org? System images with native toolchains on uClibc.org? Static busybox binaries for each target to busybox.org? How often, weekly? Run test suites?)
• Cut a release.
• Kernel
• Perl removal patches.
• Miniconfig.
• Hello world kernel.
• qemu
• Test m68k.
• QEMU weekly news.
• Qemu C Commpiler (qcc)
• Dig up the C++ paper and do another pass.
• Figure out how my desktop's once again grown 4 browser windows with a couple dozen tabs each and close 'em. (I'm slowly growing to tolerate Firefox. Yes it hangs for 20 seconds at random intervals, refusing to even repaint its windows until it's done. But the new version of kmail is doing that now too. #ineedtobuyamac)
• Watch the two netflix DVDs I have with me, and/or the accumulated podcasts, and/or the videos I downloaded from Mark.
• Learn LUA.
• Update the rss feed creation script to parse the "span" tags I've been putting in this blog for the past year or so, and create rss feeds with _just_ that content for each tag.
• Read "Kitty and the Midnight Hour" (bought 2 more books in this series at Hypericon and I'm starting it over from the beginning, so I threw that book in my backpack too).

There are 8 zillion other things on the todo list (biking here was "get exercise"), but those are the "might feel like working on them if I sit down to do something now.

So, attacking the FWL broken architectures. x86, x86_64, and mips (but not mipsel) are working. Let's see what's wrong with armv5l, that's usually a fairly reliable one:

  CC      networking/dnsd.o
networking/dnsd.c: In function 'process_packet':
networking/dnsd.c:428: error: lvalue required as unary '&' operand
networking/dnsd.c:430: error: lvalue required as unary '&' operand

Oddly enough, I'm pretty sure that's _not_ the same error that happened on securitybreach (the 8-way gentoo server), but let's get it working here first before debugging it there.

Step 1, try to build busybox from the command line with the cross compiler. So:

cd ~/busybox/git
mkdir ../temp
git archive master | tar xvC ../temp
make allyesconfig KCONFIG_ALLCONFIG=~/firmware/firmware/sources/trimconfig-busybox
PATH=~/firmware/firmware/build/cross-compiler-armv5l/bin:$PATH make CROSS_COMPILE=armv5l-

And it... built to the end just fine. Ok, what I built was more or less current -git, rather than the release version. (I only have one outstanding patch, for pgrep, and that's almost certainly not relevant here.) So maybe they fixed it, and maybe this just isn't reproducing it. (I note that git tag -l doesn't even show a 1_14_0 tag, the newest is 1_13_4, but I think that's an artifact of the svn->git conversion. I should ask on the list.) So, try again with the busybox 1.14.0 tarball the build was using, and...

Yes, it reproduces the error, and the git version of busybox doesn't. Do a diff between the git networking/dnsd.c and the one in 1.14.0 and... they're identical. So something in some header changed. The error is happening on a call to move_to_unaligned32(), and that's defined in include/platform.h so let's see what the diff is between those two...

And the file is _full_ of unnecessary whitespace changes. BRILLIANT. Ok, try again with diff -b...

Well, that explains why x86 and x86_64 work:

#if defined(i386) || defined(__x86_64__)

Now I'm wondering why mips worked. (Presumably the guts of gcc 4.2.1 didn't complain about the lvalue thing on mips.) Which brings up the question of why mipsel _didn't_ work... yup, it's complaining about the same lvalue thing.

Ok, all architectures brought down by the same busybox bug, which seems to be fixed in source control. It's git 3be23086 in the busybox repository thing, applied on April 17th. (I note that "git blame" is kind of useless if something else, like a pointless whitespace change, has touched the file since. With mercurial you can subtract one from the commit number you want to see before and do an "hg annotate -r commit filename". But you can't do previous/next math on git commit hashes. Smell the usability.

So yesterday I got x86_64 building all the canadian stuff with gcc 4.2.1, and checked it in. Then I told the 8-way server to build all the targets and went to bed, and today I find that half the architectures went boing.

Sigh. I _really_ need to get linux-2.6.30 building today so I can update whatever may have changed in the perl removal patches and submit that upstream, but this is being stroppy. Armv5l died with this during the cross-static build of uClibc:

libc/stdio/_READ.c: In function '__stdio_READ':
libc/stdio/_READ.c:39: error: 'LONG_MAX' undeclared (first use in this function)
libc/stdio/_READ.c:39: error: (Each undeclared identifier is reported only once
libc/stdio/_READ.c:39: error: for each function it appears in.)
make: *** [libc/stdio/_READ.os] Error 1

But usr/include/limits has it? So what exactly is it complaining about?

But first, I need to get smoketest.sh to kill its child processes. I want a "kill --recurseive 12345" that takes out all the child processes and their children and so on, but apparently I've hit yet another thing that the standard unix tools never bothered to implement.

I am so _sick_ of fighting the shell. The sparc target hangs when you try to test it, so testall.sh gets hung up on that one. I can run a background process that does "kill $(jobs -p)" after a timeout, or does a "pkill -P $$", I can set a trap "pkill -P $(jobs -p)" EXIT, and so on. But so far, nothing but "killall qemu-system-sparc" actually kills the qemu instance.

I'm running a script that's running a script that's calling qemu, and the child processes don't get killed when the parent process goes away. After the qemu instance hangs it doesn't get killed when its parent shell script gets killed by a signal, and the traps I'm putting in won't propogate down to it. (The buggy error messages don't help: "pkill -P pid" complains "no signal P" when the pid in question has already exited, which is _not_ helpful...

Such crap. Such total crap. This is not the bug I'm trying to fix, I keep hitting OTHER bugs that won't _let_ me get to the bug I'm trying to fix. And then I hit othe bugs when trying to debug that.

Linux is buggier now than it was under Red Hat 9. Why am I bothering?

I dug up my old copy of neverwinter nights, but character generation was such a slideshow (mouse position updating about once a second, which is kind of hard to use with a touchpad) I gave up. It worked fine on my previous laptop, which also ran Linux. (This game is so old that it was back when people bothered to put out Linux versions of games. World of Warcraft even had a Linux client during its beta. Back when people thought Linux on the desktop might actually matter someday.)

Alas, without 3D working, even with the faster processor, it's just unplayable.

So back to watching "bleach" dvds and banging on FWL, at The Donald's (where the apple pies seem to have fixed themselves, dunno if it was just a bad batch, if I was consistently getting them dehydrated, or if corporate changed its mind).

The prerequisite system-image.sh is checking for is the existence of the root-filesystem-$ARCH directory, and that's not a particularly useful test. Sure if the cross compiler didn't build, that shouldn't exist, but if the root filesystem build died halfway through...

The problem is, what specifically can I test for in the root filesystem directory that says it completed successfully? I _want_ to be able to pack up fairly arbitrary contents, so setting a flag file at the end of the build is exactly the kind of dependency between stages I _don't_ want them to have.

The root-filesystem.sh script has a natural dependeency: it can't build without the appropriate cross compiler. It doesn't care where that compiler came from, just that it's in the $PATH. The system images don't. I could add a flag to indicate that it _failed_, but that's a bit non-obvious for comfort. (When it's not asynchronous, return codes work fine, which is why this hasn't come up before.)

I'm torn between trying to properly fix this (although I have no idea how) or just hacking temporary tests into the scripts (yet again) to abort at the point I want to see the output. I guess it depends on what I'm trying to fix next. Eh, bop it on the head with a brick for now, get back to beating the behavior I want out of my old enemy, gcc.

So gcc/config.cache needs to contain "gcc_cv_as_hidden=yes" and "gcc_cv_ld_hidden=yes". I could just prepopulate the cache with this, but _why_ is the test failing? (It doesn't seem to log that part.) Hmmm...

Ah. It decided the assembler name it's using is "". That's just BRILLIANT. Right, explicitly set AS_FOR_TARGET, which for some reason 4.1.2 didn't need but 4.2.1 does. You'd think there would be a way to just specify the darn prefix for all these tools, but no...

I dug up my old copy of neverwinter nights, but character generation was such a slideshow (mouse position updating about once a second, which is kind of hard to use with a touchpad) I gave up. It worked fine on my previous laptop.

Banging away on FWL, moving everything over to gcc 4.2.1 (last GPLv2 release). Been needing to do that forever, yes it breaks stuff, trying to fix it. Need to refactor the hell out of the stage intro code to make debugging buildall.sh easier, though...

When did all this code become so _complicated_? It's just a darn build script. Where can I simplify...

Breakfast at Einstein's, then decided to bike south to the Chick-fil-a location near SJG to try to work off some of the calories from the con this weekend. (Making lots of peanut butter and jelly sandwiches, cutting them into quarters, and filling aluminum trays with them? Marvelous idea. A german restaurant with a live polka band indoors? Not so much.) Going down The Drag I got hit by a car that turned right out of the Schlotzky's parking lot without looking, but luckily I'd seen the "how the stuntmen do it" things (my behind the scenes at hollywood hobby came in useful) and rolled up on the hood like you're supposed to, and my laptop landed on top of me. The bike, alas, wasn't so lucky, and the front tire was all wobbyly afterwards, and I had to spend $60 getting it replaced. Didn't actually make it out towards Chick-fil-a until around 5:30 pm.

Reason #9437 why Linux on the Desktop sucks rocks: every couple hours, the wireless network will suddenly stop working, with no warning. Generally, it means this happened:

[15163.962304] iwl3945: Microcode SW error detected.  Restarting 0x82000008.
[15163.962326] iwl3945: Error Reply type 0x00000005 cmd REPLY_TX (0x1C) seq 0x0000 ser 0x0000004B
[15163.964337] iwl3945: Can't stop Rx DMA.

And under Ubuntu 9.04 _that_ means you're going to reboot soon. In previous Ubuntu versions, you could "modprobe -r iwl3945" and then insert the module back in, and you'd get your wireless back. But in 9.04, if you try it your kernel will panic. New and improved!

This is why I've given up on Linux on the desktop. Each new release breaks things that used to work, and this has been true for TWENTY YEARS now. On a personal level, why should I "get out and push" if the people steering wind up in a ditch or going off a cliff every single time?

Open source does not competently handle user interface issues because "shut up and show me the code" is not a useful response to aesthetic issues or usability problems. It's an inherent structural problem withour development model. Suppose a user notices something obviously STUPID, like the fact that the default action on battery drain is "shut down" and quickly suspending when you get the "system is about to shut down" pop-up just means that when you plug it into AC power and resume, you get to watch it shut down then. They complain about it. We filter out that kind of complaint as noise, and tell them "you fix it". Become an expert in our little area, then submit your fix to us and spend months iteratively convincing us to take it. Or else leave us alone you useless freeloader.

These problems are built-in to our development model. Coding disputes can be handled via objective tests, benchmarks, code size metrics, and so on, which can conclusively end arguments with numbers and proofs and logic. Dealing with more abstract issues, we fork ourselves to death with a thousand competing implementations, each of which does a couple things better than anyone else, meaning it does everything else worse.

This feeds into the way "this is a horrible default that bit me" is met with "here's how you can customize your personal copy, to fix it _for you_". No attempt at a general solution is made, because maybe somebody WANTS that behavior even if most people get bit by it. Creating a usable environment requires huge customization effort, but isn't it great how customizable our system is? You can build your own desktop. You must do so, since the default environment sucks so badly you really have no alternative. But there are so many different ones to choose from, if you just spend three years studying your available options you'll eventually find that special one built just for you. Don't like Totem? Try VLC! Don't like Pidgin? Try Jabber! Don't like Thunderbird? We've got _hundreds_ of alternatives...

So it's easy to _leave_, but hard to actually get anything _fixed_. Users who can't or won't back up their reports with scientific metrics are ignored. Or those who don't regularly perform regression testing, or who aren't going to spend six months following up on an issue, or who can't swap out individual components of their system with ones they built from source (after having upgraded everything to the latest version because fixes aren't made to the old versions; you need to be exposed to all the new bugs at once to test any fixes for your old bug)...

And even if you _do_ try to fix these things, you can't keep up. Pointing /bin/sh at dash was _new_ breakage that older Ubuntu didn't have. KDE 3.x->4.x was new breakage. Losing syntax highlighting in vi and the ability to cursor around in insert mode was new breakage. Kmail getting eaten by Kontact and then its user interface changing so hugely in the most recent release I spent 2 hours finding the "sort by thread" option: new breakage. The same network hardware this laptop has had for 3 years now breaking in new and exciting ways: new breakage. The fact 3D support no longer works for my intel graphics card (it worked fine in 8.10): new breakage. (Although a 64-bit dual core 1.73 ghz processor can just about keep up with software rendering to display video, as long as the system's otherwise idle and the window size is small enough. Even then youtube stuff flickers like mad with the "hidden" under-layer of youtube navigation buttons showing through intermittently due to the compositing happening in a single live buffer, and it tends to go all slideshow at the drop of a hat anyway. Oh, and swapping windows it sometimes leaves uninitialized memory garbage on the screen for several seconds, and sometimes the event that would redraw over it _times_out_ and it's left there in the window's buffer. Even if I drag another window over it, the garbage comes back because the redraw is from a double buffer that's got uninitialized memory in it. Kmail does this all the time with its toolbar buttons.)

There's no "steady improvement". There's no indication of clear agreed-upon _goals_, just random lateral thrashing. I keep having to either abandon existing infrastructure that stops being supported, or personally go to great lengths to keep the older stuff working in newer contexts.

Because of what I do, I mostly notice this with technical things that end users only ever notice as "black magic that failed". On a technical level it's epidemic, such as the way the IDE devices became part of the SCSI layer so device ordering is now fundamentally horked in a way it didn't used to be. (SCSI throws all devices in a big bucket and gives 'em a stir, so your USB devices and your SATA devices draw from the same pool of device names, in first come first serve order. Yes really. Back under the IDE world, /dev/hdc wouldn't move without a screwdriver.) Or the emergence of unnecessary layers like the whole HAL/dbus thing, so that your web server goes "HAL says I'm not connected to the network, so I can't let you talk to that web server you're got running on the loopback interface". (Why does it CARE? The TCP/IP layer has had a "no route to host" error for 30 years now, so if there's no network this fact should be instantly apparent when you try to use it. Adding a separate layer capable of vetoing network access is damn stupid, and whether it actually works or not is a side issue to it having no reason to _exist_ in the first place.) And don't get me started on how hideously unstable and undocumented the sysfs exports have been over the past 20 kernel releases. So on a purely technical level, the people steering Linux haven't filled me with confidence in the soundness of their judgement.

And of course there's social/process things too. The GPLv2 vs GPLv3 split didn't help anybody do anything: we're still paralyzed by infighting against our zealot arm which undermines everything the rest of us try to do, in the name of "Purity". (Similarly, I stopped cooperating with the SFLC after the FSF hijacked them.)

On the social front, the people who insist that openness is inevitable and in future all scarcity will vanish have apparently never studied history. Capitalists try to corner the market on things that were previously free _all_the_time_. Capitalism is a marvelous mechanism for managing scarcity, and in the absence of scarcity to manage, capitalism creates it artificially so it has something to manage. For example, Austin is now ringed with privately managed toll roads. I hate this, but they're still there, and they're building more. Individual instances come and it go as people play whack-a-mole with the latest incarnation, but there isn't going to be some magic day when people stop TRYING to corner the market and make a killing.

But technical and social issues aside, an awful lot of the time I notice that my Linux desktop simply doesn't work at an end-user level, and then have to invoke programmer-fu to try to work around it.

My desktop is Linux. My desktop doesn't work. Most of the time I just want to USE the thing to surf the web, watch videos, read email, edit text, and it breaks repeatedly and I have to duct tape it back together to get minimal functionality out of it. (I STILL haven't gotten the bluetooth association between my laptop and cell phone set back up. On Fade's Mac, it took about five minutes starting fromm scratch, most of which was figuring out where in the config menus to find the appropriate settings page.)

These little "end user issues" come from everywhere in the infrastructure. The network spontaneously dying so only a reboot can fix it is a kernel issue. The 3D support not working anymore is an X11 issue. When I press the eject button my my DVD, it gives me a pop-up message 'Failed to eject "/org/freedesktop/Hal/devices/storage_model_CDRWDVD_DW224EV". Given device "/org/freedesktop/Hal/devices/storage_model_CDRWDVD_DW224EV" is not a volume or drive." and that's HAL/dbus breakage. All of those are brand new failures in 9.04, on the exact same hardware that originally came running Ubuntu 6.10 from the factory. (It has more ram and a bigger hard drive since then, but that's it.) Installing a new version of ubuntu breaks stuff that used to work, every time.

The largest chunk of infrastructure I've had to abandon recently is KDE. I'm now on Xubuntu because the KDE developers did one of those "throw out all the progress we've made and start over" things that resulted in a KDE 4.x so hideously unusable that even Linus Torvalds (who has mocked the gnome developers for years as Just Not Getting It) has abandoned KDE and gone to Gnome. (In Kubuntu 8.10, KDE 3.x was no longer an option, and 4.x simply did not work.) I tried Gnome for a bit (couldn't stand it, and then the system's network configuration spontaneously ate itself so badly I had to reinstall to talk to the network again, and I went back to 8.04 for a while), and now I'm using XFCE instead.

This is just within Ubuntu, the only distro that's still _trying_ for the end-user desktop. I used to use Red Hat, but they abandoned the desktop and went off to the enterprise market. (When the call went out for a company to stand up and defend decss so Linux could play DVDs, they yanked MP3 playback support from Red Hat 8 and bogged off to the enterprise workstation market, discontinuing their consumer version entirely one version later.) I didn't so much abandon Red Hat as get abandoned by it. Then I used Knoppix, but while its "boot from CD" gimmic made it a great platform to demo Linux on various hardware it never actually transitioned to running from the actual hard drive at all well. So I abandoned Knoppix in favor of Ubuntu, then Kubuntu, then Xubuntu, which are all more or less from the same group. There are other distros I could use if I tire of Cannonical's offerings, but that's just another manifestation of "all of these suck, so we keep trying new ones".

This means I'm now using the third place desktop (gnome, kde, xfce) on the third place operating system (windows, macosx, linux). My configuration's market share doesn't even register, and NEVER WILL. I'd say XFCE was even in _fourth_ place behind the desktop the subnotebook distros (like Easy Peasy) are using for eeepc clones, except that we lost that niche (as it scaled up past rounding error, all the new seats it added were Windows) so it's comparing one flyspeck with another and the result just doesn't matter either way.

Since I've never been able to recommend Gnome with a straight face, and can no longer recommend current KDE, I'm now back in the OS/2 position where the desktop I use isn't one I can honestly recommend to anyone else. I use a weird custom system nobody else has ever heard of, and I may never actually meet another user of this particular setup in person. I might as well still be using an Amiga.

Embedded Linux is still moderately fun, but I used to do Java under OS/2 the same way. What the code you produce runs on, and the development environment you produce it in, are two different things. Linux itself will be around as long as Java (which is the new Cobol), but the Z80 processor lasted 30 years and DOS could do just about everything embedded Linux does. Cisco still sells IOS, and Solaris and even Xenix still have their users. In a way, the embedded niche is where old techologies go to die. (And every supercomputer worth the name is a hand-rolled bespoke job.) Maybe the openmoko Google phone will stop being "technology in search of a use case" and start to matter as much as the iPhone or blackberry one of these days. I'm not holding my breath, the cell phone companies have no incentive to give up control when they can sell _ring_tones_ for profit and people buy them.

It's been "the year of the Linux desktop" every single year since 1998. We are the boy who cried wolf, and even if it did start to get interesting in future nobody will ever care, or pay any attention.

This is why I need to buy a mac laptop to do my embedded Linux development on.

Poking at the FWL buildall.sh script. It's broken in several different ways, but the most fundamental one is that its debug output is horrible.

The main problem is that buildall.sh does things that build.sh doesn't, because doing a canadian cross requires multiple architectures, and build.sh builds one architecture through from start to finish. This means if I want to test any of the stuff that only buildall.sh builds, I need to test it in a different build environment that produces horrible debug output.

The horrible debug output is because it has to be able to background a lot of processes (to support FORK=1), and doing anything asynchronously makes logging and flow control fundamentally harder. Also, buildall shouldn't stop the whole build script because one architecture fails, and you have to go out of your way to make it build just _one_ architecture.

There are several ways I could address this. I could teach build.sh to handle the special case of building this extra stuff for the host architecture, which is the one instance where you don't need two architectures because the two match. I could add the option to buildall.sh to die after the first error. But ideally, I want buildall.sh to stop blanking the temp directory for later stages after earlier stages fail.

I taught the individual stages to exit when they haven't got their prerequisites, and even added an option to suppress that error. But they still blank the temp directory from include.sh, before I can make specific prerequisite tests. Moving the cleanup out of common code to solve the flow control issue would require me to duplicate it in every stage. It sucks either way, really.

The reason all all this is important is that the canadian cross stuff (or an equivalent set of prerequisites) is required to do an --enable-shared compiler, which is required to build uClibc++. (It's an unpleasant set of circular dependencies, based on the fact that libgcc_s.so is a completely horrible idea. The compiler should not add shared library prerequisites to its output, especially not for a library that links against libc.so. It wouldn't have been so bad if --disable-shared built libgcc_eh.a, but that would have been _smart_ and we're talking the gcc developers here.)

It also turns out that the reason I've been having such trouble with soft-float on arm is that they decided to add the soft float stuff to libgcc_eh.a, so it only gets built for --enable-shared. (I.E. the --disable-shared compilers have been gradually crippled by the gcc developers ever since the start of the 4.x series. Wheee...)

At a convention, and what do I do? Program on my laptop, of course.

Figured out why gcc 4.2.1 was going nuts (if you pair gcc-core in one version with gcc-g++ in another version, it gets unhappy). Now building everything with that, so I can beat soft-float out for armv4.

The mercurial git->hg converter is still broken (at least with the busybox repository). Need to bug the list about it...

And once again, buildall.sh is going "boing". Sigh. It's sad how brittle binutils and gcc are, what I'm trying to do is fairly straightforward and I have to supply a dozen separate overrides of its assumptions to do it.

Random bug in VLC: if you enable deinterlace, it kills the old window and pops up a new window. The new window has no subtitles in it, you have to disable subtitles and then re-enable them to get them back.

I'm used to Linux desktop software being buggy. This is normal.

Broke down and set kmail back up, and caught up with the accumulated week of back email. (Mostly spam, of course.)

And kmail has lost the ability to thread messages in 9.04? There used to be a folder->thread messages option in the pulldown menus, and now it's gone. And I can't find any replacement. That's just brilliant.

Reinstalling various things. I have no idea why the "hg convert" extension spontaneously disabled itself. I had the "hgext.convert=" line in my .hgrc, and it got commented out somehow. Apparently, installing mercurial from a dpkg goes through your existing .hgrc and comments out extensions you have enabled. That's deeply stupid; if I have an .hgrc possibly the data in it _means_ something, eh? Once again, Linux developers are _sure_ they know what I want to do better than I do. I need a mac. (They're probably as arogant, but far more likely to actually have good UI suggestions.)

Looking at the busybox "patch" command to see what would be involved in replacing it with the toybox one. (Either as a patch I maintain locally, or something I can push upstream to the busybox guys.) Their version is 254 lines, and mine is 361 lines. On the other hand, mine does a heck of a lot of stuff theirs doesn't (starting with supporting offsets, which is why I need it).

Ideally I'd make it config down to be as small as the old one, but it wasn't really designed that way. Hmmm...

Another thing is that after using the toybox infrastructure for a while, the busybox infrastructure looks HILARIOUSLY clunky. Put a prototype for the applet's main function on the line above the declaration of that main function. (Why?) And of course pass in argc and annotate the fact it isn't used. (Um... Ouch?)

So I found a new way to break toybox, and I don't want to put out yet another single patch bugfix dot release. If I don't want to bother with it anymore, then I need to push stuff upstream into busybox.

I pretty much just need patch, oneit, and netcat. I need my version of patch, because the busybox one can't apply anything at an offset. I need my version of netcat because the one they've got doesn't seem to have working -l support. And I boot the resulting system under oneit, which I suppose could be standalone.

The big part is that if I start working on busybox again, I'm going to have a huge urge to clean stuff up...

I tried to build qemu from source, but the version of the source I had died with a build break halfway through. (I have horrible luck with random source control snapshots, part of the "I break everything" schtick.) So I thought "ok, git update time"... except that it hung. Why did it hang? Because savannah.nongnu.org is down. That was yesterday. It's still down.

Luckily, the mailing list archive is still up (yeah, my personal email still isn't reinstalled yet. I need to break down and go back to kmail...) and there's good news. Now that the project has a new maintainer, not only did they migrate off cvs but they may finally leave savannah as well. Yay!

Watching Obama's weekly address. (He's really going to the well for his supreme court nominee.) What I'm really noticing the little flag pin on his suit and tie, and can't help but wonder if he's wearing bright red tennis shoes along with it to complete the ensemble. (Or is that just british?)

Attempting to listen to a podcast where Seannan McGuire (the lady whose livejournal I started following after she wrote a marvelous Barry Ween fanfic) is interviewing Jenn Brozek (the ex next door neighbor of Andy Weir, who wound up in his now completed webcomic as an international jewel thief with a time traveling daughter. Both Jenn and Seanan are also published science fiction authors. Jenn's an editor. Seanan's a filker with several albums out, and draws an intermittent webcomic. Interesting people, generally.

The two of them wound up at baycon together, and BayCon Podcast #4 is Seanan interviewing Jenn. It's a small internet, innit?

On an unrelated note, an interesting thing about VLC is its starting size is the actual width and height of the source video, in pixels, with 1:1 scaling. (The other players I've used just remembered the previous size relative to the rest of the desktop.) This makes it clear that the _reason_ Obama's podcasts look way the heck better than the msnbc ones is the white house podcasts take up the whole desktop, and the msnbc ones are a tiny little window in the corner. (DVDs take up maybe about half the screen.)

Mark poked me last night about several FWL todo items, so I guess I should tackle them...

Let's see, I got the nightly snapshots building with uClibc-git, I need to test those to make sure they're all working, then upload 'em to the binaries directory on uclibc.org. Once I've got the those working, I need to figure out which ones I can switch to nptl, probably adding new targets for those (ala armv5l-nptl).

And I should really deal with sparc. And I should switch over to gcc 4.2.1 as the stable version and make it always build libgcc_eh.a even when it's --disable-shared (which would be a _better_ way to fix the arm soft float issues, and also fix...

Ok, THAT was annoying. The display froze. The mouse still moved, and the audio of the video I was watching continued to play, but the video itself was frozen and nothing on the desktop responded to the mouse pointer or being clicked on, the keyboard was dead (ignoring ctrl-alt-F1 and ctrl-alt-backspace), and the power button didn't suspend. Luckily, holding the power button down for 10 seconds powered the sucker off, since that's not under Linux's control. And thus I was able to reboot.

That was crazy, the kernel was obviously still working because the audio playback would have stopped if it wasn't (and the mouse would have stopped moving), but all user I/O was frozen. Why on earth did this get redesigned? What idiot decided to route the power button (acpi event) and the desktop (X11) through a single point of failure? (Sounds like the darn "input core".)

Great, now my laptop can choose to just hang on me at any time. "Xubuntu 9.04: there are no nontechnical end users I hate enough to inflict it upon."

I need to buy a mac.

One of the main failures of Xubuntu, or possibly Desktop Linux in general, is that you can't easily pick and choose which apps you want to run. Yes, that "failures" and "can't" rather than "features" and "can".

Xubuntu comes with "movie player", which is actually a program called "totem" although it never anywhere tells you that. (You have to use ps to see the name of the running process to figure that out, and this is intentional. That's a bit like saying "web browser" and never revealing whether you're running IE, Firefox, Opera. Did I mention the Gnome developers consistently make horrible user interface decisions and then defend them to the death? Why the xfce guys are humoring the Gnome guys' poor decision making skills is an open question...)

If you press the eject button on the dvd drive, it pops up a window saying "Failed to eject "/org/freedesktop/Hal/devices/storage_model_CDRWDVD_DW224EV", and tells me that name is not a volume or drive. I have to go to a console and run "eject /dev/sr0" as root to get the disk back. This is unrelated breakage, it's because the "hardware abstraction layer" was a horrible, horrible idea Linux copied from Windows NT (see Gnome == stupid, above). But I can't tell modern software _not_ to use it.

Totem is an inferior dvd player to Kaffeine, the one Kubuntu used back in 8.04. (So the rest of what I'm talking about is a regression vs what they got working back in KDE 3, which was rendered useless when they moved to the unusable KDE 4.) Totem auto-launched when I inserted a DVD (not even popping up a "what do you want to do" menu), and then told me it couldn't access the dvd player. Turned out to be just a race condition, and on relaunch it accessed it fine. (Kaffeine did not have this race condition.)

Then it turns out that it doesn't seem to deinterlace the dvd by default. (Note I already went through the magic dance to get decss support working, which is non-obvious and something the mythical nontechnical Linux desktop users would have to phone a friend to get to work.) Google implies that back under 7.04 you could find a deinterlace option under the "view" menu, but there isn't one there under xubuntu 9.04. That menu has "fullscreen", "fit window to movie", "zoom in", "zoom reset", "zoom out", "aspect ratio", "switch angles", "show controls" (the play button and such at the bottom, no deinterlace option among them), "select text subtitles", subtitles (a sub-menu, not only redundant but it isn't aware of the actual subtitles this bleach disk has, which I navigated the DVD menu to switch on), and "sidebar" (which is read-only, showing things like the duration, resolution, and frame rate). Kaffeine puts the duration as a label ont he slider at the bottom.

Kaffeine not only had "interlace" actually as a labeled option in the menus, but when I hit "i" it said it was toggling interlace. I _think_ I enabled some kind of deinterlacing by pressing all the letter keys in sequence (accidentally fullscreening and quitting and such along the way), but A) it never notified me I was doing so, so who knows what key it was, B) the deinterlacing algorithm it's using is crap. It's better than nothing, but still full of artifacts.

Of course the problem with installing Kaffeine is it's bolted to the KDE desktop, using the QT widget set and kobjects and so on. We have so many redundant implementations of exactly the same infrastructure, and they're all slightly incompatible so can't be substituted for each other. It's like windows "dll hell", only built into the OS.

So I tried installing "vlc", another dvd player that google recommended, which presumably doesn't have these unnecessary dependencies on pieces of infrastructure it shouldn't care about. It installed fine. It seems to play the dvd. But when I put in the dvd, totem still pops up. I uninstalled the package "totem", which made no difference. There's 8 gazillion other sub-packages (totem-common) and such, several of which imply it's tied into the guts of mozilla. (This seems odd since mozilla's using the binary-only flash plugin to display essentially all video, because nothing else really works.) Ok, remove totem-common, let it randomly rip out three other packages to "resolve dependencies"...

Ok, now it does nothing when I insert a disk, and I can pull up vlc by hand from the menu to play the disk. I can live with this. (This has no menus, the whole window is displaying the video. You right click on it to get options, and video->deinterlace has seven options under it. It defaults to "disabled", yet even that seems to be deinterlacing the picture slightly better than totem was. I set it to "blend", although what I really want is "just deinterlace properly, don't bother me with implementation details since I'm not a DVD playback codec designer". (This is a really common failing of Linux software, you have to know how it's implemented in order to make it work.)

Except that with both kaffeine and totem I could use the cursor forward and back keys to skip ahead or go back in the disk, and also get slider bar along the bottom. With vlc, I can jump to the start of episodes, and that's it... Ah, I see. There's a progress bar on a separate window. I guess that's reasonable...

I still need to buy a mac, so I don't have this much upgrade hassle every 6 months. I have to make the "to kmail or not to kmail" decision again, and until then I don't have an email client...

How do the ubuntu developers get stuff this simple wrong?

When I fire up totem and tell it to play a podcast, it says it hasn't got the plugin and offers to search for it. It comes up with a window that says "GStreamer ffmpeg video plugin" and "GStreamer plugins for mms, wavpack, quicktime, musepack". Ok, fine. There's the pointless step of clicking checkboxes to select them, which pops up a dialog about legal disclaimers (clicking a checkbox should not pop up a dialog, that's horrible UI design), and then I tell it install, it grinds for a bit, and then says "Software installation failed". It won't tell me what the problem was, and there's no way to get extra information. The three buttons are "retry", "add/remove more software", and "close".

Oh, and did I mention it won't tell me the package _names_, just the description fields? If I want to do this from the command line (and thus have at least a _chance_ of better error messages), I have to figure out that "GStreamer ffmpeg video plugin" is _probably_ the "gstreamer0.10-ffmpeg" package. I still have no idea what the _second_ package is.

You could write a "user friendliness fail howto" out of this. Automation that completely hides how the underlying thing works is not useful if it means your car runs out of gas and it refuses to say so (or even that gas is a thing it might ever need).

May 27, 2009

Computer tied up most of yesterday backing stuff up, and today restoring it. Being able to do 10 megabytes/second through a USB cable isn't that impressive when you're backing up and restoring 300 gigabytes to a terabyte drive.

May 26, 2009

May 24, 2009

May 23, 2009

A little while ago I promised to explain:

So if open source used to be the norm back in the 1960's and 70's, how did this _change_? Where did proprietary software come from, and when, and how? How did Richard Stallman's little utopia at the MIT AI lab crumble and force him out into the wilderness to try to rebuild it?

Two things changed in the early 80's: the exponentially growing installed base of microcomputer hardware reached critical mass around 1980, and a legal decision altered copyright law to cover binaries in 1983.

Increasing volume: The microprocessor creates millions of identical computers

The shortage of customers was changed by the microcomputer, which was based on the microprocessor. Intel invented the microprocessor around 1971 (largely by accident; here are interviews with the engineer who designed it, the one who implemented, the customer they did it for, and their boss at the time

The first computer to sell 1 million units was Comodore's Vic 20, introduced in 1981 and beating the Apple II to the million unit mark by a few months. So within five years of the Apple II's introduction, you could potentially sell a million copies of the same software. This allowed the concept of "shrinkwrapped" software to emerge, which you could write once and then sell many identical copies of without even recompiling. Initially this supported small companies consisting of a programmer and perhaps a small support staff (WordStar, Richard Gariott's "Alkabeth"), but exponential growth in the hardware market led to matching growth in the potential software market.

The first "killer app" (a software program compelling enough to drive sales of the system it runs on) was Visicalc, the first spreadsheet. It was produced by a two person company, Dan Bricklin and Bob Frankston, operating out of Bob's attic. The first version shipped in October 1979, and was the main reason that Apple's less than 50k units its first two years were followed by a million more. (Another fun quote in that article, by Bob Frankston: "it's worth noting that back in 1979 people viewed the keyboard as an impediment to using computers. After all, only secretaries could type...") Oddly enough the book "On the Edge", which is about commodore, gives great behind the scenes information on early Apple and Visicalc stuff. The fact commodore bought the company that made the processor inside early Apple machines has a little to do with it, but mostly it was a darn small industry at the time and everybody knew everybody else...

So the reason something like Visicalc couldn't happen before 1980 is there was no customer base to sell shrinkwrapped software to, not even enough to support individual programmers operating out of their bedrooms unless they had a day job doing something else. You could make a living as a consultant writing bespoke software on commission, but your customers told _you_ what to write, and owned the result which of course included the source code.

The other thing about microprocessors is they were more uniform. The old "wire up a breadboard" computers weren't just produced in lower volume, they were more easily modified at a fundamental level.

The MIT AI lab where Richard Stallman learned to program was started in the 1950's with a pair of prototype computers, built by a recent graduate named Ken Olsen when he interned at Lincon Labs in Boston. Both were unique, each the only one of their kind ever built. The first was the MTC (Memory Test Computer), which only had only four instructions and was built for a single purpose: to stress test early "core memory" and prove it could work reliably under load. The second was the TX-0 (Transistor Experiment 0), Ken Olsen's pet project to prove that transistors could be made reliable enough to build a computer and memory out of them. (Early transistors were so sensitive to static they were even less reliable than vacuum tubes, Olsen came up with a way to buffer them so they could survive normal handling and operation.) Both prototypes were heavily modified and upgraded by the students at MIT, because although they weren't very powerful they weren't shared, either. Each belonged entirely to the AI lab 24/7, so the students there didn't have to make an appointment to use it or get permission to modify it.)

Olsen went on to found Digital Equipment Corporation to commercialize his new transistor memory, and hired a bunch of his fellow MIT graduates to do so, finding the most useful ones came from the AI lab that had received his cast-off prototypes. (The book "Hackers" by Steven Levy covers the founding of the MIT AI lab. This marvelous interview with Ken Olsen covers the TX-0 from the Lincoln Labs end, plus early transistor memory experiments, and goes on to chronicle the rise of DEC.) The TX-0 was the only one of its kind, and when Olsen founded DEC he hired a bunch of MIT graduates and sent one of the first PDP-1 systems to the MIT guys to train future graduates so he could hire them too. (The MIT guys soon wrote the first shoot-em-up video game, "spacewar" for the PDP-1, which became DEC's main sales tool for showing off the new system. DEC eventually sold around 50 PDP-1 systems.)

When DEC built its first mainframe (the PDP-6, they which shipped a total of 36 machines in its entire production run), MIT of course got one. When DEC upgraded the PDP-6 to the more powerful PDP-10 (about 700 total ever made), MIT's system was upgraded. That PDP-10 was the machine Richard Stallman did all his work on at MIT.

All of these systems, from the TX-0 to the PDP-10, were heavily modified by the MIT guys, to add new instructions and capabilities. The fact that MIT not only wrote its own operating system from scratch but customized its computer hardware to add extra instructions and registers, really wasn't that unusual under the circumstances; everybody did it back then. The computers were made by wiring together individual diodes and resistors and such on a bread board. Wiring in a few more wasn't a big deal. But it meant that programs written for a single machine ran _only_ on that machine, or perhaps one or two others that had been similarly hand-modified.

Microprocessors were different in that you couldn't customize their circuitry the way you could a breadboard full of chips and wires. There weren't just more of them but they were also more uniform, thus software didn't need to be ported between different machines of the same model, it could run out of the box. That was something new.

Only with the arrival of millions of identical microcomputers, based on mass-produced microprocessors instead of expensive hand-wired breadboards, (see the books "Soul of a New Machine" and "A few good men from Univac" for more descriptions of that era) could you write a program once and then sell the exact same program to lots of different people without porting (or even recompiling) it for each one of them. That's why the software in the 1960's and 1970's was either custom stuff individually comissioned from consultants, or it was produced by hardware manufacturers in an attempt to sell their hardware, or it was produced by the people who had the computers for their own use, and shared through user groups like DECUS and the CP/M Users Group Northwest. All this stuff came with source code.

But in 1980 Apple Inc. had a record-breaking IPO (the largest in history at the time) based entirely on sales of the Apple II. This triggered IBM to rush out its own microcomputer (the IBM PC) in hopes of squashing Apple before it grew too big to contain. (That's an interesting story in its own right, but a side track to the rise of proprietary software. Robert Cringely's book about it, "Accidental Empires", is fairly accessible and was made into an even more accessable PBS documentary called "Triumph of the Nerds".) In 1982, "The Personal Computer" was Time Magazine's Man of the Year. Exponential growth that continues long enough shakes the entire world.

Changing the law: Apple v Franklin extends copyright to cover compiled/assembled binaries instead of just source code.

The copyright issue changed in 1983, when the Apple v Franklin ruling extended copyright protections to binary code. (The writing was on the wall in that case over a year earlier, but the decision had to be upheld on appeal before commercial interests could seriously act on it.) Before that decision, source code was copyrightable but binaries weren't, so companies shipped source code to _increase_ their ownership of the code in the eyes of the law. If you just shipped precompiled binaries, you had no rights the law would recognize.

The first real "voice in the wilderness" about proprietary software was a kid named Bill Gates, with his 1976 "letter to hobbyists". Except that nobody took it seriously. (Richard Stallman claims not to have even heard about it at the time.) People only listened at the time because MITS paid to run the letter in a dozen magazines, and he wrote the letter because the world _wasn't_ like he was describing, like he wanted it to be so he could profit from it.

William H. Gates III ("Trey" to his friends) was a rich kid and the son of a lawyer who dropped out of Harvard (pre-law) to start a software company with a techie friend of his (Paul Allen). Gates's Micro-soft was founded (in 1975) in hopes of not merely selling software to the emerging microcomputer market, but explicitly cornering that market. Except that their tiny company could only afford starvation wages, to the point that both of its co-founders, Paul Allen and Bill Gates, took day jobs at their main client, MITS for the first few years of the company, and their third employee (the guy their "letter to hobbyists" credits with actually writing Altair BASIC, although their official history now differs), was a part-timer they had to lay off.

They referred to their strategy companies as "riding the bear", attaching themselves like a remora onto one dominant hardware manufacturer after another, grabbing the software crumbs falling from that hardware market and switching to the next when it lost its position. First they rode the Altair's manufacturer MITS, then switched to Tandy as their meal ticket when MITS went under. Finally they landed a lucrative contract with IBM (because Gates' mother was on the board of directors of the Red Cross with IBM's CEO and thus IBM could do business with "Mary's Boy" even though IBM wouldn't ordinarily approve such a tiny company as a vendor because it might not stay in business long enough to complete the contract; see the book "Big Blues" for details). Along the way, they sold their software (mostly BASIC) to Commodore and eery other company that would buy it.

Ironically, the first software product of micro-soft was Basic (a language developed at Dartmouth college in 1964 and freely appropriated by micro-soft). Because it was the only high level programming language MITS offered for the Altair, it quickly became the lingua franca of early 8-bit microcomputers, starting with the Altair clones from companies like Imsai but spreading to the Comodore 64 and Apple II and even IBM's PC. The thing about Basic is it was an interpreted language, where the source code _was_ the program and you ran it directly. Books and magazines distributed BASIC programs by printing source code could type into your own computer. Again, all code written in early 8-bit BASIC was open source, because _not_ being open siply wasn't an option. The language itself enforced it.

Gates was the type of person to not only see a market opportunity in bottled water, but then consider drinkable tap water as a threat to his business and respond by trying to make tap water undrinkable.

In his 1976 letter to hobbyists, Gates was incensed that microcomputer users mostly wrote their own software, and shared their programs freely with others. Users saw Gates as the type of person to not only see a market opportunity in bottled water, but then consider drinkable tap water as a threat to his business and complain about it. Back in 1976, he wasn't laughed off the stage: he was completely ignored.

It's a bit like the recent Time magazine article about how newspapers must start charging for online content by a guy named Walter Isaacson. It's considered a laughable idea today, when the norm for internet content is free access. But if that changes, Isaacson will look like a prophet.

What users didn't expect in 1976 was that Gates and others like him would respond not only with FUD (such as "shrinkwrap licenses" which were laughably unenforceable at the time) but by trying to change the law. Apple's 1980 IPO attracted Wall Street's attention to the booming computer industry, the sharks smelled money in the water, and proprietary interests have lobbied continuously to change the law for their own financial gain ever since. (Note it's still ongoing: the software patents blowup happened in the 1990's, and the Digital Millenium Copyright Act at least claimed to make shrinkwrap licenses enforceable in 1999. Today Rupert Murdoch (owner of Faux News) is trying to implement Isaacson's ideas.)

The thing about Gates is repeated failure didn't discourage him. Here's a 1980 phone interview with a young Bill Gates (audio, transcript, context) who was incensed that a recently published book about TRS-80 programming had printed an annotated dissassembly of the TRS-80 ROMs, so hobbyists could understand their machines better. Gates insisted those roms were his property, and nobody else had the right to look at them (let alone print out annotated disassembly and sell it for profit), and he lobbied congress to that effect. The entire interview is Gates talking about his attempts to change copyright law, back in 1980, into what he wanted it to be.

Similarly, when Microsoft belatedly noticed the internet in 1995, it tried to replace it with MSN, the Microsoft Network. (Remember that in Windows 95, the "unremovable" icon wasn't Internet Explorer, it was MSN. When that strategy didn't work, they bought a browser called Spyglass and renamed it Internet Explorer and made _that_ the unremovable icon, in a second slightly more successful effort to at least partially de-commoditize the internet.)

But back in the early 80's, it wasn't Microsoft that succeeded in changing the law, it was Apple. The Apple v Franklin lawsuit was Apple's response to a company (which became Franklin Ace, makers of the Speak-N-Spell) that cloned the Apple II. Franklin's computers ran Apple software, and they used Apple's ROM code verbatim in their competing product in order to do that. The courts decided that Franklin had gone too far, and that copyright must extend to cover binary code to prevent that sort of thing.

Part of the reason AT&T agreed to be broken up in 1983 (implemented in 1984, but agreed to in 1983) was to get out from under the antitrust regulation imposed on it by a 1956 consent decree, forbidding it from entering any other business beyond its telecommunications monopoly. It nominally owned Unix, by then a widely used operating system, but couldn't enter the booming computer industry as a heavily regulated telephone monopoly. When Unix first leaked out of Bell Labs in 1973, there was nothing to commercialize because there was no market. In between, the computer industry _changed_. Unix was open source from day 1, and developed a vibrant community in the 1970's based on this. (It didn't have a name for open source because it didn't need one, there wasn't any non-open-source software to speak of. When copyright first became an issue for software, the term commonly used for unencumbered software was "public domain".) When AT&T suddenly started enforcing its newly acquired copyrights, and charging _extra_ for source code, it was an enormous shocking change.

AT&T wasn't alone in this, of course. IBM's Object Code Only (OCO) policy was announced February 8, 1983 (more on this), but software predating that commonly came with source code, and often even that source wasn't copyrighted. This is why the Hercules S/370 emulator has plenty of old public domain software to run, including things like the MVS operating system (see also here). Yes, even IBM's mainframe operating systems worked by different rules back in the 1970's.

RMS tells his own story (in his own words, href=http://oreilly.com/openbook/freedom/ch01.html>biographical version) about a Xerox printer he couldn't get the driver source to. This came as a shock to him because previously, giving source code to any software was a matter of course. The world was changing under him, and he didn't like change.

So when Richard Stallman founded the FSF in 1983, the same year DOS 3.0 came out, he was reacting to the recent and ongoing rise of proprietary software. Microprocessors created an exponentially growing hardware market, the Apple vs Franklin decision gave software authors vastly expanded copyright powers, and a gold rush ensued (similar to the dot-com boom of the 90's) that hired away all the "hackers" from the MIT AI lab to work in a pair of start-up companies attempting to commercialize LISP. While his fellow students graduated and took jobs out in the booming commercial computer industry, Stallman stayed in academia as a perpetual grad student, and hid away from the problem as long as he could.

Kicked out of the nest: cancellation of the Jupiter project obsoletes the PDP-10 at the heart of the MIT AI lab.

The big change that forced Stallman out of his comfortable nest at the MIT AI lab was the obsolescence of the operating system he used, which was tied to specific hardware that was discontinued.

Over the course of three decades (1950's, 60's, and 70's), the MIT AI lab upgraded from the tiny Memory Test Computer all the way to a PDP-10 mainframe, keeping ahead of Moore's Law (which Gordon Moore first documented in 1965). Just as the PDP-6 became obsolete and was upgraded into the PDP-10, the PDP-10 itself was over a decade old by 1980, and ripe for replacement.

To replace it, DEC prototyped a new machine called the "Jupiter project", which was roughly compatible and would thus allow PDP-10 software to be easily ported to the new system. But there were only 700 PDP-10 systems ever sold, and their newest computer (the VAX, an upgrade to the PDP-11) was selling thousands of systems per _week_.

The VAX was completely incompatible with the PDP-10. The PDP-10 (like the PDP-6 before it) was a 36-bit machine with 6 bit bytes, the VAX (like the PDP-11 before it) was a 16 bit machine with 8-bit bytes. Porting assembly software from one to the other essentially required a complete from-scratch rewrite.

By 1980, Ken Olsen's DEC had now been producing and selling its own computers for 2 decades, and could hire programmers familiar with them from anywhere, so he no longer needed the MIT AI lab as a source of labor. The industry was standardizing on 8 bit systems and leaving 6 bit systems behind, and the 700 customers with PDP-10 systems were already a rounding error compared to the 600,000 PDP-11 systems eventually sold, and the VAX looked to be even more successful. Thus DEC decided to focus on 16-bit systems, told its PDP-10 customers to migrade to a VAX instead, and cancelled the Jupiter project in 1983.

With the cancellation of Jupiter, the operating system the MIT guys had written over the years, a giant pile of hand-coded assembly proudly named the "Incompatible Timesharing System" (which gives you a good idea of the AI lab's "hacker" culture), was now tied to a dying hardware architecture which would not be upgraded. The writing had been on the wall for years; no new students had bothered to learn the obsolete ITS or PDP-10 programming for a while, and the booming computer industry in Boston hired away the graduates who had been maintaining ITS, until eventually Richard Stallman was the last holdout clinging to the old ways as a perpetual grad student. When MIT announced they were retiring the now-obsolete and unmaintainable PDP-10/ITS system, Stallman's carefully insulated little bubble of ivory tower academia popped, forcing him to respond.

Richard Stallman is not and never was a visionary. He's a reactionary conservative trying to cling to the past, and when the rug was finally yanked out from under him he set about rebuilding that past as best he could. Some of the things he fought to preserve were indeed worth preserving, but looking at him as any sort of guide to the future is ludicrous. He's never been about the future, except to warn against it.

There are of course more questions (why would an ITS guy switch to Unix, why was the FSF as successful as it was, how do lisp and microkernels show the dude is totally totally not a visionary, who _else_ was doing this, and where did it all go wrong), but again they'll have to wait because this is already long enough.

So I wrestled with the design of the new qcc wrapper for a while, unsure quite how I wanted to proceed, but now it's turning into a variant of ccwrap that breaks down stages. It calls cpp to do preprocessing (and that's the stage that gets

The advantage of doing it this way is it both partitions the task nicely (eliminating duplicate code, so multiple different stages have to understand about library paths and such) but also lets me test against the existing gcc stuff before I've built the new replacements. (Writing a wrapper I can't _test_ makes plugging in other functionality a real pain, because if it doesn't work it's not immediately obvious _why_. It's also no fun preprocessing C code you can't compile, or building .o files you can't link...)

So I'm designing the top level "cc" wrapper to call cpp the -M options are all aspects of cpp, not cc1. At the moment, I'm just trying to get it to call all the gcc tools in order (cpp, cc1, as, ld, strip) without telling one of the multi-function tools to do anything one of the single function tools can do. (So even though cc1 can preprocess, and ld can strip, I'm having the specific tools that only do that do that.)

I'm also writing it to not pass on any option it doesn't understand, so if I haven't immplemented support to grab "-M" then it'll barf when given -M, instead of passing it on to cpp.

Then I can swap out this wrapper for the one in FWL and make sure it can build the whole system, and maybe gentoo from scratch on top of it.

At some point in the future, I'll probably want to coalesce stuff back together. It's possible to implement the compiler as up to six processes (cpp | cc1 | as | ld | strip, all called by cc) but passing preprocessed data from cpp to cc might not be a win vs just keeping each translated line in the processor's cache and dealing with it immediately, and outputting assembly source and then assembling it is _definitely_ slower than generating the machine code directly (although the _capability_ is still needed to support -S, which the kernel needs in order to create some of its header files).

But that's all future optimizations, which can at least wait until I get around to actually implementing the code that needs to be optimized. (I'm not too worried about anybody calling cc1 directly, and if calling cpp directly was particularly common Ubuntu wouldn't have split it out into its own package which I've never needed to install to build anything I've tried. Builds do call ld directly and expect it to strip, but that's a question of outputting _less_ stuff.)

Still vaguely uninspired about programming. Poking at qcc, but admittedly somewhat listlessly. (Currently Linux isn't seeming like a fun hobby at all; I deleted my old git howto rather than defend it against some idiot who insisted any portion he didn't understand needed to be removed from it. Apparently "I don't understand what you're saying, not that I've actually tried the technique you described to see if it worked" means "you must be wrong". *shrug* Deleted the whole thing, which seemed easiest way to give him what he was technically asking for. There are at least three other git howtos out there written by people who care. Talk to the 404.)

Followed Fade to the library this morning and got a book, Keith Olbermann's "Truth and Consequences". I was somewhat surprised he'd found the time to write a book, but apparently it's mostly transcripts of his special comments with introductions. (About 2/3 of the way through now.)

Various other errands during the day. Went out with Fade to buy a bathing suit, hit a mall.

Up to DVD 5 in bleach, which is the one where they loose track of astral form vs physical form. (Or at least he spends most of an episode with his body being doctored for wounds received as a soul reaper, despite never having established the Matrix-like "your body makes it real".)

Not that this is the first series that's needed a continuity advisor. (Red Dwarf's books tried to justify Lister having his appendix out twice, but glosses over the fact that the first Joke Lister makes upon being revived is "Three million years? I've still got that library book!" and later in the first season "I've never read... a book." Yeah, it's a comedy and these are jokes of opportunity, possibly self deprecating humor on Lister's part, and the MST3K mantra certainly applies to most of the series. Still...)

Bleach remains fun, when it doesn't go into DragonballZ territory. (Watching people yell at each other for 10 minutes isn't fun, and these people would do well to study the video game concept of "logarithmic difficulty increase", although World of Warcraft has some similar funky balance issues. I can't be _too_ upset that individual members of the strongest 10% of Soul Reaperdom could each slaughter the remaining 90%, so why the heck are they sending _those_ guys out into the human world to fight hollows? If you stop and realize that a single Stormwind guard could solve all of Westfall's problems in an afternoon, "it's required by the plot" is really the only explanation required, or offered.

There are a few tricks to understanding the series. They level by something like an XP system (or as they say, "spiritual power increases fastest when the soul is in danger"), but Ichigo's limit break is to level, so when he's almost killed he goes "ding" on his next attack (which heals him out of limit break territory). Orihime's power is based on the Mythbusters Mantra: "I deny your reality, and substitute my own" (which makes her potentially even more of a walking deus ex machina than Ichigo, although she levels more slowly). When cornered and otherwise close to levelling, all of 'em do big long "bribe the GM with a backstory solliloquy to get character development XP" to push 'em over the edge. Don't expect to get any of the jokes, the entire cultural background they're based on is different (although the ones parodying other anime series/tropes you at least have a chance on).

Currently watching the episode where Protagonist Boy regains soul-reaper-dom after starting his heartless hollow transformation. His limit break kicked in and screwed up the cutscene, and he wound up with the "hollow" and "soul reaper" flags both set. :)

So I'm writing a fresh implementation of ccwrap.c for qcc,

Wow the gcc option parsing is crap. -static and --static do the same thing, "gcc -cv hello.c" says that -cv is an unrecognized option but "gcc -c -v hello.c" is just fine... "-Lpath and -L path" are synonyms, but "-lpthread" and "-l pthread" are not...

Rather a lot of "hmmm" moments. I'm starting by reading ccwrap.c and writing a new front end "qcc" command line interpreter, which creates appropraite command lines to call cpp, cc1, as, and ld as necessary. (This means none of those commands need to know about standard paths for include files or libraries.) Note that the existing ccwrap.c calls the gcc front-end which does a lot of command line rewriting of its own. This is a harder problem, breaking up the command into stages and calling the sub-commands directly. Another issue is whether or not those sub-commands should ever call each other, or whether the way they're called should string together "cpp | cc1 | as" manually.

The issue I'm facing right now is since none of those sub-commands deal with standard includes or standard libraries, it seems silly to feed them --nostdinc and --nostdlib. (The qcc versions won't _have_ the logic for that stuff.) But the gcc version of cc1 does do that unless called in such a way that they don't have anything to do, whigh gets us bac to the "cpp | cc1 | as > out.o" thing. (On the bright side, that would distribute the work over more processors on smp systems. Is there enough work in preprocessing for this to have any benefit? The existence of ccache implies so. But the communications overhead might make it a net loss anyway. It's easy to program and keep it nicely modular, but it's not that hard to keep it in one process, either...)

Anyway, from command line option parsing standpoint, cpp needs to understand "-D -U -I -o -M* -d* -nostdinc" (and possibly also "-include -iprefix -withprefix -system"). And ld needs all the stuff from "-static -shared -L -l -Wl," and such. So presumably if qcc is dispatching right, the cc1 command line parser doesn't need to know about either...

The question is, of course, how to come up with command lines that gcc would understand, which don't require the same code to be written twice in qcc, and which doesn't impose unpleasant communication overhead between processes. (Partitioning the task! A design issue...)

I asked Mark to create a new qcc list so I can post design documents and such from the new project. Have not yet posted such a design document, but wrote a longish notes file yesterday.

Currently frowning at ccwrap.c and rewriting it from scratch in preparation of slotting the options.c material into place. There needs to be a cc applet that rewrites the command line (so "cc hello.c" becomes a much longer command line with -I and -L entries and mentions of crt1.o and such) and then calls a cc1 applet that does actual compiler thingies. But whether it should literally create a new command line and call another main function with its own command line parser, or whether it should just set up data structures internally which the other command line parser could also do, is an open question. (Leaning towards the first. Simplicity vs code duplication is one of those tradeoffs that makes you want to step back and think about the problem at a higher level because there should be a way to get them to not fight like that...)

Part of it is "should this wrapper still be able to call out to an external compiler, ala gcc?" The current wrapper tweaks $PATH because gcc won't be able to find its own sub-commands otherwise, which clearly _isn't_ required for a qcc thing calling another applet internally via a busybox built-in command style multiplexer pseudo-exec. But if the other 90% of it _does_ work that way, is losing that functionality good? (Especially if I'm going to be writing low hanging fruit like ld and ar and such first, and possibly handing off to some other cc for test purposes...)

Another thing is that ccwrap.c has c++ support, and qcc doesn't (and probably won't any time soon, since tinycc never did). If it's doing c++, it needs to call out to an external c++ compiler. (At least something like cfront.)

Eh, worry about it when I've got more code written...

I admit the #fixreplies thing is an impressive #twitfail. They spun it as a user interface improvement, except that it used to be a configurable setting which already defaulted to "off". They removed the _option_, with no warning, and called it a UI improvement. That's what we call rolling a crit-fail on your intelligence check: When you have Neil Gaiman retweeting Stephen Fry pointing to an article about how stupid your policy is (both of those people are in the 10 most followed twitterers list), you have officially screwed up.

It turns out that the change was due to technical limitation in the database scalability or some such (which means they can't put it _back_ easily), which in some ways makes it worse. This was actually a _marketing_ crit fail. If they'd said at the start "we had to yank it because it didn't scale, the code's unmaintainable or it bogs the servers, we're working on a new implementation", that's one thing. But the actual quote from their "small-settings-update" announcement was, "Today's update removes this undesirable and confusing option." I never even _used_ the option that was removed, so my config hasn't been affected by this, and I'm still kind of annoyed at being patronized like that. I don't want them fiddling with the UI "for my own good", nor do I want them insisting they know how I'd prefer to use their service better than I do. They earned the wall of flame coming back at them right now.

They've been dancing around an apology ever since, but have yet to actually do a Mea Cupla for treating their users like idiots. Nor have they promised not to crash around like tone-deaf elephants in future, perturbing the UI of a service millions of people use with no warning or beta test site. That's an easy way to get millions of people to find or set up other microblogging sites, which would mean I'd need an rss aggregator to follow microblogs. Once upon a time, the livejournal friends list was all you needed. Now as Jeph Jaques says, "Can you believe people still have Livejournals?"

It's inevitable that the idea of microblogging diffuse beyond a single website, the way free website hosting isn't just "geocities" anymore and free blog hosting no longer means "livejournal". In a lot of ways livejournal was doomed by RSS aggregation. But right now, there _isn't_ a way to do @replies across sites, and until we come up with one and spam-proof it, twitter still has a structural advantage. But making their entire userbase see them as poor custodians of the service they use probably just shortened their time of dominance by about a year.

Reading the tinycc source, collecting bits easy to yank for qcc. Got curious and browsed alloca.S. Wow, I need to brush up on my assembly.

I was a bit confused why "alloca.S" didn't compile from the command line, but then I wandered through the makefile and noticed there's no rule to build it. So I looked at the git version of tcc-grischka, and although they've got an alloca.S there too _it_ never gets compiled either.

So my original plan of "tweak and test build" to see if I can "pop %eax" instead of "pop %ecx" then "move %ecx,%eax" and so on... Kind off derailed by the fact that the original code I'm reading doesn't work, and the last time I actually tried to _author_ x86 assembly code (rather than figure out what debugger output is doing wrong) was... 1994, I think? Yeah, time to brush up...

(Hence my desire to get online and download an x86 assembly tutorial. I've got books on it at home, just not with me. And a reference != a tutorial...)

Oh _that's_ cute. Although gcc won't compile alloca.S, tinycc itself _will_. Ok, at some point I need to figure out what's wrong here, but for now...

And it died because I haven't got the 32-bit development files installed. Right. Come back to this when I have internet access...

Why am I *ahem* "not a fan" of the FSF? Ok, I'll explain.

I'm not sure you can be a computer historian and not hate the Free Software Foundation, because they're not who they say they are, they claim credit for things other people did, and these days their agenda has become counterproductive and harmful.

A revolution to preserve the status quo

The US was founded by revolutionaries, hence the name "Revolutionary War". The founding fathers tried out all sorts of new ideas, and often took multiple attempts to get it right; replacing the articles of the confederation with the constitution, then adding a big batch of amendments to the consitution in the bill of rights, swapping in the electoral college when the first method blew up in their faces, going back and forth about whether or not to have a central bank, eventually needing a civil war to finally resolve the whole slavery issue...) The US founding fathers would be the _first_ to say they were imperfect people just doing the best they could.

These days, the founding fathers are primarily invoked by ultra-conservatives, often while fighting against things like desegregation, women's rights, gay marraige, bikinis, and so on. They're about as far from revolutionaries as you can get. (As Mark Twain said, "Conservatism is the blind and fear-filled worship of dead radicals".) If you couldn't do it in their grandfather's day, you shouldn't be able to do it now (so throw out that cell phone and don't use antibiotics). You wouldn't think this attitude would come up much in the computer industry, but human nature is the same everywhere, and thus we have the Free Software Foundation.

The main reason I can't stand the FSF is they're ultra-conservatives masquerading as revolutionaries. While that's always a possibility when "the revolution" drags on for 30 years and the ideals they're fighting for fossilize in their proponents heads, that's not what happened here. The FSF actually started that way.

The FSF was founded as a conservative reactionary movement. Back in 1983 Stallman's goal was to stop (or at least personally avoid) the rise of the brand new proprietary software industry, and return to the glory days of the 1970's.

Whether or not a return to the past was a _good_ goal is a separate issue; there was nothing visionary about attempting to roll back the clock, so the people who see RMS as a "visionary" today are _badly_ misinformed. He is not a visionary leader, he was taking us back where we'd already _been_. If anything this makes him _less_ qualified to predict the future, because his most conspicuous trait is that he doesn't like change.

Unfortunately, this doesn't stop him from trying to predict the future, proclaiming dire warnings about it, and attacking others who have a different ideas about what's ahead. Nor does it prevent hordes of impressed followers (who don't know anything about computer history and think this future/past he's describing is great) from hanging on his every word.

Before there was a proprietary software industry, 1939-1979

So why is "free software" older than proprietary software? The proprietary software industry didn't exist back in the 60's and 70's for two reasons:

First, until the early 80's there wasn't a critical mass of customers to sell proprietary software to.

The MIT AI lab where Richard Stallman learned to program was started with a computer called the TX-0, a unique system built as a prototype by Ken Olsen, the guy who went on to found Digital Equipment Corporation. (The book "Hackers" by Steven Levy covers this in a lot of detail, as does this marvelous interview with Ken Olsen.) That was followed with a DEC PDP-6 mainframe, which the shipped a total of 23 machines in its entire production run. The lab was later upgraded to a PDP-10, of which about 700 total ever made. The fact that MIT modified its computers to add extra instructions and wrote its own operating system from scratch really wasn't that unusual under the circumstances; programs written for early mainframes and minicomputers often wouldn't run on any other machine in the world even if they hadn't been modified by their users, simply because their component selection and configuration was unique.

For most of the 1970's, the best-selling computer in the world was the PDP 8, which was introduced in 1965 and sold a grand total of 50,000 units during its entire production run. The PDP8 was displaced as bestseller by the Apple II, which was introduced 1977 and over the next two years sold a combined total of 43,000 units. But that was enough to put it on top: in 1980, when Apple had its IPO, it had 50% market share of the computer industry (according to the PBS series "Triumph of the Nerds").

So if you tried to sell "shrinkwrapped" binary software back in the 1970's, the largest possible market you could hope for on any platform was _less_ than 50,000 copies, and even that's only if you targeted the most popular platform in the world and somehow managed to sell a copy for every single computer of that type ever produced.

The second reason the proprietary software industry didn't exist in the 60's and 70's was that copyright law didn't cover compiled binaries. Human readable source code was copyrightable (as an original work of human authorship), but binaries produced by a compiler or assembler were "just numbers", and the law didn't explicitly allow them to be copyrighted. So if you wanted to claim any copyright on your program, you were _required_ to ship source code. Shipping source code gave someone _more_ control over the result than just shipping a binary.

Thus publishing source code (with copyright notices) became official policy throughout the computer industry (DEC, AT&T, IBM)... They actually had fewer rights if they _didn't_ publish their source code. (When Unix came out in 1974 what was special about it wasn't that it came with source code; that was normal. What was special is that the source code was in a high level language that minimized porting effort to move it from machine to machine, instead of the assembly language all the other operating systems were written in.)

Software in the 1960s and 1970s was either produced by hardware manufacturers to help sell their hardware (who happily gave out the source code, generally written in assembly language), or else it was produced by the people who bought the computers for their own use (who had no incentive to withold source code from _themselves_). Users sometimes hired consultants to a write custom software for them (in which case the source code was one of the deliverables), but back then the only way to use a computer was to learn how to program it, so users wrote most software themselves or had someone on staff to do it for them. The vast majority of software in the 1970's and earlier was written by the users, who naturally shared the results through early user groups like DECUS and the CP/M Users Group Northwest.

So how did this _change_? Where did proprietary software come from, and when, and how? How did Richard Stallman's little utopia at the MIT AI lab crumble and force him out into the wilderness to try to rebuild it?

I should probably hold off until next time for that. This has gotten a bit long. :)

Ok, I know what to do with my tinycc fork now. :)

The obvious way to replace gcc is with qcc, the QEMU C Compiler. Take tinycc, break it up the rest of the way, and slot it into the qemu source code (in a qcc subdirectory) so it can build an "$ARCH-qcc" from each current "qemu-$ARCH", using tcg (the Tiny Code Generator) as a backend.

I'd held off on gluing tcg to tinycc because forking tcg wouldn't be fun (or productive). It's still under active development, any changes I need to make to it must go upstream or they'll become a maintenance nightmare. Pushing the compiler front-end upstream into qemu makes noticeably more sense, if they'll take it. (But they already _do_ all the hard parts...)

Right now qemu application emulation can parse ELF executables and shared libraries, so qcc's linker needs to use that code (and teach it to handle .o and .a files). So in addition to redoing the code generation backend, the linker needs significant work to merge it into qemu.

I split my fork's option parsing logic off into its own file a year ago, and I still need to do the swiss-army-executable thing on it and merge it with the ccwrap.c in FWL.

That's a lot of work. Trying to figure out what order to do stuff in...

Anyway, providing a decent replacement for gcc means the only GPLv3 piece of software that actually matters is Samba, and the kernel has built in smb client support so a simple server could probably be exported from that. (It might not support Windows 95, but I can't really say I care. Current samba isn't exactly embedded code right now, is it? Oh, and the dirty trick to ignore all the case insensitivity issues: export a vfat filesystem. :)

Not quite back to my normal levels of enthusiasm (or at least bloody minded stubbornvindictive problem solving), but any realistic chance to rip out one of the foundations of the FSF's power base is always invigorating.

Still apathetic about programming in general.

Linux on the desktop is dead. Ubuntu 8.04 doesn't work with Netflix or Hulu or Youtube's television options. 8.10 was a disaster. And now 9.04 doesn't work with Intel 3D, not only the most common one but the one that used to be recommended for use with Linux. Meanwhile, Apple's market share finally hit double digits. (Whether or not Windows 7 sucking less than Vista counts as damning with faint praise or is actually significant, I reserve judgement on.)

GPLv3 has fatally split the community. The last GPLv2 version of gcc is 4.2.1 which is increasingly obsolete (doesn't do armv7 and up, meaning no thumb 2), and there's still no replacement compiler. Neither PCC nor LLVM/Clang has turned into anything useful yet, and tinycc development is just painful to watch.

Actually, tinycc development is _hilarious_ to watch in a black comedy way. Grischka admitted that he didn't take most of my code because he didn't understand it, and now _three_years_ years later he's slowly reproducing some of the obvious refactoring I did literally back in 2006. At this rate it might only take him 3 more years to get to the point my fork was at when I abandoned it.

It's painful to go back to work on BusyBox both because of lingering Bruce and because I like the toybox infrastructure so much better. And there's a lot of cleanup work to do, starting with removing the forest of #ifdefs littering the code now.

But convincing people that conceptual bloat and what you can measure with bloat-o-meter are not the same thing is exhausting. (A wrapper script around mdev is a better design decision than trying to shoehorn firmware loading into something that wasn't meant to do that. It's not mdev's job to do every possible hotplug activity, and complicating the hell out of the config file syntax is not the answer. Also, removing the mmap config file parser in favor of much slower and more complicated code that allocates memory for each line of text in the config file (and then frees it again), and does this again and again for every single device during "mdev -s"... That was wrong. I don't care if it's shared with other applets, it's still wrong.)

The reason for mdev in the first place was I disagreed with the design of udev, and that there should be an obvious and simple way to do it. I had to fight the kernel guys (Greg KH and Kay Sievers) who thought that sysfs was a private export and not a stable userspace API, so anybody trying to write their own code to do this rather than using udev was obviously crazy. More recentlly I had to fight the kernel guys (pretty much Peter Anvin) who are rewriting the kernel build system in perl. In addition to not needing an additional large prerequisite package which would rule out sane languages like Python or Lua, perl is a specific programm, not a language. It has no standard, merely a single implementation (just like Microsoft Word or Microsoft Excel). Whatever that implementation does is Perl, and it's never successfully been reimplemented (despite the Perl 6 guys trying for over a decade now). That's not a language, that's a legacy hairball.

Sigh. I feel like I'm fighting the entire Linux community, and I'm tired of it. Yeah, you can duck into the embedded world and avoid most of that, but in a proper embedded system you don't care what it's running. Might as well be DOS. The current most interesting embedded device (the iPhone) has MacOS X.

When this laptop dies, maybe I'll buy a mac. (About half the kernel developers are already doing their kernel development on macs, from what I can tell...)

Poked at build strace a bit in the evening. This is a weird program. Building natively under armv5l, it went:

In file included from syscall.c:129:
linux/arm/syscallent.h:435:3: error: #error fix me
linux/arm/syscallent.h:457:3: error: #error fix me

I'm guessing that strace 4.5.18 doesn't support the 2.6.29 kernel, at least not on an arm host. (Instead of failing by saying unknown syscall at runtime, it fails by refusing to build. That's nice. I note that 4.5.18 is the most recent version on strace.sf.net and that page has no link to a mailing list...)

The source code's README says:

You can get the latest version of strace from its homepage at
http://www.liacs.nl/~wichert/strace/ .

Which is 404 and according to archive.org has been since 2006.

I'm not sure "maintained" is a word that applies to this package.

Mark pointed out that the Gentoo guys have patches for this, and apparently have for some time. (Apparently there's been nobody upstream to send them _to_? Dunno.) Oh well, yay it's fixed, moving on...

Poking at a dropbear build script under the native root filesystem for a nightly cron job. Alas, dropbear went to autoconf for some inexplicable reason, so of course it sits there making stupid tests (very slowly):

checking for gcc... gcc

Yes, gcc is called gcc. If you were checking for "cc" (the standard name required by SUSv3; SUSv4 seems to require "c99" now), that might make some of the later tests less silly.

checking whether we are using the GNU C compiler... yes

Um, why do you care, and what was the first test for, then?

checking for C compiler default output file name... a.out

You can specify -o all the time so this never matters. Same with suffix of executables and suffix of objects files, but you can also just accept that there are exactly three interesting build platforms these days (Linux, MacosX, Windows) and you're under no obligation to support all of 'em anyway.)

checking whether the C compiler works... yes

During the build, you'll notice pretty quickly if it doesn't. You'd also get a better error message saying _why_.

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

A) This is required by the Single Unix Specification, B) there's a gcc that doesn't accept -g?

checking for gcc option to accept ISO C89... none needed

Again, your very first test was for gcc, then an explicit "are you really gcc and not another compiler pretending" (no idea why), and now it's doing this kind of crazy stuff.It just goes on and on like this. But that's not the funny part. Here's the funny part:

checking for deflate in -lz... no
configure: error: *** zlib missing - install first or check config.log ***

You have to explicitly say --disable-zlib. What exactly is the POINT of running ./configure if you have to explicitly tell it on the command line what your environment contains?

Of course when you do that it then goes on for a long time checking "is the strip command called strip" and so on, explicitly checking whether or not our C compiler has "memcpy"... and then at the end it tells us to edit options.h to select the options we want.

Sigh.

Saw Fade off the airport for Penguicon. (I'm not going this year, the con chair and I don't get along, it's probably better for the event if I stay out of his hair while he's wearing the con chair hat.) I feel bad I won't get to hang out with Garrett and Christian and Tracy and Heather and so on, but oh well. Maybe next year.

Ok, I know I'm weird, but I'd just _assumed_ the Penguicon guys had a video link set up so Wil could address opening ceremonies (and maybe even voip-phone in a panel or two) from home if he couldn't physically make it to the con again. USB web cameras are what, $15 these days? After a Red-sox-like streak had established itself, it just seemed like common sense to me. (Although I admit my money was on "meteor strike" this time. And no, the third time isn't "hindsight".)

Rather than turn this into armchair quarterbacking, here's a Plan For Getting Wil To A Future Penguicon. (I don't know who future Penguicon runners might want to invite, but in case they someday decide to try again with Wil, and he's up for the abuse, here's how I'd go about it.)

• Step 1: Go ahead and send Wil a cheap USB web camera, one that works with macintosh laptops. (This is a gift, he can keep it. Also note that most macs have one of these suckers built in. Send him a spare _anyway_.)

  Also send Wil a badge that says "Sir Not Appearing At This Con". (Make him promise he will wear it through the entire weekend of Penguicon, wherever he happens to be during that time. Demand pictures. Yes, this will be his badge if he does, by some fluke, wind up attending. Ribbons for him to hand out to Penguicon attendees, possibly by mail, are left as an exercise for the reader.)

• Step 2: Keep a second webcam on site in case Wil winds up bedridden in his hotel room (or a local hospital) instead of bedridden at home, so he can still occasionally say hi. This has the added benefit that it could potentially be used to record panels downstairs if it isn't otherwise needed. (Hey, gotta plan for everything, even the possibility of success...)

• Step 3: make a case for inviting him back a fourth time. Start by ripping the Monty Python and the Holy Grail clip about Swamp Castle, everything from "They said I was daft building a castle in the swamp" through "but the fooourrth one stayed oop!". (It's short enough it probably falls under fair use, especially with what we're going to do with it.) Have Wil use his web camera to record himself describing his first three attempts to get to Penguicon. Intercut the "burned down, fell over, and then sank into the swamp" bits with Wil's description of his previous Penguicon weekends. When competed, post this video on the Penguicon web site. (This will also be played during opening ceremonies. If necessary, use the corresponding scene from Babylon 5 about the previous stations for the next attempt.)

  Note that using the webcam for this rather than some other video camera has the advantage of testing out the webcam to make sure it works and is set up properly ahead of time.

• Step 4: Fly him in early.

  The first few years of Penguicon we flew all Guests of Honor in on Thursday wherever possible, so there was some failover in the plan. This gave the guests time to unwind, because "travel and perform the same day" is hugely stressful (as those of us who ever travel for business know). The travel itself is also less stressful when you're _not_ on an immediate deadline. (Yeah spending an extra couple hours waiting after a missed connection sucks, but there's no rush, you're not missing any panels, you can relax.)

  Given that this is a third fourth attempt, fly him in on _Wednesday_, and arrange some local event for him for Thursday so he isn't wasting his time. (Reading/signing at a local bookstore or two, perhaps? Is there a local radio program that interviews authors? Or if Detroit has anything at all like the Alamo Drafthouse he might want to introduce a special showing of one of his movies. It's a big city he doesn't get to much and we're springing for the extra hotel room day, ask him for suggestions. Maybe he wants to go to the zoo; up to him. The point is, it's ok to ask him to allocate us a little more time as long as he can make good use of it. I believe Terry Pratchett used the extra time to sit quietly in his hotel room and write the "you all meet in a tarvern" scene near the start of Monstrous Regiment, which he read for us the next day one of his talks, and we were all pretty happy with this arrangment...)

  (The Thursday night dinners with just the GoHs and concom are a nice fringe benefit of the "bring 'em in early" approach, but they're also entirely optional. I only ever actually made it to one of those myself, tended to be busy a lot right before the con for some reason...)

• Step 5: Offer odds on the web page. (Possibly Lloyds of London could help here, I dunno how that works.)

  Suspense! Drama!

• Step 6: Recuit help.

  Inviting Guests of Honor is always a question of getting good combinations that would work well together; people who are already friends, or fans of each other's work, or would go well on panels. (This part worked well this year, and Andrew Hackard stepped up to fill in for Wil.)

None of this would have actually prevented him from getting sick, of course. But it would allow the whole process to fail in the most entertaining possible manner. :)

(Actually, It occurrs to me that the logical way to get Wil to a future Penguicon is don't announce it. Do it in secret. Fly him in quietly, sneak him into the hotel, and then have Ninjas carry him into opening ceremonies (bound and gagged, if possible, and untie him on stage). Schedule either "Robert Wesley" or "Stephen E. Whitfield" (various pen names used in Star Trek) on Wil's panels and swap him in for those... Somebody on twitter suggested that we should declare he's been reconned into previous Penguicons and that this is actually his Xth appearance at the con...)

Eventually decided that building under emulation was the way to go after all, even for three small packages. For one thing, I'm not trying to get strace to cross compile. For another, it's nice to have a smoke test of the build environment to catch things like the fact that the current buildall.sh is leaving prefixes on the _native_ toolchain. (Sigh.)

So I want to build static busybox, dropbear, and strace binaries for each target. This is easy to do; I can think of at least three different ways off the top of my head. It's so easy I can sit down and write a script to do this with no references or research. Which means it's one of those class of problems where figuring out the _right_ way to do it makes my brain hurt and I get writer's block for days.

I can just fire up the native build environment, wget the source, untar it, cd into the new directory, make defconfig, make, and copy the binary back out. (Building dropbear first makes copying the binary back out noticeably easier.)

In practice, the config I want to build isn't _quite_ an actual defconfig, I switch a lot of stuff off that doesn't build on various targets (m68k hasn't got taskset) or without strange host support (selinux is evil). So I need to grab that trimconfig file out of FWL, but I can wget it straight from that URL if I like. (Put "tip" in for the version number and it auto-updates.) So I don't quite need to grab the full FWL infrastructure, but it's not entirely independent either.

Next, grabbing the dropbear source again every night is a bit impolite to the download servers because it's not going to change that regularly. Desite my desire for it to grow stunnel support, the project is mostly complete and in maintenance mode. The FWL infrastructure has cacheing already; the download and setupfor functions handle this transparently. So if I _do_ grab the FWL infrastructure, I get this pretty much for free.

I can also cross compile both of these packages, which is quick and easy and ignores the whole point of the project (that building natively under emulation is the way to go, and cross compiling is a transitional stage you get through as quickly as possible). But in each case, I've got a package that's already set up to cross compile and which produces a single static binary as its output, and the rest of the nightly cron job is a cross compiling so I've already _got_ a cross compiling build environment set up. It's even got the FWL stuff sitting there ready to use, and if I jump into the emulator I have to re-download FWL and set it back up if I want to use it.

So there's a simple way to do this which has obvious flaws, and a more complicated way to do this that probably isn't worth the complication, and this is the kind of thing that makes my brain hurt.

There are some actual fiddly bits. The download.sh script deletes everything out of packages that it doesn't know about. (This is to automatically zap old versions of the packages after you upgrade download.sh. It's actually a bit of a pain, because it means that things like qemu (which are speculatively downloaded) or alt-packages (which are sometimes downloaded and sometimes not) get deleted and then have to be re-downloaded, or else are downloaded and kept around when they don't need to be.)

I don't want to add dropbear to download.sh to unconditionally download it every time, and I'd like to move strace out too. Have some kind of cron.sh call download.sh stuff. But then I have to use a different download directory (not "packages") or the extra files will get wiped every time the original download.sh script runs. And the problem with _that_ is that the original download.sh is already downloading busybox, and having the second directory download a second copy of the busybox tarball is kind of icky.

Did I mention that root-filesystem.sh already builds busybox, and could easily be taught that BUILD_STATIC=1 applies to busybox? Except that it would build a lot of _other_ stuff (like uClibc), and would thus be an inefficient way of building that one binary (unless I wanted to have all the root filesystems it built be static), which I don't. So this is either going to be duplicated or is going to have too much fiddly configuration shoehorned into it, and both suck. Sigh...

Again, it's the _little_ things that screw me up. The ones that don't really _matter_ in the larger scheme of things, but are nevertheless _ugly_, that I go round and round on...

Don't mind me, I'm having an aesthetic reaction to a simple problem. FETCH THE CHAINSAW!

In a lot of ways programming is an art instead of a science. Debugging, especially. "I don't know what it's doing wrong" is pretty much a requirement for a bug: very few programmers intentionally write the code wrong, so it's misbehaving either because what you wrote didn't match what you had in your head or because what you had in your head didn't take something into account.

An awful lot of reworking is because "no, this isn't right" without being tied to a specific bug. You redesign some bit to make it better (often simpler and relying on fewer assumptions now that you understand which parts are actually important), and as a result bugs you haven't actually hit yet go away. And then when somebody hits a bug in an older version and ask you which patch fixed it you can't really answer because you never saw that bug. And sometimes "which patch fixed it" was a redesign, where the memory leak went away when you switched to mmap(), or the bug was in the translation module and now you've changed the other parts of the program to all talk the same language so there isn't a translation module anymore...

I bring this up because software suspend in Ubuntu 8.04 semi-regularly corrupts the vfs cache if the filesystem is active when it suspends (such as running a build that'll take another 15 minutes to complete and I don't want to wait that long just now, but I don't want to abort the build either). And I know the kernel in 8.10 didn't do that once in all the months I was using it (which is why I forgot about the problem), but doing the Ubuntu upgrades when I installed this laptop (at which point the release was almost a year old) didn't fix the bug; the old kernel is still susceptible to it. Which patch along the way to the new kernel fixed it? Probably not that simple...

I wonder what's involved in getting a Gentoo desktop with kmail on an xfce base and a working Flash plugin. Highly _unlikely_ to be particularly simple, but it's something I could much more easily tweak myself once I got it that far... (This is probably a bit like saying "It's hard to get a tank platoon 200 miles behind enemy lines, but once you do things get much easier".)

Wait until Mark gets back to town. Definitely.

So I'm fiddling with upgrading the system images to gigabit ethernet (e1000). I just did i686 and it's slightly faster, but not much. Maybe 3.5 megabytes per second. When I do a "dd if=/dev/zero of=blah bs=1M count=100" and time it, it seems to be getting maybe 11 megabytes/second. Hmmm, when I do a wget of a 120 megabyte file and add "-O /dev/null" it also gets about 11 megabytes/second. Right, there's some sort of emulated PCI bus contention going on. Whee. Still, sending stuff to/from distcc might go faster now, which would be good.

Finally wrestled canadian cross building of static toolchains into submission. Lots of darn fiddly little things to get right (one of the fiddlier, strangely enough, was making sure that the prefixes of everything wound up in the right places. For example, you can't get uClibc to produce an "i686-ld" or "i686-readelf", you have to rename 'em after the fact. And a "cc -> gcc" symlink actually has four possible states, and "i686-cc -> gcc" isn't the one you want. Some darn _weird_ errors result if you just try to run and debug this; visually inspecting the bin directory of the cross compiler is much easier. As opposed to debugging gcc, where visual inspection tells you nothing and the useful thing to do is find a failing command line and run it with -v and under strace to see what it's actually _doing_.

So now Mark can set up the cron job and build nightly binaries, except that I have to get uClibc's nightly snapshots working again. (Every time I ignore them for a month, when I come back they don't build anymore until I've tweaked the config of a half-dozen platforms and come up with fresh patches. Might be different this time, but it seems unlikely.)

Oh I'd forgotten about that bug. Occasionally software suspend on 8.04 will catch some weird race condition having to do with a dentry that's active during the suspend, and will wind up turning every access to that dentry into a segfault. (Sometimes it does one of those non-fatal panics and tells you it's a null pointer dereference in the vfs, sometimes it just kills your process with segfault and no explanation.) I remember the first time I tried to explain source control to Fade, and show her mercurial, Python spontaneously developed this affliction to screw up my demo. Now kaffeine's got it, so I can't catch up on my podcasts until I reboot. Sigh.

Hundreds of open browser tabs, unlikely to get 'em closed this week. Rebooting with the network off means they'll all be 404 and I won't be able to read cached data offline until I reload it. Rebooting with the net on will reload them all, meaning Konqueror will eat 100% of the CPU because tabs that aren't selected still run their javascript (which is a bad thing, but seems to be intentional).

And the dying thing is /usr/lib/libXdmcp.so.6 which is taking out amarok too, so I can't listen to mp3s either. Let's see, if I use Konqueror to navigate to an mp3 file and click on it... It tries to launch amarok. Idiot. It has built-in mp3 playing capability, and the flash plugin can also play mp3, but no, it wants to launch an external player. Brilliant.

Oh well, at least the stupid "Are we human, or are we dancers" song I was trying to drown out has gone off the radio now. (Because no human can dance you moronic whiny git.) Now it's gone to some other song by the same guy who did the original song Weird Al's "you're pitiful" is based on. It's a whiny falsetto theme on this station. The noise cancelling headphones, they do nothing.

The radio has now moved on to "Every time you go away, you take a piece of me with you". The frightening part is that this is is an enormous step _up_. (At the very least, I'm no longer contemplating surreptitious ways to damage the PA system here. And no, it's not the theme from the "Saw" movies. Don't think it is, anyway, despite the title I never saw 'em.)

Ooh, a Let's Play that Yahtzee did back before Zero Punctuation.

Watched the first disk of "Avatar, the Last Airbender" today. I previously thought it was an anime, but it's apparently a Nickelodeon production in a vaguely Japanese style (up to and including Japanese characters in the intro credits. I have no idea why. The practical effect of this is I can't turn the horrible english voices off and go to subtitles with Japanese voices that at least convey reasonable emotions whether or not I can understand a word. Nope, the english voice acting _is_ the original. I can only hope they improve as they get used to the characters...

I spent the first episode hoping the brother would die. The second episode _almost_ involved whatserface and the avatar guy leaving the rest of the cast behind (including the idiot brother) to go off on their own, but no. End of the disk, he's still there, which is sad. (I liked the voice actor when he was voicing Ron Stoppable, who was not a mysoginistic faux authority figure constantly blaming other people for his own screw-ups.) Still, Babylon 5, Star Trek TNG, and Star Trek the Motionless Picture (where "Nomad" has gone before) all started horribly and got better. The water bender girl and Aang's are both nice. And the bad guy's mentor is great.

Also watched the rest of the first Torchwood disk (episode 2, Death By Sex), the other commentary for Gosford Park (which is still way too complex for me to follow even half of it, and based on cultural details of a society farther from mine than the Yonomamo). And now I'm watching Bleach disk 3 (still episodes I've already seen back before Hulu stamped Watermarks all over the screen) which remains inexplicably Japanese in places, stopping for speeches about nature imagery in the middle of fight scenes.

Broke down and implemented a proper little setsid program in C, and made system-image.sh build it and call it. In theory I could have added this to toybox, but the build doesn't _depend_ on toybox being there. I could also have upgraded the busybox version to do this, and still might, but again the build shouldn't depend on the host setsid being buybox. (You can skip host-tools.sh.) And the host setsid on Ubuntu is forking a background process and returning immediately, which is useless for my purposes.

So yeah, spent a couple hours yesterday fighting with the shell in an attempt to avoid writing 9 lines of C code. (Because I shouldn't have NEEDED to, darn it.)

Ok, I haven't seriously poked at Firmware Linux in a week. Time to pick up the shovel again.

So where I left off is I rewrote buildall.sh to take advantage of the canadian cross stuff, and everything broke. The actual canadian cross part broke, but more fundamentally the old buildall functionality broke. One reason is that when system-image.sh dies, it tends to take down the script that ran it because it does a "kill 0" which takes out the current process group. Hence the first system-image.sh to fail stops buildall.sh from trying any of the later ones, and right how hw-uml is failing because user mode linux is still horked, as usual.

The way build.sh works around this is by running it via setsid (the "set session id" command explicitly creates a new process group), but needing to do that is icky. It would be nice if system-image.sh could do that internally somehow, I just need to figure out how.

I can't put system-image.sh's logic in a shell function and call that via setsid because setsid isn't a shell internal command, and a regular command can't do anything useful with the name of a shell function on its command line because it can't call _back_ into the shell that just forked and execed it as a child process. (I hit that one _all_the_time_.) I could do something incredibly ugly with "here" documents but the cure's worse than the disease there, especially with what it does to syntax highlighting, which admittedly has never worked in Ubuntu because Ubuntu hates vi for some reason and goes out of its way to break it; another reason to switch to Gentoo or something someday.

So my next attempt is to add this to the start of system-image.sh:

if [ -z "$SYSTEM_IMAGE_SETSID" ]
then
  export SYSTEM_IMAGE_SETSID=1
  exec setsid "$0" "$@"
fi

The reason that's unacceptable is now ctrl-c does nothing when system-image.sh is running. I took the exec out and added "exit $?" on the next line, but that made no difference.

The _odd_ thing is that when I call the sucker from build.sh via setsid, _that_ handles ctrl-c just fine. So what's the difference here? I tried 'setsid /bin/bash -i -- "$0" "$@"' and that confused the _heck_ out of the tty. (Ctrl-c gave me a shell prompt back, but didn't kill the running script.)

Tried calling setsid through the built-in "time" function, since that was the only difference I could see from what build.sh was doing vs this thing calling itself. No difference...

Ok, so I commented out the additions to system-image.sh and put the setsid back into build.sh and now _that_ isn't letting ctrl-c work anymore either. Which is strange because I'm pretty sure I tested that when it went in and it was providing the correct behavior. (I.E. kill system-image.sh and its children, but not the calling shell.)

I want "kill this process and all child processes, recursively". There should be a way to do that. (The gnu implementation of ps has --ppid, but the busybox ps is kind of a joke.)

Grrr. I need a way to call tcsetpgrp from the command line. Why isn't bash doing this when I call it from setsid? Because its process already is a session leader? Except that 'echo | setsid xargs -- /bin/bash blah' isn't interruptible either, and there bash is forced to be a child process of what setsid calls.

I'm using 'kill 0' because neither 'kill %1' nor 'kill `jobs -p`' work from signal context. It can't access environment variables either, so I can't save the PID and have it call that. I suppose I can reset the signal handler with an explicit list of PIDs to kill after every job I launch (make a function for it)... Except that _that_ doesn't work because killing the child process isn't recursive, it doesn't kill the sub-processes like "make" it was running (even though it wasn't a _background_ process for that shell, so you'd think the shell would do it, but no it depends on the terminal control group to propagate the signal).

Tried killing the pid of the background process as a negative number (which I believe should kill its process group), but it claims it isn't a process group. Tried running the background process through setsid to force it to _be_ a process group, but that didn't work either. (Hang on, does busybox know it's ok to do this? Does the bash builtin kill know? Trying $(which kill) and... no help. It's parsing the negative number as a signal number, so try "kill -11 -$(jobs -p)" and... it says no such pid, but with the negative number this time. Huh, I thought they'd added that support to the kernel years ago. (Try without the setsid on the child process... Nope.))

This _really_ should not be this hard. I really, really, really don't want to have to write C code for this.

Ok, spent another ten minutes installing an atexit handler in the outer instance of the script, sending a signal 11 to the child session running under setsid, catching that particular signal in there, and calling 'kill 0' if it gets it. Set up a little test where the child just waited 5 seconds, like so:

if [ -z "$SYSTEM_IMAGE_SETSID" ]
then
  echo Need to recurse
  export SYSTEM_IMAGE_SETSID=1
  setsid "$0" "$@" &
  trap "echo dying now, kill $(jobs -p); kill -11 $(jobs -p)" EXIT
  wait4background 0
  exit 0
else
  # If we exit before removing this handler, kill everything in the current
  # process group, which should take out backgrounded kernel make no matter
  # how many child processes it's spawned.
  trap "echo die lots now; kill 0 -QUIT; exit" 11

  echo do not recurse
  sleep 5
  echo done
  exit 0
fi

The above _WAITS_FOR_THE_SLEEP_TO_END_ before delivering the signal 11.

*BOGGLE* It sends the signal. And the shell doesn't process the signal until its child process exits. WHAT THE...?

Sigh. The reason this doesn't work:

if [ -z "$SYSTEM_IMAGE_SETSID" ]
then
  echo recurse now
  export SYSTEM_IMAGE_SETSID=1
  setsid /bin/bash -- "$0" "$@" &
  trap "kill $(jobs -p) 2>/dev/null" EXIT
  fg
  exit $?
fi

Is that it says "no job control" when it hits fg. Presumably because the shell isn't interactive. I can replace the last two lines of the if block with "wait4background 0" and "exit 0", but then I lose the error return code of the background process.

EPIC FLAIL!

(Well, actually as development sessions go for me, this is pretty tame. Still frustrating, though.)

Sigh. Domestic disputes, gotta love 'em. (You can stop reading now, nobody but me will care about this entry.)

My condo complex has designated parking spaces, and my neighbor's been parking over the line of their space for a while now. Annoying, but not a big deal. Today my neighbor came to yell at me because my car, parked entirely inside the lines of my space, was preventing them from pulling out.

I say "yell" because they did, and because they brought a friend with them, and claimed I'd had them towed twice (which was news to me; I asked Mark and he said he didn't tow anybody when he was here either).

I have towed cars that parked completely _in_ my space, but not for being over the line and not since last September. (It's an autumn ritual around here, as each new year starts at UT and the renters figure out which spaces are reserved and which aren't.) If somebody tows you without authorization, what you do is contact the towing company (only one services this garage, number's on the sign over the door), then you get the paperwork related to the tow which identifies who did it and has their signature. Then you sue them.

For once, the cell phone camera came in handy! (Ok, I had to borrow Fade's phone becuase my battery was low, and then I had to spend half an hour fiddling with bluetooth to figure out how to get the pictures off. But that's par for the course.) Here's the space they leave on the left, and the space they leave on my side. (This is after I moved my car to the right edge of my space, otherwise you couldn't see between 'em very well. I tried a close up, but the lighting makes it really hard to see. If you're looking for the line it's under the tires of the car on the left.) This was actually fairly mild "over the lines", he's been over by lots more, it's just this time he decided to demand that I move my car to give him more space. (Yeah, I know...)

The hilarious part was that they say they've been complaining about me to "Mrs. Thompson" for a while now, who I assume is the person they rent from. (I own my condo, they rent.) I took the pictures so I could send 'em to the condo association if it comes up again. (Which is McNeil property management, still no "Mrs. Thompson").

I suppose once somebody did a dance mix cover of "The Time Warp" it was inevitable that a music video made entirely out of clips from Dr. Who would be set to it and put on Youtube. This is the internet after all.

That's not even a particularly remarkable one. The weird science one makes better use of the Dr. Who material and matching it to the music, as do this one, and this one. There's often significant variation on the same songs, with straight takes and and ironic takes. And some were just extended versions of what wound up in the show, although you sometimes realize how carefully cut the original song was for broadcast TV. (Yes, the part of the song that wound up on TV was explicitly advocating murder, but didn't mention _sex_.) Bumped into some songs I'd totally forgotten, and I've even found some new music doing this.

Of course Dr. Who isn't the only show with with music videos. (Although Harley/Ivy is almost cannon, as is Bonnie Tyler/Hiro Nakamura really. (Although innocently looking for that brings up unwanted Luke Ski stuff, a bit like Penguicon really. Yes, I've had more than one panel scheduled next door to his "concerts", and after six years his material gets old. Nice guy in person.)

Saw "Monsters vs Aliens" today, which was entertaining but made me appreciate Coraline more. Coraline was _elegant_ in its use of 3D, when the tunnel expanded into the distance it was _supposed_ to be otherworldly, when the hands of the "piano that plays you" stepped out they _were_ intruding on coraline's space and being meanacing in a foreshadowing way. The needle coming out at the start and poking the audience in the eyes was supposed to be unsettling, as was the rest of that scene. But it would all work in 2D as well.

Today's movie wielded its 3D like a blunt instrument, and a lot of them would just be silly in 2D. Making a paddle ball fill the screen near the start of the movie was just pointless showboating. Monsters spent a lot of time using 3D very intentionally. Coraline simply _was_ 3D, and made it look effortless and natural.

Some of it's technical stuff. Several times Monsters had a bright 3D object on a dark background, and a halo filtered through the polarized lenses. I never noticed that _once_ with Coraline. (I'd also heard that one of the failure modes of 3D is that rapidly shifting depth of focus for an hour gives you eyestrain. Never came up in Coraline, happened a lot in Monsters.) A movie should makes me _notice_ the technical tricks it's doing, even when it's doing them reasonably well, because it defeats the purpose. Use the tricks to tell a story, not to show off the tricks.

I remember one of walt Disney's famous quotes was "Change the trick". This came up in an old a making of Who Framed Roger Rabbit documentary I watched, where they showed around 10 seconds of footage of a human actor riding in cartoon taxi (chase scene), and in one shot the human was on a crane with the buggy painted over it, in the next he was on a miniature vehicle with the buggy painted over it, and in the third both the buggy and the human were animated (but they were going down a dark alley so it was harder to notice the human wasn't the actor; and they still used his voice). All in the space of ten seconds. The point was so that you _couldn't_ figure out what they were doing because they kept doing different things. (See a man in a rubber suit, look again and it's now a puppet, look again and it's audioanimatronic...)

CGI eliminates a lot of this from a technical perspective, but not the need for it from a storytelling perspective.

By the way, either Ginormica's grandfathers are The Doctor and Death, or the name "Susan" attracts superpowers. Somebody should ask The Invisible Woman about this.

Poked at the canadian cross toolchain (translation: hit it with a rock) and made it work. Now I'm ripping build.sh a new one to add static cross compiler building to that.

So the problem with the canadian cross toolchains is that the pathing is horked. AGAIN. Yet another thing for the wrapper to EXPLICITLY WHACK WITH A HAMMER. Except that I probably need to patch the source to _not_ let it look at the wrong areas _first_ before falling back to $PATH. (And when it's _not_ canadian crossed, the path logic works DIFFERENTLY. Or maybe it's some targets.) Other than that, the sucker seems to work. Or at least it builds a hello world that runs under qemu.

(Ok, technically it doesn't run under qemu-mips, but that's because the current mips application emulation hangs attempting to run a hello world statically linked against uClibc. But that's not the toolchain's fault, and it runs fine under qemu system emulation just fine.)

April 15, 2009

I'd hoped to drop off Mark at the airport _before_ rush hour started, but it starts at 6am and his plane didn't leave until almost 8, so it was either deal with rush hour traffic or get him to the airport 3 hours early. Got home and slept for about two hours before Fade got up, then the phone rang with some guy who was unclear about whether or not he wanted to offer me a job, who said he'd sent me an email (and was calling to tell me he'd sent an email), but no actual email ever arrived. Email about existing work did though, so I dealt with that before going back to bed. It was noonish by that point so I didn't really sleep well, and wound up getting up earlier than I really wanted and being vaguely unproductive until after dinner. (I can work on a day schedule, or on a night schedule, but some variant of consistency does help. My internal clock flashing 12:00 leaves me a bit zombie-ish to focus on technical work. You can bury the problem with enough caffeine, but that's not a long-term solution.)

I'm so glad I got my taxes done last month. Today would really suck otherwise.

I need to re-rip Billy Joel's greatest hits, track down my genesis CD with "land of confusion" on it, and a bunch of other stuff.

"If and when the president does make his Afghanistan announcement, you can be assured that we here at The Rachel Maddow Show will cover it with an almost ridiculous, over the top, obsessive interest in the details. I'm sorry, I can't stop myself." - Rachel Maddow, 3/26/09.

I want to have Rachel Maddow's baby. I'm aware that there are a number of reasons this isn't likely: I'm male, I've never actually met her, I'm married to somebody else, Rachel's gay and in a stable relationship with a woman in the northeast somewhere... Little things like that...

April 14, 2009

Went in to the mortgage guy's office down off of Bee Caves to sign lots of paperwork for my sister's house. Hopefully, that's the last of it until closing. (Between paying taxes, funding IRAs, and the down payment and closing costs for my sister's house, I seem to have used up most of my ready cash, _and_ acquired a new $615/month bill. Oh well.)

Fade sounded distinctly forlorn when I called home around dinnertime, so I felt compelled to go make her less forlorn. (This involved sandwich, fruit, three different types of caffeine, and a cookie. She seems less forlorn now.)

More fiddling with transcoding software, banging on qemu summaries, and back to working on the canadian cross stuff too.

Gotta take Mark to the airport at 4am, so he can go visit Eric and Cathy for two weeks.

April 13, 2009

Went to vist the investment guy this morning to actually do something with the Roth IRAs Fade and I set up last week. The S&P 500 fund Wells Fargo has eats 5.75% of your investment up front, right off the top, as a management feet. (Ouch.) I wanted to buy a share of Berkshire Hathaway Class B stock (currently around $3k) and put the rest in the S&P 500 fund, but I wound up putting it all in the fund because to do that they'd have to transfer the money into a self-managed account which would take at least a day meaning I'd have to come back _again_. (Yeah, I know this is how they make their money.)

Fade went with the managed fund the guy recommended.

April 12, 2009

Ok, my working style's gotten some pretty deep groves in it. I just got to a good stopping point with the rebuild.sh and indexsections.py files for Qemu Weekly News (hey, I'm up to the end of January, 2008! Woot!). And I went, "ok, time to check these into source control"... and I haven't got source control for 'em. (Well they don't develop, except for those two pieces of infrstructure, one of which is from the old kernel docs I spent half of 2007 poking at, each data file gets finished and should never change again after that.)

Seems silly to set up a mercurial instance for two files, and/or to check in lots of static files.

Fade's been busy with online appointments for 6 hours so far today. (This is fairly normal.)

My attempt to come up with a paper for the Linux Plumber's Conference hit the snag that the scope of everything I tried to write was just too big. Even my _outlines_ spanned multiple pages, and just kept getting bigger. (I suppose I could try again tomorrow, but mostly I want to sit down and spend a couple _weeks_ writing documentation, not a couple hours. So many tangents...)

Spent rather a lot of the day doing qemu mailing list summaries instead. Lots of funky editorial decisions there, but they're mostly "should this thread be included or not, and how much". The scope of each week is "threads starting between this date and this date", and from there it's a question of _removing_ stuff, not adding it...

For example, currently I'm trying to figure out if this thread is worth covering. Four posts between two people, riffing off a LWN thread that already implemented something, wondering if qemu could do a better job and deciding "no". Long term result for qemu: none really. Is it likely to come up again? Dunno, probably not. Is it good background material explaining how qemu wound up the way it did? Well, sort of, but not very.

Or how about these two, where somebody asks a very good question... and never gets an answer.

Yeah, all this is totally subjective, but most editorial jobs are.

April 11, 2009

Good to know Moore's Law continues apace. Also nice to see idiots punished.

April 10, 2009

April 9, 2009

Of course this wouldn't have any application to brainwashing at all. (SO glad the shrub administration's worked its way through the guts of the nation.)

April 8, 2009

At home, awaiting the cable guy.

Ooh, I accidentally hit "i" in Kaffeine (I thought I the vi window had focus), and it turned "deinterlace" on, which is actually something I'd been trying to figure out how to do. (Otherwise the bleach dvd it's showing looks terrible whenever things move around on screen to fast.)

But why would deinterlace ever _not_ be on? (Isn't interlacing an artifact of CRT scanning which even modern LCD and plasma televisions need to compensate for?

Weird Al's Don't download this song remains a work of genius. (I admit I feel slightly guilty about listening to an mp3 I ripped from a legally purchased album instead of listening to the Youtube version or something, but I'm way too lazy to actually _investigate_ the file sharing networks...)

April 7, 2009

Distinctly under the weather today. Fade said she felt blah a couple days ago, now I've got it. (Very mild upset stomach, zero energy.) Not really feeling like doing anything specific, so I cleaned the kitchen. (Always a good option when nothing particularly seems worth doing at the moment.)

The cable modem's out. (There's that marvelous service Time Warner wants to charge extra for.) Somewhat inconvenient to look up time warner's number without internet access. I'm spoiled. Have to go dig up the phone book... And it's a "signal issue". A guy has to come look at it, some time tomorrow between 8 am and 9 pm.

Checked the other local wireless access points. I could associate with a couple but they route packets either. Dunno what that means. Heading down to Thundercloud, I haven't had breakfast yet anyway.

Removed mlocate and updated my "next time I reinstall" list. (Having a cron job start up and hog the disk when I resume my laptop is silly, especially for a service I never use.) I should probably poke at Xubuntu 9.04 when it comes out.

Hanging out at Epoch with Fade. Yay coffee shop internet...

April 6, 2009

April 5, 2009

This is what our new couch looks like, except for the actual couch part, and there are more cats, and the word balloon is merely implied.

April 4, 2009

April 3, 2009

April 2, 2009

It shipped.

Falling over now.

April 1, 2009

March 31, 2009

We have a couch. It is orange. I should take a picture of it.

Mark and I rented a u-haul, got the couch installed (a small phrase which contains a lot of incident), and then moved the rest of his stuff out of the old apartment into his new garage. (Most notably the washer and dryer.)

Now at Mark's pulling what's turning into another all-nighter, switching between working on the crosstool-ng patches Cisco wanted and getting FWL 0.9.6 ready for release. (I was hoping to have a FWL release out _before_ the end of the month, but as midnight approaches it's not looking likely.)

No I didn't pull two all-nighters in a row, I did an all-nigher sunday night, and now I'm pulling an all-nigher tuesday night. (Monday night was at home, Fade and I watched the rest of the Schoolhouse Rock DVD and )

It's a bit odd to be copying things I got working in FWL months ago into into a completely different open source toolchain project (which is also mostly a single person endeavor). The fact it took me about an hour to get FWL to produce such a toolchain and over a week to get crosstool-ng to do it doesn't enter into it, apparently. They want what they want, and are willing to pay to get what they want. I don't understand _why_ they want it, but it's their money...

I suspect I've been drinking too many energy drinks.

March 30, 2009

Very long day. Up overnight at Mark's last night, finishing up many things, then came home and slept a lot.

There's a release candidate set of binaries up at the new snapshots directory. We plan to start doing nightly builds, but it hasn't quite started yet.

March 29, 2009

It occurs to me that around half the eating I do is for social reasons. I went out to dinner with with Andrea at Whataburger last night (having just cooked and eaten chicken), I'm meeting Mark for dinner at Texadelphia (having just eaten leftover chicken and an apple), I regularly enjoy going out to restaurants with Fade, biking out after midnight the only places that are open are restaurants...

March 17, 2009

Biked to the north Chick-fil-a location last night. Adrienne called (locked out) while I was there. It's at least an hour and a half bike ride each way (closer to 2) so she drove up to get the key. I only got about 45 minutes of work done because they closed at 10, but yay exercise.

Thinking of biking to the south chick-fil-a location today. Made it as far as the Starbucks on 15th street.

Huh. So Dell responded to the Macbook Air by coming out with the Dell Adamo, a system which costs $200 more than the Macbook, has a slower processor, less powerful graphics chip, weighs a pound more, is slightly wider, /bin/bash: indent: command not found

I've been dropping "span" tags in my blog for something like a year now, meaning to write a cgi filter that'll actually use 'em, and for some reason today I keep doing "spam" tags instead. (Blame my email...)

Interesting. Time magazine estimates that if California legalized pot, it could balance their state budget by $2.3 billion/year. That's $1.3 in tax revenue and $1 billion saved in enforcement. (It's california's most lucrative agricultural crop, earning $14 billion/year.) Legalization would be personally inconvenient since I continue to be allergic to the stuff, but other than that it seems kind of obvious, possibly inevitable.

March 16, 2009

Dropped Adrienne off at SxSW this morning (she overslept; I simply hadn't set the alarm), then returned yet another fax to my sister's realtor, and now hanging out at the coffee shop next to Kinko's on 26th street. The barista is _stunningly_ cute. (Somewhat distracting, actually.)

Wendy at the credit union poked me about hazard insurance for Kris's new house, and I phoned and left a message with the Nationwide office on Far West.

Tested a new m68k tree from Greg Ungerer. Didn't fix the problem (uClibc build dies trying to generate the syscall list due to headers exported by make headers_install #including files it didn't install), but I got him the error message and hope for a reply.

Replied to an email about my tinycc fork (unrelated to resubscribing to the mailing list, this is a college professor at Tufts who says he and his postdoc wanted to hook my old fork up to a different backend. Could be interesting. (I still think qemu's tcg is the logical backend for the thing, but _any_ work to genericize it so the frontend and backend talk through some kind of defined interface can only help matters.)

Critiqued Pavel Machek's filesystem requirements documentation some more.

Wrote up a quick summary of last week's 440 work and sent it to Mark so he can do Impact Linux invoicing.

Heard back from the insurance guy, need to talk to an insurance guy in Minnesota. Right, called him, gave him the contact info for the mortgage lady so she can tell him what kind of insurance I need.

Tracking down how I broke the powerpc target, checking the revision history and building old versions until I find the one that doesn't work. (This would go faster if I didn't keep running out of disk space. Yeah, I managed to fill up the new bigger drive.)

I'd hoped to go to some of the free SxSW interactive parties tonight, but after being beaten to death by interrupts over the weekend I just haven't got the energy. Might go bike to someplace to huddle with my laptop...

March 15, 2009

Didn't get to update my blog yesterday. It's 8pm and I'm keeping Mark waiting while I stop at a restaurant to eat for the first time today.

Too overscheduled. Really in danger of burning out. I am now officially swap thrashing. I spend so much time being interrupted and context switching that I'm getting NOTHING DONE.

Adrienne's visiting and I have no time to spend with her. She's free for dinner tonight as soon as she finishes her nap; she came back from the conference exhausted. She's skipping the SxSW parties tonight, many of which I could go to if I had time because they're free. I really wanted to see the vintage coin-op video arcade they had (free to everybody, no badge required) at the convention center, but I never had time and it was only there for three days ending today.

Fade made it off to the airport early this morning. I didn't get to spend a lot of time with her this week, but she's only gone for a week.

Mark's wanted to meet me every evening for four days now (both because he needed help moving slightly _farther_ out of town, and because he needs help with a big documentation task Cisco assigned him with; Mark doesn't really do documentation). He's my friend and I want to spend time with him, and the technical stuff is even billable time, but with traffic it's an hour drive each way to his apartment. (Between errands, visiting Mark, driving Adrienne around, and driving Fade around, I've gone through an entire tank of gas this week. That's kind of unusual.)

I'd hoped Adrienne's visit would inspire me to clean the condo (which would have helped Fade's pre-trip stress too), and I did manage a couple hours, but I just didn't have time to finish it.

The mortgage on my sister's new house was approved, although the closing had to be rescheduled for sometime later this month (I forget when, but it's not this week and I think it's in my email). I need to go get some kind of house insurance ASAP (I forget what kind, but it's in my email). Still haven't manage to do so, maybe I'll find time tomorrow. (I need to call Kris and see how she's doing. Maybe I'll find time tomorrow.)

I've haven't had a chance to open the Lua book since the last time I wrote about it. Yes I know Gentoo From Scratch is blocked on a new useradd (because mark doesn't want to install an old version of shadow utils for some reason) and useradd is a toybox thing and toybox is blocked on me learning lua to rewrite it. I'd _love_ to have time to learn lua this week. It's very hard to do in 15 minute increments. Maybe I'll find time tomorrow.

Spamalot was in town all this past week and I missed it. I missed David Schwimmer mocking Showgirls at the drafthouse. I missed the start of the drawing class at UT I wanted to take this semester. Lawrence Lessig is in town for SxSW, maybe I'll find time to meet him tomorrow.

I've made trivial progress tackling my todo list for the FWL 0.9.6 release I wanted to do tomorrow. Maybe I'll find time tomorrow.

Went out to Whataburger last night and carved out a couple hours to poke at the 440 toolchain, because I'll have to do a progress report about that tomorrow. (I'm calling this week half-time because even with poking at two different crosstool variants I can't say I managed more than 20 hours.) I duct taped something into a FWL target that _might_ look like 440 toolchain in the right light, and fired up "qemu-system-ppc -M bamboo", and found out that the 440 processor isn't _implemented_. It's just a stub, the bamboo board just emulates the system-on-chip peripherals and otherwise uses a 405 processor emulation. (I'm embarassed this took me until yesterday to discover, but I didn't have anything to _test_ before then.)

I'd love to go visit the doctor to get antibiotics for the sinus infection that's preventing me from sleeping for more than a couple hours at a time. I've meant to go ever since I got back from Pennsylvania. (Of course the paperwork for better health insurance never got filled out anyway.) Maybe I'll find time for both tomorrow.

There are so many other things that have fallen through the cracks. Solar asked me to cut new FWL binaries on friday, and Vladmir Dronnikov's wanted those for months. Haven't found time. I need to set up the nightly build cron job, which means I need to fix static linking, see "FWL todo list" above. It's just a simple binary search regression test, I could sit down and figure it out in an hour or so. (Might be when I fiddled with the wrapper, might be when I started stripping the libraries.) And the m68k guys sent me a link to an atari emulutor that can apparently run a m68k kernel (aranym.org) but I haven't looked at it yet, nor have I followed up on the m68k headers_install issue in 2.6.29. Maybe I'll find time tomorrow.

I need to poke Eric about the C++ paper (he says he emailed me a draft earlier this week, but I didn't get it, and he's not getting my email either; sounds like a local delivery routing issue on grelber we need to track down). I haven't been to Starbucks in weeks. I need to get some exercise, and a haircut. My current car insurance card expired on the 5th (I think they sent us a new one, but dunno where it is). I need to hire a lawyer to show that Stafford idiots that when they gave me a "failure to show proof of insurance" ticket I did, in fact, have insurance (the card fell under the passenger seat). I need to deliver those shirts to Reese. I haven't spoken to Stu in weeks, I need to resubmit the perl removal patches, I may _never_ get QEMU weekly news updated...

Sigh. I don't even REMEMBER most of the things I have to do, they're just piling up into a big wave. But Mark called, and I guess I've kept him waiting long enough. Off to do documentation...

Ok, got some documentation edited over at Mark's, took Adrienne to dinner around 1 AM at the 24 hour IHOP (after visiting a 24 hour wal-mart and walgreens trying to find a supplemental battery pack for her smart phone), going to bed in hopes of getting up early in the morning and tackling backlog. May just turn my phone _off_ all day to avoid interruptions.

March 13, 2009

Lua books (2): check. Roxette "Look Sharp" album (to which I learned C): ripped, converted to mp3, and shoehorned into rockbox (which has the most horrible user interface, but oh well). Soda refills (Einstein's this time): check.

Not a bad language, really. It's firmly in the whitespace agnostic camp, not even requiring semicolons (which to be honest, aren't usually needed in C either; the compiler tells you when you've dropped one). It uses "end" in place of closing curly brackets, and makes it mandatory instead of having magic single line contexts after if statements and such. 21 reserved keywords. -- starts a single line comment, --[[ and --]] are multi-line comments. Variables are created by assigning to them, and reading a nonexistent variable return "nil" (instead of throwing an exception, ala Python).

I love this Album. Yeah it's old, but it's great to program to. (Joyride's ok too.)

Ok, I'm to section 1.3 in "Programming in Lua" and I've hit the first place I want to smack the author: "Global variables do not need declarations. You simply assign a value to a global variable to create it... if your variable is going to have a short life, you should use a local variable." Ok, _HOW_? YOU'VE NEVER MENTIONED LOCAL VARIABLES BEFORE NOW. This section offers no way to distinguish between local and global variables. I'm GUESSING it's a scope issue, but that's just a guess. Throwing me a forward reference here would be very reassuring; the author did not do this.

Wow Rockbox really is a horrible music player. (It works fine, it's just a pain to pause and unpause when I get a soda refill. Plus putting music _into_ it requires an act of congress.) I miss the KDE one. After several months of daily use, I think I can confidently say that I hate everything about Gnome. (I still have my little orange "iPod Klepto" around here somewhere, but apparently it's not in my backpack. Presumably in the car. Bit fiddly to use my giant noise cancelling headphones with a device that's maybe 1/20th their size, but it's got the "big prominent pause button" thing down. Hopefully there's a way to get it to play things in order rather than random shuffle. I'll have to look into it.)

Dear Gnome console developers: putting the close button for the last tab right above to the up arrow for the scroll bar is DEEPLY STUPID. The Konsole people figured out not to do that _years_ ago, and I just closed 3 tabs I wanted to keep in a slightly misaligned attempt to scroll up. (This is unrelated to the above statement that I hate everything about Gnome, which predated me doing this.)

The original crosstool only _claims_ to have support for the 440. If you actually try to build it, the thing breaks trying to build the first package (gcc 3.3.6, which is _really_ strange because I _told_ it to build gcc 4.1.1, which is the latest version it has). Looked at its config and it _seems_ the magic is --with-cpu=440 for gcc. Dunno if I need to tell binutils something (it didn't try to build binutils before trying to build gcc 3.3.6... I don't get it either).

Broke down and resubscribed to the tcc mailing list a couple days back to ask if CVS was dead yet. No official response so far, but I replied to the one guy who commented with a big long rant about what tcc SHOULD be doing. (Do the busybox swiss-army-executable thing and replace binutils, grab TCG from QEMU and use it as your code generator, clean up the path mess, try to build real packages... The usual. The kind of things I was working on when I left. Progress made so far: zero. Oh well.) They'll probably say CVS isn't dead, and then I'll go away for 6 months again. It's a bit like groundhogs day: I pop my head up, see CVS, and go away for 6 more months.

March 12, 2009