dburrows/ blog

archive

To reply to a blog post, email me at Daniel_Burrows@alumni.brown.edu. Please let me know whether I should post your reply on the blog page.

aptitude-related projects for Summer of Code 2010

I wasn't planning to be a mentor this year, but it looks like I might have enough time to do a decent job at mentoring after all.

So here are a few ideas I have kicking around that would probably be suitable for an SoC coder. I have lots more ideas, but these are the first ones that came to mind. These ideas are deliberately broad: as a student, you are responsible for collaborating with me to figure out the details of your project and put together a workable implementation plan. At the same time, this list is not all-encompassing; feel free to come up with your own idea and explain why I should act as a mentor for it.

  1. Improve the GTK+ interface. The GTK+ interface is only partly complete and I'd like it to be completely complete. A few open ends of interest are noted at aptitude-gtk-status-a-visual-tour, or you can just run the program yourself and see what's missing or ugly.
  2. Work on per-package Wiki pages (the idea is sketched out in my email on the subject). Most likely this can use the existing wiki.debian.org infrastructure.
  3. Create a service for tracking user reviews of Debian packages and implement support in one or more apt clients.

Note that the last two aren't aptitude-specific; they basically amount to working out a spec for clients to access the data in question, and then implementing the spec in one or more clients (possibly including aptitude, apt, synaptic, packages.debian.org, etc).

Posted Mon Apr 5 22:00:00 2010
Installing OpenWRT on a Linksys WRT54GL v1.1

I finally got a few hours free this morning to check an item off my home system administration checklist: upgrading the wireless router's firmware to OpenWRT. There were a couple motivations for this, including the fact that my SoundBridge Radio couldn't maintain a connection to my firefly server using the built-in firmware, and I had read that OpenWRT would work better.

Since this is posted on Planet Debian, I should mention why I didn't use DebianWRT. The basic answer is this text at the top of the DebianWRT homepage:

Currently the most common methods used to run Debian on these systems is to install OpenWRT or a similar firmware, add disk space either by USB storage or NFS, create a debian chroot by either running cdebootstrap from inside OpenWRT or debootstrap --foreign on a PC, and running Debian from this chroot. For example, instructions for the WLHDD.

I'm not sure whether this is because DebianWRT is experimental, or because its goal is to use routers as cheap general computers. Either way, it sounds way too complicated and/or fragile for what I'm interested in (i.e., a wireless router with better software). The goal here is to get something that does a better job than the built-in WRT firmware, but doesn't require too much tinkering to get working or to maintain. I have plenty of outlets for my tinkering urges already.

Nothing here was exactly difficult, but it was hard to find all the information I needed to get things working. Hopefully documenting the steps I went through here will save someone else some time.

Step 1: acquire the firmware

This was trickier than you might think. If you click on the download link at the OpenWRT web site, you end up at the top of an FTP server populated with a vast quantity of stuff. Worse, when you find the actual firmware images, you'll quickly discover that there are piles and piles of them divided between sixteen directories, and no guidance as to which one to pick. And picking the wrong one will turn your beautiful router into a doorstop.

Luckily, the OpenWRT documentation contains a section called "Getting Started". Unluckily, that section consists of the following text:

1.1 Getting started

1.1.1 Installation

1.1.2 Initial configuration

1.1.3 Failsafe mode

Whoops, someone forgot to execute on a TODO. :-)

Undeterred, I consulted the usual fallback reference, Google. It pointed me at several references, some of which were hidden on other parts of the OpenWRT site. Armed with these, I was able to determine that:

  1. My WRT54GL v1.1 probably uses a Broadcom 5352 chipset.
  2. The correct firmware, according to multiple sources, is probably in the kamikaze top-level director, the latest version's subdirectory, and the bcrm-2.4 directory under the version (the link is to 8.09.2, which is current as of March 6, 2010). Apparently the bcrm47xx directory doesn't have wireless support; it helpfully contains a file called NO-BROADCOM-WIRELESS to warn you off, but unhelpfully doesn't include any additional information in that file (like what exactly is broken or that you can find a working firmware in a sibling directory) ... oh well.
  3. The firmware file that you want is openwrt-wrt54g-squashfs.bin, even though it doesn't match the model number of the router. This directory could use a README file explaining which hardware is supported by each of the dozen or so firmware files it contains.

Step 2: install the firmware

After Step 1, this was a real relief. I just used the built-in Linksys firmware installer, pointed it at the .bin file, and it went.

Step 3: configure the router

The installation guide I followed was pretty much silent about what to do after I got the firmware on. Luckily, this is just a software problem, meaning it's much more familiar territory for me.

  1. First things first: I checked that I could still get a DHCP lease. It worked.

  2. Armed with that, I tried telneting to the router. I used the resulting root prompt to set a password on the root account.

  3. I logged out and tried telneting in. Apparently the router is configured to disallow root logins over telnet if you don't have a password. Good for them (although why allow telnet at all?); oops for me.

  4. Luckily, an ssh server was installed by default. I like using keys to log in, so I tracked down the documentation on configuring the server to use public-key authentication; it turns out there's a single global key file named /etc/dropbear/authorized_keys that's exactly like OpenSSH's per-user authorized_keys file. No idea what would happen if I had multiple users, but I won't.

  5. The next obstacle: I didn't have an Internet connection. For some reason, my cable modem didn't want to give the router a DHCP lease. On the off-chance that it was remembering too much, I rebooted it and ran ifup wan. That fixed the problem. I still don't know why.

  6. Not a step, but a useful note: in the process of figuring the above out, I found readlog. It's basically dmesg for syslog files; it shows the most recent lines written to syslog. This is useful because there isn't a real syslog file, due to the fact that there would be no room to store it on the 54GL.

  7. Finally, I had to get wireless working. The documentation is very helpful when it comes to describing the syntax of the wireless configuration. Unfortunately, I read the list of encryption options and missed the section right below where their meanings are explained (although, to be honest, I might not have understood the implications of the explanation without the research I did anyway).

    option encryption none, wep, psk, psk2, wpa, wpa2

    I wanted WPA2 encryption, so I entered wpa2. And nothing worked. After a good hour of trying different options on the client, swapping software components in and out on the router, experimenting with the encryption key syntax, and crawling Google, I finally found my answer. If you just want WPA2 encryption, you must not use wpa2 as the encryption type. Instead, use psk2. It turns out out that wpa2 actually means use WPA2 and also use an external RADIUS server for authentication. psk2 is the system you're familiar with from a typical consumer wireless router.

Step 4: enjoy

And with that, it works.

Unfortunately, contrary to what I wrote here originally, my Roku still doesn't work. On the other hand, having a real Linux installation is helpful for debugging it. Currently my suspicion is that the router isn't passing multicast packets between the wired and wireless interfaces (broadcast works fine, multicast doesn't). That said, it seems like if I restart Firefly just before I start playing music, I can play reasonably reliably -- as long as I don't stop, because if I do, the Roku forgets that the music server is there. Either way, I've spent about as much time fighting this as I can afford. :-/

One lingering worry I have is security; unlike Debian, which has both a security mailing list and tools to inform me when I need to install a security update, the OpenWRT firmware doesn't seem to have any mechanism for distributing security notices. True, there is an openwrt-security-announce list, but it appears to be entirely unused, as is openwrt-announce. Something to keep an eye on, then.

Also (file under note to self), I need to remember to verify that the router isn't exposing services to the outside world. The default iptables configuration is hideously complex; with just a quick glance, it could be setting up a floral shop for all I can tell. I'll need to test this empirically and maybe analyze the rules in more depth.

Posted Sat Mar 6 19:30:00 2010 Tags: installation openwrt
aptitude-gtk status, a visual tour

As some of you may know, Arthur Liu did the initial work on a GTK+ interface for aptitude last summer as part of the Google Summer of Code program. Since that program ended, I've been working to complete the GTK+ interface, along with some unrelated but similarly large changes to other parts of aptitude. It's been a while since I've written anything about aptitude-gtk, though, mostly because I set it aside for a few months to improve the dependency solver.

While I'll probably be doing more work on the backend over the summer, I also hope to get the GTK+ interface to a point where I can feel comfortable including it in a stable release of aptitude. There are two things that I want to be in place for the GTK+ interface before I release it:

  1. The interface should be functionally complete, meaning that you should be able to do anything you can do in the other interfaces.

  2. The interface should be reasonably straightforward to use and nice to look at. This is a never-ending task, but there are some pieces of the current interface that are particularly ugly or awkward that I'd like to fix before release.

I've gone through the major areas of the current UI and marked them up with notes about things that I think need to be done. This isn't a definitive or exhaustive list, but it should give some idea of where aptitude is and where it's going.


Dashboard Tab

annotated dashboard tab

The first thing you see when you start aptitude is the dashboard. This is a display that gives you immediate access to some of the most common things you'll want to do: searching for a package, viewing the available upgrades, and upgrading as many packages as aptitude can figure out how to upgrade without removing anything. This is one of the most complete parts of the interface, but there are a few minor aesthetic things that I'd like to take care of.

[UPDATE] Michael Johnson writes:

I agree with most of the notes on this screen. However, I'd like to see the Selected Package changelog stay. I often look more than one upgrade back. The summary doesn't show more than what's being upgraded. However, I do like the idea of package description tab.

The search box should definately always be available. I found it annoying when I had to click back to a different tab to get it.

One thing I definately miss from the console version of aptitude is the list of recommended and suggested packages. I don't let aptitude auto-install recommended packages, but I like to know what they are.


Packages tab

annotated packages tab

This is the view that's shown after you search for a package by name, keyword, or by other criteria. It's also fairly complete, but there are more things that could be added (mostly enhancements). Related to this is the dialog box for editing which columns are visible:

annotated visible columns editor


Dependency solver tab

annotated dependency solver tab

The dependency solver got a lot of love in the 0.5.2.1 release, and it's now basically functional. There are some minor layout issues to be fixed, and if I have time I'd like to add more ways of controlling the resolver's behavior.


Package information tab

annotated package information tab

The package information page is probably the weakest part of the GTK+ aptitude interface; it wouldn't be too much of a stretch to say that this tab is holding back a stable release of aptitude-gtk. The big problem I have with it is that I feel that the information tab should open up with an overview of the status of the package. What version is installed, what's available, what archive is it in, who's the maintainer, how big is it, etc. Instead, aptitude-gtk currently shows you a big tree of all the package's dependencies. While that's useful, it should be something the user chooses from.

I also find this tab to be visually unappealing, but I'm not entirely sure how to fix that.

[UPDATE] Gunnar Wolf writes:

As you have mentioned in your blog post (quote this mail if you so wish) that you want to make the package information screen "sexier": I think it would be natural to integrate screenshots (from http://screenshots.debian.net/) into it. Of course, it would be worth measuring whether it imposes too high a load into that system (maybe it should only get a screenshot if the user explicitly asks for it) - but surely would be a net win.

I think this is a great idea and I'm definitely planning to incorporate this feature.

[UPDATE] Michael Johnson writes:

One of the things I really like about aptitude-gtk over console-aptitude is how obvious it makes some things. The version display on this tab is one of those things. Maybe it could be more obvious, I don't know.

I agree, the dependency list is not the most important thing. And it's missing the reverse dependencies. I use that more than the dependencies. As far as the actual display of the list, I can't think of anything besides a tree. Although the current implementation does feel quite clunky.

What I'd more like to see first is the list of files in the package. But I can see how other people would have different priorties. So it should be really easy to change the default view.


Preferences tab

vast gaping hole where the preferences tab should be

One of the reasons that it is a stretch to say that the information tab is holding aptitude-gtk back is the preferences tab, which is a huge pile of not-yet-implemented-ness. In fact, it's not even designed, let alone implemented. Releasing a GUI program with no way to alter the settings strikes me as ... silly.


Preview tab

annotated

The preview tab is a bit weaker than the packages tab, in my opinion, but much stronger than the information tab. Like the information tab, I find it visually unappealing, but I'm not entirely sure how to improve it. I'm currently leaning towards the paned browser idea.

[UPDATE] Michael Johnson writes:

I've never been comfortable using this tab. It never made much sense to me. I think the last one or two times I used aptitude-gtk it started making sense, but that seems to be too long a time. I think it's the mass of text at each top level entry. The console version has brief descriptions with more info in the details pane when the entry is selected. Perhaps healthy use of tool tips or an info pane would help with this.

The paned browser idea sounds promising. I think it will be easier for new users to follow. And it provides a natural way to provide more information on what each section is about.


That's all folks!

Posted Sun May 3 09:12:39 2009
aptitude 0.5.2 released

I've released version 0.5.2 of the aptitude package manager (release notes). You can download it from Debian's experimental archive, or from the aptitude package page.

WARNING: This is an unfinished piece of software. Do not install it if you want a package manager that does everything you need perfectly all the time. It is buggy and incomplete.

This update includes major changes to the resolver backend and a new and improved GUI frontend for the resolver:

It also includes an assortment of other fixes and improvements. Click through to the release notes to see a list of what changed in this release, with screen-shots.

Posted Sat Apr 25 11:00:00 2009 Tags: aptitude release
Merge of doom
daniel@emurlahn:~/programming/aptitude/head$ hg log -r tip
changeset:   2611:c81f562751d6
tag:         tip
parent:      1665:94b597a663a7
parent:      2610:78100f86756d
user:        Daniel Burrows <dburrows@debian.org>
date:        Mon Mar 23 10:49:47 2009 -0700
summary:     Merge the entire post-lenny branch into head.
daniel@emurlahn:~/programming/aptitude/head$ hg log -p -r tip | diffstat -m | tail --lines 1
 239 files changed, 105227 insertions(+), 16334 deletions(-), 69203 modifications(!)

That's right: active aptitude development is now taking place in the head branch again, after a hiatus of about a year while I worked in a branch while waiting for Lenny to be released. Lenny is released now, and so the post-lenny and gtk branches have been merged into head and renamed to avoid accidental commits (you can find them under the names post-lenny-branch-is-closed and gtk-branch-is-closed). Future releases will come out of the head branch (http://hg.debian.org/hg/aptitude/head).

Posted Mon Mar 23 12:15:00 2009 Tags: aptitude aptitude-gtk
Sometimes, tools are useful.

I generally try not to get caught up in writing auxiliary tools for my free software projects. My experience in the past has been that given how little spare time I have each week, a tool that takes more than an hour or two to write quickly ends up becoming a time sink that prevents me from making progress on the main project. But occasionally it's worth the effort.

screenshot

I spent the last couple weeks putting together a tool to extract information about a dependency search from a log trace of aptitude's resolver. The search is displayed in a log interface that shows both the steps of the search in order, and the tree structure of the search. It took a long time to write, but after working with it for just a few hours, I was able to track down the bugs in the resolver modifications I was trying to write; without a tool like this, it would have taken a lot of painful grepping through log files to get all the information I needed.

In other news, the resolver now stores the choices it has made as a single set of choices, instead of using two different sets for the two types of choices it can make. There's no user-visible change (although I did fix another performance problem that affected safe-upgrade), but this will pave the way for adding more types of choices in a cleaner way: in particular, the resolver should get explicit support for package replacements in the near future.

The code for this is available in the tools directory of aptitude's experimental Mercurial repository, http://hg.debian.org/hg/aptitude/post-lenny. When I get around to merging it this with the main repository, it will also be available at http://hg.debian.org/hg/aptitude/head.


A brief digression regarding the implementation language

The tool is implemented in Haskell, a decision that I'm not sure was the right one. Haskell's data model was perfect for this task: having algebraic datatypes allows me to precisely represent cases where parts of the structure can't be inferred from the log file, or can only be inferred incompletely. And of course there are the other well-known benefits of Haskell (pure, a strong type system, good support for higher-order programming, etc). But all those benefits were wiped out by the time I spent fighting the runtime.

The thing is, Haskell is a lazy language. In theory, this means that it never evaluates anything you don't need and that you can work with arbitrarily large (even infinite) values and only allocate the parts you actually use. In practice, this means that you have to pay excruciatingly close attention to every single expression you write, or as soon as you run your program on non-trivial input, it will either run very slowly, eat all your memory, die with a stack overflow, or all of the above. Oh, and did I mention that laziness means that you can't have a runtime debugger, that trace statements are mostly-useless, because your program doesn't run in any predictable order, and that it's impossible to get backtraces when there's a fatal error?

I think if I were doing this over again, I'd use O'Caml. I'm less familiar with it and its syntax is clunkier, but I have used it productively in the past and it provides the core stuff I wanted from Haskell (functional programming with algebraic datatypes). If someone could make a language that was pure, had Haskell's nice syntax and type features, but was also strict, and that had decent tool support and development resources, I think I'd be in nerd heaven.

Posted Mon Mar 23 09:37:00 2009 Tags: aptitude haskell programming
Help make aptitude faster!

I'm working on aptitude's dependency resolver at the moment. Mostly I'm working on some changes to support better UI, but while I have all the code in my head I'd like to try to fix some of the nagging speed problems it has. I've already managed to clear up some simple but large problems with aptitude safe-upgrade, because I could get them to happen on my computer. The problem is, most of these problems don't happen on my computer. I have a half dozen or so specific ideas that might speed aptitude up, but I don't want to just randomly apply tweaks without some baseline tests that I can use to make sure that I'm actually making things better and not just perturbing the code for no reason. :-) Plus, seeing why aptitude is actually running slowly might help when it comes to finding ways to improve its performance.

So, this is where you come in. If you are using lenny and aptitude takes a long time to resolve dependencies on your computer outside of aptitude safe-upgrade, I would like to hear from you. All you need to do is to run this command when aptitude is slow, before you upgrade any packages:

$ aptitude-create-state-bundle aptitude-state.tar.bz2

Then, find a way to get the resulting large file to me. Email will work if your outgoing server accepts it; send to dburrows@algebraicthunk.net. That will tar up /etc/apt, /var/lib/apt, /var/cache/apt, and a few other directories: run aptitude-create-state-bundle --print-inputs to see exactly what it will include.

Thanks!

[UPDATE]: PS: please also include a brief description of the problem that you're seeing so I know what to test. Something like, e.g., when I run aptitude full-upgrade it takes thirty minutes to complete.

[UPDATE]: PPS: of course you have to run aptitude-create-state-bundle before you actually upgrade any packages! i.e., you must type n at aptitude's command-line prompt or quit the program if you're using the curses interface. Otherwise, your state snapshot will just tell me what you upgraded to, which is only half the information I need (I also need the state of all packages prior to the upgrade).

Posted Thu Mar 5 23:00:10 2009 Tags: aptitude debian
Well, that was easier than expected
daniel@emurlahn:~/programming/aptitude/post-lenny$ time /usr/bin/aptitude -sy safe-upgrade | tail --lines 3
real    0m46.016s
user    0m41.639s
sys     0m0.172s
daniel@emurlahn:~/programming/aptitude/post-lenny$ time ./src/aptitude -sy safe-upgrade | tail --lines 3
real    0m5.336s
user    0m4.096s
sys     0m0.136s

More of the same coming, although I doubt I'll be able to top those statistics. And all it took was deleting twelve lines of code. Well, that and rewriting the entire resolver framework so that those twelve lines were no longer necessary, but who's counting?

I'll post more details once I have everything in place and I'm sure it works as expected (most likely in a week or two).

Posted Wed Mar 4 08:00:00 2009 Tags: aptitude debian
Access control FAIL

Seen at work today while grovelling through some old code:

class Foo
{
public:
  Foo(); // Do not use -- no body on purpose
Posted Fri Feb 20 18:35:00 2009 Tags: c++ fail programming silly
Should we be scared of the future or not?

I recently received an email from Sigma Xi with various news items. One of them mentioned a Hawking quote:

Science fiction is useful both for stimulating the imagination and for diffusing fear of the future.

-- Stephen Hawking?

Now, Sigma Xi is generally a pretty reputable organization, but I wondered: is a scientific organization really encouraging fear of the future? Mind, I think a certain degree of concern about the future is healthy, both individually and societally, and there is a long tradition in science fiction of writing novels about terrifying futures. But Sigma Xi is usually upbeat about the wonders of science and technological progress. Besides, there's a real dissonance in that quote between becoming more creating and imaginative, and becoming more afraid of the future. Maybe it was intentional, but it seems very out-of-place. So, I started to wonder whether this was really what Hawking said. Typing a few key words into Google, I looked at where else this quote showed up.

According to the NSF, Sigma Xi's quote is correct.

According to renowned physicist Stephen Hawking, "science fiction is useful both for stimulating the imagination and for diffusing fear of the future." Interest in science fiction may affect the way people think about or relate to science...

ERIC, which is some sort of database of internal government reports, agrees:

According to renowned physicist Stephen Hawking, "science fiction is useful both for stimulating the imagination and for diffusing fear of the future." Indeed, several studies suggest that using science fiction movies as a teaching aid can improve both motivation and achievement.

So, maybe Hawking really does encourage being afraid of the future? Well, here was the third and final reference to this quote that I found, from (of all places) an unofficial transcript of a Larry King Live interview:

HAWKING: I think science fiction is useful, both for stimulating the imagination and for defusing fear of the future. But science fact can be even more amazing. Science fiction never suggested anything as strange as black holes.

So, it looks like this quote comes from a live TV interview. I don't know whether the different interpretations come from different transcripts (this is the only one I can find); defused and diffused are similar enough that it would be easy for either one to be reasonable -- and IIRC, he speaks through some sort of speech-generating machine, which would just make it easier to misunderstand him. But given that Hawking is a crazy-science-guy, I suspect that he would rather defuse fear of the future than spread it around (even if it became more diffuse as it was spread), and that the transcript above is what he meant to say. In which case, I wonder about the versions of the quote that I found everywhere else. Didn't anyone do a double-take when they were typing up those papers, Web pages and news items?

Of course, there is a third possibility, which I rather like: the quote could be a play on words, a sort of slant pun (a phrase that I just invented, which is the funnier cousin of a slant rhyme). What's particularly cool about this particular ambiguity is that (a) both sentences are sensible, (b) they have related but opposite meanings, and (c) somewhat oddly, I agree with both of them: science fiction is good at defusing unwarranted fear of the future, while also diffusing fear where it's entirely warranted. I doubt Hawking was going for the double meaning, but it's a nice thought.


And for the inevitable people who can't figure out what this blog entry is about, here's a hint:

diffuse:

Verb

  1. (transitive) To spread over or through as in air, water, or other matter, especially by fluid motion or passive means.
  2. (intransitive) To be spread over or through as in air, water, or other matter, especially by fluid motion or passive means.

defuse:

Verb

  1. To remove the fuse from a bomb, etc.
  2. To make something less dangerous, tense, or hostile.
Posted Wed Feb 11 20:45:00 2009 Tags: humor
Dear Lazyweb, does anyone understand job control?

I've been banging my head against this all day and it doesn't make any more sense than it did when I started.

I have a need to write some code that can manage job control on a terminal. More specifically, I need to run a single process and stuff it into the background at will, so that it gets suspended when it tries to read from the terminal. So, there's a controller process and a subprocess process.

 controller  ----------> subprocess
               manages

The man-pages and glibc info documentation make this look simple: disable TOSTOP if it's enabled via tcsetattr() so that the subprocess can write to the terminal, then start the subprocess in a new process group (by fork()ing and calling setpgid()). Once it's going, you can put it in the foreground or background by calling tcsetpgrp() to set the terminal's foreground process group to the subprocess (to put it in the foreground), or to the controller process (to put the subprocess in the background). Whenever the subprocess is in the background, it will be sent SIGTTIN if it tries to read from the terminal.

I have everything working -- except for the very last sentence of that last paragraph. I can see my processes being put into the right process group, and I can see them going into the foreground and the background. For instance, here the first line is the controller process and the rest of the lines are the process group of the subprocess (this one started a few other processes).

STAT CMD                           PID  PGID TPGID  PPID   SID
Ss+  ./src/aptitude              26989 26989 26989 26980 26989
S    ./src/aptitude              26991 26991 26989 26989 26989
S    /bin/sh -c /usr/sbin/dpkg-p 27027 26991 26989 26991 26989
S    /usr/bin/perl -w /usr/sbin/ 27028 26991 26989 27027 26989
Z    [dpkg-preconfigu] <defunct> 27034 26991 26989 27028 26989
S    /bin/sh -e /tmp/lynx-cur.co 27039 26991 26989 27028 26989
S    whiptail --backtitle Packag 27043 26991 26989 27028 26989

If I try to move the background process into the foreground, it is marked as the foreground process in this session:

STAT CMD                           PID  PGID TPGID  PPID   SID
Ss   ./src/aptitude              26989 26989 26991 26980 26989
S+   ./src/aptitude              26991 26991 26991 26989 26989
S+   /bin/sh -c /usr/sbin/dpkg-p 27027 26991 26991 26991 26989
S+   /usr/bin/perl -w /usr/sbin/ 27028 26991 26991 27027 26989
Z+   [dpkg-preconfigu] <defunct> 27034 26991 26991 27028 26989
S+   /bin/sh -e /tmp/lynx-cur.co 27039 26991 26991 27028 26989
S+   whiptail --backtitle Packag 27043 26991 26991 27028 26989

Note that the background process is not suspended, even though it's busy reading from the tty. If I manually suspend the process group with kill -TTIN -26991, it stops as expected:

STAT CMD                           PID  PGID TPGID  PPID   SID
Ss   ./src/aptitude              26989 26989 26991 26980 26989
T+   ./src/aptitude              26991 26991 26991 26989 26989
T+   /bin/sh -c /usr/sbin/dpkg-p 27027 26991 26991 26991 26989
T+   /usr/bin/perl -w /usr/sbin/ 27028 26991 26991 27027 26989
Z+   [dpkg-preconfigu] <defunct> 27034 26991 26991 27028 26989
T+   /bin/sh -e /tmp/lynx-cur.co 27039 26991 26991 27028 26989
T+   whiptail --backtitle Packag 27043 26991 26991 27028 26989

So the signal isn't being blocked or ignored. I can also run programs in a normal shell, outside my harness (e.g., links &) and watch them auto-suspend, but the same thing doesn't happen when I start them directly under my controller process. It's not even that they're starting as foreground processes: I start them in the background, and they never see a SIGTTIN.

Does anyone have a clue what's going on? Hopefully it's as simple as a flag I have to set somewhere...

Oh, and for extra fun, this is all happening inside a VTE terminal widget. That shouldn't make a difference (after all, it's what gnome-terminal uses, and SIGTTIN behaved as expected when I tested it there), but who knows, it might be relevant.

[UPDATE] I did a little more testing: I was bringing the subprocess into the foreground whenever the terminal was active, so I couldn't try typing into the terminal while the subprocess was in the background. If I do that, it does get SIGTTIN and it suspends. So it looks like select() doesn't count as a read for the purposes of job control. I doubt there's much I can do to get around that. :-(

Posted Sat Feb 7 17:10:00 2009 Tags: programming terminal unix
A better map of the apt state files

Last week, I posted some diagrams showing where apt and dpkg store their state files. I got a few suggestions about how to simplify them and make them more readable, and here's a tweaked version that includes those changes as well as some of my own.

diagram of apt files

This image is also available in svg and pdf formats. As before, the SVG probably won't work outside of Inkscape on Linux, and the PDF requires Acrobat Reader to display properly. If anyone knows how to convert Inkscape SVGs to PDFs that work everywhere, I'd be happy to give it a shot.

As before, I also made a slightly less flashy version that should be easier on printers, and perhaps easier on the eyes too:

simpler diagram of apt files

This image is also available in svg and pdf formats.

Posted Sun Feb 1 13:06:00 2009 Tags: apt aptitude diagram dpkg
A map of the apt and dpkg state files

[UPDATE]: I've posted a tweaked version of this diagram here.

Continuing my recent interest in making diagrams of things, this weekend's project was a visual representation of what the global state files of apt and dpkg are, what's stored in them, and how they're manipulated. As before, this includes only the core part of the system (ok, I squeezed aptitude in there too), for the simple reason that I'm not smart enough to include more stuff in the diagram without making it utterly illegible. As it is, it's already pushing the boundaries of how much you can fit on a single page of paper:

diagram of apt files

This image is also available in svg and pdf formats. Be warned, though, that the SVG might display oddly in anything but Inkscape, and the PDF only displays correctly in Acrobat Reader. If you want to print off a copy without wasting toner, if you don't have a program capable of printing the above, or if gradients offend you, you can also download a simpler version of the diagram:

simpler diagram of apt files

Also available as an svg file or a pdf file.

Unlike the last diagram I produced, this one was made in Inkscape. It took pretty much the whole weekend, most of which time I spent learning features of Inkscape that I didn't end up using. Oh well. I've come to the conclusion that Inkscape is an amazing graphics program and an utterly terrible diagram editor. Although to be fair, it seems to be one of the best, if not the best, diagram editors in Debian; also, if you slog through it and you have more graphical talent than me, the diagrams will look very good.

[EDIT]: fixed an oversight in the first version of the diagram (apt-get also writes the extended_states file).

Posted Mon Jan 26 03:30:00 2009 Tags: apt aptitude diagram
aptitude 0.5.1 released

I've released version 0.5.1 of the aptitude package manager (release notes). Once it has been installed into the archive, you can download it from Debian's experimental distribution, or manually from the aptitude package page.

WARNING: This is an unfinished piece of software. Do not install it if you want a package manager that does everything you need perfectly all the time. It is buggy and incomplete.

There are no huge user-visible changes in this release, but there are a number of smaller new features and enhancements. Click through to the release notes to see a list of what changed in this release, with screen-shots.

Posted Sat Jan 24 11:30:00 2009 Tags: aptitude release
A report from the technological present.

Today I had the occasion to do something I normally don't do: send a text message. I can report that it's a lot like writing an email. If all the bones in one of your hands are broken, that is.

Posted Thu Jan 22 21:20:00 2009 Tags: cell message phone text