Planet Larry

Often when using my computer, I will have a "project", by this I mean a directory containing an arbitrary number of files and directories. This could be a software project, or a website, or an essay, or whatever.

So when going back to a project, I might want a quick overview of what has been going on. This is what my filehistory.py script is for. It recursively orders your project files by last modified date.

The unique feature is that it mixes them all together as one stream. There is of course a tradeoff, the larger the number of files, the longer it takes, however it is quite fine for projects of up to several thousand files.

There are a few command line arguments:

• By default, the output is just the filepaths, if you use -v then it outputs timestamps as well.
• By default, it will output warnings if it finds permission errors or malformed symlinks in your project. If you want to suppress these messages, use -q.
• By default, it will output all the files within your project. If you want to specify the number of lines then use -n followed by the number. So -n 20 will output the most recently edited files.

If you followed my post about making a scripts-directory, then you can just make the Python file executable and drop the Python file into it.

It really comes into its own when you pipe it to other tools. For example, to copy the last 20 modified files to a USB stick mounted at /media/disk :

filehistory.py -q -n 20 | xargs -I x cp x /media/disk/

So feel free to check out my filehistory script. As you know I use Linux, but I think this particular script should work on Windows/Mac as well.

• Technical superiority: One product is just that much better than the rest that, while the competition is playing catch-up, the leader improves even further.


• Elegance: One product is just that much prettier or stylisher than the rest. The sold image is just that much more appealing (pretty much the reason why the wannabes all run around with Macbooks).


• Price: The one product might not be the best but it's just so cheap that it would be retarded to buy something more expensive


• Novelity: One product is just new and so different that it becomes the thing to have. It changed the rules in some respect that no one had thought about before.

1. Communication/community: Ubuntu has Jono Bacon as community manager. Then there's the forums and wikis. All in all it's like a bee hive of people all generating buzz and noise. Some of their work might be sub-par but still it's a stream of constant buzz about Ubuntu that keeps the word in the public discussion.



2. The slogan: "Linux for Human beings". I have no clue who came up with it, but that person is, in my opinion, responsible for at least 50% of the success of Ubuntu. The slogan is not geeky, it's not technical, it's talking about human beings (which we probably all are [except Harvey the invisible rabbit which is obviously reading this]) and it talks to feelings. It tells you that this is for you. No restriction, no catch.

My first Linux distribution was a very old version of Red Hat with X-Windows as a GUI. I think I was in 5th or 6th grade, and I bought it from a used book store for $10. From there, I went to Mandrake Linux 7, then Fedora, and finally to Gentoo. I was never a fan of RPM installers, as there were always issues with dependencies and uninstalling.

One of my friends, Zack had recently become a fan of Gentoo Linux. I abandoned RPM hell and became a fan of Gentoo. Despite, at the time, the best way to install Gentoo was through a stage 1 (compiling everything from start to finish), I loved it. Even if it took a day to get a usable system on a 400MHz Pentium 2, it was great.

Despite some obvious issues with dependencies in Gentoo, and many claiming Portage is just a slew of unmanageable code, I'm still in love with the speed and configuration of Gentoo. Sure, some things aren't install-and-go, but you learn more when having to configure most programs yourself.

Recently, I've been installing the latest version of Ubuntu on many of the machines I use to benchmark hardware for BIOSLEVEL. I can't say I'm a fan. I'll agree that it's fantastic that Ubuntu developers have put so much time into developing and improving many applications and daemons for desktop use, but it feels slow.

Why run native 586 code on a 686+ CPU? Why install software that doesn't take advantage of a CPU's extensions? To be completely honest, using the same hardware, a Gentoo System with Compiz Fusion running feels more responsive than an Ubuntu system without any desktop effects enabled.

While much of my testing will be done with Ubuntu, my primary Linux environment for regular usage remains Gentoo.

After first seeing it mentioned in irc last night, i took a look at former gentoo dev, kloeri’s current project called Exherbo which seems very much to be Gentoo 2.0. So far its got a number of former Gentoo dev’s onboard, including Ciaranm, who, regardless of his sometimes lacking people skills, *is* a very competent programmer. It seems their big aim is creating a gentoo-esque distro thats been built with all the experience available from working on Gentoo for years coupled with a desire to build a forward looking distro that shuns the massive amount of legacy code that other linux distros have picked up over the years.

Surprisigly though, what would appear to have been a low-key announcement in kloeris blog, has somehow been picked up by both slashdot and more surprisingly, TheRegister. While Slashdot takes the usual fairly negative view which it seems to have towards anything even *vaguely* linked to Gentoo. It was nice to read the Reg piece though and not see a totally negative or sarcastic article, they almost sound positive about exherbo!

I have to say im intrigued at the very least and am interested to see where this goes, especially as im a quiet supporter of Paludis (note to anyone reading this, im supporter not a fanboy! ), and have liked how it has forced portage to actually document how everything works via the EAPI’s, which makes it easier to figure out if a certain behaviour is a bug or actually a feature!

While ive no intention to actually try and install Exherbo at this point (as im just a mere user!), i do intend to watch and see what it grows into, and whether it can fulfil its rather lofty aims.

Nothing horribly new to report. I added a few new pictures to my Flickr feed at the bottom of the page. I'm hoping to make more use of my camera this summer, but I haven't figured out what for yet. I'm just going to have to remember to bring it with me more often.

I'm trying to get back to blogging on a regular basis. It's hard to remember to write here, especially when I'm almost always writing a review for BIOSLEVEL. What do I do if and when we run out of products to review at BIOSLEVEL? Obviously we'll be able to write Linux-related articles during any down time, but I'd also like to focus a little more here. My traffic is way way down since October of last year.

Expect more updates, a new layout, and other cool features soon.

Sometimes you have to slow down your mail delivery for some destinations or you might want to push out more mails for a certain server as it’s inside your LAN without being unkind to other people. Here’s a way of doing it:

The main.cf file of our postfix installation is the place where we define a new transport map and an according speed limit. As I am slowing down things here, the name of our test transport will be ’snailmail’ - so here we go:

— main.cf

transport_maps = hash:/etc/postfix/transport
snailmail_destination_concurrency_limit = 1

The Transport map file itself now contains the domain(s) we’re changing the limits for by assigning the new transport service:

— /etc/postfix/transport

domain.com snailmail:

Finally we’re hacking our master.cf file for defining our snailmail service to be able to make use of it:
— master.cf

snailmail unix - - n - 5 smtp

The results we’re expecting are now, that all mail traffic regarding domain.com (we’ve used that in the transport map file - remember?) is using our new delivery service. But wait - we haven’t generated the hash file for the transport map. Running postmap /etc/postfix/transport does so. Now we’re set up for the restart.

Believe it or not - we’re done - and it wasn’t even complicated.

Last summer, at GUADEC here in Birmingham, the cool young dudes first integrated Webkit as an experimental toolkit for the Epiphany web browser. Now they are working to make WebKit as the main backend for Epiphany. It is a very small team indeed, but their aim is to finish the migration by GNOME 2.26 in March 2009; if we are lucky they may have finished by GNOME 2.24 this September. The old branch with the Mozilla gecko back end is very stable and will be released as 2.24 if the Webkit migration is not finished.

This is exciting for a number of reasons, firstly we will have a fast and light, GNOME integrated browser. Secondly, and perhaps more importantly, if Webkit becomes a default GNOME library then the same browser toolkit is available in KDE desktops, GNOME desktops and Apple OS X desktops, allowing a new generation of rich client web mashup applications.

I tried Epiphany with the Webkit backend last year, it was pretty raw then. The centre of the window worked as a web browser, but it ignored the toolbars and buttons at the top of the application. So how is it 10 months later?

Pretty good I say. Of course, it is still early days, and they need a lot of help removing the old unneeded code and binding the interface to Webkit. If you know C and want to get in to an interesting open source project, this might be a good project to join as they are basically restarting a lot of it.

My adventure installing Epiphany

I wish I had a spare Gentoo machine that I could use for this, but they are all busy with important things. On Gentoo they have made ebuilds for it in the overlays that take the latest version down.

So I used a spare Ubuntu machine, I started by getting rid of anything to do with WebKit or epiphany that has been installed via Ubuntu, at time of writing these are all too old.

Then I checked out all the development versions I need:

svn co http://svn.gnome.org/svn/libsoup/trunk libsoup
svn co http://svn.gnome.org/svn/epiphany/trunk epiphany
svn co http://svn.webkit.org/repository/webkit/trunk webkit

The last took a while as it checks out the complete source tree for every platform, I didn't actually need a lot of this, so another time, if time or badwidth is a concern then I might want to try to work out a more specific checkout command.

Then while that is running, I got the other required dependencies.

sudo apt-get install libicu-dev libxslt1-dev libsqlite3-dev libjpeg62-dev \
libpng12-dev gperf bison libcurl4-gnutls-dev flex gtk-doc-tools

Then I went into the libsoup directory and compiled it:

cd libsoup
./autogen.sh
make
sudo make install

Then webkit had finished downloading by the time libsoup is done. So I went into the Webkit directory and tried to compile it with this:

cd ../webkit
./autogen.sh --prefix=/usr/local --enable-svg-experimental --with-font-backend=pango \
--with-http-backend=soup
make
sudo make install

This didn't work, so (after make distclean), I googled and tried the method used by guy called Peter Upfold. This uses QT's qmake to build WebKit-gtk, bit of a contradiction, but it works at least:

export PREFIX=/usr/local
./WebKitTools/Scripts/build-webkit --qmakearg=WEBKIT_INC_DIR=$PREFIX/include/WebKit \
--qmakearg=WEBKIT_LIB_DIR=$PREFIX/lib --gtk --qmake=qmake-qt4
cd WebKitBuild/Release/
sudo make install

Then I waited a while again.

Next update the shared library cache:

sudo ldconfig

Then I did epiphany, this was pretty quick,

cd ../epiphany
./autogen.sh --prefix=/usr/local
make
sudo make install

Then epiphany worked like a charm.

Acid2 Test, no problem:

In the Acid3 Test, Firefox 3 only scored 71. What about Epiphany, well still no problem:

Anyhow, should be interesting what they come up with in the first Epiphany WebKit release.

Please do Digg or StumbleUpon this article,

Writing text

There are many ways to markup plain text. There is SGML and its off-shoot HTML. Later came XML, a replacement for SGML, a general-purpose way of making markup languages, one of which is XHTML, which is what you are using to view this page now.

There is also LaTeX, which is a markup used for typesetting in various contexts, especially in academic fields where complex mathematical equations are used.

With all these formats, the marked up document is plain text, and then rendered into its final form, that form may be paper, or it may be a browser window. The downside to these formats is that the marked up plain text document is ugly - full of random angle brackets and letters and words entered into the text that you are trying to read. While after using HTML for a decade I can more or less block it out, the plain text form is far less readable because of the markup.

Enter reStructuredText

reStructuredText is a markup language that does things differently. It uses more natural looking markup with a focus on readability so reStructuredText can be easily read and shared in its plain text form. You can then automatically processes the document into XML (called Docutils XML), and from there it can go into XHTML, LaTeX, OpenDocument, Docbook, or any other format that you can get to from XML. So readable, yet maps to XML in a well thought out way.

If you are going to abbreviate reStructuredText, then use 'rst', don't use 'rest' because that gets confusing with Representational State Transfer, a completely different thing altogether.

I have jumped in the deep-end and I am using reStructuredText to write my posts in this new version of the site, and it is an optional markup format for people leaving their opinions in the discussion/comments section (you don't need to use it to leave a comment).

While I have just got into it, reStructuredText has been around for several years, and has been most prominently used for creating Python documentation, however, it is a general purpose and extensible markup language that can be used in many different contexts.

Some people use it to create text for the web or for standalone documents. It can also be used wherever you might have made a one-off markup format, such as on a web forum, wiki or comments section.

reStructuredText everywhere

The reference implementation of a reStructuredText parser is called Docutils which is written in Python (but this can be used from many languages). Docutils is very easy to use in your application to process reStructuredText.

If you do not have docutils on your system already then it is available from all Linux/BSD package managers. For Windows, follow the installation-instructions.

There are also third-party parsers in Perl, Java, Haskell and probably more that I have not heard of. There are also plugins for text editors and plugins for lots of web frameworks, content management systems and web log software.

As explained in the last post, Django comes with reStructuredText support (bindings to docutils) out of the box. Lastly, there are also lots of cute reStructuredText website generators that allow you generate a website from a set of reStructuredText files. For example, Michael's rest2web, Sphinx,which is used to create the new Python 2.6/Python 3000 documentation, and Damien Baty's Soho.

Very quick reStructuredText (permalink to markup)

Okay, enough blab, lets get into it. There is very comprehensive set of markup available, but I will just cover a few basic ones that are always useful.

There is no need for paragraph tags or linebreak tags; to make a paragraph, you just make a paragraph, to break a line you just break a line.

To mark something with emphasis (i.e. italic), you use a star around the word or pharse: *emphasised text* becomes emphasised text.

Two stars are for strong emphasis (i.e. bold). Like so:

Hyperlinks

To make a hyperlink, you just start the text with http:// . So http://commandline.org.uk/ becomes: http://commandline.org.uk/

If you want to do a named hyperlink, then you need two parts. First, you put an underscore after the name, for example Zeth_ will become Zeth

Then you need a target to go with that name. You specify the target with two dots, a space and an underscore:

.. _Zeth: http://commandline.org.uk/

The line with the target can go anywhere in the document. For example, some people put the target line straight after the paragraph that has the name in. However, in a long document, it is often tidier to put all the targets at the end.

Lists

To make an unordered list (i.e. a bullet list), just start the line with a star, plus or hyphen, for example:

* Green eggs
* Ham

Becomes:

• Green eggs
• Ham

For a numerated list, just start the line with the number and a dot, for example:

1. Green eggs
2. Ham

Becomes:

1. Green eggs
2. Ham

I could go on, but I think you get the idea. By default, the core of reStructuredText is broadly feature equivalent to HTML. However, reStructuredText is very flexible and allows you to go beyond that core, as we shall look at next.

Directives

To markup a piece of more advanced functionality, you use a 'directive'. Directives are where reStructuredText shines over similar markup languages. Directives are extensions to the main markup, some directives are included by default (analogous to a standard library), others can be added by yourself or from getting them from the web or wherever. I will give an example of a standard directive, then an example of a third-party directive.

A directive has four parts. First you have some markup declaring the directive itself (two dots), second comes the directive name, third is some markup that says here is the content (two colons), lastly there is the content.

Two dots, the name, two colons, the content.

.. name :: content

So to display an image, you use the image directive.

You write two dots, a space, the word "image", two colons and then the URL:

.. image:: \http://commandline.org.uk/images/whokilledtux.png

Becomes:

To share source code, firstly you start with the source code directive, this is two dots, the directive name (source code), followed by two dots; secondly, we have the name of the language the source code is in. Then we leave a blank line, then we write the source code indented (preferably with four spaces ;-)

So for example:

.. sourcecode:: python

    import os
    for i in os.uname():
        print i

Will become:

import os
for i in os.uname():
    print i

Here is the list of supported languages on this site.

There you go, if you made it this far then you can use ReStructuredText, give it a go! You can start by leaving a comment in ReStructuredText format!

So my folks Windows XP box started having difficulty booting. Most of the time it would go to boot and suddenly you'd be back at the BIOS, then the "Windows didn't shutdown properly last time" menu and then once you selected "boot normally" the cycle would begin again. Usually eventually you'd get lucky and Windows would boot (but sometimes they gave up first). Not good. This started a while ago and seemed to get worse (I think, I never use the box so I don't know). Anyways, eventually they figured (hoped actually because otherwise it was hardware failure) that their XP install was corrupted and asked me to reinstall Windows for them.

They gave me the CD that came with the box and I loaded it up and first things first, no harddrive detected. Yep, that's right, no harddrive. To be fair, XP is old, it's from 2001, and pretty much all computers come with SATA harddrives now, which didn't even exist in 2001, so it's not entirely fair to expect XP to support them. This is one of the downsides of only doing releases every 6 years. Being not terribly invested in this "install XP" project I tossed my hands up and said it's a no go and offered Linux as an alternative. My Dad was definitely interested in Linux but wasn't sure it would be all there for his business software, so I said, "no fear, let's just do a trial run, and if it doesn't work, no worries, you can, you know, always buy Vista". The thought of having to spend money to make a computer that did work, work again, for no good reason vs. the low commitment of a trail run sold him on the project.

So I burned a copy of Hardy Heron, the newest Version of Ubuntu, released just last month. It loaded up and recognized all the hardware and installed, no trouble. I'm old fashioned so I opted for a separate root and home partition, but other than that, a vanilla install. Then we booted up and everything just worked out of the box. Even the printer. So I loaded all their data on from one of my boxs from whence it had been backuped to and let them at it (after a few tweaks, like my mom's desktop).

The results?

Mom loved it. For her, it turns out the killer feature was desktop icon stretching. She really liked the larger icons. And of course, no difficulty using it for her normal uses which are nearly entirely Internet look ups. The other big hit was Eye of Gnome, the photo viewer. Last year we had about 1600 of my mom's parents family slides turned into digital photos (at no small cost) and the stock Gnome photo viewer is perfect and intuitive enough for her to use and enjoy (the newly added left right buttons to scroll a directory are crucial here, so good work guys).

My moved out sister dropped by and gave it a try, and she too was sold. For her, the large stock puzzle/games selection was the seller. It worked just fine for her net usage, but the large selection of games got her attention and kept it just on introduction for over half an hour and she only explored a few of the games in that time. She'll be back. She also was amused by compiz eye candy.

As for my Dad? Sadly neither of his business apps worked under Wine. Maximizer 8 failed to install (a little sad since it's reasonably popular and not so new) and D--------, which is from what I can gather an in house piece of software from his company, also failed to install. A few tales from his trials installing it under Windows were horrific thought, like for the first time install you are required to turn off the firewall. 0_o In house software can be scary and shoddy stuff.

So Linux failed to satisfy everyone of my family's needs, so back to Windows. I was disappointed but resigned. And then came mom to the rescue. She chimed in she didn't want to go back. She liked her new Ubuntu desktop! So that was all I needed to float the idea of dual-boot. And thankfully we had two partitions already. Dad was sold too. He does want to learn more about Linux and so looked forward to the opportunity, but also needs his business software.

So I popped in a Gentoo LiveCD, because there really is no better system recovery and maintenance CD than a Gentoo LiveCD. I copied the Ubuntu root over to the second partition, moved /home and edited fstab and grub/menu.lst and reinstalled grub (from within the Ubuntu chroot because the one thing that particular LiveCD was 'missing' was Grub) and then changed the partition type of the first partition from Linux to NTFS and volia, we were ready to go.

Then back to Ubuntu because the Gnome Device Manager is actually full of awesome and gave me all the hardware information I needed so that I could get the appropriate drivers for Windows (remember when it was the reverse and it was crucial to go into Windows System Manager to get all the hardware info so Linux could install?). Then off to the net to find the Sata drivers, and then I tossed those onto a floppy disk and tried the XP install again. Sure enough, after loading the drivers from the disk, XP install found the harddrive and was happy enough to install onto the partition I had marked for it.

Then it booted into Windows XP SP1 glory. Which was surprisingly not glorious because it had no drivers so we had a 640x480 low colour screen with no sound. And no programs. Dad and I spent the rest of that day and the next few after that running Windows Update and downloading and installing drivers and programs manually from the net. The Windows Update website actually pretty much continually stalled out and failed when it came time to get service pack 2 which was disconcerting but by then the update manager had been installed and it was able to get and install SP2 for us. Shortly after which my dad horrifyingly noticed the Windows booting bug was back! We were horrified. All this work and no fix? Was it hardware failure? But Ubuntu was still booting fine. So we kept at it and apparently now with SP3 installed it's gone back to booting normally.

So yes, that means Microsoft in one of their post SP2 updates introduced some bug that rendered our computer nearly unbootable, which drove us to Linux which was brain dead easy to install. And then when we discovered Linux didn't quite fulfill all our needs, they made the migration back to XP as painful and full of near hopeless despair as possible.

So thank you Microsoft for driving my family to try Linux because they've found they like it, and thank you after that for highlighting exactly why we don't want to be hooked on Windows. It's been educational for my family, and now ever computer in my house has Linux installed on it and my family is interested in learning about it! And I owe it all to Microsoft.

(It should be pointed out that my dad's work still has a ban on IE7 and Vista because it doesn't work with their software yet which is a mark against both Microsoft and Vista, and my dad's company's computer department because, lets face it, it is the future and it has been out for over a year. But this did mean Vista really wasn't a viable option on the fact that it would have cost us more money and it wouldn't have accomplished the one goal we needed it for, which was running my dad's business software. XP was our only option.)

Firefox 3.0 RC1 has now been released, with some fixes and changes from Beta 5. I have not looked at the changelog yet, so I do not know what has changed. I notice a few (minor) graphical differences.

I also want to report that the Acid 3 test still ends on 71/100, as it did in Beta 5. (See my previous review of Beta 4 and Beta 5) Since the trees are locked down, we can expect that Firefox 3 RTM will not correctly display the Acid 3 test.

Vor einiger sehr langer Zeit habe ich mal ueber “Saving SSH options for specific host” geschrieben. Fuer mich ist die ~/.ssh/config nicht mehr wegzudenken. Bloed ist natuerlich, wenn ich diese Datei auf meinem Rechner zuhause habe, aber auf der Arbeit etc. auch haben moechte. Auf jeden Rechner auf dem ich die Datei haben will raufkopieren, nach jedem Update ueberall neu. Ganz schoen bloed und nervig.

Viel schoener waere es fuer mich, die Datei an einem Ort zu haben, wo sie immer aktuell ist, wo ich immer rankomme, und von wo ich sie mir schnell ziehen kann. Was bietet sich da mehr an als mein Webserver?! Auf der bash ist ein “wget myServer.tld/myFile -O ~/.ssh/config” sehr schnell geschrieben

Meine ~/.ssh/config passe ich hauptsaechlich zuhause auf meinem Rechner an. Von daher reicht fuer mich eine simple Moeglichkeit diese Datei schnell und unkompliziert von da auf meinen Webserver zu bekommen. Am besten automatisch, dass ich mich nicht drum kuemmern muss. Hier meine Loesung:

1. Auf dem Webserver einen rsync-User angelegt und ihn der Gruppe “www-data” hinzugefuegt, so dass er in meine htdocs schreiben darf , und in die Gruppe “ssh-allow” hinzugefuegt, so dass er sich auch via SSH einloggen darf (in der /etc/ssh/sshd_config: AllowGroups ssh-allow)
   (adduser myRemoteRsyncUser && adduser myRemoteRsyncUser www-data && adduser myRemoteRsyncUser ssh-allow)
2. Auf meinem Desktop-PC einen User angelegt und fuer diesen einen ssh-key generiert
   (adduser myLocalRsyncUser && su - myLocalRsyncUser && ssh-keygen -t dsa -f .ssh/myKey)
3. Die myKey.pub auf dem Websever in die entsprechende authorized_keys gepackt
   (scp /home/myLocalRsyncUser/.ssh/myKey.pub myRemoteRsyncUser@myServer.tld: -> ssh myRemoteRsyncUser@myServer.tld -> cat myKey.pub >> ./ssh/authorized_keys )
4. Auf meinem lokalen Rechner rsync Befehl zum kopieren der config beim herunterfahren des Rechners automatisch ausfuehren lassen
   (in der /etc/conf.d/local.stop [Gentoo] den Befehl su myLocalRsyncUser -c “rsync -avu -e ssh /path/to/.ssh/config myRemoteRsyncUser@myServer.tld:/var/www/myRemoteSSHConfigFilename”)

Voila. Beim runterfahren wird nun die Datei automatisch auf meinen Webserver kopiert falls sich diese geaendert hat EEEENDLICH!!

dbr left a comment on this site:

I concur with the other two comments - this is one of the nicer blog'y site layouts I've seen. The comment system is also actually pleasant to use, unlike every single other one I've (not)-used \o/

—dbr (May 16, 2008)

Thanks for the compliment. I have talked before about my philosophy towards forms:

I do not make want to make it like Indiana Jones and the Last Crusade just to make a comment: roll under a flying saw, spell something in Hebrew and then take a flying leap into the abyss. Furthermore moderation of spam is my problem, not yours.

—Zeth on comment-spam (3 July 2007)

dbr continues:

One slight bug, you need to enter two backslashes to make it visible.

—dbr (May 16, 2008)

That is not a bug that is a feature! That is the escaping mechanism of ReStructuredText (as well as Python and lots of other languages). The comments form does not allow HTML but it allows ReStructuredText, as I explain below.

Ryan also left a comment:

For your blog post model, what did you do for entering posts? Do you still use the default admin interface, or did you make your own views for posting and whatnot? I haven't looked into it much, but does Django automatically include much in the way of wysiwyg text editors for text fields?

—Ryan (May 15, 2008)

Answering your questions in reverse order:

• Wysiwyg text editors

There is no default Django WYSIWYG text field, but some people use Javascript components such as TinyMCE which slots in nicely (instructions).

• Default admin interface vs own views

I have the admin interface as one way to enter posts, but I also made a simple command-line tool for entering posts, I also made some scripts for importing all my old posts from Pyblosxom.

I did create one admin view, I overrode the 'delete comment' view to create a button that adds the IP addresses of deleted spam comments to a block list.

At some point I will make a 'Preview' view and button. At the moment I can save posts as drafts but not view drafts in the site template before it goes live.

• What did you do for entering posts?

I write a new post in a real text editor such as Emacs or gedit in ReStructuredText format, which I can then either paste into the admin interface, or use my little script to squirt them into Django.

Markup in Django

'contrib' forms the 'standard library' of Django. One of these packages is called "markup". It provides filters for three markup languages: ReStructured Text, Markdown and Textile.

I chose to use ReStructured Text because using that to write my new posts because I like the format and getting confident with ReStructured Text will be useful in lots of other contexts also.

To use it in Django, just add django.contrib.markup to the INSTALLED_APPS list in settings.py. In a template, you can load it and use it like this:

{% load markup %}
{{ comment.comment|restructuredtext }}

In the next post I will explain how to write in ReStructured Text,

I got it from my sister’s digicam. Thanks to Leerz for the walkthrough. This worm is really annoying especially if you are more comfortable doing stuffs in the console.

1. Check for any bar311.exe, Autorun.inf, pc-off.bat files in mounted drives.
2. Delete if found.
3. Edit the following entries in the registry.

HKCU\Software\Microsoft\Command Processor\"Autorun"
HKLM\Software\Microsoft\Command Processor\"Autorun"

Alternatively, you can download Noob.Killer, run it, then watch and learn.

I symlinked my .vimrc to my local mirror of my website so that every time I rsync it (which is pretty often) it'll automatically update my the vimrc on this server. So that should be fun. I experiment with things in there all the time so at any given moment there are likely to be things horribly broken, but maybe someone can use some of it.

This mirror of Ciaran McCreesh's vimrc which I found linked from here (edit: updated version here) has lots of good stuff in it. In particular using :set listchars to display tabs and trailing whitespace as some funky Unicode characters is a really good idea. When I first tried that good idea I realized my favorite font ProggySquare didn't properly display most Unicode characters, which was part of my motivation to switch to Terminus. (That, and those tiny Proggy fonts aren't so great on a 1920x1200 monitor.)

After a long time putting it off, I finally hunkered down one day and figured out how the heck Vim script works. The difference between statements and expressions in Vim script language confused me for a while, which goes to show that I'm far too used to Ruby and Lisp where almost everything or everything returns a value as an expression. Vim expects expressions in certain places and colon-prefixed commands in others. But then there's normal and eval and execute and "= some of which let you do things from one mode in another mode if you mix and match them. But I think I've gotten a handle on it now.

Today I came across Limp which is a recent attempt to get Lisp to work well with Vim. It seems quite new and buggy and had dependencies on things I had to guess until I was able to install it (like rlwrap), but I still was excited about it. Until I realized that it's just a wrapper around GNU screen. SBCL runs separately, and some keystrokes send stuff from Vim to screen, but that's about it. Nice, but not nearly as nice as SLIME in Emacs. So that disappointed me. In the back of my mind I always think about how Vim could possibly be integrated with Lisp like SLIME does but I don't see any good way. Vim doesn't have the ability to embed shells like Emacs and it doesn't look like it will gain that ability any time soon. Ah well.

By some people’s logic, this how the economy is supposed to work:

1. New companies emerge all the time.
2. No companies ever close.
3. Consumers always buy the cheaper and better products.
4. No products ever become obsoleted and force the company to go out of business.

Sounds perfectly reasonable, doesn’t it?

When a new company opens in a town and provides a thousand new jobs, there’s noone protesting that this is unfair, we didn’t do anything to deserve this, that you can’t just suddenly create new jobs out of nothing, there aren’t people complaining that it’s not right, we didn’t get jobs at the new company. No, people accept it with great fanfare. Great, the economy is growing, our town will prosper! People will have more money, there’ll be less unemployment, we’ll be able to afford a higher standard of living.

And yet when, after 40 years, the company goes out of business or moves their production to a cheaper location, people say this is outrageous, 1000 jobs will be lost! There’s anger and pandemonium, how can they do this to us, we were loyal to the company for 40 years. People appeal to some sort of higher ethical body; you can’t take our livelihood away, what are we going to do with ourselves? And the town itself, which never had much industry, and really just had that one company that employed everyone in town, starts to regress. People move out in search of jobs, young people leave and don’t come back, noone moves in because there’s no local economy.

It’s a sensitive topic. Losing your livelihood is one of the more challenging life situations. But before you start screaming that it’s those damn crooked politicians and those greedy executives that have stolen your life, take a moment to think about why you had that job in the first place. In fact, let’s start with the basics: what does it mean to have a job?

It means that you are producing a product or offering a service that someone is willing to buy. It does not mean any of these things:

1. Someone is being nice to you.
2. You deserve this.
3. You’re going to keep your job because you’ve been loyal to the company.

If you actually believed any of that then you were under a complete misapprehension. Sure, sentimental concerns do come into it sometimes, like the boss’s son getting a summer job because he’s family. But in the long run, the only thing that matters is the economic reality.

If you think that’s a raw deal, think about this. Most artists aren’t wealthy, in fact most artists are struggling to get enough work to live on. A painter may think that he deserves to live a decent life as a painter, but if noone is willing to buy his work, well he’s not going to. Is that unfair? No, it isn’t, because if he’s not producing anything of value, why should anyone have to pay to keep him in business? So if a painter can’t do the job he wants to, why should it be any different for you making shoes, or catching fish, or whatever it is you do?

There used to be people working in elevators who would press the buttons. We don’t need them anymore. Shepherds aren’t in great demand either. Neither are telegraph operators. These professions have all be superseded and they’re not coming back. Many others still exist, but have been moved to where production is cheaper, like textiles.

It’s always a turbulent transition, you can be sure of that. We don’t have hunters anymore, we have domesticated animals now, no need to chase them in the woods. Think about how many hunters were out of work when this happened. But what should they have done, lynch the guy who came up with the idea of keeping animals on the property? Compared to the hunters’ relatively narrow interests (although there were many of them), domestic animals were very beneficial to the village. For one thing, you didn’t wonder where dinner was coming from, the animals were right there. So should the villagers have discarded this new idea just to make sure the hunters could keep their jobs?

I’ve got news for you. The very same thing you’re protesting against, your job being taken away, you’re doing the same thing to people everyday. That’s right, you’re not so innocent yourself. Have you ever bought a car from a different automaker, because it was cheaper? Did you ever buy peaches from Spain instead of domestic apples? Well, I’m sure it must have been a very gruelling decision for you, right? I mean to think that you could be putting car makers and farmers out of business because you’re not buying their products, that’s a tough one to swallow.

And what did you get out of it? You could afford to buy more things, because the new products were cheaper. And they didn’t break as quick, so you could use them longer. And they had some functions that the old products didn’t have, which made you happy. And just as this was happening, the old companies that couldn’t stay competitive were going out of business one by one, people were losing their jobs. But hey, you got a pretty good deal out of it, didn’t you?

Here’s what it comes down to. You’re not entitled to your job. You’ll only have it for as long as people are willing to buy your product. And even if you’ve had it for 40 years, that doesn’t mean the global market won’t make it obsolete tomorrow. There was a demand for your product, now there isn’t. You didn’t do anything to deserve getting it, and you didn’t do anything to deserve losing it.

Debian and Ubuntu are not random enough

There is has been a bug in random number generator on Debian (from Etch onwards) or Ubuntu (Feisty onwards). You should already have a security update for the number generator. If you have not yet accepted the update then do so.

The Debian and Ubuntu distributions have even made the warning pop up on user's screens. Just apply the update, see below, and replace your keys and you are done.

If you are on an operating system that has apt-get then you probably want to look at what is going on. If you are on Gentoo or another distribution then for now you can just smile quietly to yourself.

Bugs in the number generator are bad mojo because there are less combinations, depending on the severity of the bug, it makes a brute-force attack go from almost completely impossible, to either still very improbable down to theoretically possible [update: or in the Debian case, quite possible if certain things are known about the target system ].

SSH is often the first point of entry to a Linux machine (but not the last line of defense) so bugs here are particularly prominent.

However, lets not have a panic attack about it. There are a dozen ways to get into someone's machine. In proprietary software land, they probably would have just ignored this kind of theoretical exploit to keep their marketing team happy. For a proprietary software company, still existing in five years time is a higher priority than a theoretical brute-force attack using hardware of the future. Free/Open Source Software forces good security, your dirty laundry is washed in public. Today's theoretically possible attacks are tomorrow's malware. If we ignore all these things then we end up with an operating system akin to Windows.

If you are on Debian or Ubuntu, the security updates means that any new keys will be to the desired level of randomness, but your existing ones need to be ditched. The update manager does not do this for you in case you are then left unable to log into remote systems.

Swapping out your SSH keys

Cleaning this up is easy. Run:

sudo ssh-vulnkey -a

This outputs a line for each SSH key on your system:

Not blacklisted: 2048 <key fingerprint> <filename>
Not blacklisted: 1024 <key fingerprint> <filename>
COMPROMISED: 2048 <key fingerprint> <filename>
COMPROMISED: 2048 <key fingerprint> <filename>
Not blacklisted: 2048 <key fingerprint> <filename>

So the ones that came from Gentoo or another Linux distribution are okay as far as we know. The two Ubuntu ones we must delete or archive somewhere else. To delete the keys use rm.

Now we might like to generate replacements, so we can still use SSH as before:

ssh-keygen

So to make this simpler, one of the lines was: COMPROMISED: 2048 49:37:38:f4:86:28:ac:b1:7e:a6:df:bd:1d:a4:da:81 /home/warrior/.ssh/id_rsa.pub

That is the public key of the local machine. So we get rid of it:

rm /home/warrior/.ssh/id_rsa rm /home/warrior/.ssh/id_rsa.pub

Now we want a new one:

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/warrior/.ssh/id_rsa):

The brackets mean that is the default, so I press enter.

Next it finds an existing key (the private half of the existing keypair);

/home/warrior/.ssh/id_rsa already exists.
Overwrite (y/n)?

We want to overwrite it so we say we yes.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/warrior/.ssh/id_rsa.
Your public key has been saved in /home/warrior/.ssh/id_rsa.pub.

Now we are done.

1. Build a cheap and ultra portable machine that looks cool but that does not have everything in. Don't compromise too much to get millions of hardware features. A 10" display with 1000x600-ish resolution, a processor that gets the browser running at an OK speed, a very good touchpad. 1GB of RAM so you don't need swap to run. Only very little solid state harddrive.


2. Set up a service that is connected into every aspect of the machine: Allow people to have their "home directory" on your server for backups. Let them pay for it. Allow people to buy all kinds of neat services. Don't get greedy by trying to lock them into one operating system or platform but offer benefits: Build a service that allows people to have their pidgin conversation logs on your server and guarantee backups. Guarantee encryption for their personal data. Do similar stuff for other software (while still keeping the base system free of charge and updates cost-free, too).

If you have signed up to my site's RSS feed, please update to http://commandline.org.uk/feeds/full/ as soon as possible. Then you will be sure not to miss any of my exciting adventures.

This main feed should work the same as the old one, however, I have provided a number of extra feed options. if that is your bag.

Update: I am hoping all the old feeds should do something now.

or how not to serve robots.txt with PyBlosxom

So as you may have noticed, I moved this site from PyBlosxom to Django, which depending on your perspective is a fabulous thing to do or is tantamount to treason on the high seas. I will explain more about that later.

Old links to the site should, in the main, still work hopefully as I have done some regular expressions jujitsu which should hopefully send everyone to where they were supposed to be going.

However, some posts and comments will have their formatting up the creek. So I want the old version of the site to be available (at archive.commandline.org.uk) for a while longer.

Because the archived version is deprecated and on the way out, I do not want the search engines to index it. Therefore I needed to make a robots.txt file for that subdomain excluding them from indexing it.

The last version of this site, like many dynamic sites, is composed of a number of layers, part of which was a lot of my own nonsense code doing various things. Ignoring that, when a request for a packet came in it would go to WSGI which would then pass the request on to PyBlosxom which was at the bottom of it all doing the hard work.

To deploy it properly, one would normally put Apache at the front as well, but I never got around to that. In theory this is a bad thing to do. But in practice it worked really well without the huge and complicated server that is Apache in the mix. It actually ran fine for a year without stopping, and blazing fast too; if it also confused a few comment spam bots then all the merrier.

So I tried putting Apache into the mix so I could use a Location directive to direct /robots.txt to somewhere with the robots.txt file, but no joy, this would have required doing a lot of what I never got around to before.

So I then looked into how the test server was deploying the site, thinking that I could do some kind of smart regular expressions type matching like in Django or Pylons. But nope.

Hack for the win

So the next step down is PyBlosxom, so I looked out of chance in Pyblosxom/pyblosxom.py and saw the following:

def __call__(self, env, start_response):
    """
    Runs the WSGI app.
    """
    # ensure that PATH_INFO exists. a few plugins break if this is
    # missing.
    if "PATH_INFO" not in env:
        env["PATH_INFO"] = ""

    p = PyBlosxom(self.config, env)
    p.run()

    pyresponse = p.getResponse()
    start_response(pyresponse.status, list(pyresponse.headers.items()))
    pyresponse.seek(0)
    return [pyresponse.read()]

Bingo! As soon as I saw it, I just somehow, on auto pilot, typed in the following lines before the line p = PyBlosxom(self.config, env):

if env["PATH_INFO"] == "/robots.txt":
    start_response('200 OK', [('Content-type','text/plain')])
    return ["""User-agent: * \nDisallow: /"""]

And unbelievably it worked. What I had subconsciously done was to see that we have some kind of string referred to by env["PATH_INFO"]. Then further on we have an object called start_response which is being passed a status and some headers. Then we are returning the response.

I was kidding around more than anything so I just replaced everything I didn't know about with reasonable looking constants (you will know these well if you have ever done Python CGI programming).

I am sure there are millions of far better ways to serve robots.txt with PyBlosxom. But this hack works for me until I no longer need the old site anymore.

I took a few minutes today and cleaned up a few small bugs on Planet Larry and friends.

One thing I get asked for every now and then is if I have archives of past posts. Well, I do now. I just copy the HTML file of the last post to $date one each run. A simple and unelegant solution. I’ll be doing the same thing for Planet Gentoo soon.

Also, fixed the FeedBurner link on the main page — I didn’t even realize it was broken. While I was at it, I created one for Universe as well.

On the packages website, I finally fixed it so you can search against just packages again. That’s been annoying me for a while. By default the search is way too wide, I think. It will search the full atom, the package description and the package name. I have to do the package name twice because of regular expressions (starting with, ending with, exact matches, etc.). And there’s still no simple way to search for packages containing multiple words, which is also an annoying little bug. Advanced searches for GPNL and Packages has been something I’ve wanted to do for a long time, but have been putting off since I started the projects. Sheesh. Every time I sit down and start to poke at it, though, I realize just how big a beast it is, based on what I’d like to accomplish. I really need something for the interim, though.

Anyway, I better quit before this post gets any more boring. One last thing — we can use more users who are Gentoo users and have a blog on Planet Larry. Just drop me an e-mail and I’ll get you setup.

Mughlai and Jalfrezi are better than gruel

A few hundred years ago, the great mass of the British poor ate gruel, while the middle class ate bland over-boiled vegetables. However, as a naval people, the British went out around the world with their empire, and brought food and foreign chefs back with them. Now British people can and do eat food originating from the whole world. Not just the rich, normal working class people will regularly eat curry, Cantonese food, kebabs and so on that would not have been imaginable before.

If I tried to explain this to a 14th Century peasant eating his gruel, then he probably would just ignore me, having no idea what I am talking about. At best he might look at me strangely, and then go back to his life of gruel.

Windows is the gruel of the digital world. There are certain people that understand this fact and have moved on to greater and better things, however most people take what they are given and swallow it as best they can.

What is an operating system?

If you don't know what a computer program is, think about cooking. In cooking you have tools, such as an oven and a blender and you have ingredients such as vegetables and meat. The recipe allows you to use the tools to take your food (ingredients) and turn them into to other types of food (meals).

In computers, you have tools (called hardware) such as a DVD player, a screen, a keyboard, a hard drive and so on. The computer program is the mathematical recipe that allows you to take your data (text, pictures, videos, etc) and then do various things with it. So for example, you might use a computer program that takes a song from the hard-drive and then plays it out of the speakers.

An operating system is a set of computer programs that makes your computer hardware do the basic things (put text on the screen, play sound, and so on). You might then run other programs to do more advanced things. It is like your recipe book.

People who are in to cooking will try out lots of recipe books, and in doing so, they do not starve because they have bought a different recipe book, indeed the opposite happens, they cook so much that they do not have time to eat it all themselves so have to give food to their family and neighbours.

By changing recipe book, they don't suddenly become unable to cook, they in fact get better as they move on to better and more advanced recipe books.

If you turn on you computer and see Windows, then Windows is your current 'operating system'. If you have only ever used Windows, don't you think it is time to give up the gruel and try a new recipe book?

I think so, and if you think this way then you have come to the right place!

About Windows

Windows started on home computers and was commonly used for playing computer games. Though some shifty business deals in the 1980s and 1990s, Windows became pre-installed on the PC and so became the main operating system used by non-technical users.

However, there a lot of people that think that this situation is not good for humanity and we need to progress past it.

Why? Well lets look at some of the reasons.

Firstly, Windows does not promote a competitive industry. Only Microsoft can sell Windows, only Microsoft can really provide complete support for Windows. Mainstream PC shops may only stock Windows PCs.

So you have one company earning billions of their monopoly, with these excess profits, Microsoft can then give campaign contributions to politicians to make sure they don't make the industry competitive or hold Microsoft to account when they break the law.

Weak, bribed, politicians allow Microsoft to use the educational system as a giant marketing tool, indoctrinating a new generation to become helpless and passive recipients of Microsoft's, and only Microsoft's, products.

Secondly, each version of Windows is developed in secret, and then launched with billions of dollars worth of marketing to make you believe the magic; however like all magic, it is no replacement for public peer review. Microsoft don't like public peer review because they know that when compared fairly to other operating systems, Windows always loses.

The fact that there is no public peer review, and no effective competitive pressures, means that Windows is not very well engineered. When the main architecture of DOS and Windows was created in the 80s, it was already 20 years behind the state of computer science; and it has not really changed that much since.

This 'closed-off from the world in my own cave' approach to software engineering means that Windows is plagued with security problems, it uses computer resources inefficiently, wasting electricity and requiring unnecessary replacement of perfectly fine computers that could have lasted another five to ten years.

Fourthly, a software mono-culture, like a biological monoculture, is not very healthy. If a future Windows virus wipes out all of the world's Windows PCs, then 90% of the computer using population are offline, without their data and without access to government services, Internet commerce and digital information. Businesses would collapse and the western world would be plunged into a digital dark age.

Fourthly, because what Windows is doing is a secret; if you use Windows, then you are not in control of your computer, Microsoft is. Windows reports back lots of data to the USA which is then made available to whomever Microsoft wants to share it with. While most of us are not interesting to the US security agencies; Microsoft can sell your private information to anyone.

In short, Windows leaves your backdoor open to Microsoft, but even if you trust Microsoft, the US government, and all companies that Microsoft might sell your information too; the fact there are built-in backdoors means that anyone, criminals, terrorists, anyone, can potentially walk through Microsoft's backdoor to access your private data or install viruses or tracking software on your PC.

There is another way...

...Indeed there are lots of them! The opposite to Windows slavery is software freedom. And with freedom comes lots of choices, and choices are good! If you have spent a lifetime eating gruel then you might resent choice, but then remember the intolerant character in Dr Seuss' classic "Green Eggs and Ham", who resists and resists trying out new things for unjustified reasons.

The operating system I currently use is GNU/Linux (commonly just called Linu