Planet Larry

Extensions are cool

Extension frameworks are a good way to add new functionality to large applications that are messy to alter directly, they also help to quickly add functionality now, without having to wait six months for the next release.

The whole fun of extensions is that you can make your own and download random ones from the Internet, try them out for a bit, remove the ones you don't like, share the ones you do.

This of course implies two things. Firstly, that the extension interface is well thought out so that a badly written extension does not crash or unexpectedly interfere with the main application. Secondly, that the user who installs the application is adept enough to know what they are doing.

The killer feature of Firefox has been its extension framework, having hundreds or even thousands of useful extensions is what has enabled Firefox to break into the IE-dominated browser market where so many have failed.

All your extensions belong to us

Firefox 3 has changed many things compared to previous versions. One unexpected change is that the extension framework has been locked down.

I have mixed feelings about this. I understand there is a balance between on the one hand, fun and spontaneity, and on the other hand, protecting clueless people from themselves. However, for me, I hate software designed as a jail for idiots. That is why I refuse to use Windows for anything beyond testing that web applications work on it.

Firefox 3's extensions framework has a new DRM-like security barrier. You have to either submit your extension to be vetted and hosted by Mozilla, or you must use SSL, or use cryptographic keys.

I am all in favour of cryptography (as long as it is open for everyone to play), but I would be worried if this implementation makes it harder for people to write and share updates.

Computer says no

I tried to install an extension the old fashion way, by going to the author's homepage and clicking on link to install the extension.

Firstly, there a warning popped up, the same warning that was also in Firefox 2:

So I clicked "Allow" and then reclicked the install link.

Then, like a naughty child, we have to wait 4 seconds as a cooling off period, before we can click "Install".

Thirdly, even though we clicked "Allow", then were put into time-out like a child, then clicked "Install", it flatly refused to do what I had told it to do:

'Secure' here being defined as "approved by Mozilla", very few Firefox extensions are secure in the formal sense of 'trusted'.

I am root

I have to admit that this gets on my nerves somewhat. I am the master of my own computer, I don't mind software being a friend and warning me once about something. However, I cannot tolerate nanny software that takes over and tells me what I can't do and stops me from doing it.

I don't want to be a techno-snob but we should not make software for the lowest common denominator; maybe using Internet connected computers unsupervised is not suitable for everyone. We should not lockdown everyone's else computers because some people are idiots.

Computers have become the an extremely widespread technology, perhaps only second to cars in their spread. With cars, we don't let absolutely everyone drive. We don't make cars that refuse to start if the weather is not optimal, we instead let cars work in the rain and trust the driver to make allowances for poor weather.

Fortunately, in Firefox, the extension censorship can be overridden in the about:config settings dialog. So I went to Firefox 3's about:config page, and guess what? Yes you guessed it, they have added another new confirmation screen!

I am sure some of you will have strong views either way on this. Let me know!

Today I was trying to upload a lot of mail (something like 160000 mails stored in a deeply nested folder structure of about 13000 folders) to an IMAP server. Those mails were converted from the Eudora mailbox format to a more sane Eudora mailbox format using “Eudora rescue”, and then imported into Thunderbird’s “Local folders”. From there, so we thought, it would only be a matter of minutes to put those mails onto the newly setup dovecot IMAP server, with one drag-and-drop action.

But…

The upload horribly failed after the 3rd folder or so, with no error message whatsoever. No matter what I tried, it would just fail silently. The debug logging (see Debugging/Thunderbird article in the Dovecot wiki) didn’t help, and neither did dovecot’s log on the server. The problem seems to be connected with creating folders on the server somehow - because uploading a big amount (400 for a small test) mails within one folder wasn’t a problem. I couldn’t find out if Thunderbird or dovecot is to blame here…

I eventually managed to get this big pile of mail to the server by using a perl script that I found on perlmonks.org. Thanks to Davis for providing this! I downloaded it, installed the Mail::MboxParser and Mail::IMAPClient dependencies (with two ‘cpan install …’ calls), changed the configuration in the script to our site’s data and ran it. ~30 minutes later, the mails were on the server. Only thing that could have been better: All mails were ‘marked as unread’ - but that’s easy to fix.

Ekiga is the main voice over IP client on GNOME-based systems such as many free software distributions and Solaris. It also has a Windows port. So I thought I would try it out.

Giving your relatives Ekiga means they can chat to you online without you having to install strange proprietary phone clients that do strange things on your network.

To start with, you need a SIP username, I got a free SIP username from Ekiga's online-registration, but you can get one from any SIP provider, or you can be your own provider by running your own SIP server such as Asterisk.

To start with, lets just use Ekiga's free usernames.

So once you have a SIP username, we want to download Ekiga. the Windows binary is available from Ekiga's Windows-Users page.

Download the exe and click on it. This will bring up a little box asking you for your language. My first language, English, was the default so I went with that.

Next comes the Ekiga Setup Wizard:

Press next twice and you get this:

Press finish. I did this on Windows XP, which brought up a warning that Ekiga attempts to connect the Internet:

Since I want to use Ekiga to make phone calls over the Internet, I pressed "Unblock".

Next is the First Time Configuration Assistant.

It will ask you for your SIP username and password (i.e. the username we created at the beginning of this article), and it will try to detect your microphone, speakers and webcam (if you have one).

Press "Forward" a lot and read through each of the screens. when you are done it will summarise your choices:

Press "Apply" if you are happy, or press "Back" to change them.

So now hopefully you are done with all the hard work. You should see the main Ekiga interface like this:

The first icon on the right allows you to open a text chat with the person you are talking to.

So far, I only really use Ekiga to call my friends over the Internet, I have never phoned a landline, therefore I turned the numberpad off, you can do this by clicking the second item on the left, or by going to the "View" menu, and choosing "View Mode" then "Videophone".

Typing usernames into the address bar is a bit dull, so pressing the third icon on the left of the interface brings up the address book. If you add your friends to the address book then you can click on them rather than having to remember their SIP username.

Ekiga has many more features, but you can easily discover most of them yourself.

Now you probably want to test it. Put sip:500@ekiga.net in the address bar and press enter. This will call the Ekiga 500 echo service. It is a rebot who will echo back whatever you tell it, this way you can tell whether your connection, microphone and speakers are working properly.

Happy chatting!

Press Next a lot

I have used Ekiga quite a bit. It is one of the main free software voice over IP clients. It is often installed by default on many GNOME-based systems such as many free software distributions, and it is also available on Solaris.

It also now has a Windows port, so I thought I would try it out. Giving any remaining Windows-using relatives Ekiga means they can chat to you online without you having to install proprietary phone clients that do strange things on your network.

To start with, you need a SIP username, I got a free SIP username from Ekiga's online-registration, but you can get one from any SIP provider, or you can be your own provider by running your own SIP server such as Asterisk.

To start with, lets just use Ekiga's free usernames.

So once you have a SIP username, we want to download Ekiga. the Windows binary is available from Ekiga's Windows-Users page.

Download the exe and click on it. This will bring up a little box asking you for your language. My first language, English, was the default so I went with that.

Next comes the Ekiga Setup Wizard:

Press next twice and you get this:

Press finish. I did this on Windows XP, which brought up a warning me that Ekiga is attempting to connect the Internet. Being a program to chat over the Internet, I pressed "Unblock":

Next is the First Time Configuration Assistant.

It will ask you for your SIP username and password (i.e. the username we created at the beginning of this article), and it will try to detect your microphone, speakers and webcam (if you have one).

Press "Forward" a lot and read through each of the screens. when you are done it will summarise your choices:

Press "Apply" if you are happy, or press "Back" to change them.

So now hopefully you are done with all the hard work. You should see the main Ekiga interface like this:

The first icon on the right allows you to open a text chat with the person you are talking to.

So far, I only really use Ekiga to call my friends over the Internet, I have never phoned a landline, therefore I turned the numberpad off, you can do this by clicking the second item on the left, or by going to the "View" menu, and choosing "View Mode" then "Videophone".

Typing usernames into the address bar is a bit dull, so pressing the third icon on the left of the interface brings up the address book. If you add your friends to the address book then you can click on them rather than having to remember their SIP username.

Ekiga has many more features, but you can easily discover most of them yourself.

Now you probably want to test it. Put sip:500@ekiga.net in the address bar and press enter. This will call the Ekiga 500 echo service. It is a robot who will echo back whatever you tell it, this way you can tell whether your connection, microphone and speakers are working properly.

Happy chatting!

Often when using my computer, I will have a "project", by this I mean a directory containing an arbitrary number of files and directories. This could be a software project, or a website, or an essay, or whatever.

So when going back to a project, I might want a quick overview of what has been going on. This is what my filehistory.py script is for. It recursively orders your project files by last modified date.

The unique feature is that it mixes them all together as one stream. There is of course a tradeoff, the larger the number of files, the longer it takes, however it is quite fine for projects of up to several thousand files.

There are a few command line arguments:

• By default, the output is just the filepaths, if you use -v then it outputs timestamps as well.
• By default, it will output warnings if it finds permission errors or malformed symlinks in your project. If you want to suppress these messages, use -q.
• By default, it will output all the files within your project. If you want to specify the number of lines then use -n followed by the number. So -n 20 will output the most recently edited files.

If you followed my post about making a scripts-directory, then you can just make the Python file executable and drop the Python file into it.

It really comes into its own when you pipe it to other tools. For example, to copy the last 20 modified files to a USB stick mounted at /media/disk :

filehistory.py -q -n 20 | xargs -I x cp x /media/disk/

So feel free to check out my filehistory script. As you know I use Linux, but I think this particular script should work on Windows/Mac as well.

• Technical superiority: One product is just that much better than the rest that, while the competition is playing catch-up, the leader improves even further.


• Elegance: One product is just that much prettier or stylisher than the rest. The sold image is just that much more appealing (pretty much the reason why the wannabes all run around with Macbooks).


• Price: The one product might not be the best but it's just so cheap that it would be retarded to buy something more expensive


• Novelity: One product is just new and so different that it becomes the thing to have. It changed the rules in some respect that no one had thought about before.

1. Communication/community: Ubuntu has Jono Bacon as community manager. Then there's the forums and wikis. All in all it's like a bee hive of people all generating buzz and noise. Some of their work might be sub-par but still it's a stream of constant buzz about Ubuntu that keeps the word in the public discussion.



2. The slogan: "Linux for Human beings". I have no clue who came up with it, but that person is, in my opinion, responsible for at least 50% of the success of Ubuntu. The slogan is not geeky, it's not technical, it's talking about human beings (which we probably all are [except Harvey the invisible rabbit which is obviously reading this]) and it talks to feelings. It tells you that this is for you. No restriction, no catch.

My first Linux distribution was a very old version of Red Hat with X-Windows as a GUI. I think I was in 5th or 6th grade, and I bought it from a used book store for $10. From there, I went to Mandrake Linux 7, then Fedora, and finally to Gentoo. I was never a fan of RPM installers, as there were always issues with dependencies and uninstalling.

One of my friends, Zack had recently become a fan of Gentoo Linux. I abandoned RPM hell and became a fan of Gentoo. Despite, at the time, the best way to install Gentoo was through a stage 1 (compiling everything from start to finish), I loved it. Even if it took a day to get a usable system on a 400MHz Pentium 2, it was great.

Despite some obvious issues with dependencies in Gentoo, and many claiming Portage is just a slew of unmanageable code, I'm still in love with the speed and configuration of Gentoo. Sure, some things aren't install-and-go, but you learn more when having to configure most programs yourself.

Recently, I've been installing the latest version of Ubuntu on many of the machines I use to benchmark hardware for BIOSLEVEL. I can't say I'm a fan. I'll agree that it's fantastic that Ubuntu developers have put so much time into developing and improving many applications and daemons for desktop use, but it feels slow.

Why run native 586 code on a 686+ CPU? Why install software that doesn't take advantage of a CPU's extensions? To be completely honest, using the same hardware, a Gentoo System with Compiz Fusion running feels more responsive than an Ubuntu system without any desktop effects enabled.

While much of my testing will be done with Ubuntu, my primary Linux environment for regular usage remains Gentoo.

After first seeing it mentioned in irc last night, i took a look at former gentoo dev, kloeri’s current project called Exherbo which seems very much to be Gentoo 2.0. So far its got a number of former Gentoo dev’s onboard, including Ciaranm, who, regardless of his sometimes lacking people skills, *is* a very competent programmer. It seems their big aim is creating a gentoo-esque distro thats been built with all the experience available from working on Gentoo for years coupled with a desire to build a forward looking distro that shuns the massive amount of legacy code that other linux distros have picked up over the years.

Surprisigly though, what would appear to have been a low-key announcement in kloeris blog, has somehow been picked up by both slashdot and more surprisingly, TheRegister. While Slashdot takes the usual fairly negative view which it seems to have towards anything even *vaguely* linked to Gentoo. It was nice to read the Reg piece though and not see a totally negative or sarcastic article, they almost sound positive about exherbo!

I have to say im intrigued at the very least and am interested to see where this goes, especially as im a quiet supporter of Paludis (note to anyone reading this, im supporter not a fanboy! ), and have liked how it has forced portage to actually document how everything works via the EAPI’s, which makes it easier to figure out if a certain behaviour is a bug or actually a feature!

While ive no intention to actually try and install Exherbo at this point (as im just a mere user!), i do intend to watch and see what it grows into, and whether it can fulfil its rather lofty aims.

Nothing horribly new to report. I added a few new pictures to my Flickr feed at the bottom of the page. I'm hoping to make more use of my camera this summer, but I haven't figured out what for yet. I'm just going to have to remember to bring it with me more often.

I'm trying to get back to blogging on a regular basis. It's hard to remember to write here, especially when I'm almost always writing a review for BIOSLEVEL. What do I do if and when we run out of products to review at BIOSLEVEL? Obviously we'll be able to write Linux-related articles during any down time, but I'd also like to focus a little more here. My traffic is way way down since October of last year.

Expect more updates, a new layout, and other cool features soon.

Sometimes you have to slow down your mail delivery for some destinations or you might want to push out more mails for a certain server as it’s inside your LAN without being unkind to other people. Here’s a way of doing it:

The main.cf file of our postfix installation is the place where we define a new transport map and an according speed limit. As I am slowing down things here, the name of our test transport will be ’snailmail’ - so here we go:

— main.cf

transport_maps = hash:/etc/postfix/transport
snailmail_destination_concurrency_limit = 1

The Transport map file itself now contains the domain(s) we’re changing the limits for by assigning the new transport service:

— /etc/postfix/transport

domain.com snailmail:

Finally we’re hacking our master.cf file for defining our snailmail service to be able to make use of it:
— master.cf

snailmail unix - - n - 5 smtp

The results we’re expecting are now, that all mail traffic regarding domain.com (we’ve used that in the transport map file - remember?) is using our new delivery service. But wait - we haven’t generated the hash file for the transport map. Running postmap /etc/postfix/transport does so. Now we’re set up for the restart.

Believe it or not - we’re done - and it wasn’t even complicated.

Last summer, at GUADEC here in Birmingham, the cool young dudes first integrated Webkit as an experimental toolkit for the Epiphany web browser. Now they are working to make WebKit as the main backend for Epiphany. It is a very small team indeed, but their aim is to finish the migration by GNOME 2.26 in March 2009; if we are lucky they may have finished by GNOME 2.24 this September. The old branch with the Mozilla gecko back end is very stable and will be released as 2.24 if the Webkit migration is not finished.

This is exciting for a number of reasons, firstly we will have a fast and light, GNOME integrated browser. Secondly, and perhaps more importantly, if Webkit becomes a default GNOME library then the same browser toolkit is available in KDE desktops, GNOME desktops and Apple OS X desktops, allowing a new generation of rich client web mashup applications.

I tried Epiphany with the Webkit backend last year, it was pretty raw then. The centre of the window worked as a web browser, but it ignored the toolbars and buttons at the top of the application. So how is it 10 months later?

Pretty good I say. Of course, it is still early days, and they need a lot of help removing the old unneeded code and binding the interface to Webkit. If you know C and want to get in to an interesting open source project, this might be a good project to join as they are basically restarting a lot of it.

My adventure installing Epiphany

I wish I had a spare Gentoo machine that I could use for this, but they are all busy with important things. On Gentoo they have made ebuilds for it in the overlays that take the latest version down.

So I used a spare Ubuntu machine, I started by getting rid of anything to do with WebKit or epiphany that has been installed via Ubuntu, at time of writing these are all too old.

Then I checked out all the development versions I need:

svn co http://svn.gnome.org/svn/libsoup/trunk libsoup
svn co http://svn.gnome.org/svn/epiphany/trunk epiphany
svn co http://svn.webkit.org/repository/webkit/trunk webkit

The last took a while as it checks out the complete source tree for every platform, I didn't actually need a lot of this, so another time, if time or badwidth is a concern then I might want to try to work out a more specific checkout command.

Then while that is running, I got the other required dependencies.

sudo apt-get install libicu-dev libxslt1-dev libsqlite3-dev libjpeg62-dev \
libpng12-dev gperf bison libcurl4-gnutls-dev flex gtk-doc-tools

Then I went into the libsoup directory and compiled it:

cd libsoup
./autogen.sh
make
sudo make install

Then webkit had finished downloading by the time libsoup is done. So I went into the Webkit directory and tried to compile it with this:

cd ../webkit
./autogen.sh --prefix=/usr/local --enable-svg-experimental --with-font-backend=pango \
--with-http-backend=soup
make
sudo make install

This didn't work, so (after make distclean), I googled and tried the method used by guy called Peter Upfold. This uses QT's qmake to build WebKit-gtk, bit of a contradiction, but it works at least:

export PREFIX=/usr/local
./WebKitTools/Scripts/build-webkit --qmakearg=WEBKIT_INC_DIR=$PREFIX/include/WebKit \
--qmakearg=WEBKIT_LIB_DIR=$PREFIX/lib --gtk --qmake=qmake-qt4
cd WebKitBuild/Release/
sudo make install

Then I waited a while again.

Next update the shared library cache:

sudo ldconfig

Then I did epiphany, this was pretty quick,

cd ../epiphany
./autogen.sh --prefix=/usr/local
make
sudo make install

Then epiphany worked like a charm.

Acid2 Test, no problem:

In the Acid3 Test, Firefox 3 only scored 71. What about Epiphany, well still no problem:

Anyhow, should be interesting what they come up with in the first Epiphany WebKit release.

Please do Digg or StumbleUpon this article,

So my folks Windows XP box started having difficulty booting. Most of the time it would go to boot and suddenly you'd be back at the BIOS, then the "Windows didn't shutdown properly last time" menu and then once you selected "boot normally" the cycle would begin again. Usually eventually you'd get lucky and Windows would boot (but sometimes they gave up first). Not good. This started a while ago and seemed to get worse (I think, I never use the box so I don't know). Anyways, eventually they figured (hoped actually because otherwise it was hardware failure) that their XP install was corrupted and asked me to reinstall Windows for them.

They gave me the CD that came with the box and I loaded it up and first things first, no harddrive detected. Yep, that's right, no harddrive. To be fair, XP is old, it's from 2001, and pretty much all computers come with SATA harddrives now, which didn't even exist in 2001, so it's not entirely fair to expect XP to support them. This is one of the downsides of only doing releases every 6 years. Being not terribly invested in this "install XP" project I tossed my hands up and said it's a no go and offered Linux as an alternative. My Dad was definitely interested in Linux but wasn't sure it would be all there for his business software, so I said, "no fear, let's just do a trial run, and if it doesn't work, no worries, you can, you know, always buy Vista". The thought of having to spend money to make a computer that did work, work again, for no good reason vs. the low commitment of a trail run sold him on the project.

So I burned a copy of Hardy Heron, the newest Version of Ubuntu, released just last month. It loaded up and recognized all the hardware and installed, no trouble. I'm old fashioned so I opted for a separate root and home partition, but other than that, a vanilla install. Then we booted up and everything just worked out of the box. Even the printer. So I loaded all their data on from one of my boxs from whence it had been backuped to and let them at it (after a few tweaks, like my mom's desktop).

The results?

Mom loved it. For her, it turns out the killer feature was desktop icon stretching. She really liked the larger icons. And of course, no difficulty using it for her normal uses which are nearly entirely Internet look ups. The other big hit was Eye of Gnome, the photo viewer. Last year we had about 1600 of my mom's parents family slides turned into digital photos (at no small cost) and the stock Gnome photo viewer is perfect and intuitive enough for her to use and enjoy (the newly added left right buttons to scroll a directory are crucial here, so good work guys).

My moved out sister dropped by and gave it a try, and she too was sold. For her, the large stock puzzle/games selection was the seller. It worked just fine for her net usage, but the large selection of games got her attention and kept it just on introduction for over half an hour and she only explored a few of the games in that time. She'll be back. She also was amused by compiz eye candy.

As for my Dad? Sadly neither of his business apps worked under Wine. Maximizer 8 failed to install (a little sad since it's reasonably popular and not so new) and D--------, which is from what I can gather an in house piece of software from his company, also failed to install. A few tales from his trials installing it under Windows were horrific thought, like for the first time install you are required to turn off the firewall. 0_o In house software can be scary and shoddy stuff.

So Linux failed to satisfy everyone of my family's needs, so back to Windows. I was disappointed but resigned. And then came mom to the rescue. She chimed in she didn't want to go back. She liked her new Ubuntu desktop! So that was all I needed to float the idea of dual-boot. And thankfully we had two partitions already. Dad was sold too. He does want to learn more about Linux and so looked forward to the opportunity, but also needs his business software.

So I popped in a Gentoo LiveCD, because there really is no better system recovery and maintenance CD than a Gentoo LiveCD. I copied the Ubuntu root over to the second partition, moved /home and edited fstab and grub/menu.lst and reinstalled grub (from within the Ubuntu chroot because the one thing that particular LiveCD was 'missing' was Grub) and then changed the partition type of the first partition from Linux to NTFS and volia, we were ready to go.

Then back to Ubuntu because the Gnome Device Manager is actually full of awesome and gave me all the hardware information I needed so that I could get the appropriate drivers for Windows (remember when it was the reverse and it was crucial to go into Windows System Manager to get all the hardware info so Linux could install?). Then off to the net to find the Sata drivers, and then I tossed those onto a floppy disk and tried the XP install again. Sure enough, after loading the drivers from the disk, XP install found the harddrive and was happy enough to install onto the partition I had marked for it.

Then it booted into Windows XP SP1 glory. Which was surprisingly not glorious because it had no drivers so we had a 640x480 low colour screen with no sound. And no programs. Dad and I spent the rest of that day and the next few after that running Windows Update and downloading and installing drivers and programs manually from the net. The Windows Update website actually pretty much continually stalled out and failed when it came time to get service pack 2 which was disconcerting but by then the update manager had been installed and it was able to get and install SP2 for us. Shortly after which my dad horrifyingly noticed the Windows booting bug was back! We were horrified. All this work and no fix? Was it hardware failure? But Ubuntu was still booting fine. So we kept at it and apparently now with SP3 installed it's gone back to booting normally.

So yes, that means Microsoft in one of their post SP2 updates introduced some bug that rendered our computer nearly unbootable, which drove us to Linux which was brain dead easy to install. And then when we discovered Linux didn't quite fulfill all our needs, they made the migration back to XP as painful and full of near hopeless despair as possible.

So thank you Microsoft for driving my family to try Linux because they've found they like it, and thank you after that for highlighting exactly why we don't want to be hooked on Windows. It's been educational for my family, and now ever computer in my house has Linux installed on it and my family is interested in learning about it! And I owe it all to Microsoft.

(It should be pointed out that my dad's work still has a ban on IE7 and Vista because it doesn't work with their software yet which is a mark against both Microsoft and Vista, and my dad's company's computer department because, lets face it, it is the future and it has been out for over a year. But this did mean Vista really wasn't a viable option on the fact that it would have cost us more money and it wouldn't have accomplished the one goal we needed it for, which was running my dad's business software. XP was our only option.)

Firefox 3.0 RC1 has now been released, with some fixes and changes from Beta 5. I have not looked at the changelog yet, so I do not know what has changed. I notice a few (minor) graphical differences.

I also want to report that the Acid 3 test still ends on 71/100, as it did in Beta 5. (See my previous review of Beta 4 and Beta 5) Since the trees are locked down, we can expect that Firefox 3 RTM will not correctly display the Acid 3 test.

Writing text

There are many ways to markup plain text. There is SGML and its off-shoot HTML. Later came XML, a replacement for SGML, a general-purpose way of making markup languages, one of which is XHTML, which is what you are using to view this page now.

There is also LaTeX, which is a markup used for typesetting in various contexts, especially in academic fields where complex mathematical equations are used.

With all these formats, the marked up document is plain text, and then rendered into its final form, that form may be paper, or it may be a browser window. The downside to these formats is that the marked up plain text document is ugly - full of random angle brackets and letters and words entered into the text that you are trying to read. While after using HTML for a decade I can more or less block it out, the plain text form is far less readable because of the markup.

Enter reStructuredText

reStructuredText is a markup language that does things differently. It uses more natural looking markup with a focus on readability so reStructuredText can be easily read and shared in its plain text form. You can then automatically processes the document into XML (called Docutils XML), and from there it can go into XHTML, LaTeX, OpenDocument, Docbook, or any other format that you can get to from XML. So readable, yet maps to XML in a well thought out way.

If you are going to abbreviate reStructuredText, then use 'rst', don't use 'rest' because that gets confusing with Representational State Transfer, a completely different thing altogether.

I have jumped in the deep-end and I am using reStructuredText to write my posts in this new version of the site, and it is an optional markup format for people leaving their opinions in the discussion/comments section (you don't need to use it to leave a comment).

While I have just got into it, reStructuredText has been around for several years, and has been most prominently used for creating Python documentation, however, it is a general purpose and extensible markup language that can be used in many different contexts.

Some people use it to create text for the web or for standalone documents. It can also be used wherever you might have made a one-off markup format, such as on a web forum, wiki or comments section.

reStructuredText everywhere

The reference implementation of a reStructuredText parser is called Docutils which is written in Python (but this can be used from many languages). Docutils is very easy to use in your application to process reStructuredText.

If you do not have docutils on your system already then it is available from all Linux/BSD package managers. For Windows, follow the installation-instructions.

There are also third-party parsers in Perl, Java, Haskell and probably more that I have not heard of. There are also plugins for text editors and plugins for lots of web frameworks, content management systems and web log software.

As explained in the last post, Django comes with reStructuredText support (bindings to docutils) out of the box. Lastly, there are also lots of cute reStructuredText website generators that allow you generate a website from a set of reStructuredText files. For example, Michael's rest2web, Sphinx,which is used to create the new Python 2.6/Python 3000 documentation, and Damien Baty's Soho.

Very quick reStructuredText (permalink to markup)

Okay, enough blab, lets get into it. There is very comprehensive set of markup available, but I will just cover a few basic ones that are always useful.

There is no need for paragraph tags or linebreak tags; to make a paragraph, you just make a paragraph, to break a line you just break a line.

To mark something with emphasis (i.e. italic), you use a star around the word or pharse: *emphasised text* becomes emphasised text.

Two stars are for strong emphasis (i.e. bold). Like so:

Hyperlinks

To make a hyperlink, you just start the text with http:// . So http://commandline.org.uk/ becomes: http://commandline.org.uk/

If you want to do a named hyperlink, then you need two parts. First, you put an underscore after the name, for example Zeth_ will become Zeth

Then you need a target to go with that name. You specify the target with two dots, a space and an underscore:

.. _Zeth: http://commandline.org.uk/

The line with the target can go anywhere in the document. For example, some people put the target line straight after the paragraph that has the name in. However, in a long document, it is often tidier to put all the targets at the end.

Lists

To make an unordered list (i.e. a bullet list), just start the line with a star, plus or hyphen, for example:

* Green eggs
* Ham

Becomes:

• Green eggs
• Ham

For a numerated list, just start the line with the number and a dot, for example:

1. Green eggs
2. Ham

Becomes:

1. Green eggs
2. Ham

I could go on, but I think you get the idea. By default, the core of reStructuredText is broadly feature equivalent to HTML. However, reStructuredText is very flexible and allows you to go beyond that core, as we shall look at next.

Directives

To markup a piece of more advanced functionality, you use a 'directive'. Directives are where reStructuredText shines over similar markup languages. Directives are extensions to the main markup, some directives are included by default (analogous to a standard library), others can be added by yourself or from getting them from the web or wherever. I will give an example of a standard directive, then an example of a third-party directive.

A directive has four parts. First you have some markup declaring the directive itself (two dots), second comes the directive name, third is some markup that says here is the content (two colons), lastly there is the content.

Two dots, the name, two colons, the content.

.. name :: content

So to display an image, you use the image directive.

You write two dots, a space, the word "image", two colons and then the URL:

.. image:: \http://commandline.org.uk/images/whokilledtux.png

Becomes:

To share source code, firstly you start with the source code directive, this is two dots, the directive name (source code), followed by two dots; secondly, we have the name of the language the source code is in. Then we leave a blank line, then we write the source code indented (preferably with four spaces ;-)

So for example:

.. sourcecode:: python

    import os
    for i in os.uname():
        print i

Will become:

import os
for i in os.uname():
    print i

Here is the list of supported languages on this site.

There you go, if you made it this far then you can use ReStructuredText, give it a go! You can start by leaving a comment in ReStructuredText format!

Vor einiger sehr langer Zeit habe ich mal ueber “Saving SSH options for specific host” geschrieben. Fuer mich ist die ~/.ssh/config nicht mehr wegzudenken. Bloed ist natuerlich, wenn ich diese Datei auf meinem Rechner zuhause habe, aber auf der Arbeit etc. auch haben moechte. Auf jeden Rechner auf dem ich die Datei haben will raufkopieren, nach jedem Update ueberall neu. Ganz schoen bloed und nervig.

Viel schoener waere es fuer mich, die Datei an einem Ort zu haben, wo sie immer aktuell ist, wo ich immer rankomme, und von wo ich sie mir schnell ziehen kann. Was bietet sich da mehr an als mein Webserver?! Auf der bash ist ein “wget myServer.tld/myFile -O ~/.ssh/config” sehr schnell geschrieben

Meine ~/.ssh/config passe ich hauptsaechlich zuhause auf meinem Rechner an. Von daher reicht fuer mich eine simple Moeglichkeit diese Datei schnell und unkompliziert von da auf meinen Webserver zu bekommen. Am besten automatisch, dass ich mich nicht drum kuemmern muss. Hier meine Loesung:

1. Auf dem Webserver einen rsync-User angelegt und ihn der Gruppe “www-data” hinzugefuegt, so dass er in meine htdocs schreiben darf , und in die Gruppe “ssh-allow” hinzugefuegt, so dass er sich auch via SSH einloggen darf (in der /etc/ssh/sshd_config: AllowGroups ssh-allow)
   (adduser myRemoteRsyncUser && adduser myRemoteRsyncUser www-data && adduser myRemoteRsyncUser ssh-allow)
2. Auf meinem Desktop-PC einen User angelegt und fuer diesen einen ssh-key generiert
   (adduser myLocalRsyncUser && su - myLocalRsyncUser && ssh-keygen -t dsa -f .ssh/myKey)
3. Die myKey.pub auf dem Websever in die entsprechende authorized_keys gepackt
   (scp /home/myLocalRsyncUser/.ssh/myKey.pub myRemoteRsyncUser@myServer.tld: -> ssh myRemoteRsyncUser@myServer.tld -> cat myKey.pub >> ./ssh/authorized_keys )
4. Auf meinem lokalen Rechner rsync Befehl zum kopieren der config beim herunterfahren des Rechners automatisch ausfuehren lassen
   (in der /etc/conf.d/local.stop [Gentoo] den Befehl su myLocalRsyncUser -c “rsync -avu -e ssh /path/to/.ssh/config myRemoteRsyncUser@myServer.tld:/var/www/myRemoteSSHConfigFilename”)

Voila. Beim runterfahren wird nun die Datei automatisch auf meinen Webserver kopiert falls sich diese geaendert hat EEEENDLICH!!

dbr left a comment on this site:

I concur with the other two comments - this is one of the nicer blog'y site layouts I've seen. The comment system is also actually pleasant to use, unlike every single other one I've (not)-used \o/

—dbr (May 16, 2008)

Thanks for the compliment. I have talked before about my philosophy towards forms:

I do not make want to make it like Indiana Jones and the Last Crusade just to make a comment: roll under a flying saw, spell something in Hebrew and then take a flying leap into the abyss. Furthermore moderation of spam is my problem, not yours.

—Zeth on comment-spam (3 July 2007)

dbr continues:

One slight bug, you need to enter two backslashes to make it visible.

—dbr (May 16, 2008)

That is not a bug that is a feature! That is the escaping mechanism of ReStructuredText (as well as Python and lots of other languages). The comments form does not allow HTML but it allows ReStructuredText, as I explain below.

Ryan also left a comment:

For your blog post model, what did you do for entering posts? Do you still use the default admin interface, or did you make your own views for posting and whatnot? I haven't looked into it much, but does Django automatically include much in the way of wysiwyg text editors for text fields?

—Ryan (May 15, 2008)

Answering your questions in reverse order:

• Wysiwyg text editors

There is no default Django WYSIWYG text field, but some people use Javascript components such as TinyMCE which slots in nicely (instructions).

• Default admin interface vs own views

I have the admin interface as one way to enter posts, but I also made a simple command-line tool for entering posts, I also made some scripts for importing all my old posts from Pyblosxom.

I did create one admin view, I overrode the 'delete comment' view to create a button that adds the IP addresses of deleted spam comments to a block list.

At some point I will make a 'Preview' view and button. At the moment I can save posts as drafts but not view drafts in the site template before it goes live.

• What did you do for entering posts?

I write a new post in a real text editor such as Emacs or gedit in ReStructuredText format, which I can then either paste into the admin interface, or use my little script to squirt them into Django.

Markup in Django

'contrib' forms the 'standard library' of Django. One of these packages is called "markup". It provides filters for three markup languages: ReStructured Text, Markdown and Textile.

I chose to use ReStructured Text because using that to write my new posts because I like the format and getting confident with ReStructured Text will be useful in lots of other contexts also.

To use it in Django, just add django.contrib.markup to the INSTALLED_APPS list in settings.py. In a template, you can load it and use it like this:

{% load markup %}
{{ comment.comment|restructuredtext }}

In the next post I will explain how to write in ReStructured Text,

I got it from my sister’s digicam. Thanks to Leerz for the walkthrough. This worm is really annoying especially if you are more comfortable doing stuffs in the console.

1. Check for any bar311.exe, Autorun.inf, pc-off.bat files in mounted drives.
2. Delete if found.
3. Edit the following entries in the registry.

HKCU\Software\Microsoft\Command Processor\"Autorun"
HKLM\Software\Microsoft\Command Processor\"Autorun"

Alternatively, you can download Noob.Killer, run it, then watch and learn.

By some people’s logic, this how the economy is supposed to work:

1. New companies emerge all the time.
2. No companies ever close.
3. Consumers always buy the cheaper and better products.
4. No products ever become obsoleted and force the company to go out of business.

Sounds perfectly reasonable, doesn’t it?

When a new company opens in a town and provides a thousand new jobs, there’s noone protesting that this is unfair, we didn’t do anything to deserve this, that you can’t just suddenly create new jobs out of nothing, there aren’t people complaining that it’s not right, we didn’t get jobs at the new company. No, people accept it with great fanfare. Great, the economy is growing, our town will prosper! People will have more money, there’ll be less unemployment, we’ll be able to afford a higher standard of living.

And yet when, after 40 years, the company goes out of business or moves their production to a cheaper location, people say this is outrageous, 1000 jobs will be lost! There’s anger and pandemonium, how can they do this to us, we were loyal to the company for 40 years. People appeal to some sort of higher ethical body; you can’t take our livelihood away, what are we going to do with ourselves? And the town itself, which never had much industry, and really just had that one company that employed everyone in town, starts to regress. People move out in search of jobs, young people leave and don’t come back, noone moves in because there’s no local economy.

It’s a sensitive topic. Losing your livelihood is one of the more challenging life situations. But before you start screaming that it’s those damn crooked politicians and those greedy executives that have stolen your life, take a moment to think about why you had that job in the first place. In fact, let’s start with the basics: what does it mean to have a job?

It means that you are producing a product or offering a service that someone is willing to buy. It does not mean any of these things:

1. Someone is being nice to you.
2. You deserve this.
3. You’re going to keep your job because you’ve been loyal to the company.

If you actually believed any of that then you were under a complete misapprehension. Sure, sentimental concerns do come into it sometimes, like the boss’s son getting a summer job because he’s family. But in the long run, the only thing that matters is the economic reality.

If you think that’s a raw deal, think about this. Most artists aren’t wealthy, in fact most artists are struggling to get enough work to live on. A painter may think that he deserves to live a decent life as a painter, but if noone is willing to buy his work, well he’s not going to. Is that unfair? No, it isn’t, because if he’s not producing anything of value, why should anyone have to pay to keep him in business? So if a painter can’t do the job he wants to, why should it be any different for you making shoes, or catching fish, or whatever it is you do?

There used to be people working in elevators who would press the buttons. We don’t need them anymore. Shepherds aren’t in great demand either. Neither are telegraph operators. These professions have all be superseded and they’re not coming back. Many others still exist, but have been moved to where production is cheaper, like textiles.

It’s always a turbulent transition, you can be sure of that. We don’t have hunters anymore, we have domesticated animals now, no need to chase them in the woods. Think about how many hunters were out of work when this happened. But what should they have done, lynch the guy who came up with the idea of keeping animals on the property? Compared to the hunters’ relatively narrow interests (although there were many of them), domestic animals were very beneficial to the village. For one thing, you didn’t wonder where dinner was coming from, the animals were right there. So should the villagers have discarded this new idea just to make sure the hunters could keep their jobs?

I’ve got news for you. The very same thing you’re protesting against, your job being taken away, you’re doing the same thing to people everyday. That’s right, you’re not so innocent yourself. Have you ever bought a car from a different automaker, because it was cheaper? Did you ever buy peaches from Spain instead of domestic apples? Well, I’m sure it must have been a very gruelling decision for you, right? I mean to think that you could be putting car makers and farmers out of business because you’re not buying their products, that’s a tough one to swallow.

And what did you get out of it? You could afford to buy more things, because the new products were cheaper. And they didn’t break as quick, so you could use them longer. And they had some functions that the old products didn’t have, which made you happy. And just as this was happening, the old companies that couldn’t stay competitive were going out of business one by one, people were losing their jobs. But hey, you got a pretty good deal out of it, didn’t you?

Here’s what it comes down to. You’re not entitled to your job. You’ll only have it for as long as people are willing to buy your product. And even if you’ve had it for 40 years, that doesn’t mean the global market won’t make it obsolete tomorrow. There was a demand for your product, now there isn’t. You didn’t do anything to deserve getting it, and you didn’t do anything to deserve losing it.