Planet Larry

A few days ago, I spoke with John Phillips (he is a computer science professor and adviser here at Mansfield) and I found out that I had made three class-selection errors in my Fall 2008 schedule. Luckily, he was able to set me on the right track. My new schedule is linked below. Instead of Accounting II (sorry to Dave and Vinnie), Data Structures (sorry to Dan McKee), and Honors Research (sorry Sara, Andrew, and others), I will be taking Computer Organization, Finite Math, and Principles of Microeconomics.

Revised Fall 2008 Schedule (PDF).

I just visited Sourceforge with the intention of declaring my “Urdabrunn” project as dead. But instead, I decided to announce that the project has moved to my own domain, Solumslekt. The Exodus project is the natural heir and successor of Urdabrunn, and it seems natural to host it here.

The problem with the Urdabrunn project was that it was far too ambitious, like so many other dead projects on Sourceforge. It’s of course much easier to concoct grand plans than to do the actual work.

I did write some routines in Python for the Urdabrunn project, but the major work was the construction and migration of the database itself and the parts of the business logic that I wrote in plpgsql. That codebase has survived to the present day. When I decided to “go incremental” with my humble PHP reports, the interface took off in a totally different direction from what I had imagined at the time I wrote the “Into the Blue” article and submitted the Urdabrunn project to Sourceforge.

For my own part, I feel that Exodus gives me what I need from a genealogy software package. For my personal use, it’s stable to the extent that several months may pass between each time I do even small incremental changes to the codebase.

During the last week, however, I have been in a flurry of refactoring and adding translation strings. That’s because I finally have been engaged in a serious discussion about the project. There is a big difference between writing software for your own use, and having another person trying it out and asking intelligent questions.

After the big download, I kick-started the installer and I couldn’t believe my eyes: That service pack wants to have 1475 MB free space while installation - true, that number sounds scary, but it wasn’t better with the second service pack.

Since the machine boots again, I am looking through application changes - in other words, what does not work any more. But luckily it is only the ATI-Catalyst 8.2 having problems: screen rotation does not work any more. I have not spotted anything else acting strange by now…

After one week delay, the third service pack for Microsoft Windows XP arrived. You won’t believe it: It’s even available via automatic update.

But what’s inside? First of all, it’s a huge file being 313 MB which means a rather big download and a longer installation. But what’s under the hood? Can’t tell by now. I’m just installing.

http://world.std.com/~franl/worm.html

A Slashdot article reminded me of one of my favorite technical articles on the Internet entitled “A Tour of the Worm", an in-depth historical and technical look at the Morris worm. The Morris worm, mistakenly unleashed in 1988, was one of the first significant worms to strike the Internet, and it caused enough damage that it arguably has done the most relative damage of any worm since then.

Check it out if you haven’t read it, or even if you have; it’s a fascinating look at the early days of the Internet. http://world.std.com/~franl/worm.html

Here’s the thing. I hate stating the obvious. It really annoys me. On the other hand, obvious things are sometimes things that most need to be repeated. So I wrestle with myself and I finally decide that I should, because there is a shockingly large number of people out there who don’t realize how obvious this is. See if you can learn something from this mock dialog.

Vendor: Good morning, is this Harry, the CTO*, I’m speaking to?
Client: Yes, how may I help you?
Vendor: Hey Harry, this is Steve from Microsoft. I would like to talk to you about Windows Vista.
Client: What’s that?
Vendor: Why, it’s the brand new version of our Windows operating system.
Client: Oh, that.
Vendor: I was wondering if I could interest you in our product.
Client: You know what, I don’t think so, we are a very security sensitive company, and..
Vendor: But that’s precisely the reason I’m calling, I would like to tell you how you can enhance your security with Windows Vista. You see, we’ve built the operating system with security in mind and it’s the state of the art in operating systems.
Client: Hey, that sounds pretty exciting. So how does this work now, you ship us the source code and…
Vendor: No no, we don’t distribute the source code.
Client: You don’t?!?
Vendor: No, you see it’s a trade secret. (my precious etc)
Client: You’re kidding, right?
Vendor: No, really.
Client: So how do we know that it’s actually secure if we can’t see for ourselves? How do we know there isn’t anything malicious in it?
Vendor: Well you’ll just have to trust us.
*Harry hangs up*
Vendor: Hello? Harry?
*CTO - the highest placed person who makes technical decisions in a company.

How did it go? Did you get it? It was kind of a long thing, huh? Ok, stop racking your brains, I’ll give you the answer: no source code, no security.

Here’s how that works. It’s simple economics, so try to keep up. If they give you the source code, then they put their cards on the table. You can see what the code does, and if it’s doing something stupid (security hole) or nasty (like sending your data to back to the vendor), then you’ll be able to check for this. Now you may say “I don’t know how to check”, and that’s okay. But just by giving you the source code the vendor knows that you can see everything the code is doing. And if you find something nasty in there, they know you’ll never trust them again. So it doesn’t really matter if *you* don’t know how to check, because there are others who do, and sooner or later someone will find the nasty code if it’s in there. Thus, if the vendor gives you the source code, then he’ll be a lot more careful about what’s in there, because he’s risking losing your trust and your business forever. That will keep him honest.

Is there then anything surprising about finding out that Microsoft is putting in backdoors in Windows? No, because how would you know it’s there? You don’t have the source code! In case you were wondering, the words “security” and “backdoor” are mutually exclusive.

So what have we learned today? Is there somehow we could summarize all this in just one sentence? There is: If you want security, ask for the source code. If you can’t get the source code, you know that the vendor isn’t taking security seriously.

  * ERROR: net-fs/nfs-utils-1.1.2-r1 failed.
 * Call stack:
 *       misc-functions.sh, line 652:  Called install_qa_check
 *       misc-functions.sh, line 360:  Called die
 * The specific snippet of code:
 *   		[[ ${abort} == "yes" ]] && hasq stricter ${FEATURES} && die "poor code kills airplanes"
 *  The die message:
 *   poor code kills airplanes

I had to undelete someone's files from a FAT partition today. My first thought was to use good ol' Windows to do so, given that Windows is the unholy ground which spawned FAT to begin with. I remember there used to be an UNDELETE command of some sort in some old version of DOS. But this doesn't seem to exist in XP any longer.

There are however lots and lots of third-party "shareware" programs which can do this kind of thing, as Google reveals. There is in fact an overwhelming number of such shareware programs. Most of these programs are total crap and cost around $30. One program required me to burn a CD and reboot my computer from the CD before I could run it. Many of the programs "intelligently" scan a partition looking for chunks of things that look like JPEGS or WMVs. I tried a few "demos" before I gave up, not having an hour to waste finding the one program that would work. Thus bringing the current score to Windows: 948, Brian: 0.

Instead I brought the drive home and plugged it into Gentoo and used this post as a guide. I dd'ed the partition to a file, fscked around with it a bit, mounted it via loopback, and had my files back. Took 10 minutes, and worked as expected. And it didn't cost me $30.

The moral of this story: I need to burn a Knoppix disk to take to work with me.

My only quibble is that I can never ever remember what Gentoo package contains fsck.vfat. Note to self, it's dosfstools. I can never think of the search terms even to locate that package. I had to google it.

My ninth call to Westinghouse today, about my Westinghouse L2410NM 24" LCD monitor which I RMA'ed back in March, revealed that they did in fact shipmy monitor, supposedly to my house, on April 4th or so. A UPS tracking number confirms it. There are are a few things wrong with this.

1. In spite of the fact that I asked for a phone call to be updated on the status of my monitor whenever it was shipped, I received no such phone call.
2. During the four phone calls (or was it five?) I made to Westinghouse in April, AFTER my monitor was supposedly shipped to my house, no one at the company had any record that it shipped. I was told that by multiple representatives over the past four weeks that my monitor was "in processing".
3. I asked for my monitor to be shipped to workplace, not my house. My nice, safe, cozy workplace with human beings who can sign for large expensive packages. Not my empty house in a neighborhood full of drug addicts, in the property theft capital of the west. In addition to telling the phone representative this, I actually taped a 8.5 x 11 inch sheet of paper directly to the monitor itself (as well as the outside of the box) specifying SHIP TO: and my work address. Even such drastic measures were not enough to catch the attention of whatever magical monitor-repair fairies work at Westinghouse, apparently. Perhaps I should've carved that information directly into the monitor screen.
4. I could possibly overlook the above, except that, as you may have surmised, at the present time, I do not, in fact, have my monitor.

After calling up UPS to ask why their driver left a $450 computer monitor, in a shiny bright blue and white box with pictures of a computer monitor all over it, sitting on my front porch while I was at work without getting my signature, I placed call number ten (yes, I've finally hit double digits!) to Westinghouse, and managed to escalate my issue to the Westinghouse corporate office. Supposedly in 7-10 business days they will send me a brand new monitor.

Oh how I wish I had any confidence that I'm ever going to see that monitor.

In the meantime, this guy was on sale at the local store, so I bought one. Time will tell whether LG brand is any better than Westinghouse. This time, I also bought the extended warranty, having learned my lesson that it can, indeed, be worth an extra $60 to save myself some pain and aggravation later. I'm also going to think twice about buying things like this over the internet in the future. There is something to be said about being able to drive 10 minutes down the road to have your property serviced or replaced by real-life human beings, rather than paying to have things shipped around the world for a month.

I’ve made a difficult decision that I hope doesn’t make anyone feel too bad: effective immediately I’ve removed any feeds from Planet Larry / Larry the Universe that were from developers who have retired from the Gentoo project.

I feel like ex-developers carry a lot of weight with their posts and opinions, and I created the planet feeds mainly for users, by users (speaking of which, don’t be shy, and sign up).

That’s all for now.

"Intellectual Property is the oil of the 21st century"

Computers are a love/hate thing for me. I love all things digital, but I desperately need to get away from it sometimes too. So I had a nice vacation away from my computer last week. I couldn't keep myself from reading some mailing lists and hitting Slashdot once a day, but I didn't write a single line of code and didn't give my websites or work projects or anything much thought.

But now my vacation is over, and it's so easy to fall back into old habits, endlessly looking at webcomics and reading articles about Common Lisp unit testing suites and cringing at the latest drama amongst Gentoo devs and minding my message board like a crusty old beat cop making his rounds. It's the life I've chosen, and I do like it, but I do like getting away sometimes too.

I fulfilled one of my dreams last week when I finally caved and ordered a solid glass mousepad. They're pretty cheap on newegg.com, depending on the color you want. I happened to want green, and it happened to be the cheapest, so all is well. It looks very nice, and it's big and hopefully the surface won't degrade over time; I tend to eat through mousepads via a slow yet inexorable process of erosion.

Unfortunately my laser mouse doesn't work on it. However, I have learned that if I upgrade my mouse's firmware, it will magically be able to work on a solid glass mousepad. Who would've thought my mouse had updateable firmware, let alone that updating the firmware would allow it to work on new surfaces? Not I.

The bad thing is that I need freaking Windows XP to upgrade the firmware on my mouse. I don't have any computer that has XP on it and I'm afraid to try anything in a virtual machine that involves something as dangerous as fiddling with the innards of connected peripherals. So I tried to install XP on my laptop, desperate times calling for desperate measures. But of course the install failed because my XP install CD is so old (pre-SP1, received free from my college 7 years ago) that it didn't recognize most of my hardware. In fact, the XP install CD blue-screened, which set a new record for how low Windows could sink in my opinion.

So I tried slipstreaming SP2 into my install CD. But it failed because, get this, the filenames of some drivers on the CD, namely usbehci.sys, ended up in lower case rather than uppercase and the CD's install program couldn't locate them. I kid you not. Since when is anything in Windows case-sensitive? Is it running Linux? I had to burn another CD after renaming all the files into uppercase. Then the CD worked, but it couldn't find my hard drive, probably due to missing SATA drivers. At that point I gave up, and plan to take my mouse to work tomorrow to upgrade the firmware on a work machine that has XP on it.

And so the score up to this point in my life is Windows: 947, Brian: 0. Windows remains undefeated.

Thanks go out to Logitech for not letting me use Vista (or, say, LINUX) to upgrade my mouse's firmware, and of course to Microsoft, for yet another gloriously broken and frustrating computing experience.

Google’s SketchUp is an incredibly easy-to-use architectural and 3D-modeling tool. The Computer Science Club here at Mansfield has been talking about modeling the university for a 3D perspective in Google Earth, similar to what Google Earth provides for New York City and other metropolitan areas. Recently, I have been experimenting with SketchUp. A free version is available from the website and a professional version can be purchased for about $500. People enrolled in Universities can obtain the professional licenses for $50 a year. The $50 counts towards purchasing a full license, so after 10 years, it’s yours forever. Or, you can pay for the educational license for 3 years and pay the rest of the money up front.

At a first glance, SketchUp seems to be too simple to be worth anything. Upon further investigation, the simplicity seems to come from Google’s innovational perspective. Google has outdone themselves with SketchUp. It is amazingly easy to pick up and create simple objects. More complicated objects can be created with some practice. I have spent about an hour and a half working with the program. First, I watched the beginning tutorials. Then, I went straight into making objects and refining them. It’s amazingly simple. Just make a shape, pull it up to give it depth, draw other objects on it, and manipulate them. It’s amazingly simple.

In about twenty minutes, I was able to make the desk that I use here at school. Keep in mind that it isn’t 100% perfect, nor is it 100% to scale. I have never seen a program this simple. With an hour and a half of experience, I was easily able to make this (click to enlarge):

Here is the front of it:

From an angle:

From the side (notice the arches in the drawers):

And here it is next to a person:

I consider OLPC to be one of the most exciting initiatives of the last few years. When the idea was first circulated it was such an exciting call to arms to do something about the lack of education in poor regions of the world. And the project has produced what appears to be a pretty incredible product, the research of which is now recycled back into the general hardware industry, so it has brought advances that wouldn’t otherwise have happened (now).

I recall pondering the real purpose of the project, asking what is going to be achieved with these laptops. The OLPC project had a very good answer to this. They said the laptops will promote learning in areas where school books are a luxury. Furthermore, the laptop itself is completely tweakable, you press a special key and the source code of the current program pops up. This will promote learning through tweaking and experimentation, so that eventually an industry can be built on these foundations, in regions where little industry exists today and where perhaps the potential for one (in terms of natural resources) is bleak. A beautiful dream, one that could change the world in big ways.

Now Negroponte has changed his tune. Visionary that he is, he failed to convince the clients of the value of free software. So now he’s humming “forget open source, it’s all about the kids!” while preparing to run Windows on the laptop. There is a new smoke screen being constructed:

Negroponte says that the organization is working to ensure that Sugar can run smoothly on Windows.

Riiiight, running Sugar on Windows. Tell me, what exactly is the value of running Windows with an all free software stack? It’s completely useless, that’s what. The whole value of Windows is as a platform, not merely as an operating system. People buy Windows to run Windows applications, not for Windows itself. Or are we actually buying that Egyptian officials are eager to purchase Windows licenses in order to run the free software suite?

Congratulations, Negroponte, you’ve just become a licensed Windows vendor. The kids will no doubt have fun clicking on the Start menu and playing Solitaire. There is a great deal to learn from that, just nothing about the operating system or the applications, you know, actual learning.

OLPC in its original form was about empowering the users, with Windows that capability is entirely destroyed. The fact you cannot mix learning with trade secrets should be blindly obvious to anyone. Open souce is important, but it’s especially important when you want people to learn something.

Furthermore, learning doesn’t happen in isolation. It’s accelerated when it happens in a community of ideas and impulses that flow freely. Resigning OLPC president gets it when he says:

“What comes part and parcel with open source is a culture, and it’s the culture that I’m interested in,” he says. “It’s a culture of expression and critique, sharing, collaboration, appropriation.” And this culture can and should spill into classrooms, he says.

Mal eben zum festhalten:

Einfache Schleife
for i in `ls` ; do echo $i ; done

In Dateien suchen und ersetzen
cat filename | sed ’s/suchen/ersetzen/g’ > filename-new

Dateien umbenennen und hinten etwas wegnehmen
mv $i ${i%-new}

For a very long while now, several years actually, I've been a bit annoyed by the behaviour of terminals under X when you doubleclick links. What the UI considers a word is selected. Selection 'starts' at the point that is doubleclicked and 'spreads' in each direction, stopping at a char it considers to be a word delimiter. A space is probably always considered a delimiter. Sometimes a '?' too, and often ',' as well. This has been very annoying for me as I spend quite alot of time in irc (irssi in a screen, its lovely in combination with bitlbee; icq, irc etcetc in the same screen.) and every now and then someone pastes an url which I want to doubleclick and then paste into my browser. Now the selection stops prematurely as it's not uncommon for a hyperlink to contain one of the characters that is in the word delimiter list. I've thought of this as a limitation to the system I use and, although annoyed, never gave it much thought. Maybe I've been to susceptible to propagandaists telling me gnu/linux is user-unfriendly.

A couple of weeks ago I started to think a little about it and the solution is rather simple. I use Eterm more or less exclusively.

In ~/.Eterm/user.cfg I've put this:

<eterm-0.9>
begin misc
cut_chars "\t\\\`\\\"\'() *,;<>[]{|}"
end misc

This might not be perfect but it's certainly served it's purpose this far as I have been able to doubleclick on the links and then middleclick in the address field (or open a new tab) in the browser. My OS is even better =)

Graduate School Information Request

---

Your request for information has been processed.

Thank you for your interest in the University of Massachusetts Amherst.

Applications directed to addresses within the United States are mailed twice weekly with first class postage; allow 10 days for delivery. Applications directed to non-U.S. addresses are mailed weekly and sent air mail; allow at least 3 weeks for receipt. There is no charge for any of these materials.

The following materials will be sent to you here:

Department of Computer Science
KICT, IIUM
Gombak, 53100 MYS

• CMPSC program materials

Sometimes you may want to check that an email address is not syntactically invalid, i.e. it looks like a recognisable email address. I use this approach in my zetact contact form processor.

Of course, it does not mean the address actually leads anywhere, but at least you know are dealing with an email address that could exist.

This is the code I have been using, albeit I have changed it from a class method to a simple function to make this post simpler.

"""Email check using regex."""

def invalidreg(emailkey):
    """Email validation, checks for syntactically invalid email             
    courtesy of Mark Nenadov.                                               
    See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65215"""
    import re
    emailregex = "^.+\\@(\\[?)[a-zA-Z0-9\\-\\.]+\\.([a-zA-Z]{2,3}|[0-9]{1,3\
})(\\]?)$"
    if len(emailkey) > 7:
        if re.match(emailregex, emailkey) != None:
            return False
        return True
    else:
        return True

I decided it would be more Pythonic to try to do this using the built-in string methods, rather than importing the re module and using a monster regular expression. Here was my first attempt.

"""Email checks using string methods - simple version."""
def invalidemail(emailaddress):
    """Checks for a syntactically invalid email address."""
    try:
        emailitems = emailaddress.rsplit('@', 1)
        emailitems.extend(emailitems[1].rsplit('.', 1))
    except IndexError:
        return True

    if [x for x in emailitems if not x.replace(".","").isalnum()] \
            and emailaddress >= 7:
        return True
    else:
        return False

After a bit of testing and playing with this, a friend pointed me towards the relevant RFC on restrictions of email addresses. While the standard allows the use of many different special characters, in practice email addresses have to be much stricter if you actually want people in the real world to be able to send email to you.

For example, if we allow the email address []@commandline.org.uk, will whatever receives the output of this function be able to use it? As pointed out by Jan Goyvaerts, most software won't actually be able to handle obscure special characters.

We also don't want to water down the syntax check and allow junk for the sake of theoretical but non-existent addresses.

My compromise is to allow these special symbols -_.%+. in the local-part of the email address, and -_. in the domain name. I also do sanity checking on the top-level domain, it needs to be either a generic name or two characters long (country codes are all two letters).

So below is my current version, I added lots of comments and white space to make it easy to read.

"""Ditch nonsense email addresses."""

GENERIC_DOMAINS = "aero", "asia", "biz", "cat", "com", "coop", \
    "edu", "gov", "info", "int", "jobs", "mil", "mobi", "museum", \
    "name", "net", "org", "pro", "tel", "travel"

def invalid(emailaddress, domains = GENERIC_DOMAINS):
    """Checks for a syntactically invalid email address."""

    # Email address must be 7 characters in total. 
    if len(emailaddress) < 7:
        return True # Address too short.

    # Split up email address into parts.
    try:
        localpart, domainname = emailaddress.rsplit('@', 1)
        host, toplevel = domainname.rsplit('.', 1)
    except ValueError:
        return True # Address does not have enough parts. 
    
    # Check for Country code or Generic Domain.
    if len(toplevel) != 2 and toplevel not in domains:
        return True # Not a domain name.
    
    for i in '-_.%+.':
        localpart = localpart.replace(i, "")
    for i in '-_.':
        host = host.replace(i, "")
    
    if localpart.isalnum() and host.isalnum():
        return False # Email address is fine.
    else:
        return True # Email address has funny characters.
    
# Start the ball rolling.
if __name__ == "__main__":
    print invalid("warrior@example.com")

Discuss this post - Leave a comment

[update] Question solved, see bottom of post.

Since Python 2.5 the language got a new built-in method ‘all’ (and it’s nephew ‘any’). I wanted to play around with this a little, combined with generators, so I created a little testcase to test performance.

Here’s the test-case: take a list L of X random numbers in a given range [A, B], and check whether

• all elements in L are >= A
• all elements in L are >= (A + Z) where Z is a number in [0, (B - A)]

The first test should always result True, the second test could result to False.

Here’s the output of a test-run:

In [1]: import random, sys

In [2]: a = [random.randint(100, sys.maxint) for i in xrange(2000000)]

In [3]: len(a)
Out[3]: 2000000

In [4]: #Check whether all elements are >= 100 

In [5]: %timeit all(i >= 100 for i in a)
10 loops, best of 3: 515 ms per loop

In [6]: %timeit any(i < 100 for i in a)
10 loops, best of 3: 454 ms per loop

In [7]: def f(l):
   ...:     for i in l:
   ...:         if i < 100:
   ...:             return False
   ...:     return True
   ...: 

In [8]: %timeit f(a)
10 loops, best of 3: 292 ms per loop

In [9]: #Same thing for 100000, since now the list shouldn't be completely iterated

In [10]: %timeit all(i >= 100000 for i in a)
100 loops, best of 3: 4.73 ms per loop

In [11]: %timeit any(i < 100000 for i in a)
100 loops, best of 3: 4.29 ms per loop

In [12]: def g(l):
   ....:     for i in l:
   ....:         if i < 100000:
   ....:             return False
   ....:     return True
   ....: 

In [13]: %timeit g(a)
100 loops, best of 3: 2.82 ms per loop

In [14]: #For reference

In [15]: %timeit False in (i >= 100 for i in a)
10 loops, best of 3: 531 ms per loop

In [16]: %timeit False in (i >= 100000 for i in a)
100 loops, best of 3: 5.03 ms per loop

It’s as if ‘all’, ‘any’ or ‘in’ don’t break/return when a first occurence of False (or True, obviously) is found. Is this the desired behaviour, and if it is, why? The calculation time difference between using all/any/in or a custom-made function (which is, unlike all etc, not written in C) which breaks whenever it can, is pretty astonishing.

[update] Question solved. It’s pretty normal the function-based approach performs better, since it combines what ‘all’ and the generator provided to ‘all’ do, taking away the generator function-call overhead. Damn

Thanks to my friend Bill, who has had the guts and perseverance to actually take my heap of code for a spin, I have finally gotten around to clean up the act a bit. I’ve i18n’d the source_manager.php and the source_edit.php. In addition, I have done a major restructuring of the ’sources’ table that I’ve been pondering for a long time, and I figured now was the time to do it. I have dropped the column small_text, and renamed the large_text column to source_text. The idea with the small_text field was originally to provide a kind of ’synopsis’. But soon I started to use it for ‘hidden text’ instead. I have now changed the method for inputting hidden text in sources as follows: If you enclose any text in {curly braces}, it will not show up anywhere but in the Source Manager and the Source Edit screens. I think that is a more intuitive method. In addition, it simplifies the model.

I also found a bug in the previous version of source_edit.php; it didn’t show the contents of the small_text field, and it didn’t update it either.

In addition I have run regression tests and fixed a number of roadblocks. In its present version it should work almost out of the box, provided you do as outlined in the README.txt.

If you’ve been dealing with files for a while you will have noticed that there is a slight semantic gap between how humans see files and how computers do. If you’ve ever seen a file list like this you know what I mean:

Lecture10.pdf
Lecture11.pdf
Lecture12.pdf
Lecture1.pdf
Lecture2.pdf
…

Numbering these files was done in good faith, and a user understands what it means, but the computer doesn’t get it. Sorting in dictionary order produces the wrong order as far as the user is concerned. The reason is that the digits in these filenames are not treated and compared as integers, merely as strings. (Actually, . comes before 0 in ASCII, what’s going on here?)

While we’re not expecting our computers to wisen up about this anytime soon, there is the obvious fix:

Lecture01.pdf
Lecture02.pdf
…
Lecture10.pdf
Lecture11.pdf
Lecture12.pdf

You’ve probably done this by hand once or twice, while cursing.

On the upshot, this is very easy to fix with a few lines of code:

#!/usr/bin/env python
#
# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 3.
#
# revision 1 - support multiple digit runs in filenames
 
import os, string, glob, re, sys
 
def renseq():
    if (len(sys.argv) != 2):
        print "Usage:\\t" + sys.argv[0] + " <num_digits>"
    else:
        ren_seq_files(sys.argv[1])
 
 
def ren_seq_files(num_digits):
    files = glob.glob("*")
    for filename in files:
        m = re.search("(.*)(\\..*)", filename)
        ext = ""
        if m: (filename, ext) = m.groups()
 
        digit_runs = re.finditer("([0-9]+)", filename)
        spans = [m.span() for m in digit_runs if digit_runs]
        if spans:
            spans.reverse()
            arr = list(filename)
            for (s, e) in spans:
                arr[s:e] = string.zfill(str( int(filename[s:e]) ), int(num_digits))
            os.rename(filename+ext, "".join(arr)+ext)

 
 
if __name__ == "__main__":
    renseq()

Download this code: renseq.py

This works on all the files in the current directory. Pass an integer to renseq.py and it will change all the numbers in a filename (if there are any) to the same numbers, padded with zeros if they have fewer digits than the amount you want. So on the example

renseq.py 2

will turn the first list into the second list.

If say, there are filenames with numbers of three digits and you pass 2 to renseq.py, the numbers will be preserved (so it’s not a destructive rename), you’ll just revert to your incorrect ordering as it was in the beginning.

renseq.py will rewrite all the numbers in a filename, but not the extension. So mp3 won’t become mp03.

I had to fix a problem at my dad's company...
"The network was broken."

It was a NetBEUI network connecting some windows stations - it has been running for years - and now suddenly the nodes couldn't find eachother.
One of the boxes (windows 2000 iirc) had 2 network cards, one for the network, the other not used for anything (not even connected). Disabling the latter - not even touching the former - fixed half of the network.

read more

Someone in #lh today told me about Sender Policy Framework, which sounds like a badly-needed enhancement to the Internet’s email protocols. Basically, the idea is to provide a DNS record that informs MTAs “don’t trust emails claiming to be from this domain unless they’re coming from one of my actual servers".

In DNS, this looks like (in my case):

emptymatter.org. IN TXT "v=spf1 a mx ~all"

Some MTAs support SPF but need to be configured, I believe Gentoo’s postfix is one of them. If I’m going to expect other mail servers to support it, I probably should myself. I’ll have to tackle that another day…

Since Heise announced that Microsoft will release the Windows XP Service Pack 3 on the 29th, I didn’t sleep too well, as I really want to grab it as soon as possible. Sure, I use Linux, but this doesn’t mean that I am not dangling with windows boxes from time to time and I am fed up applying about 100 patches before I can even think of security.

But I was laughing too soon - as just one day after the 29th (today), I have seen a posting on the Washington Post Blog that Microsoft has delayed the start of the service pack again. In a written statement they say:

“In order to make sure customers have the best possible experience we have decided to delay releasing Windows XP SP3 to Windows Update and Microsoft Download Center.”

In other words, there seems to be no release date yet. Well - in the meantime I’m rolling my own update pack using the Heise Offline Update. Thanks a ton, guys.

Nothing exciting here. Got my tax refunds. Might build a home-made NAS with a couple terabytes of disk and put it in the basement.

On the DS, I’ve been playing Rondo of Swords and The World Ends With You. Rondo is a pleasant find, a difficult but still reasonable strategy RPG that makes one think and plan ahead, unlike games such as Revenant Wings which are much more “bring a healer and just mob everyone at the thing they’re strong against!” Also, I have a crush on Atlus by this point. There’s no denying it now. I draw their name with little hearts all around when I’m in meetings.

The World Ends With You is refreshingly original, one of those games that, even with it being Square Enix, is a bit surprising that it made it to the States. Very Japanese, and the game makes few concessions to the English audience. Sure, long gone are the times of gratitutous name changes, but even the j-pop/j-rock soundtrack remains intact, and that is, to my slightly jaded mind, a bit commendable. Now, if only the main character didn’t suffer from two vile Square Enix staples: unimaginable thinness and nearly sickening teenage angst. Neku is supposed to get better with the latter; I hope it is soon.

My games to beat are now Etrian Odyssey and Rondo of Swords, one I must beat before Etrian Odyssey II (guess which one) is released here, and the other before the Final Fantasy IV remake reaches the States. I’m excited. If I have time before those, Final Fantasy III and The World Ends With You are my RPGs to beat. FF3 is a cakewalk thus far, but its ease and its crude mechanics compared to Final Fantasy V make it hard to stay with for long.

I didn’t really intend this to become all about video games. I’ve been working on a Gentoo Wiki page for the HP 2133 which has kind of slowed down as most of the parts I’m interested in are supported as best they can be without new versions of drivers, I think. There’s some other hardware that I need to try out (the webcam, for example), but I don’t really care that much, so it’s low priority. Notebooky stuff works.

I have a Waterfield Designs bag coming soon, which I’m excited about. Don’t think it will be suitable for gaming books, but I still have that backpack which is going on 5+ years. The little trooper.

I’ve been meaning to survey the gaming group and associated friends to see what they’re using for IM these days. I think the answer for some is “nothing", with a couple saying “AIM on occasion” or “I idle on Google Talk", so I’ve not really been motivated to test those waters. I want to get a private Jabber conference room running for the group, since the IRC thing kind of sputtered off and died (I still idle there!), but I know it means getting people to switch to Jabber (or at least Google Talk) and then getting them to use a non-Google Talk client (Pidgin, I bet, but maybe Trillian would work). Sigh. If anyone has interest in switching to one network (I highly suggest a Jabber-like ["XMPP” for the techies]), or trying out conferences, or whatever, email/IM me and we’ll play around.

This really is getting rambly, and people might expect me to write long posts all the time. So I’m wrapping this up by saying that spring is finally here, and that’s why it snowed yesterday.

Use Readline shortcuts

At the bash prompt, you can use the default readline keybindings, these are similar to Emacs ones. Many of these are also available within other programs that use readline, such as the Python interpreter.

Here are some useful ones:

Ctrl-A Beginning of Line
Ctrl-E End of Line

Ctrl-U Kill (cut) everything left of cursor
Ctrl-K Kill (cut) everything right of cursor
Ctrl-W Kill (cut) the single word before the cursor
Ctrl-Y Yank (paste) the text back

Ctrl-L Clear Screen
Ctrl-D Exit
Ctrl-R Reverse interactive-search, (attempt to complete what is currently being typed using the history file)

SSH without Passwords

If you login to a remote machine often and you get bored of typing the password, then you can use public key cryptography instead.

The way it works is that the remote machine has a copy of your local machine's public key, it can then use that to check that your local machine is really your machine, and so let you in.

To start with, on the local machine, see if you already have a key pair:

ls ~/.ssh/id_?sa.pub

If not, then make one:

ssh-keygen -t dsa

Now you need to copy your public key to the remote host. On the local machine run:

scp ~/.ssh/id_?sa.pub remotehost:

Now we login to the remote server:

ssh remotehost

Append the public key to your authorized keys file

cat id_?sa.pub >> ~/.ssh/authorized_keys

Now you can login without passwords. Make sure the security of your machines is well thought out. Use disk encyption if possible.

Create a script directory in home directory

I often talk about random Python or bash scripts. The easy way to use them on Linux is to make a dedicated script directory for these.

mkdir ~/bin

Add it to your shell's path. Edit ~/.bashrc and add:

export PATH=$HOME/bin:$PATH

Now all the scripts that you add to ~/bin are always available. This makes things a lot more flexible and fun as you can try out various scripts by dropping them in ~/bin and then deleting them when you are bored of them.

Discuss this post - Leave a comment