Planet Larry

June 29, 2008

Brian Carper

Lispforum.com

Ten days ago I complained that there were no good Lisp equivalents of ruby-forum or perlmonks. It looks like someone went and made one. What good timing.

I hope it's a success, and I hope it stays newb-friendly. The amount of fake watch and shoe spam on comp.lang.lisp has reached critical mass.

Speaking of mailing lists, maybe it's just me but I've never found mailing lists to be all that enjoyable to use. They have the benefit of being a sort of lowest common denominator (everyone has email, and you can slap an HTML interface on top of one). They also have the benefit of being distributed to some degree, because everyone who gets the email serves as an archive, and if the main server dies maybe you can recover things. And mailing lists do have less overhead than MBs when it comes to running one, especially a high-traffic one, I would imagine.

But the bad things about mailing lists vs. message boards:

  • Message boards are accessible from anywhere that you have a web browser, which is everywhere. Email isn't necessarily accessible from everywhere, unless you use webmail or SSH home and use mutt or something, which not everyone can or wants to do. Or if there's a good web interface on the mailing list.
  • You can't do anything more than plaintext, which isn't entirely a bad thing, HTML email is pure evil, but being able to cleanly post images or clickable links or formatted text on a message board is a nice feature.
  • Threading never quite works correctly on mailing lists, because eventually someone will hit the wrong button in their mail client and break the thread; whereas on a message board it always works fine.
  • You can move threads around between forums on an MB, you can edit threads, you can close threads, you can delete a post if you make a mistake; but mailing lists are write-only, and once you send a message off into the ether it's posted for everyone to see forever, and no one has much control over a list beyond moderating the messages that end up getting through.
  • Avatars. Personal profiles. These things make people seem more like people and less like a nameless entity. It's friendlier and more inviting.
  • The HTML interfaces people slap on top of mailing list archives are pretty horrible 95% of the time. Probably because most people are using email clients anyways so no one cares. Message boards generally look nice and have nice interfaces for reading and posting.
  • Email sucks. Spam filters and bounced messages mean you never quite know if what you just wrote actually made it to the list. Reply to list vs. reply to sender vs. reply to all, etc. are all needless complications. How many times have you seen "UNSUBSCRIBE" sent to everyone on a list? The interface to mailing lists is not intuitive. Whereas you can always see immediately if an MB post worked or not.

And so on. I likes me my message boards.

June 29, 2008 05:51 AM :: Pennsylvania, USA  

June 28, 2008

Niel Anthony Acuna

linux raw spin locks

apparently, there are some issues with linux 2.6 and pearpc.. so i still cant
get the gentoo ppc to start properly. i thought i’d do some studying in the
meantime..

typedef struct {
        volatile unsigned int slock;
} raw_spinlock_t;
static inline void __raw_spin_lock(raw_spinlock_t *lock)
{        asm volatile(\"\n1:\t\"
                     LOCK_PREFIX \" ; decb %0\n\t\"
                     \"jns 3f\n\"
                     \"2:\t\"
                     \"rep;nop\n\t\"
                     \"cmpb $0,%0\n\t\"
                     \"jle 2b\n\t\"
                     \"jmp 1b\n\"
                     \"3:\n\t\"
                     : \"+m\" (lock->slock) : : \"memory\");
}

interpretation of the extended inline asm:

1) : “+m” (locl->slock)
“+m” means that the memory operand ‘lock->slock’ is both input and output.

2) : “memory”
from gcc manual:
If your assembler instructions access memory in an unpredictable
fashion, add `memory’ to the list of clobbered registers. This will
cause GCC to not keep memory values cached in registers across the
assembler instruction and not optimize stores or loads to that memory.

3) LOCK_PREFIX is defined at /usr/src/linux/include/asm/alternative.h:

#define LOCK_PREFIX \
                \".section .smp_locks,\\"a\\"\n\"   \   ; push current section and change section to .smp_locks
                \"  .align 4\n\"                  \   ; align to 4 bytes
                \"  .long 661f\n\" /* address */  \   ; create an entry in the up/smp alternative locking table
                \".previous\n\"                   \   ; pop back the old section
                \"661:\n\tlock; \"                    ; assert processor LOCK# signal

LOCK_PREFIX is supposed to be called within the context of a procedure (executable) which finds its way to a .text section.

i haven’t digged down deep into the internals of ‘alternatives’ yet, but a quick look at kernel/alternative.c :: alternatives_smp_unlock() shows that it’s employing a self modifying code that patches the “lock” instruction with an architecture specific NOP opcode.

without all the extended asm semantics above, the asm roughly translates to:

1:      lock       ; decrement atomically
        decb    %0 ; really decrement now..
        jns     3f ; successful in acquiring lock?
2:      rep        ; nothing
        nop        ; nothing
        cmpb    $0, %0 ; someone must release the lock first.
        jle     2b     ; busy wait...
        jmp     1b     ; lock available! try acquiring lock again.
3:     ; exit __raw_spin_lock()

and then comes the trylock variant

static inline int __raw_spin_trylock(raw_spinlock_t *lock)
{
        char oldval;
        asm volatile(
                \"xchgb %b0,%1\"
                :\"=q\" (oldval), \"+m\" (lock->slock)
                :\"0\" (0) : \"memory\");
        return oldval > 0;
}

interpretation of the extended inline asm:

1) : “=q” (oldval)
output oldval in any a,b,c or d register.

2) : “+m” (locl->slock)
“+m” means that the memory operand ‘lock->slock’ is both input and output.

3) : “0″ (0)
%0 matches the output register for oldval. initialize that with the value 0.

4) : “memory”
from gcc manual:
If your assembler instructions access memory in an unpredictable
fashion, add `memory’ to the list of clobbered registers. This will
cause GCC to not keep memory values cached in registers across the
assembler instruction and not optimize stores or loads to that memory.

__raw_spin_trylock() basically exchanges the current value in raw_spinlock_t->slock with zero and returns true if locking was successful and false if acquiring of lock didn’t succeed.

lastly, we have the raw unlock procedure:

static inline void __raw_spin_unlock(raw_spinlock_t *lock)
{
        char oldval = 1;
	
        asm volatile(\"xchgb %b0, %1\"
                     : \"=q\" (oldval), \"+m\" (lock->slock)
                     : \"0\" (oldval) : \"memory\");
}

i haven’t been able to understand the significance of the “%b” in the assembler template but looking at the asm listing of the template with and without the %b yielded the same intructions, so could anyone shed some light on this one?

basically looks like __raw_spin_trylock(), but the difference is that oldval is now initialized to 1 instead of 0.

there are many variants stemming from the idea of this implementation. raw reader/writer locks which allows multiple readers but only one writer (with no concurrent readers). and then there’s the spinlock layer above the raw spinlock implementation.. but those are for another post.

’til then.

June 28, 2008 04:04 AM :: Zamboanga, Philippines  

Martin Matusiak

spiderfetch, now in python

Coding at its most fun is exploratory. It’s exciting to try your hand at something new and see how it develops, choosing a route as you go along. Some poeple like to call this “expanding your ignorance”, to convey that you cannot decide on things you don’t know about, so first you have to become aware - and ignorant - of them. Then you can tackle them. If you want a buzzword for this I suppose you could call this “impulse driven development”.

spiderfetch was driven completely by impulse. The original idea was to get rid of awkward, one-time grep/sed/awk parsing to extract urls from web pages. Then came the impulse “hey, it took so much work to get this working well, why not make it recursive at little added effort”. And from there on countless more impulses happened, to the point that it would be a challenge to recreate the thought process from there to here.

Eventually it landed on a 400 line ruby script that worked quite nicely, supported recipes to drive the spider and various other gimmicks. Because the process was completely driven by impulse, the code became increasingly dense and monolithic as more impulses were realized. And it got to the point where the code worked, but was pretty much a dead end from a development point of view. Generally speaking, the deeper you go into a project, gradually the lesser the ideas have to be to be realized without major changes.

Introducing the web

The most disruptive new impulse was that since we’re spidering anyway, it might be fun to collect these urls in a graph and be able to do little queries on them. At the very least things like “what page did I find this url on” and “how did I get here from the root url” could be useful.

spiderfetch introduces the web, a local representation of the urls the spider has seen, either visited (spidered) or matched by any of the rules. Webs are stored, quite simply, in .web files. Technically speaking, the web is a graph of url nodes, with a hash table frontend for quick lookup and duplicate detection. Every node carries information about incoming urls (locations where this url was found) and outgoing urls (links to other documents), so the path from the root to any given url can be traced.

Detecting file types

Aside from the web impulse, the single biggest flaw in spiderfetch was the lack of logic to deal with filetypes. Filetypes on the web work pretty much as well as they do on your local computer, which means if you rename a .jpg to a .gif, suddenly it’s not a .jpg anymore. File extensions are a very weak form of metadata and largely useless. Just the same with spidering, if you find a url on a page you have no idea what it is. If it ends in .html then it’s probably that, but it can also not have an extension at all. Or it can be misleading, which when taken to perverse lengths (eg. scripts like gallery), does away with .jpgs altogether and encodes everything as .php.

In other words, file extensions tell you nothing that you can actually trust. And that’s a crucial distinction: what information do I have vs what can I trust. In Linux we deal with this using magic. The file command opens the file, reads a portion of it, and scans for well known content that would identify the file as a known type.

For a spider this is a big roadblock, because if you don’t know what urls are actual html files that you want to spider, you have to pretty much download everything. Including potentially large files like videos that are a complete waste of time (and bandwidth). So spiderfetch brings the “magic” principle to spidering. We start a download and wait until we have enough of the file to check the type. If it’s the wrong type, we abort. Right now we only detect html, but there is a potential for extending this with all the information the file command has (this would involve writing a parser for “magic” files, though).

A brand new fetcher

To make filetype detection work, we have to be able to do more than just start a download and wait until it’s done. spiderfetch has a completely new fetcher in pure python (no more calling wget). The fetcher is actually the whole reason why the switch to python happened in the first place. I was looking through the ruby documentation in terms of what I needed from the library and I soon realized it wasn’t cutting it. The http stuff was just too puny. I looked up the same topic in the python docs and immediately realized that it will support what I want to do. In retrospect, the python urllib/httplib library has covered me very well.

The fetcher has to do a lot of error handling on all the various conditions that can occur, which means it also has a much deeper awareness of the possible errors. It’s very useful to know whether a fetch failed on 404 or a dns error. The python library also makes it easy to customize what happens on the various http status codes.

A modular approach

The present python code is a far cry from the abandoned ruby codebase. For starters, it’s three times larger. Python may be a little more verbose than ruby, but the increase is due to a new modularity and most of all, new features. While the ruby code had eventually evolved into one big chunk of code, the python codebase is a number of modules, each of which can be extended quite easily. The spider and fetcher can both be used on their own, there is the new web module to deal with webs, and there is spiderfetch itself. dumpstream has also been rewritten from shellscript to python and has become more reliable.

Grab it from github:

spiderfetch-0.4.0

June 28, 2008 01:18 AM :: Utrecht, Netherlands  

June 27, 2008

Matt Harrison

Why doesn't pushd popd work in my Ubuntu shell script?

I'm in the process of moving over to my new laptop. I run into an issue where a shell script (that worked under gentoo) now no longer works. Here's the error foo.sh: 18: pushd: not found Since I'm running this as sh foo.sh, I turn on debugging wit

June 27, 2008 04:42 AM :: Utah, USA  

New lappy, Lenovo T61p (linux edition)

My trusty r52 switches between a state of 100-200 free megs of memory and no memory on the 40 gig hard drive. Rather than investing in an expensive IBM harddrive (I've heard the BIOS on my machine won't take normal OEM drives), I got a new computer.

June 27, 2008 04:30 AM :: Utah, USA  

June 26, 2008

George Kargiotakis

rxvt-unicode 256 color support with vim

Following my previous post on minimizing the resources that urxvt needs on Gentoo, I tried applying some more patches to it that I found in Gentoo’s bugzilla. Since that happened a few days ago there was no ebuild for version 9.05 yet. So I created one and applied the patch for the 256 color support. Here’s my [...]

June 26, 2008 11:14 PM :: Greece  

Thomas Capricelli

Static link with cmake under windows (Qt and others)

One of the great things about Qt is that you can compile the code under esoteric OSs (mmh?) like Windows or MacOS. Although I’ve almost never used windows, I did the packaging of yzis. I used the Nullsoft Installer, which is ok.

I had several issues with static linking, and thought maybe some of you could give some help about it. Don’t ask why I want static link, this is not the issue here.

Problem #1:

The most important one is Qt itself. I expected that to be easy, but it is not. The FindQt4 shipped with cmake 2.4.8 or 2.6 is not really aware of static lib, or is it ? Nor is the one from kde svn. I’ve tried using the one from quassel-irc, but it fails as well. If you can help me with this, the code is available.

It seems (according to #qt on irc) that it makes sense to have two different Qt installed, one static and one shared. I did that (double the space used…) and in one of them did “configure -static” and recompiled qt. Now the *.a are much bigger, and though qyzis links against those *.a, it still depends on the DLLs, according to “dependencies walker”. Gr…

Problem #2:

Then, there is the problem of gnuwin32 tools. In the lib/ directory, you can find for example

  • libintl.dll.a
  • libintl.def
  • libintl.lib

And the DLL (libintl3.dll) is in the bin/ directory. I need to ship the DLL with qyzis, because cmake found the dll, and linked against libintl.dll.a. In a perfect world, I would like cmake to use the libintl.lib and have a static link, but i dont know how to convince cmake of doing that. Do you know ?

Problem #3:

How do you find out what kind of library a file is ? According to what I’ve found googling the web, libraries can have a wild number of different names (libX.dll, libX.dll.a, libX.lib, libX.a, all of this without the lib prefix and so on…). Even worse : libX.a for example, could be either a static lib or the stub for the DLL. Do you have clarifications about this ? Do you know of a reliable way to know which kind of library it is ? (DLL, DLL stub, or purely static).

June 26, 2008 10:23 PM

June 25, 2008

Thomas Capricelli

Release of Yzis-1.0-alpha1

Ok, here it is, things have finally settled and we are happy to bring you the first alpha release for Yzis. The foundations are stabilizing, and we are now focusing on fixing bugs and porting to different architectures, OSs, and interfaces.

What you have so far :

The most noteworthy missing feature is the KDE embeddable component, we are aware of this, and this is the main thing we shall work on after 1.0 is out.

At this point, we are not yet asking for broad testing, but developers not afraid of bleeding edge are encouraged to test it out. If you can help with porting/solving bugs, this is of course even better.

You can join us on #yzis on freenode to discuss all of this.

The web site has been updated, and you should find information about getting the source from source control (mercurial) and compiling on most platforms.

http://www.yzis.org

The tarballs and the windows installer can be downloaded from this url:

http://labs.freehackers.org/projects/list_files/yzis

Below are some screenshots showing the different interfaces (qt/linux, ncurses, qt/windows):

ncurses frontend

June 25, 2008 10:54 PM

Ray Booysen

Sumatro

I’ve been pushing some new photography to my photoblog over at Sumatro.com.  I have a new set of work lined up, all being published at 9am every morning.  To keep up to date, subscribe to the RSS feed.

Would love some comments/idea/abuse on the photos if you have the time. :)

June 25, 2008 07:55 PM :: England  

Jürgen Geuter

Your automatically generated APIdoc is not a replacement for real documentation

When dealing with software you have not written yourself (or have written a year ago which can make it as alien as code someone else wrote) you have to rely on the software's documentation.

The quality of documentation varies a lot from one project/software to another: Some projects have really good and useful documentation, some just gather links to articles by third parties and some completely lack documentation, the quality of a software's documentation should always be one of the most important arguments for or against using a certain software.

One of the worst kind of "documentation" is the "API docs". It's not useless at all, you need it if you use that on class from the software and you really need to know in detail which methods instances of that class have, but some projects seem to think that it is a replacement for real documentation.

When I had to learn JAVA at university we had to do a so-called "software project" written in JAVA (it was with a group of around 10 people and ran for half a year). Back then JAVA's documentation was pretty much just their auto-generated "JAVAdoc" API documentation. (They had a few tutorials but those were written so badly that they were of no real use to someone trying to figure out how to do something.)

API docs help when you are in the middle of doing something and you need details about the entities you work with, when you have already picked your entities, but they do not help you when you are looking for something like "I need a container for a bunch of different objects that might or might not have the same type with the following properties...".

Let's look at a few project's documentation and see how good they are:

Python



The Python documentation is separated into different segments: There's the Tutorial that's even marked with "Start here" so the beginner knows where to dive in. Then there's the library reference that explains what a certain built-in module can do and how it works before diving into the details of the classes and functions that module supplies.

The Python docs are very good, it's easy for a newbie to dive in while still offering people that need info about that one special function a direct and quick access. It's a good compromise but does lack something that other languages' docs have: User comments that can sometimes elaborate something that the docs don't make clear enough (you just cannot see any problem that people might have).

PHP



PHP's documentation is not as strictly separated as Python's, but the document is separated into chapters that somewhat mirror the segments that Python's docs have. It starts with a tutorial, then goes over the language syntax with then going over the modules that PHP comes with.

PHP is easy to learn but that is also due to the really good docs: You go through the first two chapters of the docs and you know your way around. Then you can look up the special functionality that you need for whatever you do. The biggest plus point of the PHP docs is that every documentation page has a comment section where uses can contribute notes about quirks that a certain function has or that offer some example for common use cases. The md5 documentation for example shows you why your calculated md5 hash might not match one you calculated in .net or mysql. This is really useful not only to new users but also people that do know their way around but stumble on some weird quirk.

All in all PHP might be a badly designed language but the documentation is really good and helps you dive into it quickly and effectively. The user comments are (while there is of course the danger of spam) a very valuable tool to enrich your docs with information that might just not fit into the actual documentation article.

Ruby On Rails



Ruby on Rails's docs are separated in the API docs, a link to a bunch of books you can buy and a bunch of links to tutorials somewhere on the net.

This is an example for bad documentation: The team just supplies API docs that are not really all that helpful if you don't know exactly what you are looking for in the first place. Having the second aspect in your docs be advertising for books that are not available for free does make your thingy look like somewhat of a scam, the new users feels like he has to pay a tax to start using your stuff. Not good.

The fact that there are good tutorials on the net is neat and will help users, but what happens when you change something and the linked tutorial doesn't work anymore? Your users will try to use it and fail and how should they be able to determine which tutorial is right and which ain't? If they have two tutorials that work but that work somewhat different, how can they determine which can be considered "best practice"?

All in all the Ruby On Rails documentation is not good, while the framework might be opinionated (something I always like) it lacks tutorials by the devs that the user can be sure are correct and teaching best practices.

Django



Django's documentation is separated into a few main parts: The essential docs that include installation howtos, the tutorial working you through creating an application with django and and the FAQ. The next segment called "Reference" is going into detail about the different parts of django and how to work with them. The docs end with a howto on deployment and a few howtos for common problems.

The best thing about the django docs is the style. It's not weird technobabble or random API docs: It's written in plain English with a clear focus on the audience. The template language documentation for example is separated in the part for "HTML authors" and the part for "Python Programmers", with the first part being completely read- and understandable by someone with no clue about python or django (like someone who's just a designer that builds the HTML for a django application) and the latter one telling you how to do powerful thingy by extending the built-in stuff.

All in all Django's docs lack the comment feature that PHP has but probably have the best writing style and are the easiest to work with and understand.

All documentation I reviewed here is for rather popular projects and it's already obvious how different the quality in those big projects is, when you come to even smaller projects the quality can diverge even more.

Some big projects I didn't have to review in detail (like JAVA's docs that are pretty much still a trainwreck) but what you always have to keep in mind: A software with bad documentation that does not empower you will require you to buy books where someone explains what all the fuss is about. But usually there's not "the one" book, look at how many JAVA books there are (JAVA is just one example, C, C++, C# ain't much better)! With so many books around you'll have to dig through book recommendations to see which one of the books is worth reading and which might help you, which is even more time you have to invest into something plus the books usually ain't free.

When you decide on the software you want to use for your next project, have a look at the documentation for the project. Read through it so see if you feel like it's really telling you how it works. If there are books available that is a plus (see, I don't hate books! ;-) ), but you shouldn't be forced to buy books just to be able to use something; books should provide something different that the official docs, like the O'Reilly "cookbooks" that offer recipes for common problems for example. A book can also wrap the documentation up in a different way which can help you understand them better.

Do you know other projects that have a particularly good or bad documentation? Please share them in the comments.

June 25, 2008 09:54 AM :: Germany  

Christoph Bauer

Static webpages

Are you ‘up to date’ if you are still using a static webpage today? I wouldn’t say that there is a global answer to that question and everybody’s got to find out for himself.

Usually I’m a big fan of automatism and technology and its advantages - but is it really worth it? Let’s have a closer look at the topic:

A Content Management System (CMS) is a program, running on your webserver, generating the html output for the visitors browser, which is actually a good thing, if things are changing a lot.

But a CMS is just a piece of software that needs updates and patches too, disregarding new features and stuff as there might be undiscovered security holes which might turn out to be a rather serious problem. I don’t want to question any cms here. I just want to make you think if you really need one.

I’ll give you an example: My domain stargazer.at hosts a portal page, offering links to my services, my blog, whatever. As my services don’t change that often, there is no need for dynamical created content as things just don’t change. If I think back, the last update on the portal page happened four month ago. Since then it wasn’t touched anymore. Regarding security, this would be quite fatal. In the other hand, my blog here gets much more love and changes much more often. That’s why I am using Wordpress here and plain html on my portal page.


Copyright © 2007
Please note that this feed is for private use only. All other usage, including the distribution or reproduction of multiple copies, performance or otherwise use in a public way of the images or text require the authorization of the author.
(digitalfingerprint: 0f46ca51d0fa4e6588e24f0bf2b80fed)

June 25, 2008 08:50 AM :: Vorarlberg, Austria  

Sean Potter

Slowly coming along...

Nick was home from college this past week, and I took a lot of time away from working on the blog and BIOSLEVEL to spend some time with friends. It was a good time for sure, but I always come to regret it when I realize how little work I've done.

Hopefully I get things back on track over the next few days. I'm way behind on several reviews for BIOSLEVEL, and some of the products were just released in the last two weeks or so.

I also rebuilt two of my computers in to new housing, with a few new components that were sent to me for review. Let me say that I hate reviewing some CPU coolers. A certain company that will go unnamed for now sent two CPU coolers that otherwise seem impossible to install on an Intel machine. Stay tuned for a review, if I ever get them installed.

On top of all that, I somehow started playing World of Warcraft again.

June 25, 2008 07:59 AM

Brian Carper

Laptops at border crossings

There's an article on Slashdot about a US Senate hearing on laptop seizures at border crossings. This affects me, because I travel to Canada a lot and plan to move there within a year or so.

It's a problem because my job requires me to handle what amount to people's medical records as data files on my laptop. It's part of my job, and often I work from home. As of right now, I never take my laptop with me to Canada partly because I don't know what would happen if a border agent decided to inspect or copy all of my data. I can get in very serious trouble for breaching patient confidentiality. On the other hand I could get in serious trouble if I refused to allow a search for myself; at best I'd be turned way at the border, having wasted hundreds of dollars to travel there.

I really don't know what I'm going to do when I move. I'll probably have to wipe my computers clean before shipping them up there. Another option would be to encrypt all the data, upload it to the server that hosts my website, then download it all again after I move. It's insane that I'd have to do such a thing though. And shuffling sensitive data around to strangers' computers and servers isn't the safest thing in the world either.

How do lawyers and doctors and people with trade secrets and other people with classified or legally protected information handle border crossings? It's a bit of a conflict of interest.

June 25, 2008 06:08 AM :: Pennsylvania, USA  

June 24, 2008

Johannes Gilger

The GitHub

GitHubOk, I hinted that I would do a more thorough review of GitHub, the new and easy-to-use git repository hosting site. Although I’m still no power-user I’ve come to know the features that make GitHub worth using and so far unique.
The “Fork” feature is probably the most important one. Instead of just cloning a repository and working on it locally, you can fork it on GitHub. When you fork a project everyone can see you did, and has a nice flashy graph-view so they can see where you branched and what commits you made that are not (yet) in the upstream. GitHub forkAnd if you have introduced changes that you think would benefit the project you can send people (e.g. the original project owner) a “pull-request”. The recipient can then easily fetch/merge your changes into his project. It really doesn’t get much easier to contribute to (open-source) projects. I certainly did for the first time ;)

GitHub forks

Then there are feeds. You can watch projects, which means that your feed includes any commits/comments on the watched projects. It goes without saying that this can be quickly overwhelming for active projects. GitHubComments can be made on specific lines in a commit (or on the whole commit), which is a great feature (think of it as the equivalent to patches being discussed line-by-line in mailing-lists). I still prefer mails though ;)
The syntax-highlighting looks pretty good. I’ve already mentioned the very pale interface (as in low-contrast) and it still has not changed. But I think that most people really browse the commit history in their local clones anyway.
Each project has an attached Wiki too, so you can add a few pages (or a lot if you wish).
To conclude I can say that GitHub is a great service, since it has a free plan for public projects. I would not need it for my personal projects, but to run a or contribute to a open-source project it’s perfect. The amount you use its features is up to you. If you already have a Wiki, already have an active mailing-list and discuss patches there too then you can just use GitHub as the central source code repository (plus the forks of course). The real work is done on your local repository with git anyway, but y’already knew that I guess ;)

June 24, 2008 08:46 PM :: Germany  

Martin Matusiak

emacs that firefox!

So the other day I was thinking what a pain it is to handle text in input boxes on web pages, especially when you’re writing something longer. Since I started using vim for coding I’ve become aware of how much more efficient it is to edit when you have keyboard shortcuts to accelerate common input operations.

I discovered a while back that bash has input modes for both vi and emacs and ever since then editing earlier commands is so much easier. And not only does it work in bash, but just as well in anything else, like ipython, irb, whatever. :cap:

So now only Firefox remains of my most used applications that still has the problem of stoneage editing, and I’m stuck using the mouse way too much. It bugs me that I can’t do Ctrl+w to kill a word. Thus I went hunting for an emacs extensions and what do you know, of course there is one: Firemacs. Turns out it works well, and it also has keyboard shortcuts for navigation. > gets you to the bottom of the page, no more having to hold down <space>. :thumbup:

June 24, 2008 08:20 PM :: Utrecht, Netherlands  

Jürgen Geuter

NVIDIA: Fuck you.

(Imagine the headline being sung to the melody of the "America: Fuck yeah!" song from the "Team America - World Police" soundtrack).

After a bunch of kernel developers issued a statement concerning closed source kernel drivers arguing that closed source kernel drivers are pretty much bad for everyone (the reasons are obviuously that binary modules have a hard time to keep up with kernel development, them crashing makes the kernel look bad and the community cannot fix issues within those blobs, in addition to the fact that many consider them to be illegal) NVIDIA replied:
Nvidia reiterated that it won’t provide open source drivers for Linux because the company claims there is no need for it.


So there is no need for it? What about all the problems people are having getting your blob to run because it takes ages for you to support new Xserver functionality or because it takes you quite some time to modify your blob to work with recent kernels? What about the problems many people with your graphics cards have with suspending their machines? What about the need to hack around so your graphics cards don't start with the fan pretending to be a helicopter? What about your "we write our own aiglx implementation that works a little differently than the default one just because we like to do that kind of stunt"?

If there's no need for free drivers, why does the nouveau project exist? Why do people keep on asking your for open drivers?

NVIDIA will probably stay strong on the Windows side of things where people don't care that the NVIDIA driver crashes their Windows all the time but I hope that the people using free operating systems will make their wallets do the talking. It might not mean a lot to NVIDIA but the extra business the competition gets is a good sign to support their effords to create open and free drivers.

It's simple (especially for those who don't know a lot about linux but who wanna use it): Buy a card with free drivers (Intel or AMD) and things work out of the box. My laptop has the intel graphics chip which has free drivers and I have 3D accelleration on a liveCD without any setup. I install a system and everything is as fancy as it can be (though I don't use desktop effects cause they are more of an annoyance than a nice addition). With binary drivers you have to dig through howtos and forums and whatnot. With free drivers linux takes all that work away from you and you can focus on what you really want to do. Easy choice.

June 24, 2008 09:46 AM :: Germany  

Dan Ballard

Work around I probably shouldn’t need

I had to install amsn just so could video chat with a friend on MSN. Really? On the plus side, at least I could do it :)

Also, figured out how to use skype on Ubuntu. Skype really want /dev/dsp. And Ubuntu now uses pulseaudio. So basically if you've done anything with sound, Skype won't be able to get the sound. Which sucks. However, pulse audio ships with this handy utility 'pasuspender' which temporarily suspends pulse audio and it's lock on /dev/dsp. So to actually use skype

pasuspender skype

And skype can then seize and monopolize the sound card. So you can use it, but no other sound till you shut Skype down. So it's 2008 and we still can't share the sound card :/.

June 24, 2008 08:39 AM :: British Columbia, Canada  

John Alberts

Turbotail and multitail

I just found a couple cool programs called turbotail and multitail while searching for rbot using eix.

Turbotail is just like tail, but it uses dnotify instead of auto refreshing a defined number of seconds.  I always thought it was kind of silly to keep refreshing the screen searching for new content with tail.  Turbotail just sits there until the kernel notifies of a change in the file that you are tail’ing and then it updates what you see.

Multitail looks like a VERY robust way of viewing multiple files.  It can tail any number of files and supports text filtering and even syntax highlighting.

Turbotail works great, but unfortunately multitail crashes when I try to run it from my Yakuake console.  I get this:

--*- multitail 5.2.0 (C) 2003-2007 by folkert@vanheusden.com -*--
 
A problem occured at line 511 in function mynewwin (from file term.c):
 
Failed to create window with dimensions 55x9 at offset -27,-4 (terminal size: 167,19)

Seems to work just fine from a regular console though.  It will take me a while to actually learn all of the features of multitail.

June 24, 2008 02:32 AM :: Indiana, USA  

New Job!

After a while of looking for a new job, I finally got a new job. Well, actually, I’ve been working my new job for about 3 months now. So… I guess it’s not really a new job anymore.

I’m now a full time Linux administrator with ExLibris. Unfortunately, Red Hat is the preferred distribution. That’s to be expected. Most business want to make sure they use something that’s proven and has a clear line of support.

The new job is in Des Plaines, Il, which means I’ll have to sell my house and move a little closer. So far, the job seems pretty good. It’s doing something I like and the people are nice, and most of them seem pretty smart.

June 24, 2008 02:17 AM :: Indiana, USA  

June 23, 2008

Jürgen Geuter

Platform independence

You'll probably know this phrase either from some company trying to sell you their JAVA stuff or from some university students trying to look competent: "Our software is platform independent because we use technologyX which is platform independent."

Now this does not only look right, it also looks smart: The people chose a development platform that allows them to deploy their software on different operating systems and hardware architectures, which offers them the benefit of possibly bigger markets and target audiences. It also reduces the amount of code dealing with target platform details. Smart.

And wrong. Your code has to be platform independent or your choice of development environment means absolutely nothing.

If you wire your paths in your application like this: ..\..\directory your code is completely dependent on the platform below because you actively worked around the possible platform independence.

Almost any amount of interesting and non-trivial will include some things that can mess up your program on some platform. The most common ones are paths (because there are basically two ways to separate paths, the UNIX and the Windows way) but there are other things:

Some people coming from a Windows background don't seem to know that for a UNIX system "Abcd" and "abdc" are different files. So if you write to the first variant but try reading from the latter, it might work under Windows but not under other systems.

If you look at your programming language's documentation you might probably see that some function or API does not properly work under one or another operating system or architecture. Some might return "dumb" or at least irritating values (because the low level call that they wrap does not actually work on that operating system for example).

Another thing is that in a 64bit environment your types might behave somewhat different that what you are used to in your 32bit world.

There are many many traps that you can fall into when writing code that aims to be platform independent, the most important thing to realize is that your chosen development environment, whether it is JAVA or Python or whatever else you might choose can help you but it does not do everything magically.

When writing platform independent code, use the programming language's facilities to abstract away from the operating system. In Python for example use os.path.join to build a path, don't try guessing whether "/" or "\" is the right delimiter.

While today's programming platforms make it quite easy to get something starting on pretty much any system under the sun there are still many bugs that are hiding in the shadows. They can all be worked around but that requires awareness.

Don't be a platform dummy and claim that the technology you use makes your code something that it ain't, you might look less smart than you actually are ;-)

June 23, 2008 08:04 PM :: Germany  

George Kargiotakis

Euro 2008 open source tour

451 CAOS Theory has a mini review of what’s going on with open source among the countries that compete in Euro 2008. It’s quite interesting. Here’s the link about Greece. It has quite a point…Things don’t look very promising…

June 23, 2008 03:04 PM :: Greece  

Daniel de Oliveira

Crossover office 7 to support Microsoft Office 2007 and more


The following is the release announcemnet from Jeremy White of CodeWeavers (CrossOver):

Hi Folks,

I am pleased to announce that we have shipped CrossOver 7 for both Macintosh and Linux. New in Version 7 is support for Microsoft Office 2007, dramatically improved support for Outlook 2003 and Internet Explorer 6, and a broad range of improvements that should bring improvements to all Windows applications.

For our Linux customers, it also brings expanded support for most Adobe programs, with Photoshop CS and CS2 working particularly well.

For our Macintosh customers, this release also brings a change in our product mix. We are now providing “CrossOver Mac Standard” and “CrossOver Mac Professional”. The new Standard product will mirror

the Linux Standard product, in that it will be a lower priced product with more basic support and no multiple user support. The new CrossOver Mac Professional product replaces the existing CrossOver Mac product. It continues to have our best support, support for multiple users, and, CrossOver Mac Pro continues to come with a complimentary copy of CrossOver Games. If you have purchased CrossOver Mac in the past, you have been automatically upgraded to a CrossOver Mac Pro license.

Finally, another major benefit of 7.0 is that it includes many of the elements of Wine 1.0, which was also released today. This is a major milestone for us, and for the Wine project. Our many years of work, and your many years of supporting our work, have enabled us to help bring Wine to this milestone. I am very proud to have been part of this, and very grateful for all the support of our customers, advocates, and fellow Wine developers.

If you are an existing CrossOver customer with an active support entitlement, you can visit our web site to download this latest version: www.codeweavers.com
You will need to log in with the email and password that you used when
purchasing CrossOver. Please write to info@codeweavers.com if you need help with this process.

Thanks again for all your support, and I hope that you enjoy CrossOver 7!

Cheers,

Jeremy White
CEO
CodeWeavers


Version 7.0 Changelog:

New application support:

  • Office 2007 (Including Word, Excel, PowerPoint, and Outlook)
  • Adobe Photoshop CS and CS2
  • Added support for the “Compatibility Pack for the 2007 Office system” so that Office 2003 can open Office 2007 documents

Bug fixes:

  • Greatly improved online banking integration in Quicken 2007 and 2008
  • Greatly improved Outlook behavior, particularly with Exchange servers
  • Fixed service pack support for several versions of Office
  • Improved IE support in win2000 and winxp bottles (though win98 is still better)
  • Improved support for modern Linux distributions (especially Ubuntu)
  • Fixed a seriously horrible interaction with the Logitech Control Center documents from Office 2007
  • This version also includes countless Wine fixes and synchronizes with Wine 1.0.
  • Many small bugs should be fixed, and unsupported application behavior should be greatly improved.

Source: Wine Reviews

June 23, 2008 01:36 PM :: São Paulo, Brazil  

Leif Biberg Kristensen

Code prettification

Inevitably, as you’re learning a new skill, such as a programming language, you may want to revisit your old work and see if you can do it better. I had this pair of functions to fetch the previous and the next “page” of a source collection, based on the sort order of the source. In plain text, in order to get to the previous “page” I want the source with the maximum sort order smaller than the present one, and with the same parent id. Here is my brute-force approach from a couple of years ago:

CREATE OR REPLACE FUNCTION get_prev_page(INTEGER) RETURNS INTEGER AS $$
DECLARE
    self_page INTEGER;
    prev_page INTEGER;
    prev_src INTEGER;
    par_id INTEGER;
BEGIN
    SELECT parent_id FROM sources INTO par_id WHERE source_id = $1;
    SELECT sort_order FROM sources INTO self_page
        WHERE source_id = $1;
    SELECT MAX(sort_order) FROM sources INTO prev_page
        WHERE parent_id = par_id AND sort_order < self_page;
    SELECT source_id FROM sources INTO prev_src
        WHERE parent_id = par_id AND sort_order = prev_page;
    RETURN COALESCE(prev_src,0);
END;
$$ LANGUAGE plpgsql STABLE;

Ugly, ugly, ugly. I remember what was the main stumbling block here: You can’t use an aggregate function such as MAX() in a WHERE clause. The thing is that you don’t need all those assignments. A little code folding, replacing variables with sub-selects, takes you a long way:

CREATE OR REPLACE FUNCTION get_prev_page(INTEGER) RETURNS INTEGER AS $$
DECLARE
    pp INTEGER;
BEGIN
    SELECT source_id INTO pp FROM sources
        WHERE parent_id = (SELECT parent_id FROM sources WHERE source_id = $1)
        AND sort_order < (SELECT sort_order FROM sources WHERE source_id = $1)
        ORDER BY sort_order DESC LIMIT 1;
    RETURN COALESCE(pp, 0);
END
$$ LANGUAGE plpgsql STABLE;

The “ORDER BY sort_order DESC LIMIT 1″ is a great idiom whenever you need to use an extreme value as part of a WHERE clause.

Even if the two versions of the code above are functionally equivalent, and the PostgreSQL planner probably will rewrite the query to something like version 1, most programmers would prefer version 2. Why? I think there’s a lot to the concept that “coding is poetry”. Version 2 is more esthetically pleasing, because it conveys its inner meaning in a much more succinct way, as in the Merriam-Webster definition of the word succinct: “marked by compact precise expression without wasted words”. At least to me it does. And that’s probably some of the essence of poetry.

June 23, 2008 11:51 AM :: Norway  

Ow Mun Heng

Automatic Raid Array Rebuilding

Hi guys, long time no post. Last post was at March and it's now already June.

Been busy as usual, however, not been dabbling as much as I "should" as I've been busy with other NON-FOSS related stuffs. (psst: I'm now heavily into photography. Went to shoot some Japan GT queens!! Kawaaiii)

Anyway, since this is a (nearly) purely an FOSS based blog, I'm gonna talk about my automatic Raid Rebuilding script.

You see, what happens is this, my postgresql box, (celeron 2x500GB in Raid 1) has a tendency to keep dieing once in a while for X reasons. (I have till now, been unable to locate the reason why it's dieing so often) I've tried to the write-all, read-all using dd but thus far, has not seen errors being thrown out. So, it's been a manual instance of...

go to work. see the email : Your raid has Died!
log onto the box, do the rebuild.

After a while, this just becomes tiring and I decided to fsck it and make it automatic.

Here's the script

#!/bin/bash

FAIL_DRV=`mdadm --detail /dev/md0 | grep faulty | awk '{print $6}'`

if [ -n "$FAIL_DRV" ]
then
  echo "Detected degraded array : $FAIL_DRV"
  echo "Starting automated array rebuild process"
  mdadm /dev/md0 --fail $FAIL_DRV --remove $FAIL_DRV --add $FAIL_DRV
else
  echo "Nothing to do"
fi


Simple eh..

So, now I don't have to come to work to see it all wonky because it'll automatically rebuild itself.

Some of you may ask, how come I don't just replace the drive? Because I can't find any replacement drive which is a PATA connection and at 500GB capacity! The largest I can find are 160GB.

Bummer

June 23, 2008 01:32 AM

Jason Jones

Toilets and Servers

This past Thursday, we finally decided to rip it up.  I mean, literally - rip it up.

Then, we decided to take it out the backyard to see what the heck was clogging it all up.

It was just as we suspected.  Collin had flushed a freaking toy cell-phone down our toilet!

and yes, we took the toilet out in the backyard, twisted it, turned it, sprayed it, and finally fished out the darned thing so the toilet could work again.

Aren't two-year-olds the bomb?  Thank heavens, no, or we'd all be dead, but they do occasionally provide many hours of otherwise well-spent time into nothing but headaches.  Headaches well worth the while, for sure. :)

So... Yeah, the video above is just about our wonderful toilet experience, and also about the fact that the Gigabyte ga-m55sli-s4 does not have a working driver in Linux for the digital-out audio feature.

Yeah, my server died a week ago today (last Sunday), and I've been running everything from my back-up server since then.  The culprit is either the motherboard or the CPU because the darned thing won't even POST - at all.  No beeps, no video, nothing but fans spinning.  So, anyway...

I spent a good 8 hours working on the darned thing, just to make sure I wasn't crazy, because the motherboard I chose as a replacement did everything perfectly except the aforementioned audio digital-out.  I've got myth-tv on the server, and use it to provide my family's home theater media.  So, after 8 hours of pounding keys, I verified that I'm not crazy, because I have two of those motherboards here, and neither of them worked, while providing the exact same symptoms.

So, I got my hands on a Gigabyte GA-MA790FX-DS5 which proved to work completely perfectly.  Everything now is back to full working order, except that my server is faster and newer now.

Also, we can now watch the 1080p content coming out of channels 2,4, and 5 through our Comcast connection with the PCHDTV 5500 cards, without stuttering.  Evidently the old computer just couldn't keep up with the HD content, because now - it flows like a cool crisp mountain stream of digital goodness.  Holy cow, it's beautiful.

Anyway...  That's about it for now.

Toilets and Servers.

June 23, 2008 12:09 AM :: Utah, USA  

June 21, 2008

George Kargiotakis

The quest for a better rxvt-unicode on Gentoo

Today, while studying I decided to manually run a prelink on my system. For no good reason. Just boredom I guess. The results were pretty interesting though. Among the output there was a line that made a very big impression to me. prelink: /usr/bin/urxvt: Cannot prelink against non-PIC shared library //usr//lib/opengl/nvidia/lib/libGL.so.1 Why oh why is libGL.so.1 inside the [...]

June 21, 2008 10:09 PM :: Greece  

Speed up multiple ssh connections to the same destination

When you are doing multiple ssh connections to one host there’s a way to speed them up by multiplexing them. When you open the first network connection a special socket is created and then all other connections to the destination machine pass through the first network connection and don’t open any new ones. All that [...]

June 21, 2008 08:50 AM :: Greece  

Dan Ballard

Lisp (SBCL) on Hardened Gentoo

My server, mindstab.net, runs Hardened Gentoo. I like it. It provides nice features from grsecurity and PaX like memory randomization, non executable writable memory, etc. However, it really doesn't get along so well with Lisp. Lisp in general seems to like executable and writable memory, and SBCL at least also doesn't like randomized memory. So it took a bit of work to get Lisp onto my server.

Approach 1: Failure
I spent a bunch of time trying to patch the build process in portage to coax SBCL into building. First, of course, I used gcc-config to disable the hardened gcc profile, and just use the vanilla one. Then I created a suid root shell script to call "paxctl -m -p -r -e $1" so that the sandboxed build process could disable PaX features on the SBCL binaries. I added the command to the ebuild, and created a patch to insert the command into SBCL's build process. The process goes like this, portage download's the SBCL source and a pre-compiled SBCL binary. The patched ebuild then calls my suid root script which disables PaX on the pre-compiled binary so it actually runs (as opposed to crashing under PaX) and then a new SBCL binary is built from the source and the pre-compiled binary builds a core file from the SBCL lisp source. The patched SBCL make.sh then again calls the suid root script on the new binary, so it will run. Then it should load the new core and recompile the system for itself. Sadly, while it runs at least, it chokes on the core file and hangs while using 100% cpu. I couldn't get past this so I eventually gave up. If anyone has any suggestions that'd be great.

Approach 2: Success
So the actually solution was as follows: Download the most recent precompiled SBCL binary from the website (1.0.15 for x86), run "paxctl -p -e -m -r -x -s " on src/runtime/sbcl (to cover all the bases). Then run "sh install.sh" to install SBCL to /usr/local. That's it.

The problem with this is you can't emerge lisp packages in portage, you have to install them by hand (unless maybe you want to fake inject the package into the portage database).

I downloaded a copy of slime, untarred it and popped it in my .emacs and I had a full lisp environment ready to go, and on my hardened machine no less. Not so bad.

June 21, 2008 07:07 AM :: British Columbia, Canada  

June 20, 2008

TopperH

Howto: AuthenTec AES1610 Fingerprint reader and gentoo

My laptop comes with a fingerprint reader that can be used for authentication:
Bus 006 Device 007: ID 08ff:1600 AuthenTec, Inc.


There is a project called fprint that allows users to use it under linux. There is not yet an ebuilt for it in the portage tree, but I found it in an overlay.

The main issue I found is that libfprint does not compile with gcc-4.1.2 (which is currently stable for gentoo amd64) because of an "-fgnu89-inline" error.
This option is not yet implemented in 4.1.2 so I decided to switch to gcc-4.3.1.

I assume layman is installed and working, otherwise take a look here.

# layman -a wschlich-testing
# echo =app-misc/fprint_demo-0.4 >> /etc/portage/package.keywords/general
# echo =media-libs/libfprint-0.0.6 >> /etc/portage/package.keywords/general
# echo =sys-auth/pam_fprint-0.2 >> /etc/portage/package.keywords/general
# emerge -av fprint_demo pam_fprint


and now, as a regular user under X:

$ fprint_demo


In the enroll tab enroll the right index finger moving it smoothly . Than test it in the verify tab. It will take dozens of tries before getting a correct match, this is why we are going to use the usual password method if the match fails.

Now we have to edit the /etc/pam.d/system-auth file inserting this line:
auth sufficient pam_fprint.so

before this one:
auth sufficient pam_unix.so try_first_pass likeauth nullok

June 20, 2008 08:18 PM :: Italy  

Asus PRO60Eseries + Gentoo = LOVE

OVERVIEW:

CPU : Duo T7250
DISPLAY: 13.3" WXGA
WIRELESS: 802.11abgn + Bluetooth
HD: 250GB
RAM: 3GB
revolver ~ # lspci
00:00.0 Host bridge: Intel Corporation Mobile Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile Integrated Graphics Controller (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Contoller #4 (rev 03)
00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5 (rev 03)
00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #2 (rev 03)
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03)
00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 03)
00:1c.1 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 2 (rev 03)
00:1c.2 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 3 (rev 03)
00:1c.3 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 4 (rev 03)
00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #1 (rev 03)
00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #2 (rev 03)
00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #1 (rev 03)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f3)
00:1f.0 ISA bridge: Intel Corporation Mobile LPC Interface Controller (rev 03)
00:1f.1 IDE interface: Intel Corporation Mobile IDE Controller (rev 03)
00:1f.2 SATA controller: Intel Corporation Mobile SATA AHCI Controller (rev 03)
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
02:00.0 Network controller: Intel Corporation Unknown device 4229 (rev 61)
05:00.0 SATA controller: JMicron Technologies, Inc. JMicron 20360/20363 AHCI Controller (rev 02)


What does not work:
  • The fingerprint reader (Bus 006 Device 002: ID 08ff:1600 AuthenTec, Inc.) [EDIT now it works]
  • The LCD backlight controls (CONFIG_ASUS_LAPTOP=Y doesn't help and acpi4asus doesn't compile)
What needs workarounds:
  • ALSA
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03)
I needed to add "options snd-hda-intel model=lenovo" to /etc/modules.d/alsa and update modules.

What I have not tried yet:
  • Integrated 1.3 megapixels webcam (I don't really need a webcam)
  • [edit] now it works, see here


revolver# uname -snrv
Linux revolver 2.6.25-tuxonice-r4 #4 SMP PREEMPT Thu May 22 16:04:52

revolver ~ # eselect profile show
Current make.profile symlink
/usr/portage/profiles/default/linux/amd64/2008.0/desktop

Kernel config
make.conf

June 20, 2008 05:38 PM :: Italy  

Jürgen Geuter

The eclipse thing

I read an article recently about how to get the interactive django console running in pydev/eclipse (pydev is a plugin offering python supprt to eclipse). The whole shebang involved sacrificing around 10 goats and 7 virgins (as in computer science students ;-) ) to a blind man from Paris (do not ask me why, I don't know). It was very hacky, it was not able to support more than one installation (meaning that you had to go to the preferences for your IDE whenever you started working on another project. That does not only sound retarded, it actually is.

Django comes with a very good interactive shell that supports the two big shells that Python people tend to use: The "normal" python interpreter and iPython which pretty much anyone who has used Python for more than 5 minutes uses. iPython has support for tab completion, works everywhere and is all in all one of the most useful tools to anyone using python. So why would you want to work around that and cheat your IDE into providing something worse?

The whole IDE thing usually comes from Java, C++ and C# backgrounds: IDEs can help you write all that boilerplate code that those languages force you to spend your time on instead of doing something useful. People trained in that mindshare just cannot think about not using an IDE when it just does not work like that with dynamic languages.

IDEs for static languages work well cause everything is specified and fixed. You know what type a certain variable has so you can show problems with assignments as errors and whatnot. With dynamic languages you have absolutely no clue what type a variable could be. Of course you can have a debugger running in the background at all times having the code running while you write it but obviously that will not be feasible for long. IDEs just don't offer the benefit to dynamic languages that they offer to static ones.

Of course you can set eclipse up to do python syntax highlighting and use it as the slowest text editor ever but when you know that an IDE won't do that much good to you, getting a serious editor set up properly (and that includes learning how to use it) will make sure you don't have to wait for your IDE to catch up with what you're doing.

When you write Python what you want is a powerful text editor (like vim for example) that helps you work with the characters you type, that allows you to reformat things properly, that is scriptable (AFAIK eclipse's excuse for a text editor does not allow selecting columns for example, something incredibly useful). Then you want ipython in case you want to try something out live: Import your code and poke around in it, see if it's doing what you want. Next thing is to set up your preferred unit testing and write tests for the stuff you write. The tests do for you what the IDE might do for Java people: They make sure your code is doing what it is supposed to do. In addition if you work on anything more complex than one tiny script you should have one SCM (source code management system) installed and should know your way around with it (pick git, btw.): Make sure that you're not commenting out stuff all the time but that you really delete it if it's broken. You can get it back from the SCM but for now dead code makes things less readable. Throw a comment in there with a TODO item and just get on with it.

Eclipse is not a bad IDE actually. If I had to write JAVA I might use it. I played around with it for PHP once and it didn't completely suck (there were some nice HTML formatting plugins available), but in the end the slowness and how it made everything overly complex (even things that just aren't) made me look another way.

Text editors have been around for a long time and the two big ones (vim and emacs) together have around 50 years of experience writing text editors, they should have picked up a few thing along the way and there's a reason people still use them.

If you use a language that only is bearable with an IDE use one, but for other languages don't waste your precious resources on them.

June 20, 2008 02:49 PM :: Germany  

Attaching CDs to magazines ...



I got the new iX magazine (there was a "get three for free" campaign recently which this was the last one of), for those not knowing it, it used to be a very technical magazine about new trends in information technology, sometimes so specialized that you couldn't understand an article if you had not spend 10 hours before preparing. It was not your everyday magazine which was a property I really enjoyed: It tried not to be shallow.

Well that has changed, the last few issues all were quite bad and seem to solely focus on comparing software: They had a really shallow article on forums a few issues ago, this time it was blog software.

But this is not about the articles (I could write a whole post on why posting a RubyOnRails howto now makes you look like someone who has completely lost touch with the tech world) this is about the CD that came with it.

When I was younger I subscribed to a PC gaming magazine. They always had a CD coming with it that had all kinds of patches, demos and drivers on it (I was little and still using Windows back then ;-) ) so the CDs were really a nice thing to have: I had no Internet and the top-of-the-line connection was a 54kbit modem which ain't exactly fun to download stuff with.

Back then, CDs had a value because while the ping sucked (one month in between packets) the data transfer rate was really great (one packet gave you almost 700 MB of data).

Nowadays everyone with some interest into technology has an internet connection (at least in areas where iX is being read since it's a German magazine). Now for some people the aggregation of different blog software with their specs and short Pro- and Con-lists might be helpful but if they are interested in setting up a blog they will have an internet connection. If they wanna try out a few of the software packages they could just download them (seriously, how big is the average blog software? 3 Megs?).

Giving them a CD with it has a bunch of problems:

  • Environmental damage: You produce many CDs which are a pain in the ass to recycle (and no they do not belong into the normal trash or the German "gelber Sack").

  • People really interested in the software packages will go to the websites anyways looking for plugins, themes or just updated versions (with the speed in which many software projects move as soon as you download a version and have it burned to CDs there will be a new version out)

  • I have yet another coaster around.



The time of adding CDs to your magazine with software that people can get on the internet for free is long gone and you don't look like you are in touch with current tech if you have not realized that.

There are reasons to add CDs/DVDs to your magazine: Many movie/TV magazines have DVDs with legitimate copies of movies attached to them, that's useful (and a cheap way to get good movies for small money [sometimes]). If you get a free version of some software that you usually have to pay for it might make sense to add a CD/DVD with that to your magazine: Sometimes games magazines ship with full versions of somewhat older games for example. Those are all valid uses.

But just creating more stuff that I have to take care of recycling annoys me and makes you look amateurish. Get over it, the sales that you used to get because someone needed a driver for his hardware, his original CD was scratched but without the driver the graphics card wouldn't work and your magazine was the one that had the driver are long gone. It's cheaper to just go into an internet cafe and get the stuff if you really have no internet yourself.

The times of adding value to your product by aggregating stuff that's free are gone: There's value in your service testing things, but don't waste resources by burning all that software onto a CD.

June 20, 2008 09:55 AM :: Germany  

June 19, 2008

Thomas Capricelli

Release of cvxprocessing-1.0-beta1 (+tomography)

I’m releasing part of the code developed during my PhD. Those are tools useful in the field of convex processing, and more precisely to solve a problem called convex feasbility problem.

This is based on Qt4, and released mostly under the GPL (with an added restriction : you must cite my work if you use this). Oh.. and there are unit tests, examples, and documentation, too.

caution : rest of the post is for those mathematically oriented people, and no more related to free software.

The convex feasbility problem is to find a point in the intersection of closed convex sets inside a Hilbert space. Believe it or not, quite some real-life problems can be formulated as such, for examples some image reconstruction problems in medical imaging, or also in the field of signal processing.

There are a lot of algorithms to solve this, you can find more information about those in the bibliography of my papers, or those from my PhD advisor M. Combettes.

This code provides the necessary tools to play and test those algorithms. Besides all elementary methods on Euclidian/Hilbert Space, it provides a way to compute projections on different interesting convex sets. There are also the needed tools to experiment with tomography, which is one of the main application in my PhD. You can compute the radon transform of an image and use this as data in convex feasibility formulation.

This is of course mostly of interest for those working in convex optimization and/or tomography. I want to highligh the fact that sharing such kind of code is very rare in this community. I would have gained a lot of time at the beginning of my thesis if other people had provided such code.

link : http://labs.freehackers.org/wiki/cvx-processing

June 19, 2008 05:44 PM

Jürgen Geuter

Django Foundation

Good news in Django land: As it was posted a few days ago Django is now "ruled" by a foundation (just like Apache, Mozilla or Python). This marks an important date for Django because now entities interested in pushing Django forward can contribute not only in code but also in money towards the foundation which might lead to the opportunity of paying people to work on Django.

Also there's a roadmap for the way towards 1.0 which might calm down many of those that only consider something with the magic numbers 1.0 to be stable and usable.

The roadmap shows that the interesting changes will soon be merged into trunk (like for example newforms-admin) which will make working with Django even nicer than it is today. Exciting times.

June 19, 2008 04:45 PM :: Germany  

Nirbheek Chauhan

The much-delayed post

*Very* late, this post is. I hope it's not too late yet :)

I'm talking about something that was much-talked about, and people are probably following some of the suggestions made about this, but I think there should be some sort of standardisation.

Here's the header I use for all my GSoC code:


# vim: set sw=4 sts=4 et :
# Copyright: 2008 Gentoo Foundation
# Author(s): Nirbheek Chauhan <nirbheek.chauhan@gmail.com>
# License: GPL-2
#
# Immortal lh!
#


I encourage all of you to adopt the last two lines in your headers as well :D

June 19, 2008 07:44 AM :: Uttar Pradesh, India  

June 18, 2008

Brian Carper

Wish list

What's the Common Lisp version of Perlmonks or Ruby-forum? I have yet to find it.

comp.lang.lisp is largely crap. 50% of the traffic on that list is spam about shoes and fake watches. The other half is equally split between:

  • People debating tiny, silly semantic points of the Common Lisp Hyperspec.
  • People stuck in the 70's or 80's, talking about the good old days, ruminating about Lisp history.
  • Flame wars.
  • New people asking for help. Some get good honest advice and helpful answers, many are flamed and ridiculed into next week if they even hint that they dislike the parentheses.

The Common Lisp community (if you can call it that) is a bunch of really smart guys, but they all live isolated in hermit shacks up in the mountains and they spend their time doing magic tricks with Lisp that few people ever see, and if you wander too close they throw rocks at you.

What's the Common Lisp equivalent of perldoc or rdoc? We have the Hyperspec. It's an impressive document, but it's a bunch of painful HTML that looks like it was created in the early 90's, probably because it was. It reads like a dusty, dry, technical document probably because it is. What it's not, is friendly or easily readable.

Perl has CPAN, Ruby has rubygems, what does Lisp have? Either a hand-rolled system definition script, or if you're lucky an ASDF install file. ASDF is the semi-standard Lisp way of installing libraries, except that it doesn't quite work in Windows, it doesn't check dependencies or handle different versions of a package very well, and it doesn't work the same on all Lisp implementations. Many people in the so-called community think it's not very good.

The fellow running Lispcast makes another good point. Where can you download Lisp? It's not obvious.

You could say "OK Brian, good idea, now get to work!" The problem is that even if I had the time or willpower, I'm not the smartest guy in the world. I honestly don't think I could design and run and maintain a CPAN. And even if I did, would anyone use it? But I do know that there ARE plenty of smart, enthusiastic people using Lisp. Yet high-quality friendly code is largely not being produced.

Peter Christensen wrote about "langauge snobs" and the importance of community. One point made is that some really ugly, horrific languages have been extremely successful simply because they've been accessible and fun. An example given is the scripting language in Second Life, which has over 2.5 billion lines of code written in by tens of thousands of amateurs and has accurately modeled a realistic 3D environment with thousands of users at any given time. All in an ugly language some guy invented AND implemented in one week. The developers admit that the language is total crap, but it doesn't matter. 1) It has very good and accessible documentation, 2) it has a very newbie-friendly community, and 3) and it's easy to pick up, throw together some code and get immediate results. Three things Common Lisp lacks.

This is something I've said myself many times: an active, supportive, enthusiastic community is essential for the health of any programming language. Common Lisp simply doesn't have one and it's a shame.

I still secretly hope that Clojure or NewLisp or Arc turn out to be a huge success. They are the kinds of things Lisp needs today.

June 18, 2008 07:47 AM :: Pennsylvania, USA  

Joset Anthony Zamora

Sun Tech Days 2008 Day 2 - OpenSolaris 2008.5 Track

Sun admitted that Solaris’ late move to open source is a big mistake. They said that while they were busy making money, they were already losing the server market, Linux is taking over very fast.

Having tried several versions of Solaris and OpenSolaris, in my honest opinion, opening its source code to the public is a good move. :D

What’s cool in OpenSolaris 2008.5?

1. IPS - a network package management system that resembles an apt-get type of command. Thanks to Ian Murdock.

2. Bourne Again Shell - C Shell is no longer the default one. This makes majority of the Linux users comfortable when shifting to OpenSolaris.

3. OpenSolaris Developer Expert Assistance - A dedicated online support service for developers that provides technical assistance for code support, programming questions, diagnostic advice, how-to’s and best practice guidance.

4. OpenSolaris Subscription Support - Telephone and online technical support. Provides automatic notification of security updates.

June 18, 2008 03:41 AM :: Zamboanga, Philippines  

June 17, 2008

Patrick Nagel

Firefox “Download Day”

They asked for it - they got it… a nice DDoS ;) (spreadfirefox.com and mozilla.com were unreachable for a couple of minutes after 17:00 UTC).

Anyway… go ahead and download it, it’s a good piece of software. I tried out beta3 and the RCs, and I like the new version so much, that I’m now using Firefox as my default browser again (after more than two years with Konqueror as my default browser).

Download Day 2008

Nice birthday present by the way, Mozilla! Thanks! :)

June 17, 2008 06:17 PM :: Shanghai, China  

Bandan Das

Homework time

The best browser ever, landed on earth today but while big daddy was busy asking everyone to do their homework to get ready for the party, he failed to do his own.

First and foremost, as all of us came know a little later, the download day was not just 17th June, 2008; it was 17th June 2008 at 10:00 am PDT. No one ever cared to inform that important piece of shit. All big daddy wanted was a 1 million downloads in 24 hours, he didn't care how frustrated you would be to find out that you don't see your download at 12 am of June 17th, local time. And no Sir, I am not asking you to take care of all the time zones here, a simple countdown timer on the website would have been enough. Moral of the story: Big Daddy, you came up with something really impressive, you made us do our homework, but you failed to do your own!

Now, everyone knows that the digg effect is not just limited to sites on Digg's front page. Anyone can feel it; especially when you want 1 million downloaders to come to your site when you release the most anticipated browser ever, what on earth were you thinking ?! It's never going to be a breeze. Everyone knows that! Again, you failed to do your homework! Moral of the story: You promised us all the good things on June 17th, and all the good things you did give, but; the overall experience was somewhat dissapointing :(

Ok, now that my rant is over, I am off to start enjoying my brand new browser :)

read more

June 17, 2008 06:04 PM :: India  

Sean Potter

Gentoo Linux on my MacBook

I purchased my MacBook back in November of 2007, shortly after the Santa Rosa chipset update. I'd use Mac OS X a little previously, mostly in the form of trying out OSX86 on my old laptop. After using it for several months as my primary mobile platform, I decided it was time for something different.

OS X is nice and all. User-friendly, intuitive, and works fantastic with the hardware Apple ships. I still feel it's lacking in some areas where Linux excels, such as configurability and the space for customization. Not to mention that there's essentially no decent games out for OS X. I can install Linux and throw on a native version of UT2004 and play away now.

Installation wasn't horribly difficult, but Mac OS X was needed at least initially, and I didn't realize this. Gentoo now boots without an issue on the MacBook. All that I need to finish doing is installing the programs I need and setting up the WiFi.

I'm hoping that I have all the hardware working correctly in the next week or so, and that includes the web cam and touchpad. My only other concern is battery life. It'd be fantastic if I get the same amount or more battery out of the machine with Linux.

June 17, 2008 04:37 PM

Nirbheek Chauhan

AutotuA Weekly Status Report - I (and more ;)

Yes, I am alive and kicking. Although at a much slower pace than I would've liked ;)

So I sent my first weekly report over yesterday, you can either read the (excessively long and probably boring and or confusing) weekly report or you can, well, do something else :P

And guess what, next time onwards, all you have to do is checkout the AutotuA news page or subscribe to the "gsoc" label on this blog to stalk me.

Oh, right, this will also probably be my first post on the FLOSS India Planet!

Hello everyone!~ I'm Nirbheek Chauhan (also called as "slacker #1" by some). I was one of the co-ordinators of this tiny little event in IIT Kanpur's tech-festival Techkriti.

You might have heard about it and maybe seen the awesome speakers (and posters ;) of the event.

You've probably had the pleasure of conversing with the mastermind behind the whole event.

And maybe, just maybe, you've heard about "FOSSKriti" :D

PS: We'll (hopefully) be back next year, so this is shameless advance publicity ;p

June 17, 2008 04:29 PM :: Uttar Pradesh, India  

Brian Carper

Westinghouse, the saga continues

Friday a guy on the phone said he'd call me back Monday or Tuesday to give me an update on when / whether they're ever going to send me my monitor. Monday came and went with no call. Not really surprising.

I filed a complaint with the BBB today. We'll see how that goes. At the BBB Westinghouse has around 150 complaints in the past 36 months, but 133 of them were supposedly solved "satisfactorily" and Westinghouse somehow still has the highest possible rating at the BBB. I've read some things about the BBB not being an entirely neutral entity itself, but who knows. I'll start filing complaints with other consumer groups if I need to.

A good handful of people have left comments here at my blog saying they aren't going to buy anything from Westinghouse themselves, which is great to hear. I may mention my blog to Westinghouse next time I call them, if there is a next time. Is not sending me the monitor I paid for really worth losing a bunch of customers?

The sad thing is that I really do need a monitor with component and composite inputs, and they are somewhat rare (the local store had none except Westingcrap brand). However I have found a Gateway model that has them, so maybe that'll work out. I'd gladly take a refund from Westinghouse rather than a monitor at this point.

June 17, 2008 07:56 AM :: Pennsylvania, USA  

June 15, 2008

Michael Klier

Mail Client Quest

I've spent almost the whole last week testing different mail clients because I somewhat wasn't happy with mutt (especially the address book functionality sucks). Also, now that there are new methods in categorizing data, via tags/labels for example, the possibilities of processing my mails effectively with mutt seemed limited.

Until now I've used my INBOX as a staging area to organize tasks which are associated around the projects I am involved. I simply kept mails which required action on my behalf in my INBOX. Be it that I just have to reply or that I have to look at something or fix a bug, you get the picture. Since I get lots of emails (~1000 on an average week) things got quite messy in times and I sometimes forgot about some of those mails/tasks. What I ideally wanted to have is a way to label messages and then search my INBOX for mails which have a certain label assigned, or limit the number of mails I see by certain key words. The IMAP protocol supports user defined labels which would be a starting point, but mutt doesn't support them :-( (it should be noted here that mutt wasn't designed to work with IMAP in the first place).

I first took a look at sup. Sup is written in Ruby and promises a completely new approach to mail. Most of it's features are inspired by gmail and mutt. It supports labeling of messages, vim style key bindings and some really nifty quoting features like auto folding in mailing list threads. It's usage is quite intuitive when you come from mutt and I've used it exclusively over the last week. But, all those nice shiny features aside, sup has some major drawbacks. To allow labeling of messages it manages it's own index where it keeps all the related meta data. The problem with that, is that it doesn't update the mail status on the server side. Means, mails that appear as read in sup are still unread in the IN