rastplats

It’s time to close the book on 2005, not a minute too soon..

rm -rf 2005/

Recently I got a new laptop, an IBM/Lenovo Thinkpad x41 to replace the somewhat aged HP Omnibook 500 I’ve been using for the past two years or so. As with any Thinkpad, the x41 is a marvel of quality and even with it’s mere 80% keyboard it packs the true Thinkpad feeling in the keys. The big setback came when trying to install my operating system of choice, FreeBSD, on it. The FreeBSD sysinstaller didnt boot off a standalone USB CD and the x41 doesnt sport any builtin removable media. The solution seemed to be to set up PXE booting but a helpful user from The FreeBSD Laptop Compatability List presented an even easier way, booting the ISO image off the NTFS volume.. now thats a hack for sore eyes. Since there are no tutorials on how to get FreeBSD running the x41 I decided to write up a small article on the subject which can be found here although it is far from complete yet.

Apart from configuring the new computer, christmas was pretty eventless and was carried out using the same procedure as most years. I’m however not a big fan of christian holidays so I prefer them that way. At least we got some (a lot) of snow today which was nice.

The past two days have been spent on a short businesstrip to Sundsvall in order to wrap up and close the project I’ve been working on since late this summer. While the negotiations and discussions regarding the performed work went fairly well, the planned work for the afternoon, which was to end the entire deal, had to be cancelled due to technical problems which should have been caught way earlier. Utterly painful.. time is running out given that my employment ends at January 20. At least I did get to see some snow which is quite rare in the far south of sweden where I normally reside.

Speaking of christmas, this seems like a sole candidate for the present of the year.

As most of you know I’ll be moving to Australia in about a month but the reason for it seems to be held in some uncertainty. Almost everyone seems to think I’m fulfilling a dream or have other equally romantic notions about it, I wish that was the case but perhaps it’s time to come clean.

The truth is that I’m not all that concerned with where I’m going or what I’ll be doing there but what I’m leaving. The whole point of moving to the far side of the globe is to run from a life that I’ve lost interest in and control over. Staying behind would mean slowly crumbling to dust.. moving is more a means of surviving than anything else in my case, I’m not going I’m leaving.

Keep your fingers crossed that I find something down under..

We had the yearly christmas dinnerparty at work tonight which turned out quite nice although these types of events make in painfully clear that I am entirely incompatible with social interaction. There is however no doubt about that I am going to miss hanging around with this crowd when I move.

I just came back from the Beauty and the Beast musical, at Det Ny Teater in Copenhagen, which was an amazing show. The performance was topclass and the scenery and costumes beyond what I’ve seen before, a well spent evening. I can’t say that I had any problems finding the character with which to identify in the play although when it comes to being noticed our situation is quite the opposite.

The entire day at work today was spent at a seminar/workshop on Network Resource Management (NRM) in general and Network Inventory Management in particular. While the workshop was mostly spent on the topic of bikesheds it was very interesting to hear that the experts on NRM implementations all evangelise the same path that I’ve been taking the last few weeks at work. My work involves extracting data from flatfile formats such as excel and visio into a SQL metamodel using syntactic rulesets deployed using regular grammar. By doing this we will achieve i); A fulltext indexed database view of our information body and ii); Syntactic datavalidation of the currently unvalidated formats. This road was what the experts advised as well although their plan involved converting into the target database model directly just to validate the data. My approach is to allow the data to be exported into any datamodel by applying SQL rulesets to the regular grammars. Although somewhat similar in direction, somehow I doubt that the expert consultants do it all with a few hundred lines of Perl and some 2000 rows of stored procedures.. and doing it truly general purpose on both input and output.

I finally feel that I’ve found the outline for my article on metamodel implementaions using SQL, how to use syntactic analysis in combination with a metamodel to migrate from flatfile workflows to database models. Now all I have to do is muster the energy to write it..

For a project at work I need to convert Excel .xls files to a more suitable format for datamunging on a UNIX platform, csv for example. To achieve this I set out hacking a small converter using the POI project from Jakarta, according to the documentation traversing the fields in the file should be trivial. So it was, unfortunately it was equally trivial to blow the stack as well. Reading a 900kb xls file took too much memory to fit in the normal 64Mb heap allocation for the Java VM. Sigh. Even more painful was it since this was the smallest file out of the ten files I needed to convert. Assuming that this a 1Mb file just barely didnt fit the 64Mb allocation, to convert the largest file on my list I would need almost 800Mb of heapspace.. when all I did was reading one row at a time and writing it to a new file? Since I need to convert the files on the fly a few times per day this is utterly useless. It’s a good thing that this is opensource, it will be interesting to find out how they manage to blow up the data 64-fold in memory.

Fortunately the package provided an alternative way to traverse the file which consumed memory within the limits of sanity and in the end I had a small hack that converts a xls file into several csv files, one for each found sheet. If you need to do the same thing you can get it here under a BSD license.