Random bytes on technology and open source

Py3k letters - Part I, The PEP Talk

2008-03-21T13:55:00.009+05:30

This post is all PEP-talk. What pep talk you may ask ? Well not the usual pep-talk but talk about PEPs. "PEP" stands for "Python Enhancement Proposals" and is the way in which Python language developers and enthusiasts go about suggesting, agreeing on and incorporating changes to the Python language.

For Python 3.0, the key PEP is PEP 3000. All Py3k PEPs are numbered starting from 3000 upwards. PEPs 3000-3099 are special, since they are PEPs about PEPs, so called "meta-PEPs". PEP 3000 is like the god of all meta-PEPs, since it is meta-PEP of all meta-PEPs. Think about it something like the father of all meta-PEPs, the originator, the source.

There are two more PEPs which are special. PEP 3099 and PEP 3100. The former is an anti-thesis of a PEP actually, since it refuses to budge - it is a list of things that has no plans of changing in Py3k. Interesting isn't it ?

The latter is the mirror image of the former. It is a laundry list of things that will change in Py3k, which are not big enough to have a PEP of their own. The fact that it follows PEP3099 is not its fault. It just so happens. In fact, this PEP was christened as PEP3000, but as other PEPs were born and began to strive for space and the PEP talk became more and more noisy, the BDFL and his gang of merry men re-christened this PEP all the way up by a century. I am not sure why this happened, but that is history and one cannot change it. However the PEP is not complaining and neither should you. Strange are often the ways of PEP-land...

Now, Py3k was not born as an idea in a single day (neither was Rome built in one), so you can imagine that there was a lot of pep-talk which led to its conception. And though PEP-land is often strange in its ways, it is not as strange as to defy logic completely - it is not. You would expect these PEPs to be older than the one which is bears the name 3000 in its birth certificate, and you would be right if you thought so.

These older PEPs, which are the big brothers of PEP 3000, influenced the growth of PEP 3000. From older to younger, they are,

1. PEP 238, which divided the kingdom of Pythonistas over the something as simple as the division operator. The contention was how to float and divide at the same time, in just one swoop. It turned out that that this was a hard problem to solve and finally after many a battle, the decision was made - float and divide in a single swoop and only divide in two swoops . The ramifications of this decision will be huge for many programs which are inhabitants of the Python land, especially belonging to the numerical and scientific family trees.

2. PEP 328, which made a lot of funny noise with statements that is any combination of the strings "from", "import", ".", "..", and "as" . All to solve the problem of the right syntax of relative imports - that is imports as in importing code, not like importing sugar or oil, which is a more easier problem to solve apparently. This problem is much more abstruse, and aims to bring down packages from folders they live in, relative to the current folder.

3. PEP 343, which is all about disciplining the brat fraternity of "try", "except" and "finally" statements, by giving them some meaning in life (a "context" actually) and holding them together with a "with" statement. Matters get complicated when a context is managed with a contextmanager, as usually happens when managers get involved in any engineering problem.

4. PEP 352, which is about installing an original ancestor for a group of dangerous characters, whose main job is to raise issues which are euphemistically called as "exceptions". Unfortunately, chaos has reigned at the top of this hierarchy with no clearly defined ancestor who these characters can claim lineage to. Pythonistas realized (quite late) that it is dangerous to let the status quo remain and the pep-talk resulted in this PEP which aims to fix the status for good by correcting their ancestor tree. Very well. I am glad we cannot do these things in real life.

So, all the talk about pep-talk has resulted in the above peppy PEPs. Pythonistas live and swear by PEPs, so if you want to learn the Py3k way, you should walk the way of the PEPs for some time and live with them, understanding why and how they exist.

The above PEPs form the soul and core of Py3k, so get used to them and we will discuss the actual issues they solve and how they do it, starting from Part 2 of the Py3k letters. Till then happy PEPing.

Py3k letters - Part 0

2008-03-21T13:45:00.006+05:30

Python 3.0 (py3k) is on the way. The 3.0 version is going to be a significant update in the Python programming language, considering that it is almost 8 years since the release of the last major version, Python 2.0, which was released in Oct 2000.

Py3k is going to change the way the Python programmer goes about his work in many ways, with some significant "in-your-face" changes and many thousands of changes and fixes below the layers and quite some just in between.

I am starting a series of blog posts from today, which will focus on Python 3.0, specifically on the major changes in the language when compared to Python 2.x versions. The plan is to cover enough areas (hopefully), so that by the time py3k is released in August this year, there will be enough text here to serve as an aid to Python programmers in chartering py3k territory.

With that introduction, on to the "py3k letters" series, with the first edition coming in the next post. Keep tuned in.

Red letter day!

2008-02-19T18:45:00.006+05:30

I will mark today as a red letter day in my life and career. HarvestMan is one among the winners of FOSS India Awards declared by EFY today.

After four years of painstaking work and countless hours spent on developing the program, I feel happy to be at the "receiving end". A lot of gratitude goes to the EIAO team of Mikael, Nils and Morten for giving me the motivation to work on the program and keep improving it.

Congratulations to all the award winners for their spirit of free software and open source. Keep up the good work!

This is a very good initiative on the part of NRC FOSS and EFY to encourage F/OSS development in India. I really appreciate the leadership shown here by both the institutions. Apart from putting FOSS and people who innovate in FOSS in India in the limelight, the initiative will also help to advertise to the world the fact that Indians are also innovating in this area.

Hopefully, efforts like this would encourage the adoption of F/OSS on a large scale among students in engineering and technology in Indian universities and provide a big push towards more original F/OSS contributions from India.

FOSS India Awards

2008-02-11T19:56:00.000+05:30

I am making my first post of the new year 2008. This also happens to be a post after a gap of more than three months. Reasons - new job, lack of time etc.

I had submitted HarvestMan for FOSS India Awards more than a month back. However the FOSS India Awards team took their own time to provide the competing projects with a questionnaire regarding their project. I got the questionnaire in my email yesterday and today I completed it and sent it.

The awards are expected to be announced on Feb 15th, which also happens to be my birthday. Keeping my fingers X...

FOSS - Made in India !

2007-10-12T00:45:00.001+05:30

While returning home from work today, I went to the neighboring super-market and bought the LFY (Linux For You) magazine for the month on an impulse. I have not been a regular reader of this magazine since Feb 2006 when I stopped writing for them. So it was just pure impulse on my part...

I was going through the articles lazily in bed some few minutes back. I started with the review of SLES desktop and then read the article on autopackage. The project is started by a group of young enthusiasts based in U.S and Sweden which makes package installation across Linux distros easy (say "InstallShield for Linux"). I then flipped casually to the next page where there was a guest column by our very own Kenneth Gonsalves, the Chennai based open source enthusiast and activist.

He was asked the typical question which every FOSS related magazine in India loves to repeat in every alternate editions - "Why are Indians not contributing to open source ?" I glanced through his reply. Little did I realize his comments would make my day...

Here I quote his reply verbatim...

"There are a large number of indians in practically all major FOSS projects. However if you look closely, you will find that the vast majority are not resident in India. Yes, there are several hundred people in India actively contributing to projects big and small. But genuine 'Made in India' projects of international repute ? I can see only four or five: Anjuta (Naba Kumar has left the country); HarvestMan by Anand Pillai; Deepofix by Abhas Abinav; IndLinux by Karunakar and team and Coppermine by Tarique Sani..."

Boy, was I not taken aback by surprise and blushing with genuine pride!... :) I have used Anjuta and has great respect for its original developer (Naba Kumar), though little did I know about his current resident country. I think IndLinux and Coppermine are great projects and frankly I have not heard about Deepofix (my bad...). I have always considered my own contribution (HarvestMan) as a david among the goliaths. I have a sense of my place among the international open source developers. I have made a contribution worth mentioning, but I have never considered it accomplished enough to figure among "the list of original contributions" by India to FOSS. It is surely a matter of pride to see that a well known and widely acknowledged FOSS community member thinks about the project like that.

Kenneth, you clearly made my day. Thanks for the kind words. It felt really nice to see the words "HarvestMan" staring back at me from a page in an IT magazine which I was casually flipping in the midnight. Surely for a born-again techie like me, there is no better recognition than something like this.

I consider this the best compliment I have ever received in my life for something I have done. It makes more all the more excited about open source and FOSS and the spirit of sharing code and having fun at the same time.

Narcissus.py updated

2007-10-08T18:09:00.000+05:30

Since I completed the port of Rbnarcissus to Narcissus.py last week, I have been working on getting the bugs fixed. A lot of work has been done on this during the past week, and the code is now parsing a set of more than 30 javascript input samples of varying size and complexity, correctly. Also, the behaviour is very close to that of Rbnarcissus. Both the parsers now seem to fail at the same places in the code for samples which they can't parse, which is a good sign that the Narcissus.py code now approximates Rbnarcissus very well.

Since the code is now stable and somewhat usable, I have made it publicly available. The code can be browsed in the folder. There won't be any packages or formal documentation till I feel that the code is beta quality and can be made available as a Python package.

Narcissus.py

2007-09-30T23:09:00.000+05:30

Today I finished porting of Rbnarcissus to Python.

I managed to finish the porting in a total of 7 days, spending approximately
2-3 hours per day. The test parse script has also been ported. With this
I managed to produce the JS parse tree for the following simple Javascript
function.


function test()
{
   var a = 10
   var b = 20
   var c = a + b
}

As a result of parsing, the following function dictionary was printed.


{test: []}

This library will be made available as part of HarvestMan and the EIAO projects.
This is perhaps going to be the first open source pure Python parser for Javascript.

I need to do a bit more of testing on more complex Javascript code before I make the code publicly available. This might take another week or so, depending upon how much time I get to spend on this in the coming days...

Rbnarcissus Porting - Day 2

2007-09-18T01:39:00.000+05:30

In the second marathon day of Rbnarcissus porting I completed porting of another 5 functions in the Parser.rb module. What is pending are two huge functions which parses statements and expressions. That should be taken care by another day or two of hectic hacking...

BangPypers move in to python.org

2007-09-17T18:55:00.000+05:30

The entire Python family of BangPypers moved enmasse to their new home at the python.org website three days back, on Sep 14 2007.

Jeff Rush of Python Advocacy Blog was instrumental in creating the new mailing list hosted at python.org, after I sent him a request regarding the same. Jeff was immensely kind and helpful during the whole process, which got completed by end of the day. Thanks a lot Jeff!

The move came after a slew of discussions which started with this thread in early August by Anand. C (strandpyper). Quickly a kind of agreement was reached among the participants of the thread about moving out to a better place from the existing Y! group, preferably at the python.org website itself.

A lot of people participated in the discussion, giving valuable suggestions, which finally helped to reach an agreement and making the task of moving the members to the new list a painless process.

The new mailing list is public and open to anyone. This should hopefully expose the BangPypers members to the larger Python community in the international scene and give the group more visibility. It will be nice to see if any kind of larger group activities can be arranged as a part of such an exposure.

The group still lacks a coherent theme, which needs to be painted into so that it functions as a rather tight group with shared interests, than the current fragmented one. One way of doing this is to execute open source Python projects as a group, by forming small interest groups inside the larger group, which can then focus on a particular project. The monthly meetings also need to be revived, which can bring more thought into what can be done in the coming months.

If you are interested in Python and/or the BangPypers group, feel free to comment with your thoughts.

Rbnarcissus Porting - Day 1

2007-09-17T01:38:00.000+05:30

Today I completed almost 40% of the the porting of Rbnarcissus to Python. All the data structures and regular expressions have been ported along with the "Tokenizer" class.
There still remains around 13 classes to be ported which forms the bulk (60%) of the code.

If I keep up the same pace, I should be done with this in another 3-4 days. This could become a useful tool for Python programmers, having a pure Python parser for Javascript.

Watch this space for more updates.

The Ruby Way - A Python programmer learns Ruby

2007-09-08T21:11:00.000+05:30

Ruby is the language I am always putting off to learning the next day. I came across Ruby almost at the same time I started learning Python. However due to its similarities with Perl, I was never able to take an affinity towards the language.

I knew Ruby needed to be in my toolbox of languages and it was only a matter of time before I got to it. This happened last week, trying to solve a very practical programming problem.

I was trying to develop a Javascript parser/tokenizer for HarvestMan so that HarvestMan can crawl pages which defines the DOM dynamically using Javascript. I have been at this problem for some time now, but never came across a pure Python or even a C/C++ extension Javascript parser I could use. Last week I came across Rbnarcissus, a pure Ruby port of Narcissus, the open source Javascript engine written in Javascript.

I have set upon myself the task of porting this code to pure Python. I figured I knew enough Ruby to do this without any additional help, but one look at the code and I realized I needed help. I bought Hal Fulton's excellent Ruby book The Ruby Way from a book shop. (The book is a bit pricey for a low priced Indian edition, but it is worth the money.)

I have been spending the last two days with the book. I have realized a few things about the The Ruby Way when compared to the Zen of Python.

1. Ruby follows the Perl paradigm of There is more than one obvious way to do it, when compared to the Pythonic There should be one-- and preferably only one --obvious way to do it. (An import this in a Python interpreter prompt gives you the Zen of Python).

2. Ruby is a more complete object oriented language than Python and empowers its types and objects much more than Python does. However this also makes Ruby slightly more harder to learn than Python.

I feel Python is still the ideal language for a newbie who wants to learn a very high level programming language. However Ruby is much more powerful and suited for the expert programmer who expects more power out of his objects and types.

I have not got completely into the Ruby Way yet, but I am on my way. I am hoping that combining the Ruby Way with the Zen of Python will lead me to the Tao of Programming...

Some open (source) clarifications...

2007-08-31T13:50:00.000+05:30

Of late, I have been getting a few queries from people asking questions as follows:

"Have you dropped open source programming altogether ?"
"Will you stop coding in Python anymore ?"
"Have you lost interest in opensource ?"

When I thought it caused a confusion and some slight tensions with a friend who commands a lot of respect and trust and is a major facilitator of open source, I thought it was time to put it in perspective by blogging about it.

I think I am myself to blame for some of these confusions, which resulted from a recent post in my blog when I mentioned I am "taking a break from open source".

I did not intend a complete break at all...rather the post was an impulsive reaction. Let me explain.

First of all, I have not dropped open source development. I just love it too much to drop it and it has now become second nature to me after being an open source developer for nearly four years. I *cannot* stop being one overnight.

Regarding Python, it is my favorite language. I can never stop coding in Python. I continue to write open source code in Python, with most of that efforts going towards my open source project HarvestMan, whose 2.0 version is under active development. Even otherwise, my default reaction is to start a Python prompt if I have to perform some simple computation or even an arithmetic calculation!

I have not lost interest in open source at all. What happened was that I lost interest in "commercializing open source" or rather working for entities which focus on commercializing open source without contributing anything back. I had some (what appears to be now) utopian dreams in this respect, and imagined a scenario where such predator companies coexist with the people who spent their time doing open source development and contribute to a greater goal. I have realized that such dreams are pipe dreams and that most of these new "open source companies" are in it to try and make a fast buck. They have no real intentions of playing it long term or making a difference.

However when you associate yourself with such entities, you tend to associate your concept of open source with theirs some times. This was what happened to me. I spent too much time at such a place for my concepts of open source and community development to get polluted and corrupted, which ended up confusing and frustrating me a lot. When I quit, the natural reaction was a general apathy towards everything which was labeled "open source" for a while. This is something like those allergic reactions you get when you are exposed to a change in weather or surroundings; however the good thing is that an allergic reaction is not a permanent disease :)

Thankfully such reactions are not long lasting and I have come out of my black reaction finally. One good lesson I learned in the whole process was to keep my ideology separate from the ideology of the place where I work and not to mix both. If you do that you can avoid feeling frustrated when things do not work out the way you thought they will. An investment of time can be fruitful or fruitless and you may not be greatly affected; however an investment of ideology and principles can be quite frustrating if it does not bear fruit, the way you thought it would.

So, I am back to my good old ways and feeling better about it all. I will continue to be active in open source and help the community (and myself) by contributing any little code and effort I can in terms of my small projects.

I think that renaming my blog to what it was originally might be a good start and that is what I have just done. Thanks to everyone who inquired about this and well, I am being truthful to my good old ways.

Random title

2007-08-19T01:34:00.001+05:30

Looks like I am changing the title of my blog too randomly these days. I kind of like the latest one, and I might just settle on that, for a random, arbitrary period of time !

HarvestMan 2.0

2007-08-19T01:28:00.000+05:30

I have kickstarted the release process of HarvestMan 2.0, the next big version update of HarvestMan, after a gap of nearly two years. A lot of development has gone into the program during the last two years, with countless bugs getting fixed and numerous new features getting added.

As part of the process, I am making 2.0 alpha package drops available on the website. These drops are of the complete HarvestMan package with source code and documentation. You can check out the latest news on the website for more information.

If you are an existing user of HarvestMan, you should check out the alpha drops.
On the other hand, if you are new to HarvestMan, then also you are welcome to check these out, since HarvestMan-2.0 is much more robust and feature-rich than the current HarvestMan release, namely 1.4.6, which was released two years back.

Chasing dollars...?

2007-08-06T15:57:00.000+05:30

Apparently, the cat is out of the bag. Read this.

A self-demotion

2007-08-01T16:32:00.000+05:30

I just demoted myself from owner of BangPypers to moderator. Truth is that I have been getting a bit bored with open source & community in general. Moreover I have not really been able to do much Python community work for the last one year as I could do in the first year of inception of BangPypers. Since I felt I am not doing justice as being the owner of the group, I decided to give it a break.

I shall mostly demote myself to just member pretty soon. Taking a break from Python might allow me to look at other languages (open source or others) out there and allow me to do something new...

Good bye BangPypers-owner. It was nice being you...

A break from open source

2007-07-17T18:26:00.000+05:30

I am taking a break from open source, in terms of writing open source software and spending my time working with an "open source" company. My recent experiences in working full time in open source have not been very positive or pleasant. The focus has shifted - in my new avatar I won't have to think about open source as a problem to solve.

Does it mean I will stop looking into open source altogether ? No, for a couple of reasons - now a days open source is all pervasive and you need to keep in touch with it if you want to be in sync with the changes happening in the software landscape around you, of which you are a part. It still remains the best way for a developer to publish his original ideas and then take it to a larger audience with almost zero effort; also, a lot of quality projects and products are open source, which means though your product might be proprietary it is influenced by open source, directly (through borrowing code) or indirectly (through borrowing ideas/algorithms).

I will be working on HarvestMan also which will remain as an open source project.

In fact, the main changes are in two things.

1. The name of this blog has changed - It no longer has open source in it :)
2. I wont be paid to write tools for open source integration or making use of open source in commercial enterprises - I am out of that business - entirely.

Perhaps one day I will be back to that business. Of course the software landscape
keeps changing daily and the "open source companies" of the future would be mostly having totally different business models from the current ones - an interesting future to watch out for.

HarvestMan crawls up in Google rankings

2007-07-13T13:00:00.000+05:30

About 2 months after HarvestMan moved from freezope to the new site, it is nice to note that is back up in the top 10 Google queries for "HarvestMan". In fact, I note it is the 10th result as of writing this post.

Thanks Google! :)

Stacked up or hyped up ?

2007-07-10T13:24:00.000+05:30

There is a rather old article on computerworld website. The article talks about what it calls the recent "scramble" to create application stacks for enterprises using open source components. The criticism is that the market does not really exist but is a rather hyped up buzzword. One tends to draw subtle parallels with push technology of the heady dotcom days in one's mind. The technology existed, but it was trying to solve a problem that did not exist.

Open source stack vendors seems to be in a similar situation, developing solutions in search of a problem. When will this hype cycle burst ?

Barcamp Bangalore is happening

2007-07-04T15:24:00.000+05:30

And this time, many participants from BangPypers will be there. Go to the BangPypers wiki on Barcamp website to catch all the action.

Barcamp is happening on July 28 and 29 at IIM Bangalore.

Adios Amigas

2007-07-02T17:21:00.000+05:30

I submitted my resignation to my employer today. I will be seeing the last of them on the 20th of this month.

A new look

2007-06-22T13:17:00.000+05:30

My blog has a new look after Blogger "forced" me to change the template.

Power of Twisted

2007-06-19T17:12:00.000+05:30

I have been planning to learn Twisted for long, but never got a real problem to try it. Today, I was writing an XML-RPC server, and felt that this was a good time to learn Twisted, since I was not feeling very comfortable with the rather simple XML RPC server provided by Python standard library.

I read the Twisted HOWTO on XML-RPC and was off and going within 30 minutes. Within minutes I had the same XML-RPC server rewritten to use Twisted's powerful reactor framework. It took some time to figure out how to add basic HTTP authentication, but with some googling I was able to do this also in a couple of hours!

Now, I have a small framework which consists of a module which provides an extensible XML-RPC server using twisted with basic HTTP authentication. Since Twisted supports SOAP also out-of-the-box, it is quite simple to extend this to support SOAP also.

The power of Twisted is quite amazing. I am thinking of writing a version of HarvestMan which runs on top of twisted...

A scene from Dilbert

2007-06-19T10:54:00.000+05:30

Dilbert and his friends Alice, Asok and Ratbert in the cafeteria.

Ratbert: Hiya Dilbert, your face is beaming; What is news ?
Dilbert: The Boss has moved on.
Asok the intern: How does it make a difference ? These managers are the same everywhere!
Ratbert: No, you are underestimating the black powers of The Boss. He can make it or break it. I have heard, he breaks it most of the time.
Alice: I heard Catbert might also get a move on.
Ratbert: Really ?
Alice: I heard something.
Ratbert: Good riddance.
Dilbert: Yup.

HarvestMan moves to http://www.harvestmanontheweb.com

2007-05-28T14:57:00.000+05:30

HarvestMan project has moved to a new address from today. It was earlier hosted gratis on freezope. However the freezope virtual hosts have not been working properly for more than a month, which has affected the availability of the HarvestMan project website. Google also reacted promptly, taking HarvestMan project off its top queries for "harvestman". Whereas it was the number one result say two months back, it is nowhere to be seen in the top 20 results now!

The new website has the same look and feel as the original one, since I simply copied the pages over. Hope this would help the website and the project to recapture some lost page views in the coming days.

Art of Innovation

2007-05-21T11:45:00.000+05:30

I am currently reading the book by Tom Kelley with the same name. Tom Kelley is the general manager of IDEO a legendary design firm based in Palo Alto, California. IDEO is well-known as a product design firm which has created a number of innovative products and designs over the years since 1991. Some of the best examples are the original Apple mouse, the Palm V PDA, the first external module for the Handspring Visor (Eyemodule) etc. The company has won 48 IDEA (Industrial Design Excellence Award) awards, much more than any other firm in history.

The book explains the various aspects of the work culture at IDEO, giving a very vivid picture to the reader of the different elements of the work culture that makes the company have an edge in innovation when compared to similar firms. The title "Art of Innovation" is a bit misleading since the book is all about IDEO and how it is able to achieve excellence and success in its design. It focuses on the practical aspects of weaving the "innovation factor" into the business processes in a company, and does a very good and thorough job of it. A must-read for any body who is working in industries which involve creative work.

I shall be posting a more complete review of the book after I finish reading it.

What I am doing these days

2007-04-25T18:46:00.000+05:30

These are the two things I am doing these days, apart from regular work at office.

1. Developing the next version of HarvestMan
2. Developing the next version of HarvestMan

Well, that is right... It was not a typing mistake :-)

HarvestMan is going to get a major update coming May, and it will be the result of more than 1.5 years of work. In fact the program has changed so much that I am changing the major version. It is going to be HarvestMan-2.0.

There are certain surprises with this HarvestMan release. Some of the interesting changes for NLP and Computational Linguistics programmers will be the addition of a plugin API that makes developing extensions for HarvestMan a breeze. In fact, the current CVS of HarvestMan already features an extension which binds it to an existing open source indexing engine. Apart from that, the program features a number of changes from the earlier version (1.4.6) so that it is almost on its way to becoming a "platform for web crawler software development", where I envision it to be.

Apart from that, this time HarvestMan will consist of two apps in one - that's right. There will be two applications using the same codebase. The crawler application (HarvestMan, of course) and a brand new web downloader application which supports multipart downloads. Let the name of this application be a mystery for the time being. The application just might change the command-line download experience of Unix/Linux users from the typical wget one. I will write more about it in the coming weeks.

In fact all of this is already in CVS. Anyone interested can checkout the latest code from berlios repository using anonymous CVS. There is not much documentation apart from the documentation in the code, but the code is pretty stable at the moment.

This should get released by mid May.

Open source and Innovation

2007-04-24T00:43:00.000+05:30

What makes a successful open source project ? What makes a successful open source business ? How does successful open source projects make the transition to good business which still keep with the spirit of open source and open standards ?

These are questions any developer who is a serious contributor to open source would be interested in. Even if you are not an open source contributor, you will be interested in these questions, if your company is working in open source.

According to my experience so far, with my own projects and with some of the international projects I have participated in, a successful open source projects brings some new ideas to the table. New ideas need not be confused with new way of doing things or new I.P. It can be a new implementation of an existing protocol, it can be an open implementation of a proprietary standard, or it can be a project that uses existing open source components or applications to solve an existing problem (or a new one) in interesting and innovative ways. These need not always generate new kinds of intellectual property.

Yes, the key here is innovation. Good open source projects bring a fresh way of solving existing problems; they give a fresh perspective to existing way of doing things. Sometimes they are able to rewrite the rules by capturing the imagination of many hundreds of developers and thousands of supporting community members - a good example is the Firefox community. Some times, it will be a rather closeted group of skilled people in a rather niche area who finds a void in the experience of open source applications/operating systems and tries to fill the gap - a good example is the Beryl/Compiz projects which are working hard to bring display compositing to the Linux and open source crowd.

However, a common thread to all these project is this - they innovate. They innovate in fresh ideas, simplifying user experience and sometimes on performance. They often open up an entire new facet to an existing problem which makes programming a joy.

What do successful companies in open source have in common ? They understand the importance of keeping the developer crowd happy. They are keen to become good citizens of the open source community and contribute either their manpower or projects to the community - some do both. They understand that it is important not to just become consumers of open source but also stakeholders and participants.

When a company fails to understand this, or fails to create a working, effective developer policy towards open sourcing, it is prone to be assigned the category of a second or third rate citizen in the open source community. By just becoming a consumer of open source and not contributing enough, it risks alienating the coding crowd who tends to think of the company as a predator, not as an ally.

Most often, such companies never learn to use open source the right way too. By not participating enough, they fail to understand the driving force behind open source and people working in such projects. This in turn makes them less effective users of such software. For example, a company that brands itself as an open source integrator can never be quite effective if it does not understand the open source projects it is integrating and does not contribute developer resources to such projects; in fact, it is not even necessary to contribute directly most of the time. Indirect participation such as hosting meetings, contributing tools, toolchains and providing a platform for discussion and creation of new ideas are also good ways of contribution.

A company not doing any of these and still claiming to work in open source is somehow not doing the right thing. Such strategies are doomed to fail in the long term and even prove counter productive. In the long run a company like this is bound to move away from open source or bound to fail.

Some ordered thoughts on Gnosis enhancements and XPath

2006-07-04T14:11:00.000+05:30

I had dropped out of the blogging habit after Feb this year, mainly due to work pressures at office. I think it is time for me to dust out my blogging brush and start painting my blog a little bit, at least to touch up some of the loosing sheen here and there :-)

This is about a nice little utility called Gnosis utils which provides powerful XML parsing in very little code. The library is written by Dr. David Mertz who is a known authority on Python programming matters and has written some charming articles in his "Charming Python" series for IBM developerworks.

I was looking around for an XML API which provides powerful XPath parsing capabilities out of the box some time in March for a project I am doing at Spikesource. PyXML woefully lacks in this department. ElementTree, though a very good API for generic XML processing, falls short on its XPath support (no attributes etc).

Gnosis provides decent XPath support - it supports attributes, text searches but does not support attribute values, the [@attr] syntax etc.

During my work, I enhanced Gnosis XML to support some of the XPath 1.0 specs it do not support. These include,

o Support for //elem[@attr] syntax
o Search attributes by value - i.e support //elem[@attr=value] syntax
o Support //elem[last()]
o Support XPath searches in the root node

The good thing is that I have been able to extract out the Gnosis XML processing parts to a single module and add these enhancements on top of it. I plan to enhance this with the full XPath 1.0 specifications within a month or so and release it to public domain.

I hope this will address the existing void of lightweight pure Python APIs with full XPath support. The whole thing will fit inside a single module which should make it easy to use and extend.

Looking forward to completing this work!

BangPypers meet this Sunday

2006-02-24T00:14:00.000+05:30

After a long time, the BangPypers are meeting this Sunday, the 26th again. The meeting is organized in the Spikesource conference room named Ruby. How is that for some co-operation between two popular dynamic languages !

Sidharth Kuruvila is giving a talk on one of his favorite topics, namely generators in Python. Sridhar Ratna of Amazon will talk on TurboGears, his favorite Python web dev framework. Also, Anush Shetty, a very active Python community member in Bangalore shall organize a BoF session on Python Education.

It is going to be an exciting event. If you are interested in Pythonic happenings in India's silicon valley, make sure you drop in to Spikesource offices in Diamond District on Airport Road on the 26th at 4.30 pm sharp.

Unfortunately, I cannot attend this meet since I am going out of Bangalore this week-end to visit my family. Deepan from Spikesource should take care of the meeting co-ordination.

Hope you all have a fun and informative session!

See the meeting description in the group's calendar for details and contact information.

New exception model in IronPython

2005-12-16T11:13:00.000+05:30

I am a lurker in the IronPython mailing lists. Now that the project is slowly approaching 1.0 release, I have started taking an active interest in the proceedings in the list.

The recent postings on the proposed new exception handling system in IronPython makes interesting reading. Apparently the current model does not provide a nice co-existence of the exception hierarchies of CPython and the .NET CLI. This means that if you call a piece of pure CLI code from inside your Python (IronPython) code and if it raises a CLI exception, you are left stranded in no-man's land if your code only catches Python exceptions.

The proposed new model plans to separate the exception hierarchy into two hieararchies - one for .NET CLI exceptions and one for Python exceptions. The Python exception hierarchy will mirror that of the existing CPython exception class hierarchy. Under this model pure Python and CLI code catching their own exceptions will work the same. When the two worlds collide, i.e when pure Python code tries to catch a CLI exception or when pure CLI code tries to catch a Python exception, the system will convert the exception to the required type.

For example, if the Python code raises EOFError, the CLI code will see an EndOfStreamException. If the CLI code raises FileIOException, the Python code will see an IOError. The error mappings happen automatically.

Some problems still remain unsolved - like what the user should see as an exception traceback; should it be pure Python exceptions or .NET ? The current thinking is that the language should allow both forms in a user-configurable way.

One issue with this model is that in order to map CLI exceptions to Python exceptions you need to go through some kind of mapping for CLI to Python exceptions; this could slow things down a bit. Also, the model requires rethrowing of exceptions whenever the CLI and Python worlds collide.

It will be interesting to wait and watch how the new model develops.

The new exception model is described in detail in this Channel 9 wiki posting.

Back to Bangalore

2005-12-02T13:31:00.000+05:30

I came back to Bangalore after six and a half fine days in Norway on 1st early morning, 2 am. The flight from Frankfurt to Bangalore took 8 hours.

The photo album is at Y! which is online here.

After spending six days at near and below zero temperatures in Norway, Bangalore is feeling hot.

Workshop in Norway

2005-11-23T15:45:00.000+05:30

I am leaving tonight (Nov 24 2005, 2.15 am) to Grimstad, Norway for taking part in the 2nd international EIAO conference and workshop. This conference is organised by AUC as part of the EIAO project.

I will be making a small presentation on D-HarvestMan. I am returning to India on 1st Dec.

Here is a brief abstract of the proceedings in Grimstad.

2nd international EIAO conference and worskhop


24.11.05
1. Presentation of overall timeplan
2. Presentation of preliminary Observatory anatomy
3. WAM coordination (D3.1.1, UWEM, D5.1.1.1 and D5.1.1.2 )
4. Coordination of WP3, WP4 and WP5 release 1.0 time plans
5. Overall release 1.0 planning and actions

25.11.05

1. Welcome Mikael Snaprud
2. D-Crawler - Anand B. Pillai
3. Presentation of anatomy methodology Alf Fredvik
4. SW development process Design and test. Parastoo Mohagheghi, 
Per Wollebæk

26.11.05 and 27.11.05

1. Presentation of overall timeplan
2. Presentation of preliminary Observatory anatomy
3. Development of a DW anatomy
4. Description of each anatomy
5. Scheduling of each anatomy
6. SW development process (Design and test)
7. Coordination of WP6 and WP5 release 1.0 time plans
8. Overall release 1.0 planning and actions

28.11.05 
Follow ups and documentation
1. Inspection and adjustment of plans for each anatomy
2. Strategy for outreach and dissemination towards 
and after release 1.0
3. Summary and outlook - HarvestMan as a vehicle for 
research based teaching

Internet Summit in Tunis

2005-11-17T19:50:00.000+05:30

The World Summit on Information Society (WSIS), dubbed as the "Internet Showdown" by CNET, is being held under the aegis of the U.N in Tunis, the capital of Tunisia at present.

Dr.Mikael Snaprud of the EIAO project presented a paper in the conference(Past, Presence and Future of Research in the Information Society) associated to this summit on Nov 15. The paper talks about the role open source plays in ICT education and research, with the EIAO project as the background. The presentation associated with this paper is available at the EIAO Publications website.

The paper is authored by Mikael Snaprud, Agata Sawicka, Anand Pillai(myself), Nina Olsen, Morten.G.Olsen, Vidar Laupsa and Terje Gjøsæter.

D-HarvestMan prototype is born

2005-11-14T02:27:00.000+05:30

It is 2.30 am in the morning right now. I am in a good mood. The reason is that I just finished coding and testing the basic D-HarvestMan prototype for a single master, single slave configuration. And it works! The master was able to successfully bootstrap the slave crawler with a new domain and let it start downloading files from it. Hip hip hooray!

D-HarvestMan is a project to write a distributed crawler on top of the existing HarvestMan. Distributed programming is always exciting, and a distributed crawler is even more so :-)

The fox is one year old

2005-11-11T02:24:00.000+05:30

Firefox turned one year old on Nov 9, two days back. The wily fox has set the web on fire, ever since it debuted on Nov 9 2004. With the 1.5 final release on the way, it is well on course to capture further market share from the beleaguered I.E .

Three cheers to the Firefox team and wish all the best to the fox for its second year!

OOo plugin for Firefox

2005-11-07T13:54:00.000+05:30

In the context of my previous post, it is interesting to see that people have already talked about how an OOo plugin for Firefox can be a killer collaboration application on par with MS Sharepoint.

Apparently a Mozilla plugin is already available in OOo 2.0. The problem with it is that you enable the plugin from inside OOo, which requires that OOo is already installed in your machine.

From the Mozilla OOo plugin specification,

"Plug-in usability

The plug-in works only if a working OpenOffice.org installation is found on the system""

This reduces the usability of the plugin, I think. Developing a light-weight OOo plugin for Mozilla/Firefox which can be installed on the fly could be pretty useful.

Some random thoughts on a web-based office suite

2005-11-05T14:08:00.000+05:30

Ever since Sun and Google announced their partnership early in October, speculation has been rife on the possibility of a web-based office suite, aptly titled "GoogleOffice". However, there has not been any strong indication of such an effort underway. In fact a number of industry watchers were disappointed when Google and Sun announced that the initial collaboration would be on bundling the Google toolbar with Java runtime downloads.

Now that Microsoft is alligning itself as a services provider and trying to offer integrated solutions by bundling its diverse product portfolio (Office, MSN, messenger etc) along with its new service initiatives (Windows Live, Office Live), it is probably time that Google looked into countering these overtures with appropriate answers - I think there is nothing more fitting here than an office solution which integrates Gmail, Google Talk, Google Desktop and the Firefox web browser.

Perhaps such an effort is already underway in Google Labs. However, here is my vision for such a solution, a kind of blue print for a future GoogleOffice on the web.

Google office will be a collaborative suite integrating Gmail, Google Desktop, Google Talk and Firefox. Ideally it should have the following components:

1. An open-office plugin for Firefox
2. An extension to Google desktop that allows Gmail attachments to be searched and accessed.
3. An extension to Google talk that allows members to access attachments in their Gmail account and also share files and folders through Google Desktop.

Let me explain:

1. The Firefox plugin will allow one to view Openoffice documents inside the Firefox web-browser. Initially this need to support only OOo native file formats and the OpenDocument format, but MS office support would be preferable.
2. Right now attachments cannot be searched in Gmail. Google needs to add this capability to Gmail. Attachments can be searched by name, but they also need to be searchable by content.
3. Google desktop integrates with Gmail, but again attachments are not searchable or accessible. This capability need to be added to Google Desktop so that one can search and access documents stored as attachments in his Gmail id through Google Desktop.
4. The same capability should be added to Google Talk so that one can search for attached documents from Google Talk.
5. Integration between Google Talk and Google Desktop so that chatters can share documents with each other and search each others desktop, given sufficient access control privileges.

All these pieces will allow for a basic GoogleOffice over the web. Let us look at some common scenarios:

1. Someone is browsing in an Internet cafe - He wants to view an OOo document sent to his Gmail id as email attachment. Typically Internet cafes do not have OOo so he is at a loss (This happens to me quite often). This is where an OOo plugin for Firefox can help. Gmail need not know anything about this plugin. It can be a regular firefox plugin. In this case, Firefox will detect that the plugin is not installed, will download and install it automatically. Voila, you can view your OOo document inside firefox.

I dont think it will be too difficult to develop such a plugin, considering that the source code for OOo is open and the OOo file formats are well documented. This will also help in large scale acceptance of OOo file formats in a way similar to PDF. Editing capabilities will also be nice but this could be tough to implement in a browser plugin.

I would like to see plugins for all OOo file formats but specifically OOo writer (.sxw), OOo impress (.sxi) and the OpenDocument formats.

2. Adding capability to search and find attachments in Gmail will allow people to use Gmail as a sort of virtual storage for their documents (read office documents). I tend to do this even now, but because the attachments are not searchable, the experience is crippled. Initially these can be added for well known and open formats such as PDF and the OOo file formats.

3. Once Gmail is enhanced with advanced capabilities to search attachments, this capability should be integrated with Google desktop which can then index the attachments and make them searchable from the desktop. Considering the work involved in indexing large attachments, it actually makes sense to add this capabillity only to Google desktop rather than onto Gmail directly.

4. This opens up the possibility of integrating Firefox, Gmail and Google Desktop for searching, accessing and modifying office documents. One can use Google Desktop to search office doucments stored in his Gmail account, then open and edit them on his desktop inside Firefox using the OOo plugin. If OOo is installed on the machine already, it can be used instead.

5. The last missing piece is Google Talk. Google Talk should integrate with Google desktop allowing querying of documents (Gmail, desktop, photos etc) through Google Talk. Google should add capabilities to Google Talk which will make this possible not only on one's own desktop but across desktops.

That is, you can give privilges in Google Talk to selected Google Talk users (your friends, co-workers, family), to search and access documents from inside your desktop and also your Gmail id. These can be controlled by access at various levels - read-only, read-write etc.

If one finds documents of interest they can be shared and edited online. Google should provide something like a shared space for two interested parties to share documents as a part of Google talk. This space can either be part of Gmail or separate from it. However, it should allow for saving documents by multiple Google talk users, acting as a kind of collaborative space.

Thus Google Talk and Google desktop along with Gmail and Firefox can be used to build a virtual collaborative office suite on the web which if done well, can probably pose some compeition to Microsoft office solutions and the recent office collaboration initiatives. It will also allow to take the OOo efforts to the web and provide it as a part of a services offering instead of the current stand-alone product.

Perhaps this vision is a bit grand, but I don't think it is a difficult one for Google and the open source (read Openoffice) community. All the pieces are already there, they just require some additional capabilities and some plumbing to work as a unified, web-based solution.

I have not thought to deep about the technical aspects of such a solution but a very interesting thought will be the role Java and Google toolbar can play in this integrated approach. Perhaps Java can be used to develop the Firefox OOo plugin also.

I hope we can expect to see a web-based office solution from Google, Sun and the OOo community within the next 12 months.

FOSS.in

2005-11-02T13:55:00.000+05:30

FOSS.in has published the second list of speakers of the event. Some notables who are speaking include the legendary Alan Cox, Danese Cooper, David Fetter, Jeremy Zawodny, Jonathan Corbet, Andrew Cowie, Harald Welte and Brian Behlendorf.

Murugan Pal is giving a talk on "Open source Alternatives".

Also the talk by Zaheda Bhorat of Google on Google and open source seems interesting.

FOSS.in is scheduled from 29th Nov to 2nd Dec 2005 at Bangalore.

On Open Voting

2005-11-02T08:58:00.000+05:30

An article published in a local daily of Granite Bay, CA, talks about accountability and the Open Voting Consortium.

Read the article.

What is my interest in this ? Well, I happen to be part of the team that originally developed the OVC prototype, which was demonstrated in April 1 2004. The OVC project was my first experience in working for an international open source project. The architects of the system decided to use Python for the project, which was how I got interested in it.

I am also a founding member of the OVC.

International Conference on Digital Inclusion and Open Source

2005-10-22T15:15:00.000+05:30

The annual conference on "Digal Inclusion and Open Source" took place in Oslo, Norway from Oct 20-21, 2005.

A paper co-authored by me, Parastoo Mohaghegi, Mikael Snaprud & Nils Ulltveit-Moe was presented in the conference. The paper highlights the activities of the EIAO project in bringing together users and external contributors from different parts of the world in an EU sponsored project.

Read the abstract of the paper.

HarvestMan web-site redesign

2005-10-04T22:22:00.000+05:30

I have finally re-designed the HarvestMan web-site! It has been something I have been planning since the beginning of this year! When I finally did it, it took me only two days. Surprising, how much savings one can get in terms of time, if one really focuses on the task at hand.

Microsoft and JBoss shake hands

2005-09-28T13:26:00.000+05:30

Probably bad news for LAMP and other open source stacks. Complete news is here.

Pyrex

2005-09-13T13:42:00.000+05:30

I am currently learning Pyrex, a language designed to take the pain out of writing C-extension modules for Python.

I plan to rewrite performance intensive portions of HarvestMan using Pyrex. These enhancments will be available as part of the next major HarvestMan release, version 1.5.

Cathedral tries to recruit Bazaar!

2005-09-10T15:38:00.000+05:30

No kidding :-). Microsoft apparently tried to recruit Eric Raymond. If you don't know who Eric Raymond is, spend your afternoon reading up the excellent Cathedral and Bazaar essays, some of the best essays written on the open source model. He also happens to be one of the co-founders of OSI. Talk about Bush trying to recruit Osama Bin Laden for homeland security!

Read more about Microsoft's Mea Culpa in this article posted on ESR's blog.

Standalone Executables on Windows using py2exe

2005-09-09T14:53:00.000+05:30

The latest release of py2exe, namely py2exe 0.6.1 allows to create single executables on Windows. This is an improvement over the earlier versions which used to create a host of files around the main executable. I think the new feature is a welcome one, especially for a project like HarvestMan which has a number of dependencies. If you try to create an executable for HarvestMan with existing versions
of py2exe, you get quite a lot of .pyd files which are well, a bit confusing.

I am looking forward to create standalone executables for HarvestMan using the latest py2exe and provide downloads for them. I think this should boost the popularity of HarvestMan, since many Windows users I know could not be bothered with going through all the steps to install a pure Python package such as HarvestMan. Downloading and installing a single file executable is much easier.

Expect win32 downloads of HarvestMan soon. Three cheers to py2exe and Thomas Heller.

Uraga is dead - Long live Uraga!

2005-08-31T14:29:00.000+05:30

Uraga is dead. Swaroop has confirmed this in a post to BangPypers.
I long doubted this might be the case, since there was no mention of Uraga in Swaroop's blog for quite some time. The reason he cites is job pressure; as if guys who contribute to open source do not do justification to their daily job! It sounds ironic, to say the least.

I have a principle which I apply in any open source or professional work I undertake; that is, if I propose an idea, I will try at least to do a basic prototype implementation of the same. More so, if I am talking and letting the world know about it. It is a basic contract that one should have to the community with which one interacts, especially when one tries to market the community with the tag of his idea. In this case, the community is BangPypers and the idea is Uraga of course.

If you don't fulfill this basic social contract, then you are not fit to be an open source contributor, let alone an open source project initiator. I hope some of the new-age geeks who looks to open source for quick stardom realizes this.

Design Patterns - Modelling the Singleton in Python

2005-08-24T18:44:00.000+05:30

Singleton is a design pattern that seems to interest everyone, especially in the Python world.

I was doing a Google search on the ways in which Python implements the Singleton design pattern.

The results showed that in doing this in Python, you are limited only by your imagination. Unlike C++ or Java, you are not limited to a certain strategy of modeling the Singleton in Python.

I thought it was a good idea to gather the different Singleton solutions in Python and post it in a single post (pun intended) here. In this post, I list seven ways of modelling the Singleton in Python which looks elegant to me. I am not including some overly verbose or cryptic solutions which you will find if you perform the Google search above.

Though I have no preference for any particular solution, I have ordered them in the order of what I think is the least elegant solution, to the most elegant one. Of course this is purely personal! :-)

A word of caution: Except for the Borg solution, the rest of them will work only with new style classes. Also note that some solutions are exactly the same, though they look different. I have explained them as we go along.

The most basic solution overrides the __new__ method in an outer class, returning an instance of an inner class as the Singleton. Here it is:

class Singleton1(object):
    """ Singleton by overriding __new__ and using an inner
class by using new style classes """

   class __Singleton(object):
       pass

   __instance = None
  
   def __new__(cls):
       if not Singleton1.__instance:
           Singleton1.__instance = Singleton1.__Singleton()
       return Singleton1.__instance

Here is this solution in action:

s1=Singleton1()
print s1
s2=Singleton1()
print s2
s3=Singleton1()
print s3
<__main__.__Singleton object at 0x009F1390>
<__main__.__Singleton object at 0x009F1390>
<__main__.__Singleton object at 0x009F1390>

Clearly the drawback with this solution is that it is not a Singleton in its true sense. The Singleton class does not return an instance of itself, but an instance of an inner class. In other words, what appears to be the Singleton class is actually a class wrapper around an inner class, which is the actual Singleton. Not very elegant.

The next solution fixes this problem. It works directly with the guts of the class by accessing the classe's dictionary.

class Singleton2(object):

""" Singleton by using new style classes """

    def __new__(cls):
        if not '_the_instance' in cls.__dict__:
            cls._the_instance = object.__new__(cls)
        return cls._the_instance

All right. Is there something magical about the _the_instance attribute ? Nothing. So why can't we replace it with a direct class level attribute? Yes, you can though there is not much difference in both technically. However, it looks like a kind of combination of the first solution with the second one (which it is not), so here it is for illustration purposes.

class Singleton3(object):

    """ Singleton by using direct class attribute
access without using cls.__dict__. This might
look different from Singleton2, but in fact
it is the same. """

    __instance = None
   
    def __new__(cls):
        if not Singleton3.__instance:
            Singleton3.__instance = object.__new__(cls)
        return Singleton3.__instance

All right. Enough of fooling around with classes directly. Can't we do this by using metaclass magic? Looks like you can. And with most metaclass solutions, it seems to be somehow more elegant than directly putting the logic inside the class!

Here is the solution from Bruce Eckel's Thinking in Python.

class SingletonMetaClass(type):
def __init__(cls,name,bases,dict):
super(SingletonMetaClass,cls)    .__init__(name,bases,dict)
original_new = cls.__new__
def my_new(cls,*args,**kwds):
 if cls.instance == None:
   cls.instance =         original_new(cls,*args,**kwds)
 return cls.instance
cls.instance = None
cls.__new__ = staticmethod(my_new)

class Singleton4(object):
__metaclass__ = SingletonMetaClass

The idea is to override the __new__ method of the object's class right in it's metaclass's __init__ method. This is done the first time the object is created. The overrided __new__ works quite similar to the one in Singleton1,Singleton2 and Singleton3. Thereafter, everytime you create an object of Singleton4, it will call the __init__ in its metaclass where the magic happens.

The elegancy of this solution comes from the fact that, the extra code for the class is just one line, where we assign the __metaclass__ attribute. All the magic resides in the metaclass, which allows one to quickly convert his class to a Singleton by adding just this one line.

NOTE: In fact the metaclass solutions for Singletons (or any other patterns for that matter) are not class scoped solutions, but type scoped ones. It might take some time to wrap your head around that, if you come from a C++ background.

But does this look a bit cryptic? Well, I should say yes since it took some time for me to figure out what is happening here. Apparently, you don't need to take all that trouble to get it right. Here is the above solution re-written, but without the inner function and all that.

class SingletonMetaClass(type):

    """ Singleton using metaclasses by overriding
the __init__ method, 2nd version. """

    def my_new(cls, *args, **kwargs):
        if not cls.instance:
            cls.instance = object.__new__(cls)
        return cls.instance
           
    def __init__(cls, name, bases, dct):
        super(SingletonMetaClass, cls).__init__(name, bases, dct)
        cls.instance = None
        cls.__new__ = cls.my_new

class Singleton5(object):
    __metaclass__ = SingletonMetaClass

The above solution is a re-write of Singleton4, but lesser cryptic and more readable.

Is that all you can do with metaclasses and the Singleton in Python? The answer is No. Looks like there is a much more elegant way of doing this using metaclasses. It is done by overriding
the __call__ method in the metaclass, instead of the __init__ method. This solution is the ASPN Python Cookbook Recipe #412551 by Daniel Brodie. I am just copying it here.

class SingletonMetaClass(type):

    """ Singleton using metaclasses by overriding the __call__ method.
          Original code courtesy from ASPN Python Cookbook recipe number
          412551 """

    def __init__(self, *args):
        type.__init__(self, *args)
        self._instances = {}

    def __call__(self, *args):
        if not args in self._instances:
            self._instances[args] = type.__call__(self, *args)
        return self._instances[args]

class Singleton6(object):
    __metaclass__ = SingletonMetaClass

Before I conclude, I should include one of the most ingenious methods of doing the Singleton in Python, created by Alex Martelli. This is the so-called Borg non-pattern. This and the concept of non-patterns is discussed in detail here.

The Borg is unique in that it re-defines the problem Singleton is trying to solve. Instead of trying to ensure that the unique instance maps to a unique memory location, Borg ensures that the state of the various instances are shared and hence the various instances are in effect, the same. In other words, Borg focuses on object equivalence instead of object identity which is what Singleton offers.

Here is the Borg non-pattern, applied to the Singleton problem.

class Singleton7:

    """ Alex martelli's Borg non-pattern. Not exactly
    a singleton. Focus on equivalence of state rather than the
    uniqueness of the Singleton instance """

    __shared_state = {}
    def __init__(self):
        self.__dict__ = self.__shared_state

Here is the Borg non-pattern in action.

s1=Singleton7()
# Set s1's state
s1.x = 100
print s1.__dict__
s2=Singleton7()
print s2.__dict__
s3=Singleton7()
print s3.__dict__

{'x': 100}
{'x': 100}
{'x': 100}

Well, according to me, that is the most elegant one. Instead of solving the Singleton problem directly, it solves the problem that the Singleton is trying to solve, which is that of ensuring
unique state across instances.

That is the end of my first post on Python and Design patterns. I hope to add more in the future, on the rest of the Gang of four Design Patterns.

Freezope is down

2005-08-02T12:45:00.000+05:30

Freezope is down since yesterday. I was planning to make the 1.4.5 beta 1 release of HarvestMan today, but it looks like I cannot provide any updates on the HarvestMan site till freezope comes back up.

I might still make the files public at BerliOS and make the freshmeat and PyPI announcement today or tomorrow. There won't be any update at the site of course, but that can wait till freezope is back on track.

Apple? - Not on BillG's map

2005-07-27T21:27:00.000+05:30

The latest offering from Redmond, MSN Virtual Earth beta is making the buzz for all the wrong reasons.

The Register reports that Apple HQ is nowhere to be found on MSN virtual earth. Apparently, MSN virtual earth has chosen to rebuild the World Trade Center towers too. Looks like the images Microsoft used is way too old.

Clearly, MS has pushed a hastily built software that has not undergone enough testing to make even to alpha stage, as a beta. The stunt is an apparent knee-jerk reaction to the buzz Google is making with its map service and Google Earth. This is supposed to be a competition to Google Earth, but clearly the features or stability to be a credible competitor are missing. This is when Google has upped its sights and started to map the moon!

My gripes about MSN Virtual Earth.

1. The images are way too old. The image quality and resolution is way lacking when compared to Google Earth. Since MS seems to have used U.S.G.S archives which are 10 years old, this is to be expected. But hey, we are talking about one of the richest companies in the world. Surely they could have done better.

2. The map is not as responsive to panning as Google maps. The tiling does not work very well, often leaving large patches of yellow rectangles without resolving the underlying geography. Also, the maps load pretty slowly and often you are left staring at blank patches of screen for seconds before the tiled images start loading. You almost end up thinking that the server has stopped responding. Yeah, I am on 256 kbps broadband, not dial-up :-) .

3. Screws up Firefox. After some panning and zooming, some buttons on Firefox (I am using 1.0.5) does not work well. And hey, why does it disable my "Back" button?

4. "Locate me" feature. I don't understand the actual need for such a feature except for generating buzz. The feature requires ActiveX which I think is one-step backwards for Microsoft for making this technology work with other browsers such as Firefox. The I.P address based location does not work very well. It located me in Bombay, India whereas I am actually in Bangalore.

5. Sticky mouse buttons: The left mouse click tends to "stick" on the page. So even after I release the mouse button and move the mouse, the image keeps panning. I have to click the left mouse button once more to "release" the "stickiness".

6. The stupid compass: Why do you need that? It just eats up space on the map and is really not very useful.

7. Interface quality: The interface seems hastily thrown together and not well thought out. It is counter-intuitive if you are used to Google maps since the search and address box is on the left rather than on the right as in Google. To me, right position seems more intuitive and user friendly since that means the map does not overlap on the right edge of my screen and looks solid. Also, MS seemed to have not done much research on how surrounding colors affect the users ability to effectively view the maps. The choice of colors and color contrast leaves a lot to be desired. If I were the engineer, I would replace that stupid blue overlay on the top with something that increases the contrast and is less straining on the eyes.

The pluses.

1. It occupies more screen area than Google maps. Clearly this is an advantage.
2. Seems to show more places in India than Google. Hell, it even shows some obscure towns and villages in Kerala, my native state. Google maps suck in that respect.

Conclusion: Not enough reasons for anyone to switch to Virtual Earth if he/she is already using Google earth/google maps. I think MS should apologize, rename the release to MSN VE alpha, work on it a bit more and release a much better, user-friendly and fast beta. Then I would agree that it is competition.

Apple - An "i" for Innovation

2005-07-23T18:20:00.000+05:30

iMac, iPod, iTunes - Popular brand names from a company known for its innovative products - Apple.

Now it has become official. In a Businessweek poll conducted among top executives from across the planet, Apple won the top slot for the most innovative company, with nearly 25% of the votes.

3M came second with nearly 12% of the votes, followed by Microsoft. Big Blue came a distant 7th after GE, Sony and Dell in that order.

Another feather in the innovative cap of CEO Steve Jobs.

Way to go Apple! Well done.

Hasta-la-Vista Windows!

2005-07-23T17:56:00.000+05:30

Microsoft has re-christened its unborn poster child,
"Windows Longhorn" as "Windows Vista".

The marketing speak from Microsoft is that Windows Vista brings clarity to the connected world.

Personally, I think "Longhorn" was much better than "Vista" though it gave rise to subtle bovine references.

It will be no wonder if the naming turns out to be prescient and puts an end to Microsoft monopoly on the desktop, considering the rate at which features are getting axed from Longhorn aka Vista.
(Also read Steven J Vaughan Nichols' article on axing of Monad from Longhorn).

Is it time yet to say "Hasta-la-Vista Microsoft"? Let us wait and watch.

Longhorn Screenshots

2005-07-11T17:12:00.000+05:30

Some screenshots of Longhorn have made it to a few blogs today. The news has also hit Slashdot. The winbeta.org site claims that this cannot be part of Longhorn Beta1, since the build number 5203 is not part of the Beta1 builds, which means it must be a leak from inside Microsoft.

The screenshots show a system which looks like a modified Windows XP with transparency and some other eye candy and a black theme. Nothing great when you consider that this system was supposedly 5 years in the making.

Microsoft cannot even do a copy job well. ;-)

Btw, I have added a few screenshots right here with the new photo feature of blogger.

Originals, courtesy xerocool.

Europe says no to software patents

2005-07-07T10:44:00.000+05:30

On July 6, the European parliament decided
by a large majority to reject the software patents directive.
This is great news for the FOSS community and open software
innovation in general. The patent directive was a looming threat for
software innovation in Europe, since it threatened to bring
into legislation, the US style practice of supporting patents
in software and business processes.

Redhat and Sun Microsystems worked with the
Foundation for a Free Information Infrastructure (FFII) to bring about this victory for those who value free and
open source software.

This should be a blow to Microsoft's business aspirations in Europe
and should help the Linux community.

Guido did not create Python!

2005-06-02T15:41:00.000+05:30

Well, you and me know that the title of this post is not true, but that is not what Mark Joseph Edwards would have us believe, given a chance.

In an article on Windows IT Pro Magazine introducing IronPython, he writes...

"You might have heard of Python, a popular and powerful cross-platform open-source programming language. Python, which is the creation of Jim Hugunin..."

I am not sure how many Pythonistas read Windows IT Pro, but I won't suggest it to someone who is new to Python, for reasons that are very clear. :-)

IronPython, Boo and Smallscript

2005-06-02T15:16:00.000+05:30

What do the terms IronPython,Boo and Smallscript have in common?

Well, before you type an entry into dictionary.com, let me tell
you that these are all relatively new language implementations for the CLR provided by the .NET platform.

IronPython is a supposedly faster implementation of CPython for .NET, which allows you to extend Python on .NET with C# and other .NET static languages.

Boo is a new, statically typed OO language for .NET with a Python-inspired syntax, but without it's dynamic typing overhead. Boo claims to be as fast as C# or VB.net on the .NET platform.

Smallscript provides the new S# language, a modular re-design of the Smalltalk language, designed to work with .NET and the AOS common language platform which supports dynamic languages.

IronPython had initially hogged the limelight, but after the
creator Jim Hugunin was appointed by Microsoft, it has become something of a one-man-show instead of becoming a true peer reviewed open source project.

Best Essays on Software in 2004

2005-05-04T13:44:00.000+05:30

I found a post on the best essays on software in 2004 at Max Ischenko's blog today and felt that I should make a mention of it in my blog.

The discussion started in the popular Joel on software, when Joel asked his readers to post
essays which they consider are the best on software related topics in 2004..

Go to: Best essays on software in 2004.

Logix Talk

2005-05-03T11:21:00.000+05:30

I should be blogging more often, but some how I have to put a lot of mental effort to get my ideas into words most of the time, which makes me a pathetic blogger ;-)

Well, I had a nice trip to Norway last week where I met with some cool folks at Agder University College with a vision to create accessibility metrics in order to improve European web sites. I am involved in the project because they are using HarvestMan web crawler, as part of the crawler component in the project. I have to blog about it separately, since there is a lot to write. However, here are some quick links:

Web
accessibility and meta-modeling workshop on 15th April 2005

European Internet Accessibility Observatory Home Page

Hey, I am diverting... This post is about the upcoming talk on Logix, a multi-language programming system developed by Tom Locke. He says has "outsourced himself" to Bangalore for developing and popularising Logix here. The idea is to allow the Python developer to create his own languages and schema
by using a meta-language model.

I think it is ideally suited for tasks such as meta-modeling which is one important aspect of EIAO. Perhaps the EIAO folks will look into Logix for developing some of their metamodeling components. It could also be used for developing software modeling techniques similar to UML, specific to Python.

The talk is on May 7th at the regular venue for BangPypers meetings, i.e at ThoughtWorks, Diamond District. For more information see my post at BangPypers
regarding the talk here. For marking your calendars, go here.

BangPypers is turning out to be an important forum for presenting Python related
technologies and reaching out to the Python community here. The Logix talk is yet another achievement which shows the popularity of BangPypers.

If you are in Bangalore on May 7th and you like the idea of multi-language programming, it might be a good idea to drop in at ThoughtWorks by 5.00 pm to attend Tom's talk.

Yahoooooooooo!

2005-03-20T18:32:00.000+05:30

Yahoo! completed 10 years on 3rd March.
Yahoo!, which started off as a search-engine has now diversified into many areas, so that it is not perceived any more as specialising in any single vertical. Meanwhile Google has beaten Yahoo! as the top search engine.

Yahoo! is remarkable in that it has survived the dotcom bubble and the meltdown of the Internet during early 2000. When many search engines and web startups such as Altavista, Excite, Lycos, Go.com, Snap etc fell down by the wayside, Yahoo! is still going strong with a 50 billion $ market cap.

Google has been the leader in taking new technologies to the market, with Yahoo! being forced to follow-up. The new Yahoo! search has borrowed a lot of concepts from the Google, not just the look n feel. Google has been the pioneer in broadening the search market from simple keyword/directory search to specific search verticals such as newsgroup search, image search, catalog search, product search, scholar search, news search, video search etc.

However Yahoo! has improved their search tremendously since they parted ways with Google last year. Though forced to do catch-up, Yahoo! seems to be doing a good job of it now a days, with Microsoft tagging along.

So who will win the search war ultimately? One yer back I would have confidently said "Google" (Yahoo! who?) but not anymore. The field is wide open once more.

Microsoft gets "Groovy"

2005-03-13T18:10:00.000+05:30

Microsoft recently acquired Groove Networks and Ray Ozzie in one shot. Ozzie will become one of the three CTOs at Microsoft, taking care of collaborative software, for the evil empire.

So what is the big deal? From Microsoft point of view, it is another one of the acquisitions which appears to give an "edge" to its products. Groove is basically a peer to peer computing system over a secure internet connection which allows people to share documents over a network, creating a kind of virtual office. Microsoft has been an early investor in Groove, providing it funding as early as 2001. Ozzie has also aligned himself nicely with Microsoft since his days as the chief architect of Lotus Notes at IBM. Thus the acquisition is not a big surprise at all.

Ozzie has been successful in aligning his software ideas with the biggies in the game, since 1984. He worked for Lotus in developing Lotus Notes and later on continued his work at IBM, when IBM acquired Lotus. He left Lotus and IBM to start Groove in 1997 and aligned his work with Microsoft. Bill Gates, who is not too lavish in encomiums, have praised Ozzie as one of the best programmers in the universe. However tall that might sound to be, one has to accept that Ozzie is a shrewd guy with his software, and knows to position himself correctly in the market.

Ozzie is surely one of the most influential programmers of the 90s and can be credited with inventing the groupware class of software products, along with pioneers such as Mitch Kapor. It will be interesting to see how his ideas and vision will work at Microsoft, and whether the addition of Ozzie as CTO, will improve the technical quality of Microsoft products in areas where they lack sorely such as security. Ozzie has had a lot of success as an independent programmer and team leader, but it remains to be seen how that experience helps in shaping the future of collaborative software at a company of the size of Microsoft.

Economics of the Bazaar

2005-03-12T00:26:00.000+05:30

If you have been associated to opensource software in an intimate way, say as a programmer, business developer or the marketing/sales person, you must have asked yourself this question at some point.

"How can opensource software be a viable business? How do you make money by creating software whose source code is free?"

Bruce Perens, one of the founders of the OSI (Open Source Initiative) has some interesint insights on this. According to him, open source reduces the costs for your business IT infrastructure so that you save money which can be used for creating software that differentiates your business from others. In other words, you can spend your money on researching and developing your actual technology instead of spending it on buying up software infrastructure. The money you save becomes additional profit or can be spent on other areas where it is required.

Well I never looked at open source from that point of view. I think this comes from the my programmer's mindset where the focus is immediately set on the free and available source code rather than thinking about the economics behind it.

Bruce's article contrasts the open source model with other software development models and points out the benefits of the open source economy. The complete article can be found here. A must read for any open source afficianado.

Daily Python Url!

2005-03-11T10:53:00.000+05:30

I am a regular reader of Daily Python Url which gives a snapshot of the daily happenings in the Python universe. It is powered by Blogger.

For the last week, the blog has been almost dead. I was getting the same old page, last posted on 3rd March. Today I came to know the reason :-)

2005-03-10

(no posts? some editors are on vacation, the rest of us are just lazy. and blogger sucks. postings will resume shortly. stay tuned.)

Well guys, shrug off your laziness and get a fresh start! I think that the Daily Python Url is too important for the Python community to be given such a cold shoulder!

IBM Redux

2005-03-07T12:49:00.000+05:30

I just finished reading the book "IBM Redux" by Doug Garr. The book is all about the business turn around by IBM in the last decade under the leadership of the previous CEO, Lou Gerstner.

The book is a great read on how corporations behave and gives many insights into how a gigantic corporation like IBM works. It is quite eloquent in its praise for Gerstner, though his not-so-amiable personality and ego-centric nature is pointed out as personal drawbacks. However, the book lays no doubt on who is responsible for the bounce back by IBM from the bottomless quagmire it found itself in the beginning of the 90s.

It also has a few interesting anecdotes such as IBM's role in the 1994 Atlanta Olympics and the PR disaster which ensued. A few pages are also devoted to the much hyped Garry Kasparov - Deep Blue face-off in 1997.

The book follows a rather chronological style, starting from the events that lead to Gerstner's heading IBM, and ending with the peak of Gerstner's rule at 1999, when he is sitting pretty at the top and is percieved widely as the saviour of IBM. The author follows the style of chipping in with a few anecdotes and history pieces here and there which sometimes is against the flow of reading. However, he cannot be charged on missing out a single piece of event which impacted the computer & software technology landscape during 1993-1999, where IBM played a part, either as the vanquisher or as the vanquished (Remember OS/2 Anyone?).

All in all a good read and a book recommended to anyone interested in the history of technology.

Some links worth reading about Lou Gerstner and his life.

http://news.com.com/2100-1001-828095.html
http://www.forbes.com/2002/11/11/cx_ld_1112gerstner.html
http://www.thecarlylegroup.com/eng/team/l5-team861.html
http://www.britannica.com/eb/article?tocId=9114982

The BangPypers

2005-03-05T15:39:00.000+05:30

First of all, the introduction -> The BangPypers are a group of Python enthusiasts in the city of Bangalore where I live.

How did BangPypers come into being? It all started on Dec 27 2004, when I was browsing the Daily Python Url blog and saw a post mentioning that "Chennai Python Meetup is on...". I followed the link and found out about the Meetup.com site. I was surprised to learn that there were already many Python meetup groups at the site http://python.meetup.com. Well I mused that if the Chennaites can come up with a Python meetup group, the Bangaloreans can come up with one too and do it in a better manner, considering that Bangalore is the tech hub of India.

I scourged the python meetup site for any Bangalore Python meetup groups. There was one, a dysfunctional group that called themselves "Bengalru Python meetup group" with 5 members and looking for an organizer. I initially joined them and posted a message.

The next day, I checked the site. There was no reply for my post and I realized that the group was quite dead. Bengaluru might be the original name of Bangalore in Kannada, but it is of course more popular and better known as "Bangalore" itself.

I created the "Bangalore Python meetup group" on Dec 28. The first few members to join were my friends who I invited - Anish Damodaran, Indrajith. I also sent an invitation to Premshree Pillai and posted in comp.lang.python about the news.

There was good response over the many days that followed with 16 members joining till Jan 7 2005. Swaroop published about the group in his blog on Jan 7th and the response increased, which shows the popularity of his blog. I have captured some interesting statistics on this in a post to the BangPypers group.

We had the first meeting at Ebony Restaurant, Barton Center, M.G Road on Jan 22nd. 10 guys including me attended. We decided to form the Yahoo! group and also a website. The idea for Project Uraga was also born. It was a fun night, though I had some personal expenditure which I expect the group to make up over time. :-)

I created the Y! group with the name "BangPypers" on 24 Jan 2005. We have a lot of activity in the group and it still must be one of the Yahoo! groups with the highest traffic. The group currently has 75 members most of who are in Bangalore. There are a sizeable number of non-Bangaloreans too. The first month had 101 messages, Feb had a whopping 364 messages! March has just started and we already have 25 messages over the last 4 days.

We had the first "technical meetup" of the group at ThoughtWorks on Feb 19 2005. You can read all about it here .

A website at http://www.bangpypers.org is also registered. Many thanks to Ramdas S of Developer IQ for doing the same. The website work has not started off however. We also have a Google group of the same name. All posts to Y! BangPypers gets mirrored to the Google BangPypers.

If you love Python and you are a Bangalorean, you should join BangPypers :-) . Well, you are invited to join even if you are a non-Bangalorean. BangPypers is the second biggest Python meetup group in the world, so we are on the world-map of Python, just like Bangalore is on the world-map for technology.

My Blog is up!

2005-03-05T11:47:00.000+05:30

After trying to set up a number of blogs on many different sites, I have finally found time
and patience to create a blog of my own... hip hip hooray!

The setup is not complete. You can find details about me and my work once I set it up properly.