Py3k letters - Part I, The PEP Talk

This post is all PEP-talk. What pep talk you may ask ? Well not the usual pep-talk but talk about PEPs. "PEP" stands for "Python Enhancement Proposals" and is the way in which Python language developers and enthusiasts go about suggesting, agreeing on and incorporating changes to the Python language.

For Python 3.0, the key PEP is PEP 3000. All Py3k PEPs are numbered starting from 3000 upwards. PEPs 3000-3099 are special, since they are PEPs about PEPs, so called "meta-PEPs". PEP 3000 is like the god of all meta-PEPs, since it is meta-PEP of all meta-PEPs. Think about it something like the father of all meta-PEPs, the originator, the source.

There are two more PEPs which are special. PEP 3099 and PEP 3100. The former is an anti-thesis of a PEP actually, since it refuses to budge - it is a list of things that has no plans of changing in Py3k. Interesting isn't it ?

The latter is the mirror image of the former. It is a laundry list of things that will change in Py3k, which are not big enough to have a PEP of their own. The fact that it follows PEP3099 is not its fault. It just so happens. In fact, this PEP was christened as PEP3000, but as other PEPs were born and began to strive for space and the PEP talk became more and more noisy, the BDFL and his gang of merry men re-christened this PEP all the way up by a century. I am not sure why this happened, but that is history and one cannot change it. However the PEP is not complaining and neither should you. Strange are often the ways of PEP-land...

Now, Py3k was not born as an idea in a single day (neither was Rome built in one), so you can imagine that there was a lot of pep-talk which led to its conception. And though PEP-land is often strange in its ways, it is not as strange as to defy logic completely - it is not. You would expect these PEPs to be older than the one which is bears the name 3000 in its birth certificate, and you would be right if you thought so.

These older PEPs, which are the big brothers of PEP 3000, influenced the growth of PEP 3000. From older to younger, they are,

1. PEP 238, which divided the kingdom of Pythonistas over the something as simple as the division operator. The contention was how to float and divide at the same time, in just one swoop. It turned out that that this was a hard problem to solve and finally after many a battle, the decision was made - float and divide in a single swoop and only divide in two swoops . The ramifications of this decision will be huge for many programs which are inhabitants of the Python land, especially belonging to the numerical and scientific family trees.

2. PEP 328, which made a lot of funny noise with statements that is any combination of the strings "from", "import", ".", "..", and "as" . All to solve the problem of the right syntax of relative imports - that is imports as in importing code, not like importing sugar or oil, which is a more easier problem to solve apparently. This problem is much more abstruse, and aims to bring down packages from folders they live in, relative to the current folder.

3. PEP 343, which is all about disciplining the brat fraternity of "try", "except" and "finally" statements, by giving them some meaning in life (a "context" actually) and holding them together with a "with" statement. Matters get complicated when a context is managed with a contextmanager, as usually happens when managers get involved in any engineering problem.

4. PEP 352, which is about installing an original ancestor for a group of dangerous characters, whose main job is to raise issues which are euphemistically called as "exceptions". Unfortunately, chaos has reigned at the top of this hierarchy with no clearly defined ancestor who these characters can claim lineage to. Pythonistas realized (quite late) that it is dangerous to let the status quo remain and the pep-talk resulted in this PEP which aims to fix the status for good by correcting their ancestor tree. Very well. I am glad we cannot do these things in real life.


So, all the talk about pep-talk has resulted in the above peppy PEPs. Pythonistas live and swear by PEPs, so if you want to learn the Py3k way, you should walk the way of the PEPs for some time and live with them, understanding why and how they exist.

The above PEPs form the soul and core of Py3k, so get used to them and we will discuss the actual issues they solve and how they do it, starting from Part 2 of the Py3k letters. Till then happy PEPing.

Py3k letters - Part 0

Python 3.0 (py3k) is on the way. The 3.0 version is going to be a significant update in the Python programming language, considering that it is almost 8 years since the release of the last major version, Python 2.0, which was released in Oct 2000.

Py3k is going to change the way the Python programmer goes about his work in many ways, with some significant "in-your-face" changes and many thousands of changes and fixes below the layers and quite some just in between.

I am starting a series of blog posts from today, which will focus on Python 3.0, specifically on the major changes in the language when compared to Python 2.x versions. The plan is to cover enough areas (hopefully), so that by the time py3k is released in August this year, there will be enough text here to serve as an aid to Python programmers in chartering py3k territory.

With that introduction, on to the "py3k letters" series, with the first edition coming in the next post. Keep tuned in.

Red letter day!

I will mark today as a red letter day in my life and career. HarvestMan is one among the winners of FOSS India Awards declared by EFY today.

After four years of painstaking work and countless hours spent on developing the program, I feel happy to be at the "receiving end". A lot of gratitude goes to the EIAO team of Mikael, Nils and Morten for giving me the motivation to work on the program and keep improving it.

Congratulations to all the award winners for their spirit of free software and open source. Keep up the good work!

This is a very good initiative on the part of NRC FOSS and EFY to encourage F/OSS development in India. I really appreciate the leadership shown here by both the institutions. Apart from putting FOSS and people who innovate in FOSS in India in the limelight, the initiative will also help to advertise to the world the fact that Indians are also innovating in this area.

Hopefully, efforts like this would encourage the adoption of F/OSS on a large scale among students in engineering and technology in Indian universities and provide a big push towards more original F/OSS contributions from India.

FOSS India Awards

I am making my first post of the new year 2008. This also happens to be a post after a gap of more than three months. Reasons - new job, lack of time etc.

I had submitted HarvestMan for FOSS India Awards more than a month back. However the FOSS India Awards team took their own time to provide the competing projects with a questionnaire regarding their project. I got the questionnaire in my email yesterday and today I completed it and sent it.

The awards are expected to be announced on Feb 15th, which also happens to be my birthday. Keeping my fingers X...

FOSS - Made in India !

While returning home from work today, I went to the neighboring super-market and bought the LFY (Linux For You) magazine for the month on an impulse. I have not been a regular reader of this magazine since Feb 2006 when I stopped writing for them. So it was just pure impulse on my part...

I was going through the articles lazily in bed some few minutes back. I started with the review of SLES desktop and then read the article on autopackage. The project is started by a group of young enthusiasts based in U.S and Sweden which makes package installation across Linux distros easy (say "InstallShield for Linux"). I then flipped casually to the next page where there was a guest column by our very own Kenneth Gonsalves, the Chennai based open source enthusiast and activist.

He was asked the typical question which every FOSS related magazine in India loves to repeat in every alternate editions - "Why are Indians not contributing to open source ?" I glanced through his reply. Little did I realize his comments would make my day...

Here I quote his reply verbatim...

"There are a large number of indians in practically all major FOSS projects. However if you look closely, you will find that the vast majority are not resident in India. Yes, there are several hundred people in India actively contributing to projects big and small. But genuine 'Made in India' projects of international repute ? I can see only four or five: Anjuta (Naba Kumar has left the country); HarvestMan by Anand Pillai; Deepofix by Abhas Abinav; IndLinux by Karunakar and team and Coppermine by Tarique Sani..."

Boy, was I not taken aback by surprise and blushing with genuine pride!... :) I have used Anjuta and has great respect for its original developer (Naba Kumar), though little did I know about his current resident country. I think IndLinux and Coppermine are great projects and frankly I have not heard about Deepofix (my bad...). I have always considered my own contribution (HarvestMan) as a david among the goliaths. I have a sense of my place among the international open source developers. I have made a contribution worth mentioning, but I have never considered it accomplished enough to figure among "the list of original contributions" by India to FOSS. It is surely a matter of pride to see that a well known and widely acknowledged FOSS community member thinks about the project like that.

Kenneth, you clearly made my day. Thanks for the kind words. It felt really nice to see the words "HarvestMan" staring back at me from a page in an IT magazine which I was casually flipping in the midnight. Surely for a born-again techie like me, there is no better recognition than something like this.

I consider this the best compliment I have ever received in my life for something I have done. It makes more all the more excited about open source and FOSS and the spirit of sharing code and having fun at the same time.

Narcissus.py updated

Since I completed the port of Rbnarcissus to Narcissus.py last week, I have been working on getting the bugs fixed. A lot of work has been done on this during the past week, and the code is now parsing a set of more than 30 javascript input samples of varying size and complexity, correctly. Also, the behaviour is very close to that of Rbnarcissus. Both the parsers now seem to fail at the same places in the code for samples which they can't parse, which is a good sign that the Narcissus.py code now approximates Rbnarcissus very well.

Since the code is now stable and somewhat usable, I have made it publicly available. The code can be browsed in the folder. There won't be any packages or formal documentation till I feel that the code is beta quality and can be made available as a Python package.

Narcissus.py

Today I finished porting of Rbnarcissus to Python.

I managed to finish the porting in a total of 7 days, spending approximately
2-3 hours per day. The test parse script has also been ported. With this
I managed to produce the JS parse tree for the following simple Javascript
function.

function test()
{
   var a = 10
   var b = 20
   var c = a + b
}

As a result of parsing, the following function dictionary was printed.

{test: []}

This library will be made available as part of HarvestMan and the EIAO projects.
This is perhaps going to be the first open source pure Python parser for Javascript.

I need to do a bit more of testing on more complex Javascript code before I make the code publicly available. This might take another week or so, depending upon how much time I get to spend on this in the coming days...

Rbnarcissus Porting - Day 2

In the second marathon day of Rbnarcissus porting I completed porting of another 5 functions in the Parser.rb module. What is pending are two huge functions which parses statements and expressions. That should be taken care by another day or two of hectic hacking...