Arrival Haikou, Got Payment!

First, I have arrival Haikou tonight, and then have eaten some delicious. :)

Secondly, the GSoC payment has sent to my store value card right now! Hopefully every GSoCers have got it. :)

Back!

Well, finally I have came back to the earth!

After a super busy month, many things get done - the exams, TOEFL, GSoC midterm evaluation, rent room in Singapore, ... Another good news is my GSoC project still on track! Though, well, I have a little trouble of implementing the Python buffer support - I havn't a clear plan about what I am going to do...

Tonight I will go to Haikou, and then contiune my summer in this hot beatiful city. :)

Got my card yet!

After waiting and waiting, finally I got my card from Google! Yoohoo!

Thank you Google for the Childern's Day gift! :p

BTW, an ad: help Firefox 3 to Set a Guinness World Record!

Hello, Planet Python/SoC!

I'm so glad to see that my blog has been added to Planet Python/SoC! Thanks Titus Brown for doing so.

Just saying a hello to everybody, and happy coding! :)

Debugging Python Extension Module

There's some useful technique for debugging Python extension module, gathered during these days work.

• Clewn is a way to combine VIM and GDB.
  
• Set break on PyErr_Format or PyErr_SetString so you can trace a Python exception raised from the extension module
• Use 'make EXTEA_CFLAGS="-ggdb3 -O0"' to compile Python so you can have debugging symbol in the compiled executable. (Thanks my mentor Richard told me that.)
  
• There is a Misc/gdbinit in Python's code base. It defined many useful macro for GDB. Put the file in your working directory and rename it to .gdbinit to use it. The most useful macro is "pyo", which prints many useful information of a PyObject.
• Valgrind is also a good debugging tool. (Btw, it has won the Trolltech Open Source Development Award recently.) There's a Misc/valgrind-python.supp for it.
• [Python-Dev] disappearing exceptions explained some odd behavior I've ever encountered. Seems even the print() statement also swallow exception. (At least in Python 3.0)

Anyway, happy hacking!

Submitted my first issue to python.org, and some good news

You can see it here: http://bugs.python.org/issue2899

It is about a problem in SWIG's test cases. It used string.find(s, 'abc') to test whether 'abc' is contained in s. However, I don't know what it has been written in this way. Maybe for compatibility with older Python like 2.2, 2.3?

And a good news, after these day's working, nearly all test cases passed! Currently only 2 cases related to PyFile and 2 cases related to exception are still failed. I'll continue to work on them.

Another good news is I have received my book from Google yesterday. A really beautiful book! :)

Vim tips for SWIG

I've ever annoyed with the mixed tab and spaces in SWIG's code. Just talked it with William Fulton in #swig-gsoc. He suggested some tips.

This is a way to specific coding style by project:

au BufRead */[SWIG|swig]*/* set tabstop=8 | set shiftwidth=2 | set softtabstop=2 | set expandtab " SWIG

And a Syntax file for SWIG, add this in ~/.vimrc or ~/.vim/filetype.vim to use it:

" SWIG filetype file
if exists("did_load_filetypes")
   finish
endif
augroup filetypedetect
   au! BufRead,BufNewFile *.swg  setfiletype swig
   au! BufRead,BufNewFile *.i    setfiletype swig
augroup END

Finally, a big thanks to William! :)

Yeah, sigfaults disappered!

Finally I have solved most of the segmentation faults in yesterday's midnight. Just two lines of code!

Now I understood that PyObject_New() only allocate memory, but haven't do the works like creating the object's __dict__ and etc. We should use PyBaseObject_Type.tp_new() to create a "full" object. Or maybe PyType_GenericNew() also works?

Another mistake I've made is using obj->tp_dict() directly to access the object's dict. *Must* use _PyObject_GetDictPtr() instead! Otherwise, we only get the dict of the object's type!

Well done, now more than 90% test cases passed! Cheers!

Looked into the failed testcases

I've just looked into the failed testcases these days. Reasons of the failures can be fallen into below several categories:

• PyInstance_NewRaw: should find a replacement
• A lot of things related to the Unicode string changes
• IO module changed (see PEP 3116), some PyFile_* dissappeared
  
• How to define swig::LANGUAGE_OBJ?
• Some old testcases code should be rewritten (eg. they should raises an exception object instead of a string)
• Exception implementation: in Python 3 all excetion classes must inherit from BaseException, so we can't directly use a C/C++ struct as an exception now.

This would be a todo-list for me in the next weeks.

The test suite and disable 2to3 fixers

I'm writing SWIG makefile rules for testing today. Now I can see the test progress running well when I type "make check-python3-test-suite". It wonders me that so many tests can be passed! However, the failed tests should be the hardest part.

I have spend some time to work on Python 3's relative import, which mentioned on PEP 328, and PEP 366 seems a solution to my problem. But there is no 2to3 fixer to apply the PEP 366, so I had to disable the "import" fixer. The 2to3 has no option to disable a fixer now, so, this is how I did that:

$ 2to3 `2to3 -l | grep -v -E "Available|import$" | awk '{print "-f "$0}'` foo.py

This is the first time I tried to use awk, and the first time to write such a complicated command line. However, there should be a more clever way to do it.

The test suite still running, very slow, maybe due to that awful command line?

My own SVN branch! (and "svn merge")

Thanks to William, I have set up my own SVN branch now! Previous modifications also committed when starting the branch.

But unluckily since I did "svn cp" on my local working copy, but forgot "svn up" before it. So I have branched an older revision (r10394), and SWIG trunk's head revision is r10404 now!

So I tried to merge changes from trunk. I did:

$ svn merge -r 10394:10404 ../../trunk

But there are many annoying "Skipped Examples/python/shadow/some_file " lines, mostly occurred when "Examples/python/shadow" was deleted. After google, seems it means there are non version controlled files in these directory, so svn decide to keep them.

A dirty solution is:

$ rm -rf gsoc2008-bhy
$ svn co https://swig.svn.sourceforge.net/svnroot/swig/branches/gsoc2008-bhy

Then redo the merge, everything works well.

Google's fax machine ran out of paper

Today I have tried to fax my GSoC related documents to Google. I dialed the correct number, the line was rang again and again, but no response. After tried many times, finally I given up. Then I saw some words posted by LH in GSoC-student mail list:

Fax machine might be out of paper. We'll check it tomorrow. (Yes, in about 15 hours.)

Cheers,
LH

It's noon for me when I trying to fax, but it should be midnight in Mountain View, I think. So I should wait for them to wake up. :(

Starting the work

Today I have started to work on my SWIG's Python 3.0 support project. Since some work already done when preparing the proposal, I'd like to write a summarization here.

The works have been done:

• Getting familiar with SWIG's code base.
• Getting familiar with GNU automake, which SWIG used as build system.
• Modified the typemap, .swg, and etc code, so my SWIG can generate "Hello, world" level Python 3 wrapper now.
• A detailed time table has been written.
• Added checking for Python 3 configuration in configure.in (done today)

And the problems encountered right now:

• How to get rid of unbound method removal in Python 3? A mail posted to python-3000 mail list, waiting for replay.
  
• How to write makefile script so when we type "make check-python3-test-suite", it can run 2to3 automatically to convert *_runme.py codes to Python 3, then do the test.
• And I think I need a svn branch now.

Ok, too late and I'm feeling sleepy. I'll off to bed now. Good dream.