Unicode encode decode error in python email

Been bothered with these on and off errors in our application while sending email via python:
File "/usr/lib/python2.4/email/Generator.py", line 182, in handletext self._fp.write(payload)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2014' in position 4986: ordinal not in range(128)

The reason is that Python's email module needs some information about charset used when a MIMEText object is created.

Here is a link which explians how to solve this:
http://mg.pov.lt/blog/unicode-emails-in-python


Links to understand unicode in Python:

http://effbot.org/zone/unicode-objects.htm
http://www.amk.ca/python/howto/unicode

Enduro3

So we competed in and completed the Enduro3 race. Sujoy, Sunil and I cycled some 45 kms over mud tracks and hilly roads, trekked another 50, and eventually survived the adventure race unhurt, and still friends. We started around 10:30 am on Friday and finished around 6:30 pm on Saturday, having slept some 4 hrs in between.

The race:
Cycling:
From Paud Road to NDA.
On some muddy terrain inside NDA. Was good fun.
Back on the road, up and down, up and down and up up - ending at Panshet Water Sports Center.
(One of us and his cycle have since been divorced.)

Trekking:
Start from Panshet, walk along the ridge, get totally lost for a couple of hours, reach one end of the lake. (9:30pm)
Stop for the night.
Do river crossing. Only one in the team had to and Sujoy, you rocked then!
Sleep till 2 am. Trussed up in a dryfit t-shirt, the poncho, a full cotton t-shirt, a nice jacket, a monkey cap and inside a sleeping bag. felt good.
Wake and start trekking around 3 am.
Sleep again from 5 to 6 am because we got lost.
Wake (6 am) and trek trek around the lake, on the road, up koshimgarh and trek trek along the ridge till we reached Panshet WSC. (6.30 pm)

The lake and mountains were beautiful. The morning trek was quite refreshing and we saw a lot of birds. The sudden burst of wings from a group of quails from under my feet took me totally by surprise. So close were they, our bodies could feel the drumming vibrations of the frantic wing beats. Also saw a hovering black shouldered kite, a grey jungle fowl, a large green barbet(?), loads of red whiskered/vented bulbuls, purple sunbirds, some flowerpeckers, puff throated, jungle babblers, magpie robins, and rufous backed shrikes.

The race was tiring, and we lost our way more than once. But it was a lot of fun and it definitely tested our endurance and it feels good to have completed the race and be writing this blog.


Information for the next time (IT category):
Water was available at a lot of places and was not a problem.
Even though the race director laughs at you in the briefing session when you ask him about carrying sleeping bags, know that they are absolutely needed.
Do NOT expect flags where you need them.
Finish cycling as fast as possible so that you can navigate properly in daylight on the trek. On a moonless night like we had, it was impossible to look at the lake's shape and find our bearings from the map.
The river crossing is in a deep part of the lake and you have to swim (and not wade) across. You get a life-vest so you won't drown and there are ropes at water level to pull yourself along. The water is placid and does not have a current. The distance is around 100m I think.
The river crossing and rifle shooting timings are only useful when you have a tie. So really don't bother much with these.
The rifle shooting was dropped for our event because of time constraints(?).
Glucose is the best. Food may be taken but you would be better off with chocolate bars and energy bars.
Carry a couple of 1L water bottles per person.
Practise with your own cycle, and from well in advance. Don't fatigue your muscles just a day or two before the race.

Javascript sleep() or wait() in Sahi

My previous post on 'Javascript sleep() or wait()' is visited a lot of times and I think I should write how the problem was eventually solved in Sahi.

Sahi needs to playback scripts written in javascript on the browser.
Let us consider a small example:

We are trying to automate an AJAXy mail client application which has an Inbox button, which when clicked, loads a part of the page, and this newly loaded content has a link 'Unread'.

_click(_button('Inbox'));
_click(_link('Unread'));

When the first line is executed, the Inbox button gets clicked and the script has to wait till the relevant portions of the page load, before it can click 'Unread'.

To solve this, Sahi has a script parser which modifies this script to look something like this:

steps = new Array();
steps[steps.length] = "_click(_button('Inbox'));";
steps[steps.length] = "_click(_link('Unread'));";

And to execute these steps, it does:

var currentStep = 0;
function execute(){
if (currentStep == steps.length) return;
window.eval(steps[currentStep]);
currentStep++;
window.setTimeout('execute()', 1000);
}

execute();

Hope this helps people who are looking for a solution for javascript sleep.

java -cp does not work?

Was trying to get Sahi's batch and shell scripts to work properly since I get a lot of queries regarding database drivers not being found instpite of adding to the classpath.

The problem is this. I need to add a mysql driver to Sahi's classpath so that the scripts can access the driver to do some data driven testing.

So I had to add the mysql driver to this:

java -jar ../lib/sahi.jar

Trivially, I added a -cp option so the command looks like this:

java -cp ../extlib/mysql-connector-java-5.0.4-bin.jar -jar ../lib/sahi.jar

But this does NOT work. Why? Because an executable jar should have its classpath in its manifest file and not outside.

The way to solve this is to use:

java -cp ../extlib/mysql-connector-java-5.0.4-bin.jar;../lib/sahi.jar net.sf.sahi.Proxy

So I made these changes on my windows machine, and it worked. Then logged on to my newly installed ubuntu to check if the shell script works with the same changes.

Made the necessary semi-colon to colon conversion in the classpath, so it looks like this.

java -cp ../extlib/mysql-connector-java-5.0.4-bin.jar:../lib/sahi.jar net.sf.sahi.Proxy

The sahi.sh file has a few other lines too.

When I ran sahi.sh, it gave me a NoClassDefFound error for net/sf/sahi/Proxy! After hunting around quite a bit on the net to see if there was such a problem on linux, I realized that nobody on the forums seems to understand the problem even when somebody reported it.

Eventually I realized that since I had edited the file in windows, it was adding ^M characters at the newlines which was not being recognized by the shell. So I used another editor and got rid of them by deleting and reentering the newlines. (I could also have used dos2unix, but did not have it installed on my machine)

I also realized why the people in the forums did not understand the problem. Most of the replies were from people using linux all the time and the question posters would have mostly been using windows but since the problem was on a linux machine would have posted the question on linux forums!

Anyway, now things work properly :)

Sahi at FOSS.in

I will be presenting on the internal workings of Sahi at FOSS.in at Bangalore. This is not a demo level session but will involve a deep dive into the internals of the proxy, the script, extending the proxy, adding new APIs and other unmentionables. Hope to meet some interested and interesting people there.

Cycling trip to god knows where

The cycle had been well over hauled. Had to replace the fork, both tyres and tubes. But the result was good. A very smooth, no hassles ride. The gear shifts were working well and the rolling was good. Except for the loose pedal shaft which had worn out and kept shaking and threatened to come off anytime. Saturday morning I set out from home at around 9, got the pedal shaft replaced and was on my way by 9.25. Was to meet Shrik near my office which was a good 17 kms away. Reached the office by 10.10. 45 minutes was a decent time and made me feel that I could go the office on cycle at least once a week. Had a vada pav and lime juice.

Shrik's Hero Octane looked good, and we started at 10.30 towards Hinjewadi. Past Hinjewadi, towards Paud, and we were in the villages and we kept going uphill and then eventually had a long long descent. Rested at the end of it. Had a limca and decided where to go next. Turned right on some local advice and went on a really bumpy bad road for quite a distance. The scenery around was good though it was all villages. Suddenly we were pleasantly surprised to see three woolly necked storks on the side on an inundated field. Took a few snaps and moved on in search of some hotel to eat.



We saw a lot of sugar cane laden tractors and got a sugarcane from one of them. Had some of it and washed ourselves in a nallah. Eventually found a hotel, but we could not get any meals there. Just had vada sambar, pav and a tea. The hotel guy suggested we go a little ahead and go to a Satya Sai temple nearby if we wanted to have a nap. So off we went to the temple, but it turned out to be one steep climb. Half way through we came across a short cement platform and we parked right there and went off to doze in the shade. Half an hour later, a little refreshed, we proceeded and came across Tikona on the right and a lovely lake on the left. It was beautiful. Ahead, after a friendly water break with some locals, we hit a steep long ghat road which was quite a challenge to go up. At the top, rested for a while with tea and a cream roll. Moved on from there down a lovely steep winding road, and then climbed again all the way to the Kamshet Dam and then downhill again to Pavana nagar village. At a fork where Kamshet was 10 kms to the left, we decided to go right because we were told that the highway was closer in that direction. Was that a mistake! We cycled for a long long distance with no sign of a highway. Twilight became night and we were either guided by the less than half moon, or blinded by the oncoming vehicles. Had a very needed halt where we hogged five bananas each and to wash off the sweet taste I had a tomato too! Tried to hitch a ride back with a few vehicles but no luck. So we kept going and eventually, we went over the Bombay Pune expressway, and down further to join the NH4. We went to the nearest dhaba, which was quite comfortable with beds to rest, had a beer and some not so great food, and chilled out for a while.

Around 8.30 we started cycling towards the nearest railway station, Begadewadi, which was hardly a km away from the dhaba thankfully. Got on the Lonavla-Pune local at 9.20 and after getting a few excited and curious glances and comments about our cycles, Shrik got down at Khadki and I at Pune. Pedalled back from the station to home and reached around 10.30, by when Shrik had also reached home. Had a bath and crashed.

Sahi no longer needs waiting for AJAX calls

A new version of Sahi has been released. Download from https://sourceforge.net/project/showfiles.php?group_id=151639

More details on the release are here: http://sahitest.blogspot.com/