PunBB Textpattern based web site hacked and fixed

A few days ago I noticed that search results on Google for sahi, started showing strange content which was not visible on the webpage or in the view-source.

A little hunting around showed that some of the common included php files on the website had been modified, such that when the page was accessed using google-bot's user-agent values, the page added the extra spam content. This made Google index the website with the spam title instead of the real one. Here is a great detailed post on this hack on Google's Webmaster Central website.

The hack had modified the PHP files by adding a single line which looks like this:

error_reporting(0);eval(base64_decode('JGxMOXdG..long long string..'));

When the string is decoded, it shows HTML content with all the spam links.

The modified time of the file was the same as other files, so who ever did this made sure the modified time was set to the original one. I removed the extra content and placed a request to delete the page from Google's cache (For details on removal, look at http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=164734&rd=1 under "The page has changed and I want the outdated information removed")

Everything worked fine. Till a few days ...

Google results again started showing weird results, and another hunt showed similar strings added. Removed the content, was fine for a few days and again ... This was becoming painful and stressful. I had now got into this habit of logging into the shell and running

grep -RI base64_decode . | more

to see if there were any long entries, and then replacing that file with the untainted one from my backup. But the root cause of the problem was still unknown.

And then there was one particular file which on my machine was called groups.php
This had another eval(gzinflate(base64_decode('...'))) and when run, it showed a page called "r57shell 1.40". This page gave full access to whatever PHP exposed under the web users permissions, which meant one could access the filesystem, database, mail, ftp and what not. This was brilliant and I would have admired it more had it not been so harmful to me. I guess this was used to periodically modify various php files on my website to inject spam content. This may have been added during the brief period I was using ftp instead of sftp. I am not sure yet though. A couple of other similar harmful files also surfaced, (called white.php and added to punBB's lang/eng/ directory) which had some other back doors opened.

Removed these files and started searching for a solution which would track such problems and alert me, when I came across an "Anti Virus" link on a2hosting's CPanel. Added back the problem files, ran the AV (powered by ClamAV) and sure enough, those files were detected as Trojans. I wish I had known about this a couple of months back.

To prevent further such attacks,
all file permissions were changed to 644 and directory permissions to 755
all ftp users were removed and only sftp will be used henceforth

Also created a small php file which would show files with base64_decode in it

<?php
$output = shell_exec('grep -RI base64_decode /my/homedir/www');
echo "<pre>$output</pre>";
?>

I hope this post helps others who face similar problems.

Windows 7 fix for slow copy speeds from USB

I was recently copying over about 1.5 GB of data from my USB stick to Windows 7 (32bit) and was surprised to see that it took over 25 minutes to do the copy. I researched a little bit and this is what helped me fix it.

1) Run these commands to turn off TCP autotuning. (http://www.speedguide.net/articles/windows-7-vista-2008-tweaks-2574)

netsh int tcp set heuristics disabled
netsh int tcp set global autotuninglevel=disabled

Effect: good increase in speed

2) Turn off Quick Heal antivirus during copy
Effect: good increase in speed

3) In Device Manager, right-click the USB drive in Disk drives folder, then select Properties, switch to Policies tab, and choose "Better Performance". Click OK to keep it. (http://social.technet.microsoft.com/Forums/en-US/itprovistahardware/thread/60a7b507-9bda-4e66-8693-b85498cd860c/)
Effect: good increase in speed

Overall the copy time came down to about about 4 minutes (From stable speed of 500KB/s prior to changes, to 5.4 MB/s)

Note for the smart asses: before one starts inferring more than is necessary, let me clarify that Win7 and Quick Heal are both great products and serve my purpose very well.

GTAC 2010

After having been denied participation initially, I did get to attend the second day of GTAC thanks to Simon Stewart helping me get in. I met a lot of interesting people, and spoke at length to the WebDriver folks, comparing notes, discussing road-map and about possible use of WebDriver underneath Sahi. It is going to take us a little while to do an honest appraisal and see whether that is the right direction to take.

GTAC was good over all, in the quality of crowd. The talks were interesting. James Whittaker definitely lost my vote when he said something like "Honestly I was not thrilled when I heard GTAC was in India. I did not expect much, <pause>, and I must say, I am not disappointed". Seriously man, next time think before you say something like that, or be dishonest. Btw, some morons clapped at this.

There was one other instance where I thought I need to add my comments. During a discussion, where I mentioned that going back to the web developer to add ids increases turn-around time, Simon jumped to say that it is wrong to have teams separate, they should be sitting next to each other and should get it fixed immediately. This may be the ideal situation (I strongly question that too), but it is definitely not what is practised or feasible in the industry. While interactions are good, minimizing the testers dependence on the developer is required. They may wish to discuss business logic, but really why should they know the internal ids that need to be added to the view. It is just a waste of time. Adding ids is also not always possible when using off the shelf javascript UI frameworks. Why burden everyone with ids when tools like Sahi can do without the ids? I wish the discussion had stuck to testability and not gone off to project management.

Those aside, I think GTAC was good. Thanks GTAC for letting me be there at least for the second day!

Syntax Highlighting for Sahi Code

I have added syntax highlighting to my blogs now. Used Alex Gorbatchev's SyntaxHighlighter 2.0 which is awesome. I added a Sahi brush to SyntaxHighlighter so that keywords in Sahi would also be highlighted.

This is what shBrushSahi.js looks like:

SyntaxHighlighter.brushes.SahiScript = function()
{
 var keywords = 'abstract boolean break byte case catch char class const continue debugger ' +
     'default delete do double else enum export extends false final finally float ' +
     'for function goto if implements import in instanceof int interface long native ' +
     'new null package private protected public return short static super switch ' +
     'synchronized this throw throws transient true try typeof var void volatile while with';

 var schedulerFns = '_alert _assertEqual _assertNotEqual _assertNotNull _assertNull _assertTrue _assert _assertNotTrue _assertFalse _assertExists _assertNotExists _callServer _click _clickLinkByAccessor _dragDrop _resetSavedRandom _setSelected _setValue _simulateEvent _call _eval _setGlobal _wait _popup _highlight _log _navigateTo _callServer _doubleClick _rightClick _addMock _removeMock _expectConfirm _setFile _expectPrompt _debug _debugToErr _debugToFile _mouseOver _keyPress _focus _keyDown _keyUp _mockImage _execute _assertContainsText _enableKeepAlive _disableKeepAlive _dragDropXY _deleteCookie _createCookie _clearPrintCalled _saveDownloadedAs _clearLastDownloadedFileName _rteWrite';
 
 var browserFns = '_accessor _button _check _checkbox _image _imageSubmitButton _link _password _radio _select _submit _textarea _textbox _event _getGlobal _random _savedRandom _cell _table _containsText _containsHTML _byId _row _getText _getCellText _div _span _spandiv _option _lastConfirm _reset _file _lastPrompt _lastAlert _get _style _byText _cookie _position _print _printCalled _label _lastDownloadedFileName _rteHTML _rteText _re _prompt _getCellText _getSelectedText _scriptName _isVisible _listItem _parentNode _parentCell _parentRow _parentTable _in';
 
 var otherFns = '_getDB _readFile _logException _logExceptionAsFailure _stopOnError _continueOnError _include';
 
 schedulerFns += (' ' + otherFns);


 this.regexList = [
  { regex: SyntaxHighlighter.regexLib.singleLineCComments, css: 'comments' },   // one line comments
  { regex: SyntaxHighlighter.regexLib.multiLineCComments,  css: 'comments' },   // multiline comments
  { regex: SyntaxHighlighter.regexLib.doubleQuotedString,  css: 'string' },   // double quoted strings
  { regex: SyntaxHighlighter.regexLib.singleQuotedString,  css: 'string' },   // single quoted strings
  { regex: /\s*#.*/gm,          css: 'preprocessor' },  // preprocessor tags like #region and #endregion
  { regex: new RegExp(this.getKeywords(schedulerFns), 'gm'), css: 'color4' },    // operators and such
  { regex: new RegExp(this.getKeywords(browserFns), 'gm'), css: 'color5' },    // operators and such
  { regex: new RegExp(this.getKeywords(keywords), 'gm'),  css: 'keyword' }   // keywords
  ];
 
 this.forHtmlScript(SyntaxHighlighter.regexLib.scriptScriptTags);
};

SyntaxHighlighter.brushes.SahiScript.prototype = new SyntaxHighlighter.Highlighter();
SyntaxHighlighter.brushes.SahiScript.aliases = ['sahi', 'sahiscript'];

And this is what I added to my blogger template:

<script src='http://sahi.co.in/static/syntaxhighlighter/scripts/shCore.js' type='text/javascript'/>
<script src='http://sahi.co.in/static/syntaxhighlighter/scripts/shBrushJScript.js' type='text/javascript'/>
<script src='http://sahi.co.in/static/syntaxhighlighter/scripts/shBrushSahi.js' type='text/javascript'/>
<link href='http://sahi.co.in/static/syntaxhighlighter/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://sahi.co.in/static/syntaxhighlighter/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>
<style>
.syntaxhighlighter .color4, .syntaxhighlighter .color4 a{color: #000A7F !important;}
.syntaxhighlighter .color5,.syntaxhighlighter .color5 a{color: brown !important;}
</style>
<script type='text/javascript'>
    SyntaxHighlighter.config.clipboardSwf = 'http://sahi.co.in/static/syntaxhighlighter/scripts/clipboard.swf';
    SyntaxHighlighter.config.bloggerMode = true;
    SyntaxHighlighter.all();
</script>

Note that, in blogger, <br> tags will be visible in the code in place of new lines if you do not add

SyntaxHighlighter.config.bloggerMode = true;

So now Sahi scripts on the blog will look like this:

function search(){
    _setValue(_textbox("q"), "sahi");
    _click(_submit("Google Search"));
    _assertExists(_link("Sahi"));
}
function goToForums(){
    _click(_link("Sahi"));
    _click(_link("Forums"));
}
function logout(){
    if (_condition(_link("Logout"))){
        _click(_link("Logout"));
    }
}
function login($username, $password){
    _click(_link("Login"));
    _setValue(_textbox("req_username"), $username);
    _setValue(_password("req_password"), $password);
    _click(_submit("Login"));
    _assertExists(_listItem("Logged in as "+$username));
}

search();
goToForums();
logout();
login("sahitest" "khuljasimsim");

Chrome rocks!

Chrome rocks absolutely!

It is blazingly fast, and feels ultra light weight. After using it for a few weeks now, I am totally addicted.

The landing page which shows snapshots of all pages recently viewed, is very convenient to start off and I feel great to just be able to click on a thumbnail and get to the site with out examining the url on my url bar or clicking on a bookmark. In fact the landing page is something that has really made a lot of difference in my browsing experience. I don't get lost as often as I used to and it saves me a lot of time and effort.

I love the incognito mode which allows me to login simultaneously as different users, even into web applications which use permanent cookies. This is great for developers testing web applications.

I like the ability to drag a tab and create its own window and vice versa.

Space usage on chrome is very efficient and the concept of getting rid of the windows title bar is a very good one. The concept of showing page popups as part of the same tab but hidden down below is also quite useful.

Honestly speaking I don't care much about being able to type the search terms or url into a single box. Surprising that Google talks about that more than the other features.

My default browser was Firefox, but now I am converted.

Sahi V2 20080831 Released

After close to a month of development, Sahi V2 20080831 has been released today. (http://sahi.co.in/w/)

This release uses Rhino as the scripting engine, thus moving most of the script execution to the proxy. This should go a long way in simplifying Sahi scripts. Scripts now execute on the proxy, and only stuff that needs to execute on the browser is sent to the browser. Thus scheduler functions are sent to the browser for execution. One change which has come in is that custom functions which may have been added for identification of browser elements, now need to be wrapped in a <browser></browser> tag so that they are also sent to the browser.

This build also has some important changes to the SocketPool which will fix issues related to too many sockets being used and errors due to BindExceptions. Suite execution has been changed such that even if the browser crashes, the suite will continue with the next script and thus not hold up a build. DB methods now close connections properly.

There will still be a few rough edges and I hope users will report bugs so that they are easily fixed. Meanwhile, help spread the message through your blogs or email forums.

Sahi V2 20080831 Released

After close to a month of development, Sahi V2 20080831 has been released today. (http://sahi.co.in/w/)

This release uses Rhino as the scripting engine, thus moving most of the script execution to the proxy. This should go a long way in simplifying Sahi scripts. Scripts now execute on the proxy, and only stuff that needs to execute on the browser are sent to the browser. Thus scheduler functions are sent to the browser for execution. One change which has come in is that custom functions which may have been added for identification of browser elements, now need to be wrapped in a tag so that they are also sent to the browser.

This build also has some important changes to the SocketPool which will fix issues related to too many sockets being used and errors due to BindExceptions. Suite execution has been changed such that even if the browser crashes, the suite will continue with the next script and thus not hold up a build. DB methods now close connections properly.

There will still be a few rough edges and I hope users will report bugs so that they are easily fixed. Meanwhile, help spread the message through your blogs or email forums.

Sahi - Latest developments

Copy of post on forum: http://sahi.co.in/forums/viewtopic.php?id=261

I had not been active on the Sahi forums (http://sahi.co.in/forums) for sometime. Thanks a ton to StringyLow, tinchie8, lepierrot, pankaj.nith and others for keeping this forum active, and replying to posts.

Meanwhile I have been working on a version of Sahi which attacks one of the basic problems with Sahi.
Scopes of variables, scheduler and normal functions and the the way steps are queued and executed and the way their integrity needs to be maintained across page loads.

The problem had been that the scripts (after parsing) were executed on the browser itself, and when a page unloads, the state of the script's execution needed to be persisted on the proxy and then resurrected when the next page loaded. While this allowed the ease of using javascript for scripting, when scripts became bigger, the browsers and the proxy had to do a lot more persisting and resurrecting.

As the logical next step, I wanted to move this script execution to the proxy. I now use Rhino, an excellent javascript engine, to execute the scripts on the proxy. Only steps that need to be executed on the browser are sent to the browser. Javascript can still be used for Sahi scripting. Even though the script still is parsed, it is much simpler to understand script execution than it was before. Rhino also comes with a debugger which can prove useful to Sahi script debugging.

The version is slated to be released soon. I am looking for volunteers to test and give me feedback about the new version. If you have existing scripts, the scripts may need to be modified a little to make it work with the new version.
If you are interested, please post back or email me at narayan at sahi.co.in.