Tuesday, March 21, 2006

Sahi vs Selenium vs Watir

A brief comparison. Please let me know if any information is incorrect.
Added Watir related data from Chris McMohan's comments:

FeatureWatirSeleniumSahi
Browser basedyesyesyes
Browser type independent (IE, Firefox etc)IE & FireFoxyesyes
Scriptable/programmable (manageable/refactorable)yesnot natively, but using rubyyes (uses javascript)
RecorderyesOnly on Firefox. yes
Object spynonoyes (hovering on any element,shows its accessor)
Simultaneuos playback of multiple scripts(reduces playback time)yesnoyes
Multiple domain support (go from google.com to thoughtworks.com)yesnoyes
Frames and popup supportyeshas problemsyes
Ant support (helps continuous integration)yesyesyes
Multi language (non-ascii character)supportyesyes
HTTPS supportyes(Protocol independent)yes
Learning curveLearn RubyRecorder reduces the need to write code.TextPad Clip Libraries can be used too.
Language of scriptingRuby - not native to the browserRuby - not native to the browserJavascript - native to the browser andhence easily extendable.
Web-app independentyesno - needs to be deployed with appyes
OS Independentnoyesyes


Comments from Chris McMohan which I have incorporated into this comparison:
Actually, Watir has
a Firefox interface (version 1.0 just released, called "FireWatir")
a Recorder
methods intended for use at the command line like "show_links" so an object spy isn't necessary
Simultaneous playback (via Ruby threads)
Frames support (really excellent frames support)
Popup support native in version 1.5 (coming soon)
Ant support (not sure what you mean, but Ruby can be integrated with CruiseControl)
non-ascii character support (via Ruby)
Learning curve: Watir *is* a DSL for scripting the IE DOM. Ruby is just a bonus. :)

13 comments:

Chris McMahon said...

Actually, Watir has

a Firefox interface (version 1.0 just released, called "FireWatir")

a Recorder

methods intended for use at the command line like "show_links" so an object spy isn't necessary

Simultaneous playback (via Ruby threads)

Frames support (really excellent frames support)

Popup support native in version 1.5 (coming soon)

Ant support (not sure what you mean, but Ruby can be integrated with CruiseControl)

non-ascii character support (via Ruby)

Learning curve: Watir *is* a DSL for scripting the IE DOM. Ruby is just a bonus. :)

Avishek Sen Gupta said...

I stumbled upon IeUnit a few days back. It's only for IE, but has a lot of features that you describe.

Mind posting a comparison on that? :-)

Anonymous said...

Apropos the HTTPS support column, WATiR drives a browser (IE or FireFox) so the protocols it supports are basically the ones that the underlying browser supports. Both IE and FireFox work perfectly fine with HTTPS and by implication so does WATiR. To ask whether WATiR supports HTTPS is not very meaningful.

Anonymous said...

Brilliant! Keep up the good work.

Alex said...

There is also SWExplorerAutomation (SWEA)(www.webunittesting.com ).
SWEA supports frames (cross domain), windows and html dialogs (IE), popup windows, file downloads, page to image trasformations, visual data extractions using XPATH, DHTML pages. The SWEA Recorder allows visually record test scripts and generate NUnit code. The SWEA solutions (c# or VB.NET) can be run from ASP.NET or windows service (CruiseControl.NET).

Last Obelus said...

Actually, https support IS an issue in Watir.

If, for example, you have a site with frames that loads an https url in one of the frames, Watir will not be able to access that frame.

This is because the browser thinks of the https url as a different domain (even if the actual domain is the same), and denies access to protect against "cross-browser scripting"

Luke Closs said...

It seems most of your Selenium information is out of date.

There has been lots of recent activity on the Selenium project, adding in frame and https support. In addition, Selenium is scriptable in Perl, Python, Java, .net and ruby.

openqa.org/selenium is the place with all the details.

Anonymous said...

Does Watir support a .Net webbased application?

Anonymous said...

Do any of the tools support testing applets in web applications?

Helder dos Santos Ribeiro said...

Watir has a working recorder?! Where?

Magazines said...

I have been using HttpUnit for years but now would like to try automating browsers for functional testing. Thank you for the comparison table.

Anonymous said...

Note this comparison is on Selenium Core, not Selenium RC (which is more comparable to Sahi).

Check out RC before discounting Selenium.

Tim said...

Selenium actually has a sub-project (if you can call it that) called Selenium Remote Control (or Selenium RC) which allows you to create tests in several languages (i.e. Java/.NET/Ruby/Python). With it you can create tests and test suites using any testing framework (i.e. JUnit/NUnit/TestNG). It works by using a server which drives the browser and a client API that talks to the server. The server is written in Java and the client is written in several languages listed above.

Selenium RC also has experimental HTTPS support which I've sucessfully used to create tests that used a secure certificate.

The learning curve is whatever you make of it, meaning if you already have experience with JUnit, it's pretty easy to pick up the Selenium API. The ease of writing tests depends on how well you put id's in your HTML. Without id's you'll have to use X-Path or DOM or something.

The scripting language that drives the browser is actually JavaScript.

If you write a test with Selenium RC, it does NOT need to be deployed with the app.

I hope that clears things up a bit. Check out the home page for more information.