Benchmarks
We've played a little with some early crude benchmarking. Three Four scenarios were benchmarked in Celerity and Watir.
- Scenario 1: 2000 span elements (local file)
- Scenario 2: Looping a Google image search result
- Scenario 3: Counting the number of "diggs" on digg.com's front page
- Scenario 4: Locating different HTML input elements (local file)
Results: Overview
This table provides a quick overview. For more accurate numbers, see below.
| Scenario | Watir (total) | Celerity (total) | Time reduction |
|---|---|---|---|
| 1 | 316,97 s | 0,59 s | 99,81 % |
| 2 | 278 s | 86 s | 69 % |
| 3 | 128 s | 33 s | 74 % |
| 4 | 185,00 s | 4,67 s | 97,48 % |
Notes
- Celerity 0.0.2 on JRuby 1.1 and Watir 1.5.4 on Ruby 1.8.6 was used.
- Neither Watir nor Celerity downloaded any images in these scenarios. Watir (IE) had them cached, and disabling images in IE did not seem to affect the benchmarks (compared to when they are cached).
Results: Actual output from benchmark scripts
Watir on Ruby: Scenario 1
n = 100
user system total real
Loop through all spans (n = 1) 62.469000 16.312000 78.781000 (198.016000)
Last span by id (String) 15.968000 4.953000 20.921000 ( 59.500000)
Last span by id (Regexp) 17.140000 5.015000 22.155000 ( 59.454000)
total : 316.97000002861
average: 105.656666676203
Celerity on JRuby: Scenario 1
n = 100
user system total real
Loop through all spans (n = 1) 0.438000 0.000000 0.438000 ( 0.446355)
Last span by id (String) 0.016000 0.000000 0.016000 ( 0.013113)
Last span by id (Regexp) 0.141000 0.000000 0.141000 ( 0.132035)
total : 0.5915031433105469
average: 0.19716771443684897
Watir on Ruby: Scenario 2
n = 5
user system total real
Google image search results 44.234000 11.047000 55.281000 (277.734000)
total : 277.733999967575
average: 277.733999967575
Celerity on JRuby: Scenario 2
n = 5
user system total real
Google image search results 86.016000 0.000000 86.016000 ( 86.005572)
total : 86.00557208061218
average: 86.00557208061218
Watir on Ruby: Scenario 3
n = 5
user system total real
Diggs on front page 7.344000 2.000000 9.344000 (127.907000)
total : 127.90700006485
average: 127.90700006485
Celerity on JRuby: Scenario 3
n = 5
user system total real
Diggs on front page 33.422000 0.000000 33.422000 ( 33.427076)
total : 33.4270761013031
average: 33.4270761013031
Watir on Ruby: Scenario 4
n = 1000
user system total real
text input by id (String) 1.093000 0.281000 1.374000 ( 2.765000)
text input by id (Regexp) 1.063000 0.250000 1.313000 ( 2.766000)
text input by name (String) 1.625000 0.281000 1.906000 ( 4.282000)
text input by name (Regexp) 1.594000 0.281000 1.875000 ( 4.266000)
select list by id (String) 1.813000 0.328000 2.141000 ( 4.704000)
select list by id (Regexp) 1.766000 0.235000 2.001000 ( 4.672000)
select list by name (String) 1.671000 0.312000 1.983000 ( 4.656000)
select list by name (Regexp) 1.797000 0.313000 2.110000 ( 4.671000)
checkbox by id (String) 5.844000 1.203000 7.047000 ( 14.047000)
checkbox by id (Regexp) 5.515000 1.203000 6.718000 ( 13.937000)
checkbox by name (String) 5.578000 1.063000 6.641000 ( 14.031000)
checkbox by name (Regexp) 5.328000 1.062000 6.390000 ( 14.359000)
checkbox by id (String) and value (String) 9.406000 2.188000 11.594000 ( 28.578000)
checkbox by id (Regexp) and value (Regexp) 9.718000 2.078000 11.796000 ( 25.437000)
checkbox by name (String) and value (String) 6.390000 1.203000 7.593000 ( 16.453000)
checkbox by name (Regexp) and value (Regexp) 9.578000 1.719000 11.297000 ( 25.375000)
total : 184.998999834061
average: 11.5624374896288
Celerity on JRuby: Scenario 4
n = 1000
user system total real
text input by id (String) 0.063000 0.000000 0.063000 ( 0.053699)
text input by id (Regexp) 0.360000 0.000000 0.360000 ( 0.357198)
text input by name (String) 0.313000 0.000000 0.313000 ( 0.322434)
text input by name (Regexp) 0.297000 0.000000 0.297000 ( 0.298454)
select list by id (String) 0.031000 0.000000 0.031000 ( 0.028299)
select list by id (Regexp) 0.297000 0.000000 0.297000 ( 0.303616)
select list by name (String) 0.219000 0.000000 0.219000 ( 0.228946)
select list by name (Regexp) 0.250000 0.000000 0.250000 ( 0.242370)
checkbox by id (String) 0.031000 0.000000 0.031000 ( 0.032020)
checkbox by id (Regexp) 0.360000 0.000000 0.360000 ( 0.348517)
checkbox by name (String) 0.469000 0.000000 0.469000 ( 0.469942)
checkbox by name (Regexp) 0.485000 0.000000 0.485000 ( 0.495507)
checkbox by id (String) and value (String) 0.015000 0.000000 0.015000 ( 0.029422)
checkbox by id (Regexp) and value (Regexp) 0.312000 0.000000 0.312000 ( 0.322679)
checkbox by name (String) and value (String) 0.547000 0.000000 0.547000 ( 0.536925)
checkbox by name (Regexp) and value (Regexp) 0.609000 0.000000 0.609000 ( 0.600956)
total : 4.670983791351318
average: 0.2919364869594574