An effort to add browser benchmarking using ci.linaro.org and Lava was started in late 2012. The main requirement was it had to run in a CI loop so it had to be totally automatic. Certain versions of Firefox had a command line option to render a page and then exit but that would only allow taking the total time of a benchmark suite and not of individual components.
Selenium webdriver seemed to provide the best framework for a browser independent solution for running benchmarks and scraping the results.
Selenium requires an extension for firefox which it is able to install as needed. For Chrome it needs a gateway to translate from selenium wire protocol to Chrome wire protocol. This is a binary called chromedriver which can be built as part of chrome but is not included in the Ubuntu build of the Chrome browser. Chrome browser packaging was modified to include chromedriver and an armhf build of chromedriver has been added to the browser benchmark source here: http://bazaar.launchpad.net/~linaro-foundations/linaro-ubuntu/lava-test-browser-benchmarks/files/head:/bin/
The changes to Ubuntu Chrome packaging were also submitted in a bug but that went nowhere: https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1069930
Using the chromedriver, an out of tree Lava test was developed and it worked initially on both Chrome and Firefox. Later it stopped working on Firefox. https://bugs.launchpad.net/linaro-ubuntu/+bug/1084527
This could be worked around or may be fixed with a new Firefox selenium extension.
Further investigation may prove that there is a better framework for this than Selenium.
If this task gets taken up again these things should happen.
- Revisit if Selenium is the right approach. If it works for Chrome and Firefox then it is great. There may not be another alternative.
JohnRigby/BrowserBenchmarking (last modified 2013-01-31 15:59:24)