perfjankie
PerfJankie is a tool to monitor smoothness and responsiveness of websites and Cordova/Hybrid apps over time. It runs performance tests using browser-perf and saves the results in a CouchDB server. It also has a dashboard that displays graphs of the performance metrics collected over time that you help identify performance trends, or locate a single commit that can slow down a site.
After running the tests, navigate to the following url to see the results dashboard.
http://couchdb.serverl.url/databasename/_design/site/index.html
Here is a dashboard created from a sample project.
Why ?
Checking for performance regressions is hard. Though most modern browsers have excellent performance measurement tools, it is hard for a developer to check these tools for every commit. Just as unit tests check for regressions in functionality, perfjankie will help with checking regressions in browser rendering performance when integrated into systems like Travis or Jenkins.
The results dashboard
Setup
Perfjankie requires Selenium as the driver to run tests and CouchDB to store the results. Since this is based on browser-perf, look at setting up browser-perf for more information.
Usage
Perfjankie can be used as a node module, from the command line, or as a Grunt task and can be installed from npm using npm install perfjankie
.
Node Module
The API call looks like the following
var perfjankie = ;;
Other options that can be passed include preScript
, actions
, metrics
, preScriptFile
, etc. Note that most of these options are similar to the options passed to browser-perf. Refer to the browser-perf options for a mode detailed explanation.
Grunt Task
To run perfjankie as a Grunt task, simple load task using grunt.loadNpmTasks('perfjankie');
, define a perfjankie
task and pass in all the options from above as options to the Grunt task. Here is an example.
Command line
Run perfjankie --help
to see a list of all the options.
Quick Note - to only update site the first time, run the following from the command line.
$ perfjankie --config-file=local.config.json --only-update-site
The config file can contain server configuration and can look like this.
Hosting dashboard on a different server
You can also host the HTML/CSS/JS for displaying the results dashboard on not on CouchDB, but a different static server, possibly behind a CDN. In such cases,
- Use the npm module and host the contents of the
site
folder. - Open index.html and insert the following snippet in the
<head>
section
This will ensure that all requests for data are made to the other CouchDB server. Also ensure that the CouchDB server has CORS turned on.
Login before running tests
You can login a user, or perform other kinds of page setup using the preScript or the preScriptFile options. Here is an example of a login action that can be passed in the preScript option.
Migrating data from older versions
If you have older data and want to move to the latest release of perfjankie, you may also have to migrate your data. You can migrate from older version of a database to a newer version using
$ perfjankie --config-file=local.config.json --migrate=newDatabaseName
This simply transforms all the old data into a format that will work with the newer version of perfjankie. Your version of the database is stored under a document called version
, and the version supported by your installed version of perfjankie is the key dbVersion
in the package.json
What does it measure?
Perfjankie measures page rendering times. It collects metrics like frame times, page load time, first paint time, scroll time, etc. It can be used on
- long, scrollable web pages (like a search result page, an article page, etc). The impact of changes to CSS, sticky headers and scrolling event handlers can be seen in the results.
- components (like bootstrap, jQuery UI components, ReactJS components, AngularJS components, etc). Component developers just have to place the component multiple times on a page and will know if they caused perf regressions as they continue developing the component. For more information, see the documentation for browser-perf
Development
Dev setup
Any changes should be verified with unit tests, see test
-folder.
To run the tests you local couchdb installed with a database, see test/res/local.config.json
for details:
- start couchdb
- start a local selenium grd:
java -jar node_modules/selenium-server/lib/runner/selenium-server-standalone-2.53.0.jar -Dwebdriver.chrome.driver=$(pwd)/chromedriver/lib/chromedriver/chromedriver
- run tests via
npm test