Mozilla A-Team: Writing tests for Peptest
« previous entry | next entry »
Nov. 30th, 2011 | 11:50 am
With ahal's impending return to studentdom and myself coming back from paternity leave, I will be taking over Peptest development and maintenance. To get myself up to speed, I wrote some tests.
The easiest test to write is one that looks for unresponsiveness while simply loading a page. I've noticed that the site for my favourite blog, The Daily What, causes some pain in Firefox, what with all the videos and images and so forth. I wrote a very simple test to see if I was onto something:
Indeed, while there were no very long pauses, there were a string of short ones. Remember, we care about pauses longer than 50 ms, which Peptest identifies for us:
Not awful, but not great. I filed bug 706250 to investigate this.
Next, I decided to delve into some Project Snappy bugs to see what else I could find.
Changing the URL in the above test was all I needed to confirm that the page in comment 62 of bug 61684 was still an issue. This time I got about 50 unresponsive periods, the longest being 2.6 s. Ouch.
Bug 430106 is a little more interesting. Someone reported problems switching back to a tab in which a large image was loaded. The simplest way I could replicate this was by loading the example URL in one tab, loading any old page in a second tab, waiting for about 20 seconds, then switching back to the first tab. In peptest form,
When I ran this, I saw a visible delay before the image was repainted. Peptest confirmed this:
The unresponsiveness appears to be relative to the size of the image, as an image of about twice the dimensions, that is, 3.4 times as many pixels, resulted in a delay about 3.4 times as long.
One of the next steps for Peptest is to add JS-function tracing so we can figure out the exact sources of unresponsiveness. This, however, requires a fix for bug 580055, which in turn depends on bug 702740. As soon as patches for those bugs have landed, we'll add support to Peptest.
For more information on Peptest, see the wiki article and/or check out the code, which has recently been moved to hg.mozilla.org under mozilla-central/testing/peptest.
The easiest test to write is one that looks for unresponsiveness while simply loading a page. I've noticed that the site for my favourite blog, The Daily What, causes some pain in Firefox, what with all the videos and images and so forth. I wrote a very simple test to see if I was onto something:
Components.utils.import('resource://mozmill/driver/mozmill.js');
let c = getBrowserController();
pep.performAction('open_page', function() {
c.open('http://thedailywh.at');
c.waitForPageLoad();
})
Indeed, while there were no very long pauses, there were a string of short ones. Remember, we care about pauses longer than 50 ms, which Peptest identifies for us:
PEP TEST-START | test_dailyWhat.js PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 103 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 199 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 112 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 204 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 105 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 57 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 79 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 194 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 202 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 68 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 182 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 63 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 84 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 118 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 51 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 55 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 67 ms PEP WARNING | test_dailyWhat.js | open_page | unresponsive time: 215 ms PEP TEST-UNEXPECTED-FAIL | test_dailyWhat.js | fail threshold: 0.0 | metric: 322.362 PEP TEST-END | test_dailyWhat.js | finished in: 9426 ms
Not awful, but not great. I filed bug 706250 to investigate this.
Next, I decided to delve into some Project Snappy bugs to see what else I could find.
Changing the URL in the above test was all I needed to confirm that the page in comment 62 of bug 61684 was still an issue. This time I got about 50 unresponsive periods, the longest being 2.6 s. Ouch.
Bug 430106 is a little more interesting. Someone reported problems switching back to a tab in which a large image was loaded. The simplest way I could replicate this was by loading the example URL in one tab, loading any old page in a second tab, waiting for about 20 seconds, then switching back to the first tab. In peptest form,
Components.utils.import('resource://mozmill/driver/mozmill.js');
let c = getBrowserController();
while (c.window.gBrowser.tabs.length < 2) {
c.window.gBrowser.addTab();
}
// Load large image in first tab.
c.tabs.selectTabIndex(0);
c.open('http://flickr.com/photos/thomasstache/2429920499/sizes/o/');
c.waitForPageLoad();
// Load any page in second tab.
c.tabs.selectTabIndex(1);
c.open('http://www.mozilla.org');
c.waitForPageLoad();
// Wait for memory to be freed from first tab.
c.sleep(20000);
pep.performAction('switch_tab', function() {
c.tabs.selectTabIndex(0);
// Wait for image to repaint.
c.sleep(2000);
});
When I ran this, I saw a visible delay before the image was repainted. Peptest confirmed this:
PEP TEST-START | test_largeImgTabSwitchLocal.js PEP WARNING | test_largeImgTabSwitchLocal.js | switch_tab | unresponsive time: 54 ms PEP WARNING | test_largeImgTabSwitchLocal.js | switch_tab | unresponsive time: 835 ms PEP TEST-UNEXPECTED-FAIL | test_largeImgTabSwitchLocal.js | fail threshold: 0.0 | metric: 700.141 PEP TEST-END | test_largeImgTabSwitchLocal.js | finished in: 24083 ms
The unresponsiveness appears to be relative to the size of the image, as an image of about twice the dimensions, that is, 3.4 times as many pixels, resulted in a delay about 3.4 times as long.
One of the next steps for Peptest is to add JS-function tracing so we can figure out the exact sources of unresponsiveness. This, however, requires a fix for bug 580055, which in turn depends on bug 702740. As soon as patches for those bugs have landed, we'll add support to Peptest.
For more information on Peptest, see the wiki article and/or check out the code, which has recently been moved to hg.mozilla.org under mozilla-central/testing/peptest.
