How to read a WebPageTest waterfall chart

I often find myself looking at WebPageTest (WPT) waterfall charts, but as I seem to have the memory of a chimpanzee (not a goldfish, that’s a myth), I tend to forget some of the details and what they all mean. So I decided to pull together many bits of information into a single blog post I can refer to at a later date. If you find it useful, or think I’ve forgotten anything, please let me know.

Basic Layout

Here’s the basic layout of the waterfall chart user interface:

Basic layout of the Web Page Test waterfall chart includes the 1:key, 2:request list, 3:request timeline, 4:CPU utilisation, 5:Bandwidth in, 6:browser main thread, 7:page is interactive.

1 - Key

The key shows three types of information:

  1. Information about the connection status (DNS lookup, connection established, SSL negotiation)
  2. The type of resource being requested (e.g. HTML, images etc)
  3. Miscellaneous events (wait, JavaScript execution)

Each resource has 2 colours, a light shade and a dark shade. The light shade signifies the point at which the browser has made the request for the resource. The dark shade is the point at which the resource is actually downloading. See this comment from Pat Meenan for more information.

The “wait” visualisation is a new addition to WPT. It shows you the time between when the browser first discovered the resource on the page, up to when the request for the resource was made by the browser to the server.

2 - Request list

A list of assets the browser has discovered on the page and the order in which they were requested. Note the request number on the far left, as well as the yellow lock if the request is being made over a secure connection (HTTPS).

3 - Request timeline

The timeline shows the time along the horizontal (x) axis verses each request made on the vertical (y) axis. From this you can see the lifecycle of a request made by the browser. From discovery (wait), through to request being made, and finally to asset being download.

Ideally you want to make sure this timeline covers as little time as possible, as this indicates better overall performance. The thinner the timeline, the quicker the page loads for a user.

4 - CPU Utilisation

A simple graph showing the CPU utilisation of the browser process running on the device. It displays how much CPU the current webpage is using at any point in time. It ranges from 0 - 100% utilisation. See this comment from Pat Meenan for more information.

5 - Bandwidth In

This is an indicator of when data is coming into the browser. The visualisation is helpful to see when the browser is doing useful work vs wasted time. The absolute scale can be ignored, as it isn’t very accurate. Use the “Capture network packet trace (tcpdump)” option in the advanced tab on the WebPageTest homepage if you want more accurate results. See this comment from Pat Meenan for more information.

6 - Browser Main Thread

This graph visualises what the browsers main thread is doing at any specific point in time (x-axis). The y-axis shows the percentage from 0 - 100%. The colours were copied from the Chrome DevTools CPU graph (under the performance tab). Here’s what each of the colours mean:

  • Orange - Script parsing, evaluation and execution  
  • Purple - Layout  
  • Green - Painting  
  • Blue - HTML parsing  

Using this graph it is possible to see if the CPU is becoming a bottleneck in one of the above areas.

7 - Page is Interactive

This graph gives you an indication of when the main thread is blocked. The red blocks indicate that the main thread has been blocked for 100ms (which will also block inputs like button presses). Green indicates the thread isn’t blocked. Note: it still may be possible to scroll during red blocked phases, as scrolling is usually handled off the main thread for most browsers. See this comment from Pat Meenan for more information.

Vertical lines

You can see the key for each of the coloured vertical lines below the “Waterfall View” header as seen below:

Key for the vertical lines displayed on the timeline. From left to right: Start render, RUM first paint, DOM interactive, DOM content loaded, On load, Document complete.

But what do each of them mean?

Start Render - Green line  

This is the first point at which a user will see pixels painted to the page. The pixels could be from anything at all (background colours, borders etc), not necessarily content. Before this point the screen was blank. This metric is measured by analysing individual video frames that are captured during the page load. See comment from Pat Meenan for more information.

RUM First Paint - Light green line  

This is the point where the browser renders anything to the screen that is visually different from before navigation (i.e. the blank screen for WPT). This metric is reported via the browsers API, so when it thinks it painted the first content. Because of this, the line is only visible if the browser supports the Paint Timing API.

DOM Interactive - Yellow line  

The point at which the browser has finished parsing all the HTML, and DOM construction is complete. Unfortunately it’s not a reliable metric.

DOM Content Loaded - Pink line  

The point at which the HTML has been loaded and parsed, and the browser has reached the end of the document. All blocking scripts have been loaded and run. The DOM at this point is completely defined. See comment from Pat Meenan for more information.

On Load - Lavender line  

The point at which the window load event fires. All objects are in the DOM and all images and scripts have finished loading.

Document Complete - Blue line  

The point where the onload event has fired and all the static image content has loaded. Content changes that are triggered by JavaScript execution may not be included.

Horizontal timings

So let’s now concentrate on the request timeline (3). What do the horizontal blocks mean and what metrics do they refer too? Well, if you click on an individual request you will see a popup with a lot more information, as seen in the example below:

By clicking on an individual request in WebPageTest it will give you much more information including timing details, request details, response details and if enabled when testing, the original response body.

So lets take a look at a few of the requests from this waterfall view, as it gives us a number of quite varied requests to look at.

Request 1 - The HTML

Here the browser is requesting the HTML document, so at this point in time it is also having to setup the connection to the server. In the request details we are given the following timings:

  • Discovered: 0.011 s
  • Request Start: 0.116 s
  • DNS Lookup: 27 ms
  • Initial Connection: 25 ms
  • SSL Negotiation: 43 ms
  • Time to First Byte: 315 ms
  • Content Download: 40 ms

I’ve annotated the request to show what each of these timings mean:

Request number one with timings annotations added, including all the timings listed above.

Adding the DNS, Initial Connection, SSL negotiation, Time to First Byte (TTFB) and the Content download times gives you the 450ms that is displayed directly after the request finishes.

It’s worth noting that WPT follows a specific convention in the request details panel:

  • If the time corresponds to a duration, it is measured is milliseconds (ms), e.g. the DNS lookup took 27ms.
  • If the time corresponds to a starting point, it is measured in seconds (s), e.g. the request started at 0.116s.

Request 7 - A third-party JavaScript file

This request is different from the other requests examined because the file is coming from a different domain. The request details give the following timings:

  • Discovered: 0.473 s
  • Request Start: 0.702 s
  • DNS Lookup: 28 ms
  • Initial Connection: 39 ms
  • SSL Negotiation: 153 ms
  • Time to First Byte: 48 ms
  • Content Download: 9 ms

Notice how the browser needs to go through the whole connection negotiation again (DNS, Connection, SSL negotiation) because the file exists on a different domain. This adds a fair chunk of time to the request (28 + 39 + 153 = 220ms).

Request 7 loads a JavaScript file from a third-party domain. This causes a new TCP connection to be established. 200ms after the JavaScript is downloaded the graph shows JavaScript execution that is also visible on the Browser main thread.

The other interesting point about this request is the script executes around 200ms after the download is complete. There’s no information about this execution in the details panel, but you can see it in the waterfall as light pink lines after the request and orange in the “Browser Main Thread” panel (6) which signifies “Script parsing, evaluation and execution”.

Request 15 - A sponsor PNG

With this request the browser has discovered a PNG image and requests it from the server. In the request details we are given the following timings:

  • Discovered: 0.652 s
  • Request Start: 0.824 s
  • Time to First Byte: 214 ms
  • Content Download: 28 ms

Wait time is calculated by subtracting the discovered time from the request start time. The wait time is the time taken from when the browser first finds the asset, to the time when it has the capacity to send a request to the server.

Request 15 downloads a PNG image. The waterfall now includes the new waiting period from when the asset was discovered by the page, to when the browser makes the request.

The duration after this request is the time taken from the request being made, to when the request is completed (Time to First Byte + Content Download). Since a connection to the domain has already been established, there’s no need for DNS, Connect, SSL negotiation.

Request 23 - GIF file moved

Although request 23 looks quite unremarkable, there are a couple of things going on. The background of the request is yellow. This is to signify a server response status code that isn’t the usual 200. In this instance it is a 302 status code, which signifies the GIF file has been temporarily moved. In fact all responses with a 3xx status code will have a yellow background. The request details show the following information:

  • Error/Status Code: 302
Request 23 shows how WPT displays 302 redirects in the form of a highlighted bar in yellow and a different status code after the request has completed.

Notice how request 23 doesn’t require a TCP connection to be established. This is because it has already happened for this domain on request 20.

Error status codes 4xx and 5xx are displayed in a similar way, only the background is red, like in the example below (note this image is from a different test):

Request 7 shows what happens on a WebPageTest waterfall graph when it encounters a status code starting with a 4xx (error) or 5xx (Internal Server Error). The background of the request is coloured red, and the status code at the end displays 404.

The request details show the following information:

  • Error/Status Code: 404

Notice the colour of the returned resource response in this instance. Rather than being the expected purple colour for an image, it is blue signifying that it is an HTML page. Or, in other words, it is the server responding with the 404 page because the asset can’t be found.

Download chunks

Another visual peculiarity you may be curious about is the vertical stripes in each of the requests. As mentioned earlier, the lighter colour signifies the request has been made and the browser is waiting for a response. The darker colour indicates that bytes for the specific resource are being delivered to the browser. Now sometimes this doesn’t happen all at the same time. This results in what looks like zebra striping, where the browser is being drip-fed bytes over time. These are called download chunks (red arrows).

Download chunks can be seen in requests and they look like zebra striping. This indicates that the browser is being drip-fed bytes for each asset. Chunking usually occurs when lots of assets are competing for bandwidth, or the HTML early flush technique is being used.

This is very visible when an HTML early flush technique is being used (request 2 - see scenario below for more details) or if a large number of assets are being downloaded in parallel and are competing for resources (requests 3-9). Note the bandwidth graph at the bottom of the UI is maxed out from 1.6 to 2.5 seconds.

You may be asking “but this chunking also occurs after the bandwidth usage drops (2.6 seconds+), so what is happening there?”. Well the number of parallel connections has dropped so less is downloading in parallel. But the connections that have been created in requests 12 - 15 are still in the TCP slow start phase, so the the assets are still competed for (now limited) bandwidth.

Common scenarios

Here’s a list of common patterns seen in a WPT waterfall chart. I’ll add more of these over time as I encounter them.

DNS-prefetch

DNS Prefetch is part of the Resource Hints Working Draft. It gives a developer the ability to tell the browser that a DNS lookup is going to be needed for another domain in the near future. So instead of waiting, start the lookup immediately. By the time the domain is actually required, the browser will only need to complete the TCP handshake, and optional SSL negotiation. It looks similar to the preconnect example below, in that it is “floating” in the timeline. But here only the DNS lookup (green) is visible.

The waterfall shows the dns-prefetch occur quite some distance from where it would usually happen (with the TCP and SSL negotiation). It is also worth noting where it occurred in the timeline, directly after the HTML download completed.

Notice where the dns-prefetch occurred in the timeline: almost immediately after the HTML has finished downloading and is parsed. It is easy to see the difference if you compare it to the connection negotiations happening in requests 5 and 7, where preconnect is being used.

Preconnect

Preconnect is part of the Resource Hints Working Draft. It allows a developer to give the browser a “hint” that a domain will need to be connected too in the near future. By connecting to the domain early, the connection won’t need to be established later in the waterfall, thus allowing assets from said domain to be requested and downloaded quicker when they are required.

The image shows the preconnected domain establishing the connection earlier than required, before the asset is requested. This preconnect allows the request to happen immediately.

As you can see in the image above, the preconnect looks to be “floating” in the timeline. It happens way before the actual request for the image is made. This is the browser using the preconnect hint to connect ahead of time before it is required. For more information on the preconnect hint I recommend reading this blog post by Andy Davies.

Prefetch

Prefetch is part of the Resource Hints Working Draft. It allows a developer to tell the browser to prefetch a resource (e.g. CSS, JS, HTML document) in the current navigation, as it may be required at a later date. For example, if you know the majority of your users navigate to a specific page from your homepage (e.g. maybe your login page), you could decide to prefetch it so it will already exist in the users browser cache when it is required. In the example below I am prefetching another HTML document that sits along a users journey:

With Prefetching

The waterfall shows a request for two HTML files. One for the initial connection (request 1), another for the prefetch we have specified (request 19). The prefetched file is simply cached, it isn't parsed.

The prefetch is visible at request 19 in blue (HTML). It is worth noting that this prefeched HTML is simply stored in the browser cache. It isn’t parsed by the browser. You can verify this in the waterfall chart UI by looking at the browser main thread graph. At the point the the HTML is prefeched, there’s no main thread activity registered.

Without Prefetching

Only one HTML request can be seen on the waterfall chart because prefetch isn't being used.

WebPageTest gives us some information in the popup to let us know it is a prefetch hint:

  • Priority: IDLE (under the details tab)
  • purpose: prefetch (under the request tab)

It’s important to remember the priority of a prefetch. In WebPageTest, when testing in Chrome it is listed as priority IDLE. This maps to Lowest priority in DevTools (according to the Resource Fetch Prioritization and Scheduling in Chromium document). So a prefetch is an optional and often low-priority fetch and will be loaded as late as possible by the browser. This differs from preload which is a mandatory fetch and gets a High priority in the browser. A resource loaded using preload is layout blocking, so use it sparingly else it could actually slow down perceived performance.

Prerender

Prerender is part of the Resource Hints Working Draft. It gives a developer the ability to tell a browser what a users likely next navigation could be (assuming the developer is tracking this in some form of analytics). In December 2017, with the release of Chrome 63, Google overhauled how prerender worked. Here’s a brief before and after explanation:

Pre-Chrome 63: Chrome would look for the <link rel="prerender"> element and create a hidden page for the URL listed by it. Unseen by the user, this page would be downloaded including all dependant subresources and any JavaScript would be executed. Should the user happen to navigate to the page, the prerendered page would be swapped out with the current tab, giving the user the impression of an instant page load. There were a few negatives to this method. The first, memory usage was high due to having to maintain the state of the hidden page. This made it unsuitable to low-end devices. And second, since user-facing and stateful actions should not occur with a prerender (since the user can’t see them), it was a very complex task to achieve this when the whole point of a prerender was to render the page. So the decision was taken to depreciate and remove the then current implementation.

Chrome 63+: Since the release of Chrome 63, the prerender hint is still recognised and followed by Chrome, only it is handled in a much different way. Chrome now uses a technique called “NoState Prefetch” when it sees a prerender link element. If seen it will be followed only if these two conditions are met: the user is not on a low-end device, and they are not on a mobile connection. If followed, the page will be downloaded and scanned for subresources to download. These will be downloaded and cached at the lowest priority possible (IDLE). No JavaScript will be executed on the prerendered page, and should the user actually navigate to the page, the browser will need to load the cached resources into a new browser window (not swapped out as before). A lot more information about “NoState Prefetch” can be found here.

So what does this prerender look like in a WPT waterfall chart:

With Prerendering

You can see the request for the prerendered HTML page, and the subsequent subresources loading. Many of these subresources already exist in the browser cache since they were download as part of the original homepage request.

In the waterfall chart you can see the usual page and resources loading from request 1 through to 19. Request 16 is where the prerender occurs. Here you can see a request for a second HTML page. Once completed, this then triggers requests 20 through to 29. Notice how many of these requests have a yellow background with a 304 status code. This is telling us they are identical to a resource that already exist in the browsers cache. They exist here because the homepage HTML above them (request 1) put them there only a few 100ms before. Notice how there’s very little happening in the browser main thread graph (other than the homepage parsing which is happening because request 30 (CSS) completed). This confirms that the prerendered assets and subresources are simply being stored in the browser cache for later use.

As with prefetch, WPT gives us a little information in the popup for each prerender resource to let us know the requests aren’t from a standard user navigation:

  • Priority: IDLE (under the details tab)
  • purpose: prefetch (under the request tab)

Note: It doesn’t explicitly tell us it comes from a prerender hint, only from a prefetch. Since “NoState Prefetch” is now being used, this actually makes sense.

Without Prerendering

Without the prerender link element you get your standard page load. No requests are made to other pages or subresources.

Without the prerender link element you get your standard page waterfall chart. Only requests to the current page subresources can be seen.

Preloading

Preloading is a W3C Candidate Recommendation and is used to increase the loading priority of selected assets. A developer can tell the browser: “this resource will absolutely be needed soon, so load it right away”. This technique is often used when loading web fonts.

Without preload, when loading a web font the browser first needs to download the HTML and CSS, then parse both to create the render tree. Only at this point can the browser request the font. This can lead to what is known as a Flash of Invisible Text (FOIT) and Flash of Unstyled Text (FOUT). A way around this issue is to request the web font file immediately using the preload directive.

With Preloading

A preloaded request comes directly after the HTML download starts (request number 2), when the browser parses the <head>. The request for this asset is started as soon as it is discovered.

Without Preloading

The usual font loading occurs after the CSS has downloaded and is parsed. You can see the font has request number 11, verses number 2 when preloaded.

If you compare both of the images above you will see the request for the preloaded WOFF2 font is made as soon as the HTML starts to be downloaded at request number 2 (dark blue strip). The browser parsed the <head> tag, saw the preload directive and made the file request immediately.

Compare this to the second image, where the browser downloads the font after waiting for the HTML and CSS to be downloaded and parsed. Only at this point can the WOFF2 font request be made. As you can see from the image, when preloading isn’t used the font is at request number 11. I’ve written more about font preloading here if you are interested.

HTTP/1.1 vs HTTP/2

HTTP/2 is the next iteration of the HTTP protocol after HTTP/1.1. Due to the fact that HTTP/2 uses a single TCP connection and multiplexes files over this single connection, it is easy to spot the difference in the resulting waterfall charts:

HTTP/1.1

HTTP/1.1 waterfall showing the stepped image requests from the page due to multiple TCP connections having no knowledge of each other.

HTTP/2

HTTP/2 waterfall showing the simultaneous image requests by the browser. The browser will stream the image data back via a single connection.

A browser using HTTP/1.1 requests images via separate TCP connections, and this tends to happen at slightly different times (hence the stepped nature of the waterfall). A browser using HTTP/2 on the other hand requests all the images at the same time. It is the server that decides when the images will be sent back to the browser, and in what order.

OCSP

Online Certificate Status Protocol (OCSP) is an internet protocol used for obtaining the revocation status of SSL certificates. One way for a browser to certify a certificate is to connect to an OCSP server for verification. When this happens WebPageTest will show you in the waterfall as seen below:

With OCSP

OCSP requests seen in Web Page Test appear before the request for the HTML. These request add a large amount of time to the SSL negotiation and the HTML download.

This OCSP check is bad for performance. The verification requires a DNS lookup and an initial connection to the OCSP server. Only once the certificate has been verified, can the SSL negotiation take place to the original domain. As you can see in the image, the whole waterfall is being pushed back. It takes almost 2 seconds before the HTML page can even be requested!

Without OCSP

When the OCSP doesn't happen request number 1 is the request direct to the web server, setting up the connection (DNS, Connect, TLS), then requesting the HTML file.

If you compare the with & without OCSP waterfalls, you can see that the length of the SSL negotiation is much shorter for without OCSP (300ms instead of 1000ms+) and therefore the request for the HTML file happens much quicker (at 1 second verses 1.95 seconds). The OCSP check adds 950ms to the initial HTML request on a 3G Fast connection. That’s a huge number!

If you notice this on your WebPageTest timelines you should look into enabling OCSP stapling on your server. Note: If you are using Extended Validation certificates (EV), OCSP stapling won’t fully solve the issue, see this technical Twitter thread for more details on this.

Firefox enhanced tracking protection

Firefox enabled enhanced tracking protection by default as of version 69 (June 2019). The agents on WebPageTest updated around the same time. In some rare cases the tracking protection requests could be seen in the WPT waterfalls (request 1-3):

Firefox enabled enhanced tracking protection in version 69. A DNS lookup and the requests could be seen before the initial connection in some rare cases.

According to Pat Meenan these requests should now be filtered out by default, so they will never be seen in the waterfall charts.

Service worker precaching

The use of service workers is gradually increasing, and one of the many features they allow is a fine-grain control on what assets are cached and for how long. They also give a developer the ability to precache files for future use (e.g. for offline capabilities). An important detail to remember when precaching assets using a service worker, is the browser may need to download the same files twice. Once for the HTTP cache (the standard browser cache), and again for the service worker cache (Cache API). Theses are two totally separate caches, and don’t share assets. These duplicate requests can be seen in a WebPageTest waterfall chart:

The two sets of identical assets are downloaded twice, once for the HTTP cache then again for the service worker cache.

In requests 17 and 18 you can see the service worker JavaScript being requested, downloaded, and initialised. Immediately after this, the service worker looks through it’s precache JSON file and requests any assets listed. Note: In the example above the Workbox library is being used to simplify the service worker workflow.

Chrome stair-step

Chrome has included a prioritisation technique that is named due to the pattern it creates in waterfall charts. It involves Chrome examining the assets in the <head> tag (before the page even has a <body>), and requesting, downloading, and parsing these requests first. The browser even goes so far as to delay requests for assets in the body until the <head> requests are complete. It is easier to see this stepping in a HTTP/1.1 graph, as seen in the example below (although it also occurs in HTTP/2):

Chrome stair-stepping occurs when the browser delays requests from the body element in favour of completing the requests from the head element. This can be seen in the WPT waterfall as a distinct step in the timings.

In the above image from the BBC News website, 8 out of the first 9 requests are made for assets in the <head>, with only 1 request to a JavaScript file located in the <body>. The “step” isn’t very long in terms of duration, only around 200ms. But it gives the browser enough time to concentrate all CPU and bandwidth on downloading and parsing these assets, so the <head> is then setup and ready to go before the <body> assets are downloaded and parsed.

Not much has been written about this “layout-blocking” phase in Chrome, but it can be seen in detail in the Resource Fetch Prioritization and Scheduling in Chromium document by Pat Meenan, and also in Chrome’s resource scheduler source code.

HTML early flush

HTML early flush was mentioned above in the download chunks section. It is when a web server sends a small slice of the HTML document, before the whole HTML response is ready. What this then allows the browser to do is parse the HTML it has received, and look for assets that it can then request early (compared to waiting for the whole HTML document to download, parse, and then request).

With early flush

With early flush the server sends a small chunk of HTML to be parsed. This parse then triggers additional requests much earlier than when it would usually happen (and therefor speed up the website).

In the example above, the small download chunk of HTML the browser receives (request 2) contains the <head> tag which links to JavaScript, fonts, JSON, preload directives, and dns-prefetch resource hints. This HTML is parsed and 16 requests are almost immediately triggered very close to each other. Note: Notice how I didn’t list CSS. CNN.com has inlined the CSS in a <style> tag (so no request to a CSS file is required). They then trigger CSS requests via JavaScript later in the waterfall once the JavaScript has loaded and parsed.

Without early flush

Without early flush the browser must wait for the whole HTML document to be downloaded before it can be parsed, and then additional requests can be made. Notice how the HTML download complete lines up vertically with the request being made directly after it.

If you compare the waterfalls for with & without early flush (from different sites unfortunately): With flush you will notice how the requests for assets is made during the HTML download response. Compared to without, where the browser must wait for the whole HTML document response to complete. Only then can it be parsed and requests made to other page assets.

Flushing allows the browser to make requests earlier in the waterfall, and therefore makes the page load and feel faster (if implemented correctly).

Long DOM content loaded (DCL) time

The vertical lines on the waterfall chart are usually very thin, maybe only a couple of pixels thick. This shows they take very little time to actually happen. But in some cases the DOM content loaded line takes up a large chunk of time, and you will very easily see it on your waterfall chart! Take a look at the two (different) charts below:

Normal DCL

DOM Content Loaded (DCL) vertical line is very thin, maybe only 1-2 pixels wide showing it took very little time.

The pink DOM Content Loaded (DCL) vertical line is usually almost instantaneous. Very thin and barely visible on the waterfall chart as it usually occurs around the same time as other events like DOM Interactive & Start Render. This is what a healthy DCL looks like.

Long DCL

DOM Content Loaded (DCL) vertical line is very thin, maybe only 1-2 pixels wide showing it took very little time.

Now compare the previous waterfall chart to the one above and you can easily spot the issue. The DCL line is covering almost 1 second on the waterfall chart! That’s not a healthy looking line. In the chart you will see some unusual goings on that can help explain what is happening. The light pink line on request 4 is showing a lot of JavaScript execution is happening, and it just so happens to be the exact same width as the expanded DCL line. The browser main thread is solid yellow, which confirms that a script is parsing. The CPU graph is at around 80-90% capacity over the same time period and the page is blocked. Curiously the “Bandwidth In” graph is hardly registering anything, so whatever is causing the issue has already downloaded and is being parsed.

So what is actually happening? Well, there’s some very expensive JavaScript running in the file at request 4. Often this is caused by JavaScript attached to the jQuery $(document).ready() method (or some equivalent). So if you ever see a DCL line eating a large chunk of your waterfall chart, you now know where to look.

More to be added soon…

As I discover more common waterfall scenarios I will add them here. If you know of any common ones that are missing, please do let me know!

Hidden gems

So there are features that are sitting in plain sight on the WebPageTest UI that you may not have even noticed before. Here are a few that I’ve found useful:

Now it may seem obvious to some, but it is worth pointing out nonetheless. There is a direct relationship between the filmstrip view and the corresponding waterfall chart below it:

The filmstrip and the waterfall chart are related. There's a red line to the left of the waterfall that corresponds to the red line that moves across the waterfall chart as you scroll. You also see thick orange borders around each image if something has changed on the screen.

To the far left of the filmstrip you will see a 1px vertical red line. As you scroll the filmstrip horizontally you will see an identical red line moving across the waterfall chart. These are directly related. Together they show you what the page looked like at that exact point in the waterfall chart (you would not believe how long it took me to notice this feature!).

Another feature that can be seen is the use of a thick orange border around some of the images. This orange border signifies that something has changed on the screen (compared to the previous one). This is very useful if you are trying to identify even minor changes between screenshots (like an icon loading).

You can see both of these features in action in the screenshot. There is a thick orange border around the image at 0.9s, as it shows a major change to the page compared to the image at 0.8s. Looking closer at the waterfall we can see the red line is approaching and the vertical green line (start render). The image at 0.9s is actually the start render time for the page.

How to generate customised waterfall chart images

Almost all of the waterfall images in the article you see above have used this feature that is tucked away at the bottom of every waterfall chart. WebPageTest gives you the ability to customise what is included in a waterfall chart:

The 'customize waterfall' link leads you to a page with a whole host of options you can use to customise the waterfall image.

Clicking the ‘customize waterfall’ link directs you to a set of customisation options (see image below):

The 'customize waterfall' link gives you the ability to select what requests and graphs you want to include on the output image, the time the waterfall covers, as well as it's dimensions. These are just a few options available.

As you can see in the image above, I have customised the output image. We have:

  • Set a custom image width
  • Set a custom time that the whole waterfall chart covers (note this will lead to cropping of requests further down the waterfall)
  • Only selected certain requests to be shown either individually or as a range (notice the ellipsis between these that identify missing items)
  • Unchecked ‘Show CPU Utilization’ so the ‘CPU Utilisation’ and ‘Browser main thread’ graphs are hidden

All these options allow you to remove much of the noise in the chart so you can focus on the areas you wish to identify and highlight. Once you are happy with the image, simply right click and save the image to your device as you would any other image.

How to add custom marks using the User Timing API

Here’s a useful feature you may not know about. Using the User Timing API you have the ability to mark certain points in your page load. These points will be discovered by WebPageTest and displayed accordingly in the results.

For example, if you wanted to know when the browser got to the end of the <head> tag, you could add the following code right before the closing tag:

<head>
<!-- head stuff here... -->
<script>window.performance.mark('mark_head_parsed');</script>
</head>

The browser then sets a mark at this point which can be read by WebPageTest. The resulting WebPageTest run will show you results similar to this:

Adding custom marks via the User Timing API gives output metrics similar to this in the WebPageTest output.

As you can see from the image above I have set four User Timing marks on the page and one of them is called mark_head_parsed. You can add as many marks as you need, it really depends on what you are trying to measure. Now if you head over to the ‘customise waterfall’ link (as mentioned in the section above), you will see one or more purple triangles and vertical lines on the waterfall chart. These are the User Timing marks we just set:

The custom timing marks can be seen in purple on the waterfall chart if you customise the waterfall image.

In the image above the timing marks are now visible (you get the option to toggle them on and off if required). So you may be thinking: “So why can’t I see these marks on the interactive waterfall chart?”. The reason for this is because they have been disabled by default on the interactive chart. Many third-party JavaScript scripts were including marks in their code, which was polluting the graph with a whole load of noise that were irrelevant to most WebPageTest users. So it was decided to disable the visibility of the marks by default. Marks can now only be seen within the ‘customise waterfall’ link.

Note: If you happen to be running a private instance of WebPageTest, you can configure it to display the User Timing marks to be on by default for the interactive waterfall chart.

Conclusion

So there you have it, a brain dump into a blog post about some of the aspects of WebPageTest that I’ve found a little mysterious. As this has been a learning exercise for me too, if there is anything I have misinterpreted, please do tweet me and let me know!

If you would like to learn a lot more about WebPageTest, I highly recommend the book: “Using WebPageTest” by Rick Viscomi, Andy Davies, Marcel Duran.


Post changelog:

  • 11/10/19: Added waterfall for Firefox enhanced tracking protection scenario.
  • 12/10/19: Added service worker precaching scenario.
  • 13/10/19: Added Chrome Stair-Step scenario.
  • 14/10/19: Added the dns-prefetch scenario.
  • 15/10/19: Added note about OCSP stapling and EV certificates (thanks Ryan Townsend for flagging).
  • 16/10/19: Added information about download chunks (thanks Pat Meenan for raising and checking). Clarified the error code presentation and added a “Without OCSP” waterfall chart for comparison (thanks Andy Davies).
  • 17/10/19: Added the HTML early flush scenario.
  • 18/10/19: Added prefetch scenario.
  • 20/10/19: Added prerender scenario (thanks Simon Hearne & Ryan Townsend for the input).
  • 24/10/19: Added new section called “Hidden Gems”, including “How the filmstrip view and waterfall chart are related”, “How to generate customised waterfall chart images”, and “How to add custom marks using the User Timing API” (Thanks again to Pat Meenan for clarification with this).
  • 08/12/19: Added the Long DOM content loaded (DCL) time scenario. Originally written for my article “Reading a WebPageTest Waterfall Chart” on the Web Performance Calendar 2019.
Loading

Webmentions