How to Exclude Playwright Test Traffic from Google Analytics

Published on Dec 11, 2022.

#tech #automation #ga4 #google analytics #gtm #playwright #testing

We collect website or webapp usage statistics in order to understand our users better and make well-founded business, marketing and user experience decisions based on it. Analyzing the data is a big topic by itself, but no matter how sophisticated our reports are, they work with the data we gathered. If our data is incomplete, bloated or too noisy, quality of user analytics reports decreases and negatively affects our business and user experience decisions.

Automated tests traffic is not representative of our real users. Let’s see how we can filter it out from Google Analytics (GA) data collection.

Enough talk, take me straight to the solution!

Internal Traffic Filter Based on Client IP Address

Probably, this is the most obvious and also very simple solution — if you have a list of internal IP addresses to exclude (think HQ, remote developers and CI servers), you can exclude all the traffic coming from them with the native Internal Traffic filter and Internal Traffic Definition available in Google Analytics Tag settings.

However, if your automation tests are running in the cloud, IP addresses of those virtual machines are likely to change over time. In such case you will have to keep updating the list of internal IP addresses manually to keep the internal (development and testing) traffic out.

It gets even more complicated with remote colleagues accessing your website from various locations across the world, while (hopefully) enjoying the comfort of their homes.

Custom Product Build without Google Analytics

Second option is to introduce a special testing build for your website or application, which would have Google Analytics initialization script removed.

In modern web development very often we already have at least two build targets — production and development. The new, testing one, will be just like the production but with small tweaks removing (or disabling) the GA code.

Two problems arise:

We have to build and deploy that specific testing build before running any of our automated tests. This is not always ideal — often we want to run tests against the regular production version of our website which might be already up and running.
Some automated tests are designed to run against the public production version of the product, were we simply can’t use a custom build.

This method comes with limitations but teams, already having a specialized testing build, can benefit from this approach the most.

Set Google Analytics Debug Mode in GTAG Script

If you want to use the custom testing build and are using the older Google Tag (GTAG) script on your website, then you can conditionally add a debug_mode: true parameter to GTAG config event:

const isTestingBuild = process.env.NODE_ENV === 'testing';

// need to pass undefined instead of false here
gtag('config', 'G-XXXXXX', { debug_mode: isTestingBuild || undefined });
js

When set to true, debug mode will mark all GA events generated by that build (regardless of where it is deployed and who is using it), as developer traffic — the same way GA events are marked when using the Google Analytics DebugView mode.

To exclude events marked as developer traffic from Google Analytics reports we need to update Developer Traffic Filter settings.

Debug Mode URL Query Param

Let’s assume that we can not limit the pool of internal IP addresses and custom-no-google-analytics build doesn’t fit our testing workflow or the way our product is built and deployed.

Is there anything we can do with the test code and, perhaps, a few lines of runtime product code so that our application or website could recognize that it is being used by the automated test runner? Regardless of the environment and client ip.

Of course, we can!

We can add a custom query param to the URL of every page we are testing, i.e.: example.com/?ga_debug_mode=true. Then javascript code of our product will pick up the query param in runtime and disable (or label as developer traffic aka debug mode) all GA events created during that session.

Implementation is a little tricky however, since there is no Playwright configuration field we could leverage to add our custom query param automatically to the URL of every page we test.

Updating every page URL manually in the code doesn’t sound particularly exciting nor future-proof. Also, we don’t want to pollute the list of product URLs we have in our analytics data, in case something doesn’t go as planned.

Rejoice, there is a better way indeed!

Playwright `storageState` and Runtime Debug Mode

Instead of a URL query param, let’s leverage the storageState config file option to populate the localStorage of the every browser instance spun up by the Playwright with the custom ga_debug_mode property:

import type { PlaywrightTestConfig } from '@playwright/test';

const storageState = {
  cookies: [], // not used but required by type definition
  origins: [{
    origin: '*', // doesn't narrow hostname at all. Use with caution!
    localStorage: [
      {
        name: 'ga_debug_mode', // feel free to modify the property name
        value: 'true', // this is a string
      },
    ],
  }],
};

const config: PlaywrightTestConfig = {
  use: {
    baseURL: 'https://example.com/',
    browserName: 'chromium',
    headless: true,
    storageState,
  },
};
ts

Our website or application runtime code will check the localStorage.getItem('ga_debug_mode') value during the initialization phase and — if it is set to 'true' — disable Google Analytics completely or turn the GA debug mode on (more on that below).

We need to turn the reporting off or mark all the future GA events with debug_mode: true event parameter before Google Analytics (or GTM) initialization script runs.

Now we will look into every option we have to disable GA in runtime, based on the way Google Analytics is installed on your website (with or without the Google Tag Manager) and preferred method of filtering out the developer traffic — by skipping GA code altogether or marking GA events with debug_mode parameter.

Skip Google Analytics Initialization in Runtime

You can disable GA or GTM initialization code by wrapping its setup code snippet with the following if statement:

<script>
  if (window.localStorage.getItem('ga_debug_mode') !== 'true') {
    // GTM or GTAG Initialization code
  }
</script>
html

This way there will be no user events reported to Google Analytics engine at all during that particular browsing (in fact — testing) session.

Google Analytics Debug Mode with the Older GTAG Script

This GTAG only method is very similar to the snippet we used for the custom-no-google-analytics testing build. The only difference is that debug_mode event parameter value is conditional and gets set in the runtime:

const gaDebugMode = window.localStorage.getItem('ga_debug_mode') === 'true';

// need to pass undefined instead of false
gtag('config', 'G-XXXXXX', { debug_mode: gaDebugMode || undefined });
js

This way, when ga_debug_mode: 'true' localStorage value is provided by Playwright, all GA events of the testing session will get marked with debug_mode: true event parameter, so that we could filter them out later with the native Developer Traffic GA filter.

Google Analytics Debug Mode with Google Tag Manager

When using Google Tag Manager, Google Analytics Config event is fired by GTM automatically and there is no explicit GA initialization code we can tweak directly.

Instead, we will be adding a user-defined GTM variable which retrieves the ga_debug_mode value from browser localStorage and passes it to the GA4 Config Tag as a Tag Field when instantiating Google Analytics Tag on your website.

Let’s create a user-defined Custom JavaScript GTM variable named GA Debug Mode (JS script) with this snippet:

function gaDebugMode() {
  return window.localStorage.getItem('ga_debug_mode') === 'true' || undefined;
}
js

Here is the screenshot of a complete variable configuration:

Next, we need to add a new debug_mode Tag Field to the GA4 Config Tag and select our GA Debug Mode (JS script) variable as its value:

At this point Google Tag Manager should be passing the debug_mode field read from browser’s localStorage when initializing Google Analytics code. All subsequent user (or, in our case, test) generated GA events will get marked with debug_mode: true event parameter, so that we could filter them out later.

To validate that it works as expected, go to the GTM Preview Mode and open your website (remote or local). Then, click on the Container Loaded event in the left column. You should see the GA4 Config card in Tags Fired group: GA Config GTM Event

Click it to open the Tag Details panel and select Values radio button in the top right corner.

This is how it looks in my setup with debug mode turned off:

Now, open the DevTools JS console in browser with an active GTM debuging session and run the following command to emulate localStorage value populated by Playwright test runner:

localStorage.setItem('ga_debug_mode', 'true');
js

Reload the page, and open GA Config card details panel the same way as we did earlier. This is how it should look now: Note that value for the debug_mode field is now set to true.

Once everything works fine, don’t forget to remove the ga_debug_mode property from the localStorage of your browser with the following one-liner:

localStorage.clearItem('ga_debug_mode');
js

Filter Developer Traffic from Google Analytics Reports

Now, when all GA events created by automated testing runs are marked with debug_mode: true flag, we can filter them out from the GA data collection (and, consequentially, GA reports).

To do so we need to activate the native Developer Traffic filter in Google Analytics. Head to the Admin > Property Settings > Data Settings > Data Filters.

By default, the Developer Traffic filter mode is Testing which doesn’t drop debug/developer traffic GA events, but marks them with Test data filter name dimension, so that you could confirm that it works as expected before permanently removing them from the incoming data stream.

Google Analytics Data Stream Filter Developer Traffic

And that’s it — now we are officially done with this problem!

Conclusion

To avoid polluting Google Analytics data with test run events we need to:

make our product code aware of the fact that it is being used by automated test runner. We have two options — having a separate testing build target, or passing a flag to the website javascript runtime code.
use the flag passed (or custom build) to turn the analytics off, or tag all subsequent GA events with debug_mode flag and filter them out with the Developer Traffic GA filter.

We went through three different ways to pass the testing flag:

dedicated testing build target
debug_mode query param for the URL being tested
ga_debug_mode localStorage property populated by the Playwright test runner

We also looked into differences of turning GA Debug Mode on for the older GTAG script and Google Analytics set up by the Google Tag Manager.

Thanks for reading so far (way too long, I know) and Happy scripting!