SvelteKit Accessibility Testing: Automated CI A11y Tests # SvelteKit Accessibility Testing: Automated CI A11y Tests #

🧱 What we’re Building #

In this post, we’ll see how you can use Playwright to run integration tests as you develop or each time you build your site. Playwright is an automated testing tool like Cypress. We just use it for accessibility testing here, but you can also use it for more complete end-to-end and integration testing. On top, we level up the testing using the pa11y tool, running axe and htmlcs tests. Then we add polish with Husky, merging the tests into your continuous integration process. If that sounds like something you can get excited about, then why don’t we get cracking?

⚙️ SvelteKit Accessibility Testing: Setup #

We will run through using the SvelteKit MDsveX starter  but if you already have a SvelteKit site, it might make more sense for you to follow along but working on a test branch of your own site. If you are using the starter, let’s get going by cloning it locally:

    
git clone https://github.com/rodneylab/sveltekit-blog-mdx.git sveltekit-accessibility-testing
cd sveltekit-accessibility-testing
pnpm install

  

Next we’ll install the extra packages needed (do this whether you're working on the starter or your own site):

    
pnpm add -D @playwright/test pa11y pa11y-ci playwright-core
pnpm exec playwright install

  

Playwright can run tests in various browsers, and the second step above installs all the browser packages Playwright needs. This can take a few minutes, depending on your internet connection, so you might want to brew a coffee once you start that step running. You will also need to rerun that step each time you update Playwright.

playwright.config.js #

Once you have all of that installed, let's move on to adding a Playwright config file. Create playwright.config.js in your project’s root folder and add the following content:

    
1 import { devices } from '@playwright/test';
2 
3 /** @type {import('@playwright/test').PlaywrightTestConfig} */
4 const config = {
5   forbidOnly: !!process.env.CI,
6   retries: process.env.CI ? 2 : 0,
7   use: {
8     trace: 'on-first-retry',
9   },
10   projects: [
11     {
12       name: 'chromium',
13       use: { ...devices['Desktop Chrome'] },
14     },
15     {
16       name: 'firefox',
17       use: { ...devices['Desktop Firefox'] },
18     },
19     {
20       name: 'webkit',
21       use: { ...devices['Desktop Safari'] },
22     },
23   ],
24   // webServer: {
25   //   command: 'pnpm run build && pnpm run preview',
26   //   port: 5173,
27   // },
28 };
29 export default config;

  

There are a couple of ways you can run the tests. If you want to run tests with a dev server spinning in the background, then the config above, as it is, is what you need. Alternatively, once you’re ready to commit, you can stop your dev server and have Playwright automatically build and test the site. In this case, uncomment lines 24 – 27 (also change the port in line 26 to suit your needs).

The final piece of setup is to add an environment variable to make things a little more configurable. If you are working on your own site, you will need to install the dotenv package. Either way, update the .env file to include the following line:

    
SITE_TEST_URL="http://localhost:3000"

  

Once again, change the port if needed.

🏡 Home Page Test #

We will place all of our test files in a new tests folder in the project root. Create a pages folder within tests then add an index.spec.mjs file (if you prefer TypeScript create index.spec.ts). Add the following content:

    
1 import { expect, test } from '@playwright/test';
2 import 'dotenv/config';
3 import pa11y from 'pa11y';
4 
5 const path = '/';
6 const url = `${process.env.SITE_TEST_URL}${path}`;
7 
8 test.describe('it has no accessbility issues', () => {
9   test.beforeEach(async ({ page }) => {
10     await page.goto(url);
11   });
12   test('it has no issues identified by htmlcs or axe', async () => {
13     const results = await pa11y(url, {
14       runners: ['axe', 'htmlcs'],
15     });
16     if (results.issues) {
17       results.issues.forEach(({ code, message, runner, selector }) => {
18         console.log({ code, message, runner, selector });
19       });
20     }
21     expect(results.issues.length).toBe(0);
22   });
23 });

  

This is a basic Playwright test for our home page. It does not look too different to test files from other testing tools, so we won't spend too much time on it. If there is anything that needs clarification, please drop a comment below, so I can update the post.

In line 14 you see we run both axe and HTML Code Sniffer tests. Together these two runners will find 35% of accessibility issues  . I have noticed that the axe tests can fail on contrast ratio when you use background gradients (even when manual testing shows contrast ratios are good). You will probably want to tweak configuration  on such pages, if you run into this issue.

You can test the home page now, from the Terminal:

    
pnpm exec pa11y http://localhost:5173/

  

We will look at some scripts and integrated testing in a moment. Before that, try breaking accessibility to reassure yourself the tests work. As an example, you could temporarily remove the title in the HTML head. In the starter, this is injected in src/lib/components/SEO/index.svelte. Just comment out the <title>{pageTitle}</title> at line 27 to test.

You can create a similar test file, to the one above, for other pages. In the starter, we have a contact page, so copy the contents above to tests/pages/contact.spec.mjs and just update line 4 in the new file:

    
1 import { expect, test } from '@playwright/test';
2 import pa11y from 'pa11y';
3 
4 const path = '/contact';
5 const url = `${process.env.SITE_TEST_URL}${path}`;

  

Next, let’s see what we can do for blog posts.

🖋 SvelteKit Blog Post Accessibility Testing #

We will create a single test file for all blog posts, so you can see how to parameterize Playwright tests. This pattern will save you creating new test files as you write fresh blog posts. Make a new folder: tests/posts and in there create index.spec.mjs:

    
1 import { expect, test } from '@playwright/test';
2 import 'dotenv/config';
3 import { existsSync, lstatSync, readdirSync } from 'node:fs';
4 import { join, resolve } from 'node:path';
5 import pa11y from 'pa11y';
6 
7 export const BLOG_PATH = 'src/content/blog';
8 const __dirname = resolve();
9 export function getSlugs(location) {
10   const directories = readdirSync(location).filter((element) =>
11     lstatSync(`${location}/${element}`).isDirectory(),
12   );
13   const articles = [];
14 
15   directories.forEach((element) => {
16     const contentPath = `${location}/${element}/index.md`;
17     if (existsSync(contentPath)) {
18       articles.push(element);
19     }
20   });
21   return articles;
22 }
23 
24 const postLocation = join(__dirname, BLOG_PATH);
25 const slugs = getSlugs(postLocation);
26 
27 test.describe('it has no accessbility issues', () => {
28   for (const slug of slugs) {
29     const url = `${process.env.SITE_TEST_URL}/${slug}`;
30     test.beforeEach(async ({ page }) => {
31       await page.goto(url);
32     });
33     test(`${slug}: it has no issues identified by htmlcs or axe`, async () => {
34       const results = await pa11y(url, {
35         runners: ['axe', 'htmlcs'],
36       });
37       if (results.issues) {
38         results.issues.forEach(({ code, message, runner, selector }) => {
39           console.log({ code, message, runner, selector, url });
40         });
41       }
42       expect(results.issues.length).toBe(0);
43     });
44   }
45 });

  

Here at the top, we have a function to generate a list of all the slugs for blog posts. You might need to adapt this to work on your own site. Then in line 28 you see the for loop which runs the tests for each slug. If you update this code for your own site, be cognisant that the existing for loop contains async code, so you will need to be careful if you opt for using the forEach array method. In the test name at line 33 we include the slug parameter to give each test a unique name.

Try testing a blog post or two. Next we will look at some integration scripts. Add more test files if you have different types of pages in your own site.

Husky Continuous Integration Scripts #

We are almost done. Tinker with the following steps to suit your own taste. First, add two new testing scripts to package.json:

    
1 {
2   "name": "sveltekit-blog-mdx",
3   "version": "2.0.0",
4   "scripts": {
5     // ...TRUNCATED
6     "test:a11y": "pa11y-ci --sitemap 'http://localhost:5173/sitemap.xml' --sitemap-find 'https://example.com' --sitemap-replace 'http://localhost:5173'",
7     "test": "playwright test"
8   }, // TRUNCATED...
9 }

  

Update the ports and URLs here depending on your site and local setup. I would run the test script with each commit, to check my work as I go along (or even between commits, depending on what I am working on). This will run all the test files in the tests folder. Then, I would double-check before pushing, which involves running test:a11y on all pages in the sitemap. I would build the site and run it in preview mode, making sure the sitemap is up-to-date first. With the starter, you can update the sitemap by running pnpm run generate:sitemap.

With those scripts defined, let’s add Husky scripts. These will run when we commit and push the code. If you are running on your own site, install husky to get this going.

When you are ready, update .husky/pre-commit and .husky/pre-push:

    
1 #!/bin/sh
2 . "$(dirname "$0")/_/husky.sh"
3 
4 pnpm run prettier:check && pnpm run lint:scss && pnpm run test

  

    
1 #!/bin/sh
2 . "$(dirname "$0")/_/husky.sh"
3 
4 pnpm run check && pnpm run test:a11y

  

Then finally to complete setup in the Terminal run:

    
pnpm exec husky install

  

That’s it. I hope it has all worked well for you. Let me know if you had any issues or if anything needs more clarification.

🙌🏽 SvelteKit Accessibility Testing: Wrapping Up #

In this post, we have seen a nice way of adding automated a11y testing to your Svelte project. In particular, we have seen:

• how to configure Playwright with SvelteKit;
• a way to use pa11y with axe and htmlcs tests in Playwright; and
• how to parameterize Playwright tests.

The full code for the app is available in the repo on Rodney Lab GitHub  .

I hope you found this article useful and am keen to hear how you plan to use the new tests in your Svelte projects.

🏁 SvelteKit Accessibility Testing: Summary #

Can you add automated accessibility test to your app's continuous integration process? #

Yes. It is possible to configure a testing tooling called Playwright to run automated accessibility testing. Playwright is typically used for running end-to-end and integration testing, though combining it with specialized accessibility tools, you can use it to keep your site accessible. In particular, we have seen how you can run the well-know axe tests with help from the pa11y accessibility testing tool. You can also run htmlcs tests with this setup.

Does Playwright work with SvelteKit? #

Yes, Playwright plays well with SvelteKit. Once you have installed the Playwright testing package, you just need a single configuration file to start writing the tests themselves. Playwright is able to test your app, running various browsers including Chrome, Firefox, and Safari. Tests run in parallel if you have multiple process cores on your machine, speeding up the feedback cycle.

How can you run automated axe tests in SvelteKit? #

You can run automated axe accessibility tests in SvelteKit using Playwright and pa11y. Axe is probably the best known accessibility testing framework, and it is not too much effort to work it into your SvelteKit continuous integration process. You can set tests to run each time you commit some code in Git, as well as each time you push code to your repo. The Husky tool helps with this integration.

🙏🏽 SvelteKit Accessibility Testing: Feedback #

Have you found the post useful? Would you prefer to see posts on another topic instead? Get in touch with ideas for new posts. Also, if you like my writing style, get in touch if I can write some posts for your company site on a consultancy basis. Read on to find ways to get in touch, further below. If you want to support posts similar to this one and can spare a few dollars, euros or pounds, please consider supporting me through Buy me a Coffee.

Rodney

@askRodney

Ideally you’ll get an a11y professional to audit your ❤️ SvelteKit app. This is not always an option though and automated a11y testing tools can help you out. You can:

- add tests to you CI process with Husky,

- run pa11y tests with Playwright,

- catch about 35% of issues.

pic.twitter.com/4DH1gZJcGp

— Rodney (@askRodney) March 23, 2022

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on Twitter and also askRodney on Telegram  . Also, see further ways to get in touch with Rodney Lab. I post regularly on Astro as well as SvelteKit. Also, subscribe to the newsletter to keep up-to-date with our latest projects.