I am trying to use Accessibility plugin that es with Protractor. From what I see it does checking for a11y of last page that I am located. Is there a way to have 2 test scripts executed one after another one and provide different reports or put all in one report but separated.

Example:

access.js

access1.js

Output file:

resultJsonOutputFile: 'result/result.json'

I tried to this way in conf.js:

specs: ['../test/access.js', '../test/access1.js'],
or
specs: ['../test/access*.js'],

but still get result for last script executed

I tried also creating suites:

suites: {
   homepage: '../test/homepage/access.js',
   catalogpage: '../test/catalogpage/access1.js'
  },

but when I check JSON file, if 2 scripts executed, then 1st one is ok with no issues and provides error for 2nd script. However, if to run 1st script alone, Protractor provides errors

Also I tried to create in one js file as different scenarios, but still same issue

I am trying to use Accessibility plugin that es with Protractor. From what I see it does checking for a11y of last page that I am located. Is there a way to have 2 test scripts executed one after another one and provide different reports or put all in one report but separated.

Example:

access.js

access1.js

Output file:

resultJsonOutputFile: 'result/result.json'

I tried to this way in conf.js:

specs: ['../test/access.js', '../test/access1.js'],
or
specs: ['../test/access*.js'],

but still get result for last script executed

I tried also creating suites:

suites: {
   homepage: '../test/homepage/access.js',
   catalogpage: '../test/catalogpage/access1.js'
  },

but when I check JSON file, if 2 scripts executed, then 1st one is ok with no issues and provides error for 2nd script. However, if to run 1st script alone, Protractor provides errors

Also I tried to create in one js file as different scenarios, but still same issue

• javascript
• angularjs
• protractor
• accessibility

4 Answers 4

With the current implementation, the accessibility plugin is set to run exactly once per invocation of the Protractor runner, on the last page. So unfortunately, no modification of the suites or test files will make it run more than once.

You can create separate configuration files for each set of test files you'd like to run, or using shardTestFiles to make sure that each file is run in its own process. See the referenceConf for more details on sharding.

Alternatively, you could use aXe to do your accessibility testing. In order to use it with e2e tests in protractor and Webdriver, do the following:

npm install --save-dev axe-webdriverjs

Then in your e2e test files, you do:

var AxeBuilder = require('path_to_the/axe-webdriverjs');

to get hold of the AxeBuilder and then wherever you need to run a test, you:

AxeBuilder(browser.driver)
    .analyze(function (results) {
        expect(results.violations.length).toBe(0);
    });

The above example is using Jasmine but you can extrapolate for any other assertion library.

Also: there is a sample project you can clone and run here https://github./dylanb/UITestingFramework

Disclaimer: I am associated with the aXe project and therefore not neutral

I ran into that problem too - as another poster stays the plugin isn't really designed to operate that way.

I wrote a derivative of that plugin which does what you're looking for - protractor-axe-report-plugin.

You make a call to runAxeTest (or runAxeTestWithSelector) whenever you have a page open in the browser that you want to test, and it generates reports using the aXe engine.

Continuum can be used for your use case where it seems the accessibility plugin that es with Protractor cannot. Here's some documentation on a Protractor-based sample project that uses Continuum. It can be downloaded from webaccessibility. under 'Continuum for Protractor'. If you look at the source code of the sample project, it basically just boils down to this:

const continuum = require('../js/Continuum.js').Continuum;
continuum.setUp(driver, "../js/AccessEngine.munity.js");
continuum.runAllTests().then(() => {
    const accessibilityConcerns = continuum.getAccessibilityConcerns();
    // accessibilityConcerns.length will be 0 if no accessibility concerns are found
});

(For more information on the above, you can check out the API documentation.)

You can execute that continuum.runAllTests bit wherever in your tests that you like. That includes multiple times from within the same test too, if desired, which if I understand correctly is ultimately what you're after.

Of course, no automated accessibility testing tool is a replacement for manual accessibility testing. It seems like you're just looking to get a baseline level of pliance right now though, so Continuum seems appropriate for your use case to tackle the low-hanging fruit.