Wdio Image Comparison Service Versions Save

Image comparison / visual regression testing for WebdriverIO

@wdio/[email protected]

3 months ago

🚀 New Features

  • 43ed502: Add font loading features:
    • add waitForFontsLoaded so the module automatically waits for all fonts to be loaded, enabled by default
    • add enableLayoutTesting so all text will become transparent so
      • font rendering issues won't cause flakiness
      • image comparison can be done on the layout

Examples:

On element level

container-local-chrome-latest-1366x768

On Screen level

viewportScreenshot-local-chrome-latest-1366x768

🐛 Bug fix

  • 6e3b0b2: Fix standalone support Standalone/remote now needs to be used like this
import { remote } from 'webdriverio'
import VisualService from '@wdio/visual-service'

const visualService = new VisualService({
    autoSaveBaseline: true
})

const browser = await remote({
    logLevel: 'silent',
    capabilities: {
        browserName: 'firefox',
    }
})

// Add this to "Start" the service
visualService.remoteSetup(browser)

await browser.url('https://webdriver.io/')

await browser.checkScreen('examplePaged', {})

await browser.deleteSession()

Patch Changes

Committers: 1

@wdio/[email protected]

3 months ago

🐛 Bug Fixes

  • 14b6ae6: Two fixes
    • Add support for Browserstack real device names
    • Fix hide/remove elements

Committers: 1

[email protected]

3 months ago

🐛 Bug Fixes

Committers: 1

@wdio/[email protected]

3 months ago

🐛 Bug Fixes

  • Updated dependencies [c8fdcd3] to be able to use the element visibility/display property

Committers: 1

[email protected]

3 months ago

💥 Breaking Changes

  • fd74a35: (feat): set default baseline folder next to test file If you don't provide a baselineFolder in the service options, see below for an example
export const config: Options.Testrunner = {
    //...
    // ========
    // Services
    // ========
    services: [
        // ===================
        // Image compare setup
        // ===================
        [
            'visual',
            {
                //...
                baselineFolder: join(
                    process.cwd(),
                    './path/to/your/baseline/'
                ),
                //...
            },
        ],
    ],
   //...
}

then the default screenshots will be stored in a __snapshots__-folder next to the spec file executed to run the Visual tests. The folder ./wic/baseline/ was at the root of your project. Make sure you adjust your tests to it

🐛 Bug fix

  • e93a878: fix default snapshot path to be overwritten through method/service options
  • Updated dependencies [fd74a35]

Committers: 2

@wdio/[email protected]

3 months ago

💥 Breaking Changes

  • fd74a35: (feat): set default baseline folder next to test file If you don't provide a baselineFolder in the service options, see below for an example
export const config: Options.Testrunner = {
    //...
    // ========
    // Services
    // ========
    services: [
        // ===================
        // Image compare setup
        // ===================
        [
            'visual',
            {
                //...
                baselineFolder: join(
                    process.cwd(),
                    './path/to/your/baseline/'
                ),
                //...
            },
        ],
    ],
   //...
}

then the default screenshots will be stored in a __snapshots__-folder next to the spec file executed to run the Visual tests. The folder ./wic/baseline/ was at the root of your project. Make sure you adjust your tests to it

🐛 Bug fix

  • e93a878: fix default snapshot path to be overwritten through method/service options
  • Updated dependencies [fd74a35]

Committers: 2

[email protected]

3 months ago

💥 Breaking changes (ef386b6):

  • resizeDimensions on the element can now only be an object, it has been deprecated for a while

💅 New Features

  • Next to supporting Web snapshot testing this module now also supports 💥 Native App 💥 snapshot testing. The methods saveElement|checkElement | saveScreen | checkScreen and the matchers toMatchElementSnapshot | toMatchScreenSnapshot are available for Native Apps

[!NOTE] This module will automatically detect the context (web | webview | native_app) and will handle all complex logic for you

The methods saveFullPageScreen | checkFullPageScreen | saveTabbablePage|checkTabbablePage will throw an error when they are used in the native context for native mobile apps and will look like this

$ wdio tests/configs/wdio.local.android.emus.app.conf.ts
Execution of 1 workers started at 2024-01-30T06:18:24.865Z
[0-0] RUNNING in Android - file:///tests/specs/mobile.app.spec.ts
[0-0] Error in "@wdio/visual-service mobile app.should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode"
Error: The method saveFullPageScreen is not supported in native context for native mobile apps!
    at /wdio/visual-testing/packages/webdriver-image-comparison/src/commands/saveFullPageScreen.ts:26:15
    at step (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:33:23)
    at Object.next (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:14:53)
    at /wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:4:12)
    at saveFullPageScreen (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:47:12)
    at Browser.<anonymous> (file:///wdio/visual-testing/packages/service/dist/service.js:101:24)
[0-0] FAILED in Android - file:///tests/specs/mobile.app.spec.ts
    "spec" Reporter:
------------------------------------------------------------------
[/wdio/visual-testing/apps/app.apk Android #0-0] Running: /wdio/visual-testing/apps/app.apk on Android
[/wdio/visual-testing/apps/app.apk Android #0-0] Session ID: c1101184-e3d5-42b5-a31f-8ebaa211f1a1
[/wdio/visual-testing/apps/app.apk Android #0-0]
[/wdio/visual-testing/apps/app.apk Android #0-0] » /tests/specs/mobile.app.spec.ts
[/wdio/visual-testing/apps/app.apk Android #0-0] @wdio/visual-service mobile app
[/wdio/visual-testing/apps/app.apk Android #0-0]    ✖ should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode
[/wdio/visual-testing/apps/app.apk Android #0-0]
[/wdio/visual-testing/apps/app.apk Android #0-0] 1 failing (1.5s)
[/wdio/visual-testing/apps/app.apk Android #0-0]
PORTRAIT-mode
[/wdio/visual-testing/apps/app.apk Android #0-0] The method saveFullPageScreen is not supported in native context for native mobile apps!
[/wdio/visual-testing/apps/app.apk Android #0-0] Error: The method saveFullPageScreen is not supported in native context for native mobile apps!
[/wdio/visual-testing/apps/app.apk Android #0-0]     at /wdio/visual-testing/packages/webdriver-image-comparison/src/commands/saveFullPageScreen.ts:26:15
[/wdio/visual-testing/apps/app.apk Android #0-0]     at step (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:33:23)
js:14:53)
[/wdio/visual-testing/apps/app.apk Android #0-0]     at /wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:8:71
[/wdio/visual-testing/apps/app.apk Android #0-0]     at new Promise (<anonymous>)
js:4:12)
saveFullPageScreen.js:47:12)
[/wdio/visual-testing/apps/app.apk Android #0-0]     at Browser.<anonymous> (file:///wdio/visual-testing/packages/service/dist/service.js:101:24)

Spec Files:      0 passed, 1 failed, 1 total (100% completed) in 00:00:11

error Command failed with exit code 1.
  • autoSaveBaseline is true by default, so if no baseline images are present it will automatically create a new baseline
  • Mobile screenshots of the complete screen now automatically exclude all native OS elements like the notification bar, home bar, address bar, and so on, the settings blockOutSideBar | blockOutStatusBar |blockOutToolBar are now all defaulted to true

🐛 Fixed bugs:

  • element screenshots could also get resized dimensions, which would cut out a bigger portion around the element. This was failing when the dimensions got out of the boundaries of the official screenshot. This has now been fixed with:
    • not going outside of the screenshot
    • log extra warnings

@wdio/[email protected]

3 months ago

💥 Breaking changes (ef386b6):

  • resizeDimensions on the element can now only be an object, it has been deprecated for a while

💅 New Features

  • Next to supporting Web snapshot testing this module now also supports 💥 Native App 💥 snapshot testing. The methods saveElement|checkElement | saveScreen | checkScreen and the matchers toMatchElementSnapshot | toMatchScreenSnapshot are available for Native Apps

[!NOTE] This module will automatically detect the context (web | webview | native_app) and will handle all complex logic for you

The methods saveFullPageScreen | checkFullPageScreen | saveTabbablePage|checkTabbablePage will throw an error when they are used in the native context for native mobile apps and will look like this

$ wdio tests/configs/wdio.local.android.emus.app.conf.ts
Execution of 1 workers started at 2024-01-30T06:18:24.865Z
[0-0] RUNNING in Android - file:///tests/specs/mobile.app.spec.ts
[0-0] Error in "@wdio/visual-service mobile app.should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode"
Error: The method saveFullPageScreen is not supported in native context for native mobile apps!
    at /wdio/visual-testing/packages/webdriver-image-comparison/src/commands/saveFullPageScreen.ts:26:15
    at step (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:33:23)
    at Object.next (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:14:53)
    at /wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:4:12)
    at saveFullPageScreen (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:47:12)
    at Browser.<anonymous> (file:///wdio/visual-testing/packages/service/dist/service.js:101:24)
[0-0] FAILED in Android - file:///tests/specs/mobile.app.spec.ts
    "spec" Reporter:
------------------------------------------------------------------
[/wdio/visual-testing/apps/app.apk Android #0-0] Running: /wdio/visual-testing/apps/app.apk on Android
[/wdio/visual-testing/apps/app.apk Android #0-0] Session ID: c1101184-e3d5-42b5-a31f-8ebaa211f1a1
[/wdio/visual-testing/apps/app.apk Android #0-0]
[/wdio/visual-testing/apps/app.apk Android #0-0] » /tests/specs/mobile.app.spec.ts
[/wdio/visual-testing/apps/app.apk Android #0-0] @wdio/visual-service mobile app
[/wdio/visual-testing/apps/app.apk Android #0-0]    ✖ should compare a screen successful for 'Pixel_7_Pro_Android_14_API_34' in PORTRAIT-mode
[/wdio/visual-testing/apps/app.apk Android #0-0]
[/wdio/visual-testing/apps/app.apk Android #0-0] 1 failing (1.5s)
[/wdio/visual-testing/apps/app.apk Android #0-0]
PORTRAIT-mode
[/wdio/visual-testing/apps/app.apk Android #0-0] The method saveFullPageScreen is not supported in native context for native mobile apps!
[/wdio/visual-testing/apps/app.apk Android #0-0] Error: The method saveFullPageScreen is not supported in native context for native mobile apps!
[/wdio/visual-testing/apps/app.apk Android #0-0]     at /wdio/visual-testing/packages/webdriver-image-comparison/src/commands/saveFullPageScreen.ts:26:15
[/wdio/visual-testing/apps/app.apk Android #0-0]     at step (/wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:33:23)
js:14:53)
[/wdio/visual-testing/apps/app.apk Android #0-0]     at /wdio/visual-testing/packages/webdriver-image-comparison/dist/commands/saveFullPageScreen.js:8:71
[/wdio/visual-testing/apps/app.apk Android #0-0]     at new Promise (<anonymous>)
js:4:12)
saveFullPageScreen.js:47:12)
[/wdio/visual-testing/apps/app.apk Android #0-0]     at Browser.<anonymous> (file:///wdio/visual-testing/packages/service/dist/service.js:101:24)

Spec Files:      0 passed, 1 failed, 1 total (100% completed) in 00:00:11

error Command failed with exit code 1.
  • autoSaveBaseline is true by default, so if no baseline images are present it will automatically create a new baseline
  • Mobile screenshots of the complete screen now automatically exclude all native OS elements like the notification bar, home bar, address bar, and so on, the settings blockOutSideBar | blockOutStatusBar |blockOutToolBar are now all defaulted to true

🐛 Fixed bugs:

  • element screenshots could also get resized dimensions, which would cut out a bigger portion around the element. This was failing when the dimensions got out of the boundaries of the official screenshot. This has now been fixed with:
    • not going outside of the screenshot
    • log extra warnings

@wdio/[email protected]

4 months ago

Major Changes

  • 36d3868: Support for WebdriverIO v8

Minor Changes

  • 36d3868: (feat): add visual matcher

Patch Changes

[email protected]

4 months ago

Major Changes

  • 36d3868: Support for WebdriverIO v8

Patch Changes

  • 488d424: (docs): update readme