Streamline visual regression tests by making CI maintain visual snapshots #324
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
Playwright visual snapshot differences were detected.View the Playwright report to review the visual differences. |
Playwright visual snapshot differences were detected.View the Playwright report to review the visual differences. |
|
/approve-snapshots |
Updating snapshots. Click here to see the status. |
Playwright visual snapshot differences were detected.View the Playwright report to review the visual differences. |
|
/approve-snapshots https://fresco-sandbox-h4e61po4f-network-canvas-f4790d84.vercel.app/ |
Updating snapshots. Click here to see the status. |
Playwright visual snapshot differences were detected.View the Playwright report to review the visual differences. |
|
/approve-snapshots https://fresco-sandbox-9332l2bnh-network-canvas-f4790d84.vercel.app/ |
Updating snapshots. Click here to see the status. |
🎉 Successfully updated and committed Playwright snapshots! 🎉 |
There was a problem hiding this comment.
Pull Request Overview
This PR streamlines visual regression testing by refining how CI handles visual snapshots, ensuring that local and CI snapshots align. Key changes include updating the Playwright configuration with multiple reporters and additional timeouts, modifying E2E tests to toggle between active tests and fixmes for visual snapshot validations, and enhancing the CI workflow to comment on PRs with snapshot reports.
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| playwright.config.ts | Updated reporter configuration and added global timeouts. |
| e2e/protocols.spec.ts | Enabled visual snapshot test with soft expectations and fixed test statuses. |
| e2e/participants.spec.ts | Reordered test statuses for visual snapshots and participant actions. |
| e2e/global.setup.ts | Removed the slow flag, updated timeout formats, and modified click timeout values; commented out a toast assertion. |
| .github/workflows/playwright.yml | Added required permissions and steps to capture and comment PR data on visual snapshot failures. |
Files not reviewed (1)
- package.json: Language not supported
Comments suppressed due to low confidence (1)
e2e/global.setup.ts:142
- The removal of the participants imported toast verification reduces the test coverage for validating a successful import. Consider re-enabling or updating this check to ensure that the import success is properly verified.
// // participants imported toast
This PR implements a streamlined approach to visual regression testing. It fixes an issue where snapshots taken when tests are run locally do not match snapshots taken by CI. This refactors the process so that the CI automatically detects snapshot errors and comments with the report. Then, the update snapshots action implemented in #335 and #334 updates the snapshots on comment.