Skip to main content

​
Prerequisites

Before setting up CI integration, ensure you have:
  • A GitHub repository with Playmatic tests
  • Playmatic CLI installed: npm install -g playmatic
  • API access: Ensure you’re authenticated with Playmatic

​
Setting up GitHub Actions

Playmatic provides an automated setup command that configures your entire GitHub Actions workflow.

​
Automatic Setup

Run the CI setup command in your project root: 
playmatic ci-setup
This command will:
  1. Detect your repository: Automatically identifies your GitHub repository
  2. Create CI API key: Generates a dedicated API key for your repository
  3. Generate workflow file: Creates .github/workflows/playmatic.yml with optimal settings
  4. Guide secret setup: Provides instructions for adding the API key to GitHub Secrets
The automatically generated playmatic.yml file has two sections that you can configure for the workflow. Workflow triggers for deciding when you want to run the Playmatic tests in the CI, and Workflow settings for test running settings.
playmatic.yml
name: Playmatic E2E Tests

on: # Workflow Triggers
  push:
    branches: [ main ]

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      ...
        with: # Workflow Settings
          api-key: ${{ secrets.PLAYMATIC_API_KEY }}
          environment: 'preview'  
          test-paths: 'playmatic-tests'  
          base-url-override: ${{ github.event.deployment.url }}

​
Workflow Triggers

​
Running on PR merged

The generated workflow automatically runs tests when code is merged to your main branches. This ensures that your main branch always has passing tests after changes are pushed.
playmatic.yml
on:
  push:
    branches: [ main ]

​
Running on new PR

Tests can also run automatically on new pull requests to catch issues before merging. This provides immediate feedback to developers and prevents broken code from reaching your main branch.
playmatic.yml
on:
  pull_request:
    branches: [ main ]
In order for Playmatic to run tests on new PRs, you need a preview environment. For preview deployments (Vercel, Netlify, etc.), uncomment and configure the base URL override:
playmatic.yml
- name: Run Playmatic tests
  uses: playmaticai/playmatic-action@v1
  with:
    api-key: ${{ secrets.PLAYMATIC_API_KEY }}
    environment: 'preview'
    base-url-override: ${{ github.event.deployment.url }} #dynamic deployment URL
Ensure you are passing in the dynamic preview URL via base-url-override, otherwise Playmatic will not be able to test the changes.

​
Running on schedule

To run tests on a regular schedule (for monitoring production environments), add a cron schedule to your workflow:
playmatic.yml
on:
  schedule:
    # Run every day at 6 AM UTC
    - cron: '0 6 * * *'
    # Run every Monday at 9 AM UTC
    - cron: '0 9 * * 1'
Common cron patterns:
  • 0 0 * * * - Daily at midnight UTC
  • 0 */6 * * * - Every 6 hours
  • 0 9 * * 1-5 - Weekdays at 9 AM UTC
  • 0 0 1 * * - First day of every month
Use crontab.guru to help build and verify your cron expressions.

​
Workflow Settings

The generated playmatic.yml file includes these key settings:
  • environment: Specify which test environment from your playmatic.config.ts to target
  • test-paths: Directory containing your Playmatic tests (default: playmatic-tests)
  • base-url-override: Override the base URL for preview deployments
playmatic.yml
name: Playmatic E2E Tests

on:
  ...

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      ...
        with: 
          api-key: ${{ secrets.PLAYMATIC_API_KEY }}
          environment: 'staging'  # Change to match your test environment
          test-paths: 'playmatic-tests'  # Change if tests are elsewhere
          base-url-override: ${{ github.event.deployment.url }}  #Change to match your deployment action

​
Test Execution Summary

After each CI run, you can view a detailed summary report in the GitHub Actions logs with test results, execution times, and any self-healing that occurred: 
📊 Summary
├── Total: 2 tests
├── Passed: 1 ✅
├── Failed: 1 ❌
├── Duration: 1m 56s
└── Self-healed: 1 steps

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📝 Test Details

❌ analytics.spec.ts (1m 17s)
   Failed step: "Navigate to Analytics page from sidebar"
   Code cache failure reason:
         page.getByRole: Unable to locate
   Computer Use failure reason:
        I was unable to find the analytics tab on the side bar, suggesting the feature has been removed or hidden.

✅ test-editing-fast.spec.ts (1m 52s) - 1 step self-healed
   Self-healed step:
   └─ "Navigate to login page"
      └─ Code cache failure reason:
           page.goto: Timeout 30000ms exceeded.
           Call log:
             - navigating to "https://petite-crews-run.loca.lt/login", waiting until "load"

​
Notifications

Configure test result notifications through the Playmatic dashboard:
  1. Navigate to Alerts: In your team dashboard, go to the Alerts page
  2. Configure notification preferences:
    • All results: Get notified for every test run
    • Only issues: Get notified only when tests fail
    • Off: Disable all notifications

​
Slack Integration

Connect your Slack workspace to receive notifications:
  1. Click Connect to Slack in the Alerts settings
  2. Authorize the Playmatic app in your workspace
  3. Use /invite @Playmatic in channels where you want notifications

​
Email Notifications

Add additional email recipients beyond the default user who triggered the run:
  1. Enter email addresses in the Email Alerts section
  2. Configure when to receive alerts (all results, only issues, or off)

​
Troubleshooting

​
Common Issues

API Key not working: Ensure the PLAYMATIC_API_KEY secret is correctly added to your repository settings. Tests not running: Check that your workflow file is in .github/workflows/ and committed to your repository. Environment mismatch: Verify the environment setting in your workflow matches a test environment defined in your playmatic.config.ts. Dependency installation fails: Update the install command in your workflow to match your project’s setup (npm, pnpm, yarn, etc.).

​
Next Steps

Configuration

Configure environments and runtime settings

Running locally

Learn how to run tests on your local machine