Skip to main content

​
What are AI actions?

AI actions are smart test actions that are driven by vision and LLM reasoning, and executed via Playwright. AI actions help the Playmatic agent create reliable tests for complex verifications and dynamic applications.
All AI actions are designed to be bounded by deterministic code (e.g., one click) so that tests can run predictably at runtime
Currently, Playmatic has two AI actions:
  1. aiClick(description, page): Click on an element on the page based on a natural language descriptor.
  2. aiVerify(description, page): Assert a verification based on a natural language descriptor.
These AI actions are imported via the Playmatic SDK and used in the same way as actions in Playwright steps. 
import { aiClick, aiVerify } from '@playmatic/sdk';
import { test } from '@playwright/test';

test('Verify Calendly page booking', async ({ page }) => {
  await test.step('Navigate to Calendly page', async () => {
    await page.goto('https://calendly.com/playmatic/30min?');
  });

  await test.step('Click Book Demo and interact with calendar in popup', async () => {
    // Use AI actions to click and verify
    await aiVerify('calendar or scheduling interface is visible', page);
    await aiClick('an available date slot in the calendar', page);
    await aiVerify('a date is selected or highlighted', page);
  });
});

​
When are AI actions used?

AI actions are only used by Playmatic in test development when the agent determines that existing Playwright selectors cannot create a stable test. The agent always prefers traditional Playwright actions.
AI actions typically take a few seconds to execute. Avoid using them if you can achieve the same result with normal Playwright selectors.
Some examples of when the Playmatic agent will use AI actions: Third-party embeds: Services like Calendly, Typeform, or Stripe checkout that render in iframes or use complex internal structures. Shadow DOMs: Components that encapsulate their DOM structure (common in web components) where standard selectors can’t reach internal elements. Canvas-based interfaces: Visual editors, drawing tools, or games that render content on canvas elements without traditional DOM nodes.

​
Best Practices for Writing Descriptors for AI Actions

Each AI action is driven by the natural language descriptor. When you write the descriptor, keep in mind that the level of specificity that you use will determine the behavior of the action. For example, aiClick("submit button") will be more durable than aiClick("the blue submit button on the right sidebar").
We recommend the following best practices for writing descriptions:
  • Be specific but natural: “Click the submit button” vs “Click the blue button on the right.”
  • Describe the purpose: Reference elements by their purpose and not just their appearance

​
Next Steps

Environment Management

Learn about managing test data and variables with Playmatic.

Running Locally

Learn how to run Playmatic tests locally.
⌘I