Browse documentation

Operating System Control Architecture

Zaya follows a strict Observe → Identify → Act → Verify execution loop.


Fallback Hierarchy

  1. Native Control: Win32 process spawn, ShellExecute, window focus.
  2. Browser DOM: Playwright element locator hierarchy.
  3. Windows UI Automation (UIA): Direct inspection of accessible controls, buttons, text fields.
  4. Set-of-Mark (SoM): Overlaying numerical tags onto Candidate UI elements.
  5. Vision Segmentation: Multimodal vision model bounding box calculation.
  6. Coordinate Fallback: Direct cursor click with post-action difference verification.

Something unclear or not working?

Troubleshooting guide ↗Report a problem ↗