Operating System Control Architecture
Zaya follows a strict Observe → Identify → Act → Verify execution loop.
Fallback Hierarchy
- Native Control: Win32 process spawn, ShellExecute, window focus.
- Browser DOM: Playwright element locator hierarchy.
- Windows UI Automation (UIA): Direct inspection of accessible controls, buttons, text fields.
- Set-of-Mark (SoM): Overlaying numerical tags onto Candidate UI elements.
- Vision Segmentation: Multimodal vision model bounding box calculation.
- Coordinate Fallback: Direct cursor click with post-action difference verification.