A little possibility, everywhere
Find your facet.
Begin with the everyday. Explore the advanced when you’re ready. Each capability has its own requirements.
Nothing hidden behind the crystal
The complete feature collection.
Browse all 56 catalogued capabilities. Development labels describe components; they do not certify every flow in the public beta. Planned features are not available yet.
stablecross platform
Gemini Live Bidirectional Voice
Full-duplex conversational voice streaming over WebSockets with real-time speech synthesis.
◇ Data route: cloud
Details & requirements +
Streams microphone and response audio through the configured Gemini Live model for conversational voice. Response time depends on the provider and network.
- Bidirectional audio streaming
- Configurable voice settings
- Conversational turns
Requires: Gemini API Key, Microphone, Audio Output
Read the guide ↗stablewindows
Barge-In & Acoustic Interruption
Speak at any point during Zaya’s response to immediately halt queued speech and pivot.
◇ Data route: local
Details & requirements +
Local WebRTC VAD and energy-based speech detection catches owner speech during output playback, truncates speech buffers, and resumes active listening instantly.
- Zero latency cutoff
- Acoustic echo suppression
- No provider round-trip needed
Requires: PyAudio, WebRTC VAD
Read the guide ↗stablewindows
Local Wake-Word Gating & Push-to-Talk
Private on-device acoustic keyword spotting or hardware hotkey trigger.
◇ Data route: local
Details & requirements +
Keeps audio stream offline until the trained wake phrase or physical push-to-talk hotkey is held, preventing unintended ambient listening.
Requires: Local Hotkey Hook, OpenWakeWord Engine
Read the guide ↗stablewindows
Modular Local AI Engine
Split-pipeline local intelligence replacing cloud services with on-device models.
◇ Data route: local
Details & requirements +
Combines local WebRTC VAD, faster-whisper speech transcription, local LLM inference via Ollama / LM Studio / llama.cpp, and onnx Piper voice synthesis.
Requires: faster-whisper, Piper ONNX, Local LLM (Ollama/llama.cpp)
Read the guide ↗stablecross platform
Multi-Key & Multi-Provider Failover
Automatic rate-limit recovery and multi-key rotation on 429 quota exhaustion.
◇ Data route: optional cloud
Details & requirements +
Maintains prioritized key rotation pools and provider fallbacks. When Gemini hits quota limits, Zaya seamlessly shifts to alternate keys or secondary providers without dropping conversation state.
Requires: Configured Keys in api_keys.json
Read the guide ↗stablewindows
Hierarchical Computer Operator
Multi-tiered desktop manipulation following an Observe → Identify → Act → Verify paradigm.
◇ Data route: local
Details & requirements +
Tiered execution engine attempting Native Control first, followed by Browser DOM, Windows UI Automation (UIA), Set-of-Mark visual tagging, vision segmentation, and precise coordinate fallback.
- Never clicks blind coordinates
- Post-action verification check
- Atomic action rollback on failure
Requires: Windows UI Automation, PyAutoGUI / SendInput
Read the guide ↗stablewindows
Application & Window Lifecycle
Open, minimize, focus, tile, and gracefully terminate desktop software.
◇ Data route: local
Details & requirements +
Direct Windows process and ShellExecute integration with title matching, multi-monitor coordinates, and process tree tree-kill safeguards.
Requires: win32gui, psutil
Read the guide ↗betawindows
Floating Desktop Widgets
Create small desktop tools such as clocks, timers and calculators using Widget Forge.
◇ Data route: local
Details & requirements +
Create small desktop tools such as clocks, timers and calculators using Widget Forge.
Requires: Widget runtime
Read the guide ↗betawindows
Companion Expressions & Controls
Use the taskbar character, expressions and Settings & Controls to personalize the desktop experience.
◇ Data route: local
Details & requirements +
Use the taskbar character, expressions and Settings & Controls to personalize the desktop experience.
Requires: Desktop companion
Read the guide ↗betawindows
Runtime Logs & System Diagnostics
Inspect local runtime logs, resource information and diagnostic status when something needs attention.
◇ Data route: local
Details & requirements +
Inspect local runtime logs, resource information and diagnostic status when something needs attention.
Requires: Local desktop session
Read the guide ↗betawindows
Selected Text Assistance
Read selected text and use it as context for your current request.
◇ Data route: local
Details & requirements +
Read selected text and use it as context for your current request.
Requires: Accessible selected text
Read the guide ↗betawindows
Floating Content & Ghost Overlays
Display supported content and helper overlays alongside your desktop work.
◇ Data route: local
Details & requirements +
Display supported content and helper overlays alongside your desktop work.
Requires: Desktop display runtime
Read the guide ↗betawindows
Computer Settings Assistance
Use supported system-settings actions while retaining owner permissions and confirmations.
◇ Data route: local
Details & requirements +
Use supported system-settings actions while retaining owner permissions and confirmations.
Requires: Windows settings access
Read the guide ↗stablecross platform
Hybrid Browser Automation & Recovery
Native browser launching alongside deep Playwright DOM automation.
◇ Data route: local
Details & requirements +
Launches your standard browser (Brave, Chrome, Firefox) for daily links, and creates isolated persistent Playwright contexts for deep programmatic navigation, form completion, and content extraction.
Requires: Playwright Chromium / Firefox
Read the guide ↗betawindows
Web Search & Research
Search the web and gather context through configured search and browser tools.
◇ Data route: optional cloud
Details & requirements +
Search the web and gather context through configured search and browser tools.
Requires: Internet connection, configured providers
Read the guide ↗betawindows
Weather Reports
Request weather information through the configured weather tool.
◇ Data route: optional cloud
Details & requirements +
Request weather information through the configured weather tool.
Requires: Internet connection, weather service
Read the guide ↗betawindows
Flight Search Assistance
Look for flight options through the flight-finder integration. Availability depends on its configured services.
◇ Data route: optional cloud
Details & requirements +
Look for flight options through the flight-finder integration. Availability depends on its configured services.
Requires: Internet connection, flight search integration
Read the guide ↗stablewindows
On-Demand Visual Reasoning
High-resolution desktop or webcam snapshot analysis for contextual reasoning.
◇ Data route: optional cloud
Details & requirements +
Captures targeted window or desktop regions only when explicitly requested by user commands, analyzing UI layout, error popups, code snippets, or architectural diagrams.
Requires: Desktop Capture, Multimodal LLM / Gemini Vision
Read the guide ↗Perception
01 capabilitiesstablewindows
Continuous Local Screen Perception
Zero-cloud screen awareness monitoring active applications and UI region changes.
◇ Data route: local
Details & requirements +
Performs differential frame hashing, bounding-box motion tracking, and active window inspection locally at low CPU overhead. No live video stream is ever transmitted to cloud providers.
- Completely private on-device
- Diff-based low overhead
- Selective OCR streaming
Requires: mss, OpenCV Contrib
Read the guide ↗betawindows
File & Folder Management
Find, read, create, move and organize local files and folders with owner-controlled actions.
◇ Data route: optional cloud
Details & requirements +
Find, read, create, move and organize local files and folders with owner-controlled actions.
Requires: Filesystem access
Read the guide ↗betawindows
Document Reading & Generation
Work with supported documents and generate files for your tasks. Format support depends on installed libraries.
◇ Data route: optional cloud
Details & requirements +
Work with supported documents and generate files for your tasks. Format support depends on installed libraries.
Requires: Document libraries
Read the guide ↗betawindows
Owner-Controlled Terminal
Run terminal tasks in a working directory with the real execution policy and confirmation gate.
◇ Data route: local
Details & requirements +
Run terminal tasks in a working directory with the real execution policy and confirmation gate.
Requires: System shell
Read the guide ↗stablecross platform
Verified Git Workflow Controller
Repository inspection, branch creation, commit signing, and verified push operations.
◇ Data route: local
Details & requirements +
Validates repository clean states, diff boundaries, author identity checks (sajancodes), and runs status checks before executing pushes to origin.
Requires: Git CLI
Read the guide ↗stablecross platform
Supervised OpenCode Orchestrator
Multi-pass coding assistant with structured briefs, tool-free review, and independent test verification.
◇ Data route: local
Details & requirements +
Compiles owner tasks into technical briefs, manages hidden OpenCode CLI worker processes, tracks git diffs, executes test suites, and enforces a strict three-pass correction ceiling.
Requires: OpenCode CLI, Git, Python / Node
Read the guide ↗betawindows
Code Search & Helpers
Search project code and use configured code-generation helpers alongside supervised OpenCode work.
◇ Data route: local
Details & requirements +
Search project code and use configured code-generation helpers alongside supervised OpenCode work.
Requires: Project access, configured AI provider
Read the guide ↗stablewindows
Mission Supervisor & Durable State Engine
Long-running autonomous execution with bounded retries, proof-of-work, and restart recovery.
◇ Data route: local
Details & requirements +
Orchestrates complex multi-step user goals through an explicit 10-state state machine: planning, running, waiting confirmation, waiting clarification, waiting external, paused, interrupted, completed, completed unverified, failed, cancelled.
Requires: SQLite Mission Store
Read the guide ↗betawindows
Reminders & Scheduled Calls
Schedule reminders and request a phone call at a specified time. Phone delivery needs a paired, reachable device.
◇ Data route: optional cloud
Details & requirements +
Schedule reminders and request a phone call at a specified time. Phone delivery needs a paired, reachable device.
Requires: Reminder scheduler, Android pairing for calls
Read the guide ↗betawindows
Calendar & Meeting Scheduling
Query schedules and create or reschedule meetings through the configured calendar integration.
◇ Data route: optional cloud
Details & requirements +
Query schedules and create or reschedule meetings through the configured calendar integration.
Requires: Calendar connection
Read the guide ↗betawindows
Background Task Monitoring
Follow ongoing work and inspect runtime status, progress and failures.
◇ Data route: optional cloud
Details & requirements +
Follow ongoing work and inspect runtime status, progress and failures.
Requires: Running desktop session
Read the guide ↗betawindows
Proactive Owner Notices
Configure supported proactive notices and background checks for the owner, with delivery depending on the active session.
◇ Data route: optional cloud
Details & requirements +
Configure supported proactive notices and background checks for the owner, with delivery depending on the active session.
Requires: Desktop session, configured monitoring
Read the guide ↗stablewindows
Structured SQLite & FTS5 Knowledge Store
Continuous local conversation archive with semantic BM25 indexing and alias resolution.
◇ Data route: local
Details & requirements +
Stores private dialogue history in conversation.db. Fresh sessions start clean with zero context bloat; recalls at most five relevant messages on demand via indexed SQLite FTS5 queries.
Requires: SQLite3 with FTS5
Read the guide ↗betawindows
Personal Profile & Preferences
Save supported personal preferences and profile information for your assistant.
◇ Data route: local
Details & requirements +
Save supported personal preferences and profile information for your assistant.
Requires: Local profile store
Read the guide ↗betawindows
On-Demand Conversation History
Browse local conversation archives and request targeted recall without loading a previous session on startup.
◇ Data route: local
Details & requirements +
Browse local conversation archives and request targeted recall without loading a previous session on startup.
Requires: Local conversation archive
Read the guide ↗Hand Control
01 capabilitiesstablewindows
Optical Hand Landmark Tracking
Real-time 21-joint 3D hand skeleton inference running in an isolated sub-process.
◇ Data route: local
Details & requirements +
Uses Google MediaPipe Hand Landmarker model in an isolated process (.venv-hands) running at 30 FPS. Evaluates gestures: open palm (cursor), index poke (click), two fingers (selection), fist (scroll), thumbs up (Enter).
Requires: MediaPipe 0.10, OpenCV, Webcam
Read the guide ↗Virtual Keyboard
01 capabilitiesstablewindows
Depth-Invariant Air Keyboard
Floating mid-air typing interface with palm-relative depth adaptation and debouncing.
◇ Data route: local
Details & requirements +
Projects a translucent cyan keyboard plane. Automatically normalizes keystroke depth based on hand scale from the camera, enforcing single-strike latching and refusing to inject keystrokes into credential windows.
Requires: Ghost Keyboard Engine, SendInput
Read the guide ↗stablewindows
3D Spatial Intelligence & CAD Workspace
Interactive WebGPU/WebGL 3D environment with GLB/glTF streaming and natural language planning.
◇ Data route: local
Details & requirements +
Babylon.js powered spatial canvas supporting progressive geometry streaming, exploded component views, X-ray inspection, semantic part labeling, and natural language scene transformations.
Requires: Babylon.js, Vite / WebGPU / WebGL
Read the guide ↗stablecross platform
WhatsApp Hermes Linked Gateway
Persistent multi-device WhatsApp link supporting Observe, Draft, and Auto modes.
◇ Data route: local
Details & requirements +
Maintains a linked-device WhatsApp session via Hermes bridge. Supports per-contact mode assignment, urgent vocal escalation, and strict activity privacy boundaries (shares current song/task, never private chat history).
Requires: Hermes Gateway, Node.js
Read the guide ↗stableandroid
Native Android Companion (Kotlin / Jetpack Compose)
A mobile connection to your laptop’s running Zaya instance.
◇ Data route: local
Details & requirements +
Native Kotlin app mirroring the companion state, live transcripts, active mission progress, WhatsApp messages, and file exchange over an encrypted local WebSocket bridge.
Requires: Android 10+ (API 29+), Encrypted Keystore
Read the guide ↗stableandroid
Zaya-Initiated Calls & Live Video Context
Incoming phone rings for urgent owner alerts and opt-in camera sharing.
◇ Data route: local
Details & requirements +
Zaya can proactively place priority calls to the owner phone for urgent alerts or completed tasks. During an active call, camera frames stream to the laptop for spatial visual reasoning.
Requires: Camera2 API, Android Telecom Framework
Read the guide ↗betaandroid
Away & Presence Preferences
Tell Zaya when you are going out or back so phone-call delivery can use your stated availability.
◇ Data route: local
Details & requirements +
Tell Zaya when you are going out or back so phone-call delivery can use your stated availability.
Requires: Owner session, paired phone for delivery
Read the guide ↗Phone Video
01 capabilitiesbetaandroid
Opt-In Phone Camera Context
Share phone camera frames during a call for visual context. Quality and latency depend on the device and connection.
◇ Data route: optional cloud
Details & requirements +
Share phone camera frames during a call for visual context. Quality and latency depend on the device and connection.
Requires: Paired Android companion, camera permission
Read the guide ↗betawindows
Desktop Music & Media Controls
Control playback, track changes, mute and volume through Windows media controls.
◇ Data route: local
Details & requirements +
Control playback, track changes, mute and volume through Windows media controls.
Requires: Compatible media player, Windows
Read the guide ↗betawindows
Video & YouTube Playback
Open videos and work with supported playback controls using your desktop and browser.
◇ Data route: local
Details & requirements +
Open videos and work with supported playback controls using your desktop and browser.
Requires: Browser or supported player
Read the guide ↗stablewindows
System Audio Ducking
Automatically attenuates background desktop sound during active speech.
◇ Data route: local
Details & requirements +
Windows CoreAudio session mixer drops background application audio to 20% when either owner or Zaya speaks, smoothly ramping back to full volume after release.
Requires: Windows CoreAudio / pycaw
Read the guide ↗betawindows
Double-Clap Toggle
Use the optional microphone-based double-clap control where configured.
◇ Data route: local
Details & requirements +
Use the optional microphone-based double-clap control where configured.
Requires: Microphone, enabled clap detection
Read the guide ↗betawindows
Plugin & Tool Extensions
Extend the desktop through configured plugins and discovered action tools. Additional integrations need their own setup.
◇ Data route: optional cloud
Details & requirements +
Extend the desktop through configured plugins and discovered action tools. Additional integrations need their own setup.
Requires: Plugin configuration
Read the guide ↗stablecross platform
Owner Trust Boundary & ToolResult Architecture
Owner-only authority model treating external content as untrusted data.
◇ Data route: local
Details & requirements +
Strict privilege isolation: Web content, OCR text, WhatsApp messages, and tool outputs are strictly classified as DATA, NOT AUTHORITY. Destructive operations require on-screen owner confirmation.
Requires: Windows DPAPI, ExecutionPolicy
Read the guide ↗stablewindows
Windows DPAPI Encrypted Vault
Windows user-account encryption for stored provider credentials.
◇ Data route: local
Details & requirements +
Uses Windows CryptProtectData to protect stored provider credentials under the Windows user account. This does not protect against a compromised, signed-in user session.
Requires: Windows Crypt32
Read the guide ↗Remote Control
05 capabilitiesbetaandroid
Phone Contacts & Call Actions
Look up contacts and request supported calling actions on your paired phone.
◇ Data route: local
Details & requirements +
Look up contacts and request supported calling actions on your paired phone.
Requires: Android permissions, paired companion
Read the guide ↗betaandroid
Phone Notifications & SMS
Access supported phone notifications and messaging actions after granting the required Android permissions.
◇ Data route: local
Details & requirements +
Access supported phone notifications and messaging actions after granting the required Android permissions.
Requires: Notification access or SMS permissions
Read the guide ↗betaandroid
Phone Device & App Controls
Request supported app launches, media actions, flashlight controls and device status from your laptop.
◇ Data route: local
Details & requirements +
Request supported app launches, media actions, flashlight controls and device status from your laptop.
Requires: Paired Android companion, device permissions
Read the guide ↗betaandroid
Phone Screen Sharing
Share a phone screen capture for visual assistance with explicit Android screen-capture consent.
◇ Data route: local
Details & requirements +
Share a phone screen capture for visual assistance with explicit Android screen-capture consent.
Requires: MediaProjection consent, vision provider
Read the guide ↗betawindows
Owner-Requested Port Tunnels
Manage supported port tunnels for explicitly requested remote access. Exposing a service requires owner approval.
◇ Data route: local
Details & requirements +
Manage supported port tunnels for explicitly requested remote access. Exposing a service requires owner approval.
Requires: Tunnel provider, owner authorization
Read the guide ↗Distribution
01 capabilitiesbetawindows
Windows Beta Installer
Install the published Windows beta using the official setup file and screenshot guide. The beta is unsigned.
◇ Data route: local
Details & requirements +
Install the published Windows beta using the official setup file and screenshot guide. The beta is unsigned.
Requires: Windows 10/11 64-bit
Read the guide ↗betawindows
Configured Game Update Assistance
Use the game-update tool for supported, configured update workflows.
◇ Data route: optional cloud
Details & requirements +
Use the game-update tool for supported, configured update workflows.
Requires: Supported game configuration, internet connection
Read the guide ↗betawindows
Official Release & Update Checks
Find official releases and check update availability. Unsigned desktop betas require manual installation; automatic installation accepts signed releases only.
◇ Data route: optional cloud
Details & requirements +
Find official releases and check update availability. Unsigned desktop betas require manual installation; automatic installation accepts signed releases only.
Requires: Official release channel, internet connection
Read the guide ↗◇Your first hello
Bring a little ice
to your everyday.
Start with the Windows beta. Choose your provider, set your permissions and make Zaya your own.
Get the Windows beta ↗