Jarvis Personal Manager

A local, voice-driven personal assistant that handles your day across system control, development work, school scheduling, content creation, and business/venture tracking. Say "Hey Jarvis" to wake it, talk naturally in the browser HUD that opens, and it routes your request to the right domain, uses that domain's tools, and speaks back to you.

Runs entirely on your own machine except for the LLM call itself (Google Gemini's free API tier — no usage-based billing) and ElevenLabs for a richer voice (optional, free tier: ~10k characters/month).

Architecture: Domain-Routed Tool Calling

Instead of a flat list of 37+ tools (the old Jarvis approach), Jarvis 2.0 uses a lightweight two-hop flow:

1. Router: A quick Gemini call classifies your utterance into one of six domains — system (volume, lock/sleep, app launching, screen vision), dev (Claude Code session orchestration), school (schedule/assignment management), content (script drafting, video clip generation), business (venture/project/client tracking), or general (chit-chat / anything that doesn't need tools).
2. Domain handler: A second Gemini call runs with only that domain's tools and system-prompt section, executes any tool calls, and produces the spoken response.

This keeps each domain focused and lets them evolve independently. See docs/superpowers/specs/2026-08-22-jarvis-personal-manager-design.md for the full architecture.

Features

• "Hey Jarvis" wake word — fully offline detection via openWakeWord, no cloud, no account.
• Natural, continuous conversation — wakes, greets you, listens, responds, and stays open for follow-ups until you go quiet (45-second idle timeout).
• Domain-scoped tool calling — dev-domain tools for launching Claude Code, school-domain tools for reading/writing your calendar, business tools for tracking ventures, etc.
• Persistent, structured storage — a SQLite database replaces the old memory.json, handling facts, business entities, notifications, clip-generation jobs, and dev session tracking (tasks and timers still use their own JSON files).
• Proactive alerts — timers/reminders fire on their own when due, even if you're not mid-conversation.
• JARVIS-style HUD — animated reactor core with live time/weather/status readouts.
• $0 to run — Gemini's free API tier is rate-limited, not metered.

Quick start

See SETUP.md for full setup instructions. The short version:

pip install -r requirements.txt
playwright install chromium
copy config.example.json config.json
# Fill in gemini_api_key (or set GEMINI_API_KEY in .env), elevenlabs_api_key/voice_id (optional),
# and domain-specific keys (schoolplan credentials, claude_code_workspaces, etc.)
python server.py          # in one terminal
python wake_word_trigger.py   # in another

Then say "Hey Jarvis" near your mic.

Project layout

File/Folder	What it does
server.py	FastAPI backend — WebSocket chat, two-hop Gemini routing, ElevenLabs TTS, proactive alerts
wake_word_trigger.py	Background listener for "Hey Jarvis" (openWakeWord) — offline, no cloud
llm/client.py	Gemini LLM client interface (router + domain-scoped tool-calling)
domains/	Domain modules — system, dev, school, content, business, registry
memory_store.py	SQLite-backed persistent storage (facts, ventures, notifications, clip jobs, dev sessions)
timers.py	Timer/reminder management; get_due_items() is polled for proactive alerts
proactive.py	Queue for unprompted Jarvis responses (timers/reminders hand off here)
frontend/	The HUD web UI (no framework — plain HTML/CSS/JS, same as old Jarvis)
scripts/launch-session.ps1	Starts both server.py and wake_word_trigger.py for auto-start at login
config.example.json	Copy to config.json and fill in

What changed from the old Jarvis

• Domain routing: Utterances are classified into one domain; only that domain's tools are available per turn.
• Structured storage: memory.json → jarvis.db (SQLite), auto-created on first run via memory_store.init_db(). tasks.json and timers.json are unchanged — tasks.py and timers.py still read/write them directly.
• New config keys (see SETUP.md for details):
  • claude_code_workspaces — project name → folder path map for the dev domain.
  • schoolplan_url, schoolplan_user, schoolplan_password — credentials for school domain.
  • publikclip_path — path to publikclip for video clip generation in content domain.
  • elevenlabs_monthly_char_quota — tracks free-tier usage; gracefully falls back to browser TTS if exhausted.
• Reused from v1: Wake-word detection, FastAPI/WebSocket skeleton, HUD frontend, rate-limit handling, ElevenLabs TTS, system control tools.

Not cloning the actual movie voice

ElevenLabs' voice library has plenty of British-butler-style voices that land a similar vibe, but this project doesn't attempt to clone the real JARVIS voice from the films — that's a real actor's protected performance. See SETUP.md for picking a voice.

License

Personal project, no license file — ask before reusing.

C:\Users\there\desktop\transfer\jarvis.worktrees\rebuild-personal-manager\README.md