Jarvis
Windows voice assistant: Python agent owns wake word, speech, Gemini conversation, and tools; an Electron shell owns tray, hotkeys, and permission prompts.
Overview
Jarvis is a Windows personal assistant split into a Python agent and an Electron shell. The agent owns the microphone, wake word, Google Speech-to-Text, Gemini conversation, and tools. The shell owns tray, window, hotkeys, and Allow/Deny dialogs over a localhost WebSocket.
Challenge
Cloud-only assistants cannot see the local desktop, and fully local bots either lack a good LLM or run with no permission gate. Users need a voice core that can use tools without silently taking over the machine.
Solution
Clear ownership: Python 3.11+ agent with Gemini and GCP STT; Electron UI for presence and consent. Every tool call can require Allow/Deny. Credentials stay in gitignored JSON and .env, never in the shell.
What we built
- Wake-word and microphone pipeline in the agent
- Gemini chat brain with tool use
- Electron tray, window, and hotkeys
- Allow/Deny prompts before sensitive tools
- Google Cloud Speech-to-Text via service account
Outcomes
- Voice and UI are separate processes that can fail independently
- Tools never run without an explicit consent path
- Works as a Windows companion, not a browser tab
Stack
Wake word + chat brain | Allow/Deny tool gate
Build something in this family
Talk to NTechnologies about a POS, agent, marketplace, or internal tool.
