SDKs & Libraries
Voice SDK
Voice-activated assistant integration for hands-free development.
Voice SDK
The Voice SDK (@codmir/voice-daemon) enables voice-activated interactions with codmir. Say "codmir" to activate, then speak your command.
Overview
The voice system consists of:
- Wake Word Listener - Detects "codmir" activation phrase
- Voice Daemon - WebSocket server coordinating voice pipeline
- Ball Overlay - Visual feedback UI (PyQt5)
- TTS Worker - Text-to-speech responses
Architecture
┌─────────────────┐ ┌─────────────────┐
│ Wake Word │────▶│ Voice Daemon │
│ Listener │ │ (port 9410/11) │
└─────────────────┘ └────────┬────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Ball │ │ AI │ │ TTS │
│ Overlay │ │ Agent │ │ Worker │
└──────────┘ └──────────┘ └──────────┘Installation
Prerequisites
- Ubuntu 22.04+ or compatible Linux
- Python 3.10+ with PyQt5
- Node.js 20+
Setup
# Navigate to voice daemon
cd workers/voice-daemon
# Install Node dependencies
pnpm install
# Install Python dependencies for overlay
pip install PyQt5
# Build TypeScript
pnpm buildUsage
Start Services
# Using the CLI helper
codmir-activate start
# Or manually
systemctl --user start codmir-voice-daemon
systemctl --user start codmir-voice-overlayCommands
codmir-activate start # Start voice services
codmir-activate stop # Stop services
codmir-activate status # Check service status
codmir-activate test # Run overlay in demo modeWebSocket API
The voice daemon exposes a WebSocket server on port 9410.
Connect
const ws = new WebSocket('ws://127.0.0.1:9410');
ws.onopen = () => {
console.log('Connected to voice daemon');
};
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
console.log('Event:', data.type, data.payload);
};Events
// Wake word detected
{ type: 'wake', payload: { timestamp: number } }
// Listening started
{ type: 'listening', payload: {} }
// Transcription result
{ type: 'transcript', payload: { text: string, final: boolean } }
// AI response
{ type: 'response', payload: { text: string } }
// Speaking started
{ type: 'speaking', payload: { text: string } }
// Idle (ready for next command)
{ type: 'idle', payload: {} }
// Error
{ type: 'error', payload: { message: string } }Send Commands
// Simulate wake word (for testing)
ws.send(JSON.stringify({ type: 'simulate_wake' }));
// Send text command (skip voice)
ws.send(JSON.stringify({ type: 'command', text: 'Create a new ticket' }));
// Cancel current operation
ws.send(JSON.stringify({ type: 'cancel' }));HTTP Webhook
Wake word detection posts to http://127.0.0.1:9411/webhook/wakeword:
curl -X POST http://127.0.0.1:9411/webhook/wakewordBall Overlay States
The floating ball UI indicates assistant state:
| State | Appearance |
|---|---|
idle | Subtle pulse, gray/blue |
listening | Active pulse, bright blue |
thinking | Spinning animation |
speaking | Ripple effect |
error | Red flash |
Configuration
Environment variables:
# Voice daemon
CODMIR_VOICE_WS_PORT=9410
CODMIR_VOICE_HTTP_PORT=9411
CODMIR_AI_ENDPOINT=http://localhost:3000/api/voice
# Overlay
CODMIR_OVERLAY_POSITION=bottom-right
CODMIR_OVERLAY_SIZE=64Systemd Services
Voice Daemon Service
# ~/.config/systemd/user/codmir-voice-daemon.service
[Unit]
Description=codmir Voice Daemon
After=network.target
[Service]
Type=simple
WorkingDirectory=/path/to/workers/voice-daemon
ExecStart=/usr/bin/node dist/index.js
Restart=on-failure
[Install]
WantedBy=default.targetOverlay Service
# ~/.config/systemd/user/codmir-voice-overlay.service
[Unit]
Description=codmir Voice Overlay
After=codmir-voice-daemon.service
[Service]
Type=simple
WorkingDirectory=/path/to/workers/voice-daemon/overlay
ExecStart=/usr/bin/python3 ball.py
Restart=on-failure
Environment=DISPLAY=:0
[Install]
WantedBy=default.targetIntegration with Desktop
The voice assistant can control:
- Terminal - Run commands
- Browser - Open URLs
- File Manager - Navigate files
- codmir App - Create tickets, run builds
// Example: Voice command handling
daemon.on('command', async (text) => {
if (text.includes('create ticket')) {
await createTicket(text);
}
if (text.includes('run tests')) {
await runTests();
}
});Troubleshooting
Daemon Won't Start
# Check logs
journalctl --user -u codmir-voice-daemon -f
# Check port availability
lsof -i :9410Overlay Not Showing
# Check X11 display
echo $DISPLAY
# Run overlay manually
cd workers/voice-daemon/overlay
python3 ball.pyWake Word Not Detecting
Ensure the wake word listener is running and configured for "codmir" activation phrase.