The stack
Inside Pandora's Box
Every subsystem, integration, and tool that lives inside the platform — surfaces, LLM providers, integrations, production tools, memory, security, self-improvement, foundational shared layers. Grouped by what they do, marked by what they need.
Built-in
Required dependency
Optional dependency
External service
Group 1 · Surfaces
How you interact
Where messages come in and replies go out. Per-tenant or personal. Owner-allowlisted at every gate.
Built-in
Web App
Browser-first dashboard for the personal agent. Unified inbox, calendar, drafts, voice, watch summary. Tailscale-accessible from any device.
External service
Telegram
Per-tenant conductors plus a personal-agent bot. Owner-allowlisted at the conductor before any classification runs. Most common surface.
External service
WhatsApp
Alternative chat surface via the same conductor pattern. Same owner allowlist gating as Telegram.
Optional
Voice
STT in, TTS out. Phone or Pixel Watch. Conversational, hands-free. Always returns a written transcript for the audit log.
Optional
Pixel Watch Companion
Wear OS app for voice on the wrist. Tiles, complications, persistent notifications.
Built-in
Browser Terminal
Web-served terminal surface for the admin agent. Use to drive deploys, monitor services, run admin sessions. Local network only.
Group 2 · LLM providers
Multi-model by design
A frontier model for reasoning, a local model for high-frequency classification, and a small set of specialist models for voice and vision. Cost-routed.
Required
Claude (primary)
Anthropic Claude via Pro / Pro Max subscription. Used for reasoning, drafting, multi-step planning. The default model for most agent actions.
Optional
Gemini Live
Google Gemini Live for voice-call awareness and frame-by-frame visual analysis. Adds: real-time voice conversations, vision over a phone camera or screen share.
Built-in
Ollama (local)
Local model (default: gemma3:12b) for classification, summary, extraction. Zero per-call cost. Automatic fallback to Haiku if unavailable.
Optional
ElevenLabs
High-quality TTS for narration and voice replies. Custom voice clones supported.
Optional
Groq
Fast Whisper-based STT for voice input. Sub-second transcription.
Group 3 · Integrations & MCP
Where the work lands
External systems the agents read from and write to. Most go through the Model Context Protocol (MCP) servers so the wiring is auditable and the credentials stay scoped.
External service
Microsoft 365
Mail, calendar, files via the Microsoft Graph MCP server. Per-tenant OAuth tokens — never shared between tenants. Most-used external surface.
External service
Gmail
Inbound classification + draft generation. Read-only by default; sending always staged for human review.
External service
SharePoint
Scoped folders per tenant for document operations. Built on Graph; same auth surface as M365 mail.
Built-in
Filesystem (MCP)
Local file access scoped to per-tenant directories under
/opt/pandoras-box/<tenant>/. Enforced at the OS level (750, separate UID).Built-in
Database (custom MCP)
Per-tenant CRM and project databases. SQLite under the hood. Per-tenant SQL with strict
WHERE tenant = own at the framework layer.Optional
Brave Search
Privacy-respecting web search for research mode. Used by the personal agent for live-fact lookups.
Optional
Polygon.io
Market data feed for the trading-research agent. Quotes, fundamentals, news.
Group 4 · Production pipelines
Build, produce, publish
End-to-end pipelines for content, knowledge, review, and browser automation. Each is a shared capability that agents activate as a module rather than reimplementing.
Optional
Media Production Pipeline
PRS-free original music, voice narration, image and video generation, and YouTube publishing automation in a single chain. Output of this is what powers Free Background Focus.
Optional
Offline Knowledge Library
Local-first reference library via Kiwix. Full English Wikipedia plus medical, survival, DIY and finance reference packs. All queries answered locally, no internet required.
Built-in
Reviews Dashboard
Web surface for staged work product — approve, revise, or reject. The human-in-the-loop gate for everything the agents produce before it goes anywhere external.
Built-in
Browser Automation
Headless browser driver for tasks the chat can't do directly — clicking through web flows, scraping a page, filling a form behind a login. The Assistant's hands.
Group 5 · Memory + storage
Persistent context
Memory is a layer, not a hack. Recent history, semantic recall, drops, important standing facts, notes vault, session summaries, optional temporal scoring — every session loads from multiple sources, every session writes back.
Built-in
Memory Hub
Central memory coordinator for the personal agent. Routes loads and write-backs across all memory sources.
Built-in
Vector Store
SQLite vector extension for semantic recall. Embeds turn-by-turn history for similarity search at session-start.
Optional
Obsidian Vault
Markdown knowledge base for People, Threads, Commitments. Human-readable, version-controllable, syncs to any device. Editable outside the agent.
Built-in
Drops
Passive-context folder structure dropped to by other agents (emails, calendar items, files). Personal agent picks these up at session start.
Built-in
Recall Scoring
Temporal-aware retrieval scoring. Optional bolt-on; runs in shadow mode by default and is operator-activated per category once calibrated.
Built-in
Session History
Per-session conversation log with full-text search (SQLite + FTS5). Searchable from inside the agent and from the admin terminal.
Group 6 · Security + oversight
Six independent gates
Every layer is enforced in code, not by policy. Defense in depth.
Built-in
Oversight Daemon
Independent process. Runs as its own OS user. Approves or blocks every queued job before execution. Cannot be instructed by any agent.
Built-in
Content Classifier
GLiGuard 0.3B model. Screens outbound content across six axes (prompt safety, response safety, refusal, toxicities, jailbreak). Shadow mode by default; light-gate after calibration.
Built-in
Audit Log
Append-only event log of every oversight decision. Used for after-the-fact review and pattern detection.
Built-in
Lockdown Manager
Quarantine and lockdown executor. Per-agent isolation on trigger. Manual + N-strike automatic + tamper-triggered paths.
Built-in
Watchdog
Twice-daily integrity checks: heartbeat, DB integrity, log sizes, env permissions, system-prompt tamper, secret scan, scope creep, stuck jobs. Runs 08:00 and 20:00.
Built-in
Dependency Scan
Weekly scan of GitHub releases and npm audit for the oversight + shared layers. Generates a mitigation plan; operator approves before any patch.
Group 7 · Self-improvement
A skill governance loop
The system improves itself on a cadence, not continuously. Six scheduled workers; results aggregated to the operator on Sunday morning; nothing changes without approval.
Built-in
Reflection
Hourly. Observation only — surfaces patterns in agent behaviour without acting on them.
Built-in
Calibration
Saturday. Measures skill performance against the baseline; identifies regressions.
Built-in
Prompt Optimiser
Saturday. Generates proposals on tool descriptions and prompt sections — where did wording cause misrouting or sub-optimal behaviour?
Built-in
Upstream Scan
Weekly. Tracks dependency drift — Anthropic SDK releases, upstream model changes, npm package versions of interest.
Built-in
Weekly Review
Sunday 08:00. Aggregates optimiser proposals, skill flags, security mitigations into a single Telegram digest for the operator.
Built-in
Consolidation
Nightly. Writes session digests to long-term memory; ages out raw conversation logs older than 90 days.
Group 8 · Foundational shared
The base layer every agent uses
Agent-agnostic primitives. Consumed by every higher layer; never nested under the first agent that happened to use them.
Built-in
Bridge
Anthropic Claude adapter — per-session stable subprocess that routes all Claude API calls via the operator's subscription. One bridge per chat-session ID.
Built-in
Module Catalogue
YAML manifest registry for all installable modules.
shared/catalogue/modules/*.yaml. Each module declares tools, skills, rules, stores.Built-in
Skill Library
Versioned canonical store with a signed manifest. Per-agent cache pulled at session start with hash verification.
Built-in
Module Activation
Per-agent activation state. Single source of truth (
agent-activation.json) for what each agent has on. Reviewable, version-controllable.Want to see it together?
The six-layer overview
All of the above grouped into the six architectural layers. Read the framework page for the full walkthrough.
