Open Source • MIT Licensed • TypeScript

The sub-10ms voice agent framework for TypeScript.

Replace slow, token-heavy LLM loops with Joint Embedding Vector (JEV) routing and deterministic state machines. Zero hallucinations. Zero runtime model inference fees.

Get Started↗Star on GitHub

src/concierge-agent.ts

● ~4.8ms JEV routing|$0.00 token cost|100% deterministic

import { createAgent } from "felona-voice";

// Define conversational agent with deterministic action nodes
const agent = createAgent("SupportConcierge")
  .system("You are a real-time voice support assistant.")
  .action("track_order", "Check delivery status and carrier ETA", async (query, state) => {
    return `Order ${state.orderId} is currently out for delivery via FedEx.`;
  })
  .action("technical_help", "Diagnose power, wifi, or connection status", async () => {
    return "Please hold the reset button for 10 seconds. Does the LED turn solid blue?";
  })
  .fallback("Sorry, I didn't catch that. Could you please repeat your question?");

// Predict next node via Joint Embedding Vector (JEV) cosine dot product in ~5ms:
const reply = await agent.interact("where is my shipment?");
console.log(reply.action.id); // "track_order"
console.log(reply.text);      // "Order ACM-9281 is currently out for delivery via FedEx."

Pluggable Audio Ecosystem

DeepgramNova-2 Streaming

CartesiaSonic (<90ms TTFB)

ElevenLabsTurbo v2.5

OpenAIWhisper & Embeddings

TwilioSIP & Media Streams

AssemblyAIStreaming STT

WebSockets16kHz Linear PCM

The Paradigm Shift

Sub-10ms neural routing vs the 1,200ms LLM loop

Traditional voice agents pipe every utterance into slow, token-heavy LLMs. Felona Voice decouples intent routing from text generation, evaluating next actions in ~5ms.

Traditional Voice AIMonolithic LLM Stack

1,200ms – 1,800ms

Exceeds the 300ms human conversational barrier. Causes awkward pauses, overlapping speech, and high token costs.

STT Audio Transcription~150ms

LLM Inference & Generation~900ms – 1,500ms

TTS Speech Synthesis~150ms

$0.05 – $0.15 per call minute in token fees

Vulnerable to prompt injections and hallucinations

Felona VoiceJEV Neural Routing

~246ms Turn Latency

Beneath the human voice perception threshold (300ms). Natural conversational turn-taking with zero runtime token costs.

STT Streaming Audio~150ms

JEV Vector Match + Action~5ms

TTS Streaming Audio~90ms

$0.00 runtime model inference fees (Zero tokens)

100% Deterministic execution with typed state machine

Feature / Dimension	Traditional LLM Pipeline	Felona Voice (JEV Engine)
Decision Latency	400ms – 1,200ms per turn	~5ms (Normalized Cosine Similarity)
Runtime Inference Cost	$0.05 – $0.15 per call minute	$0.00 (Zero LLM Tokens)
Hallucination Risk	High (Statistical word generation)	0% (Explicit TypeScript Action Nodes)
State Machine Constraints	Unpredictable prompt adherence	Directed State Graphs with Typed Channels
Deployment Architecture	Vendor lock-in or heavy GPU clusters	Lightweight Node.js / Bun process (MIT Open Source)

Architecture Pillars

Built for sub-10ms voice agents

Three tightly integrated systems designed to give you complete ownership and deterministic control over conversational voice AI.

JEV Neural Core

Encodes conversational context into 128-dimensional vectors and resolves user intent via normalized cosine dot products in ~5ms.

• Zero runtime token costs ($0.00 model fees)

• Zero external API dependencies in cold-start mode

• Built-in confidence thresholds & ambiguity margin guards

Core Documentation↗

Directed State Machine

Constrain conversational flow with directed graph edges. Only candidate actions reachable from the active state are evaluated.

• Strongly typed state channels with TypeScript generics

• Mathematically eliminates hallucinations & prompt injection

• Auto-generates Markdown tables, Mermaid, and ASCII diagrams

State Machine Guide↗

Streaming Audio & Telephony

Pluggable audio pipeline for real-time bidirectional PCM streams with native WebSockets and Twilio SIP integration.

• Deepgram Nova-2 streaming STT (<200ms)

• Cartesia Sonic & ElevenLabs neural TTS (<90ms TTFB)

• Built-in zero-dependency Energy VAD turn detection

Audio Pipeline Docs↗

State Machine & Visualization Suite

Deterministic State Machine Explorer

Deterministic state graphs constrain what actions can be triggered at each conversational turn. Inspect diagrams via visual canvas, Mermaid flowcharts, terminal ASCII, or Markdown specs.

Click any conversational node to inspect transitions & handlersTotal Nodes: 6

START CALL

🟢 Entry Point

greet

Welcome caller warmly, identify Acme Corp, and ask how to assist with orders or product issues

Outgoing: 5 transitions

action

order_status

Check delivery status, transit location, carrier tracking number, and arrival ETA for order

Outgoing: 5 transitions

action

tech_troubleshoot

Diagnose device glitches, blinking light, power cycle, factory reset, or wifi connectivity

Outgoing: 4 transitions

action

refund_request

Initiate return, refund request, damaged package compensation, or return shipping label

Outgoing: 3 transitions

action

transfer_specialist

Escalate to senior supervisor or human support lead when customer asks for manager or complex issue

Outgoing: 1 transitions

🛡️ Fallback

fallback

Out-of-scope questions, trivia, weather, background noise, or unhandled speech

Outgoing: 5 transitions

Node Inspector

greet

Welcome caller warmly, identify Acme Corp, and ask how to assist with orders or product issues

Allowed Next State Transitions:

→ order_statusTrack Order

→ tech_troubleshootHardware Troubleshoot

→ refund_requestProcess Refund

→ transfer_specialistTransfer to Human

→ fallbackFallback Guard

Action Handler Implementation:

async (input, state) => {
  return "Hello! Welcome to Acme Support. How can I help you today with your order or product?";
}

Speed Benchmarks & Cost ROI

Engineered for Sub-10ms Speed

In voice AI, every 100 milliseconds of pause feels like an eternity. Compare real-world decision latency and calculate your monthly infrastructure savings.

Decision Turn Latency

Lower is better (ms)

Felona Voice (JEV Engine)

~5 ms

Zero network hops, zero token generation latency

Local Small Language Model (8B)185 ms

Fast Cloud LLM (e.g. GPT-4o-mini)420 ms

Traditional Voice Chain (Full Prompt LLM)1,150 ms

Exceeds natural human conversation pause threshold (250ms)

Human Cadence: Felona Voice's 5ms routing leaves the entire latency budget for STT/TTS, achieving total conversational gaps under 250ms.

ROI Cost Calculator

Estimate your monthly telephony routing cost savings vs cloud LLM inference fees.

Monthly Completed Calls50,000 calls

Average Conversational Turns6 turns / call

Estimated Monthly Savings:

$4,500 / mo

Calculated on 300,000 decision turns ($0.015/turn LLM cost vs $0.00 Felona JEV).

Developer Experience

Built for TypeScript Developers

Clean APIs, zero boilerplate, full static typing, and pluggable audio streaming providers.

concierge-agent.ts

Zero-boilerplate fluent API. Create an intelligent voice agent in just 3 lines of code.

import { createAgent } from "felona-voice";

// Create voice agent with 3 fluent actions
const agent = createAgent("Concierge")
  .system("You are a friendly concierge.")
  .action("book_table", "Book a dining table or restaurant reservation", async () => "Table booked for 7 PM!")
  .action("room_service", "Order food or fresh towels", async () => "Room service is on its way.")
  .fallback("Sorry, I am not able to understand that. How can I assist you?");

// Test instantly without spinning up a server (zero external API keys required!):
const reply = await agent.interact("can I get clean towels?");
console.log(reply.text); 
// Output: "Room service is on its way." (~5ms decision latency!)

Sub-10ms Voice Agents

Start building with Felona Voice

Install the open-source package, define your conversation state graph in pure TypeScript, and run streaming voice agents with zero token fees.

Read Documentation↗Star on GitHub