# SensAffect Developer API Specification ## Version 0.1 This document defines a practical developer-facing API for SensAffect as an emotional infrastructure layer. It is designed to be dropped into a TypeScript/Node/React project and expanded over time. The API is intentionally built around three ideas: 1. **State, not labels** 2. **Trajectory, not single-turn snapshots** 3. **Regulation, not analytics only** --- ## 1. Design Goals The API must: - accept multimodal observations incrementally - compute a normalized emotional state - derive kinematic features across time - estimate risk and recovery trajectories - select recommended regulation actions - remain explainable and deterministic - integrate with existing THRPY / RealRuntime / AURA layers --- ## 2. Core Concepts ### 2.1 Observation A single sensed input moment. ```ts export interface SensAffectObservation { ts: number; source: "voice" | "text" | "ui" | "memory" | "system"; confidence?: number; valence?: number; // -1..1 arousal?: number; // 0..1 dominance?: number; // -1..1 sentimentScore?: number; // -1..1 resonance?: number; // 0..1 topic?: string; tags?: string[]; metadata?: Record; } ``` ### 2.2 Emotional State The fused state computed from recent observations. ```ts export interface EmotionalState { ts: number; valence: number; // -1..1 arousal: number; // 0..1 dominance: number; // -1..1 confidence: number; // 0..1 emotionScore: number; // 0..100 resonance: number; // 0..1 stability: number; // 0..1 energy: number; // 0..1 labels: string[]; contributors: StateContributor[]; } ``` ### 2.3 Kinematics How state changes over time. ```ts export interface EmotionalKinematics { dtMs: number; velocity: EmotionalVector; acceleration: EmotionalVector; jerk: EmotionalVector; speed: number; accelerationMagnitude: number; jerkMagnitude: number; direction: "improving" | "worsening" | "volatile" | "stable"; recoveryProbability: number; // 0..1 escalationProbability: number; // 0..1 } ``` ### 2.4 Regulation Decision What the system should do next. ```ts export interface RegulationDecision { mode: | "steady" | "deescalate" | "ground" | "uplift" | "cooldown" | "contain" | "handoff"; confidence: number; reasons: string[]; targets: RegulationTargets; actions: RegulationAction[]; } ``` --- ## 3. Type Definitions ```ts export interface EmotionalVector { valence: number; arousal: number; dominance: number; } export interface StateContributor { source: SensAffectObservation["source"]; weight: number; summary: string; } export interface RegulationTargets { visualPace: number; // 0..1 slower to faster visualBrightness: number; // 0..1 audioLfoHz: number; // e.g. 0.06 - 0.12 audioWarmth: number; // 0..1 cool to warm responseCadence: number; // 0..1 slower to faster verbosity: number; // 0..1 terse to expanded safetySensitivity: number; // 0..1 } export interface RegulationAction { type: | "set_aura" | "set_audio" | "set_response_style" | "trigger_grounding" | "trigger_breathing" | "raise_safety_watch" | "handoff"; payload: Record; } ``` --- ## 4. Public API Surface ### 4.1 `createSensAffectEngine` Creates a stateful engine instance. ```ts export function createSensAffectEngine(config?: Partial): SensAffectEngine; ``` ### 4.2 `ingest` Adds a new observation to the engine. ```ts engine.ingest(observation: SensAffectObservation): SensAffectSnapshot ``` Returns the latest snapshot including fused state, kinematics, and regulation. ### 4.3 `getSnapshot` Returns the latest computed snapshot. ```ts engine.getSnapshot(): SensAffectSnapshot | null ``` ### 4.4 `reset` Clears internal state. ```ts engine.reset(): void ``` ### 4.5 `exportTimeline` Returns recent timeline entries for debugging or visualization. ```ts engine.exportTimeline(): SensAffectSnapshot[] ``` --- ## 5. Snapshot Shape ```ts export interface SensAffectSnapshot { ts: number; state: EmotionalState; kinematics: EmotionalKinematics; regulation: RegulationDecision; windowSize: number; } ``` --- ## 6. Configuration ```ts export interface SensAffectConfig { maxObservations: number; // default 120 smoothingFactor: number; // default 0.35 resonanceWeight: number; // default 0.15 riskArousalThreshold: number; // default 0.78 riskNegativeValenceThreshold: number;// default -0.5 volatilityJerkThreshold: number; // default 0.22 recoveryVelocityThreshold: number; // default 0.04 escalationVelocityThreshold: number; // default 0.04 } ``` --- ## 7. Behavioral Contract ### 7.1 Ingestion rules - Observations may omit some values. - Confidence defaults to `0.5` when omitted. - Missing VAD components are ignored in weighted fusion. - Observations are fused over a rolling window. ### 7.2 State fusion rules - The engine computes weighted means for V/A/D. - Observation confidence influences contribution. - Resonance is fused as a separate modifier. - Emotion score is normalized to `0..100`. - Stability decreases when jerk and variance increase. ### 7.3 Kinematic rules - Velocity = difference between consecutive states / time - Acceleration = difference between consecutive velocities / time - Jerk = difference between consecutive accelerations / time - Escalation is more likely when arousal velocity is positive, valence velocity is negative, and jerk is high. - Recovery is more likely when arousal falls, valence rises, and volatility is low. ### 7.4 Regulation rules Typical policy examples: - **High arousal + negative valence + rising speed** → `deescalate` - **Low arousal + negative valence + flat trajectory** → `uplift` - **Volatile + unstable + high jerk** → `contain` - **Stable + positive drift** → `steady` - **High risk thresholds crossed** → `handoff` --- ## 8. Integration Points ### 8.1 RealVoice / RealRuntime integration Map existing VAD outputs into `SensAffectObservation`: ```ts engine.ingest({ ts: Date.now(), source: "voice", valence: realVoice.valence, arousal: realVoice.arousal, dominance: realVoice.dominance, confidence: realVoice.confidence, metadata: { pitch: realVoice.pitchMean, tempo: realVoice.tempo, }, }); ``` ### 8.2 Memory / EMC² integration Inject resonance from semantic similarity or subconscious undercurrent: ```ts engine.ingest({ ts: Date.now(), source: "memory", resonance: emc2.resonance, confidence: emc2.confidence, metadata: { matchedThemes: emc2.themes, }, }); ``` ### 8.3 AURA integration Use `regulation.targets` to drive visual parameters. ```ts const snapshot = engine.getSnapshot(); aura.setPace(snapshot.regulation.targets.visualPace); aura.setBrightness(snapshot.regulation.targets.visualBrightness); ``` ### 8.4 Psychoacoustic engine integration ```ts audio.setLfoHz(snapshot.regulation.targets.audioLfoHz); audio.setWarmth(snapshot.regulation.targets.audioWarmth); ``` ### 8.5 Response orchestration integration ```ts llmOrchestrator.setCadence(snapshot.regulation.targets.responseCadence); llmOrchestrator.setVerbosity(snapshot.regulation.targets.verbosity); ``` --- ## 9. Event Hooks (Optional) ```ts export interface SensAffectHooks { onSnapshot?: (snapshot: SensAffectSnapshot) => void; onModeChange?: (mode: RegulationDecision["mode"], snapshot: SensAffectSnapshot) => void; onRisk?: (snapshot: SensAffectSnapshot) => void; } ``` --- ## 10. Example Usage ```ts import { createSensAffectEngine } from "@/lib/sensaffect"; const engine = createSensAffectEngine(); engine.ingest({ ts: Date.now(), source: "voice", valence: -0.42, arousal: 0.81, dominance: -0.18, confidence: 0.74, }); engine.ingest({ ts: Date.now() + 1200, source: "text", valence: -0.51, arousal: 0.77, confidence: 0.83, topic: "work-pressure", }); const snapshot = engine.getSnapshot(); console.log(snapshot?.state.emotionScore); console.log(snapshot?.kinematics.direction); console.log(snapshot?.regulation.mode); ``` --- ## 11. Suggested File Layout ```txt src/ lib/ sensaffect/ index.ts types.ts emotionalKinematics.ts regulationLoop.ts ``` --- ## 12. Future Extensions - multi-user field modeling - dyadic interaction state - agent-to-agent emotion mirroring - confidence calibration against ground-truth labels - per-domain governance modules - safety policy packs - visual cluster debug surface bindings