# Real-Time Safety → LLM Flow Integration

**Last Updated:** 2026-01-24  
**Status:** ✅ Complete - Real-time bidirectional flow with safety layer

## Overview

Your THRPY system now performs real-time safety checks on transcribed text and immediately flows to LLM for instant responses. The complete flow is:

**STT → Vocal Inference → Text Inference → Safety → LLM → TTS**

All steps happen in real-time for natural conversation flow.

## Architecture

### Complete Real-Time Flow

```
User Audio Input
    ↓
[STT] Speech-to-Text (Whisper ONNX)
    ↓
[Parallel Inference]
    ├─ Vocal Inference (audio → emotion)
    └─ Text Inference (text → sentiment/emotion/topics)
    ↓
[Safety Layer] Safety Check (<50ms)
    ├─ Fast Gate (<20-50ms)
    └─ Full Assessment (<50ms)
    ↓
[LLM] Generate Response (streaming)
    ├─ If Safe: Normal therapeutic response
    └─ If Flagged: Restricted/crisis response
    ↓
[TTS] Text-to-Speech (Supertonic)
    ↓
User Audio Output (with synchronized word highlighting)
```

## Safety Integration

### Safety Check Flow

**Step 1: Fast Gate (<20-50ms)**
- Local crisis detection
- Keywords: self-harm, violence, emergency
- Result: `safe_to_start` boolean

**Step 2: Full Assessment (<50ms)**
- Comprehensive risk analysis
- Risk level (0.0-1.0)
- Risk categories
- Crisis types
- Intervention protocols

**Step 3: LLM Response**
- If safe (`risk_level < 0.7` and not urgent): Normal therapeutic response
- If flagged: Restricted response with crisis protocol

### Safety Decision Logic

```python
is_safe = (
    safety_assessment.get("risk_level", 0.5) < 0.7
    and not safety_assessment.get("is_urgent", False)
)

if is_safe:
    # Generate normal therapeutic response
    llm_response = await chat_service.generate_response_instant(...)
else:
    # Use restricted/crisis response
    llm_response = "I'm here to help. Let's make sure you're safe..."
```

## Message Flow

### Transcription with Safety & LLM

```json
{
  "type": "transcription",
  "text": "I've been feeling anxious lately",
  "vocal_inference": {...},
  "text_inference": {...},
  "safety_assessment": {
    "risk_level": 0.3,
    "is_urgent": false,
    "risk_categories": ["anxiety"],
    "confidence": 0.85
  },
  "llm_response": "I understand that anxiety can be really challenging...",
  "llm_ready": true,
  "is_final": true
}
```

### LLM Response Chunks (Streaming)

```json
{
  "type": "llm_chunk",
  "text": "I understand",
  "accumulated": "I understand",
  "safety_assessment": {...}
}
```

```json
{
  "type": "llm_chunk",
  "text": " that anxiety",
  "accumulated": "I understand that anxiety",
  "safety_assessment": {...}
}
```

### Safety-Flagged Response

```json
{
  "type": "llm_chunk",
  "text": "I'm here to help. Let's make sure you're safe...",
  "accumulated": "I'm here to help. Let's make sure you're safe...",
  "safety_assessment": {
    "risk_level": 0.85,
    "is_urgent": true,
    "risk_categories": ["self_harm"],
    "intervention_required": true
  },
  "safety_flagged": true
}
```

## Performance

### Latency Breakdown

| Step | Latency | Notes |
|------|---------|-------|
| **STT** | 100-300ms | Whisper ONNX (real-time) |
| **Vocal Inference** | 50-150ms | Parallel with STT |
| **Text Inference** | 100-300ms | Parallel with STT |
| **Safety Check** | 20-50ms | Fast gate + full assessment |
| **LLM Response** | 200-1000ms | Streaming (first token <200ms) |
| **TTS** | 50-200ms | Supertonic (ultra-fast) |

**Total End-to-End**: ~500-2000ms (first response token)

### Real-Time Optimization

- **Parallel Processing**: Vocal + Text inference run in parallel
- **Streaming**: LLM response streams tokens as they're generated
- **Fast Safety**: Safety check completes before LLM starts
- **Ultra-Fast TTS**: Supertonic generates audio at ~167x real-time

## Code Locations

### Backend

- **Unified Service**: `chat-api/app/unified_audio_streaming_service.py`
  - `process_voice_input()` - Batch mode with safety → LLM
  - `process_voice_input_stream()` - Streaming mode with safety → LLM
  
- **Safety Client**: `chat-api/app/safety_client.py`
  - `analyze_message()` - Safety assessment
  
- **Chat Service**: `chat-api/app/services/chat_service.py`
  - `generate_response_instant()` - Instant LLM response after safety check

### Frontend

- **WebSocket Hook**: `frontend/src/hooks/useUnifiedAudioWebSocket.ts`
  - Handles transcription, safety, and LLM response messages

## Usage Example

### Complete Real-Time Flow

```typescript
const { sendAudioInput } = useUnifiedAudioWebSocket({
  userId: "user123",
  onTranscription: (result) => {
    // Real-time transcription with inference
    console.log("Transcription:", result.text);
    console.log("Vocal emotion:", result.vocal_inference?.label);
    console.log("Text sentiment:", result.text_inference?.sentiment?.label);
    
    // Safety assessment
    if (result.safety_assessment) {
      console.log("Risk level:", result.safety_assessment.risk_level);
      console.log("Is urgent:", result.safety_assessment.is_urgent);
    }
    
    // LLM response (when ready)
    if (result.llm_ready && result.llm_response) {
      console.log("LLM response:", result.llm_response);
      // Automatically triggers TTS with synchronized word highlighting
    }
  },
  onLLMChunk: (chunk) => {
    // Real-time LLM response chunks (streaming)
    console.log("LLM chunk:", chunk.text);
    console.log("Accumulated:", chunk.accumulated);
  }
});

// Send audio input
// Flow: STT → Inference → Safety → LLM → TTS (all automatic)
sendAudioInput(audioBytes, false);
```

## Safety Protocols

### Normal Flow (Safe)

1. User speaks → STT transcribes
2. Inference runs (vocal + text)
3. Safety check passes (`risk_level < 0.7`)
4. LLM generates normal therapeutic response
5. TTS speaks response with word highlighting

### Crisis Flow (Flagged)

1. User speaks → STT transcribes
2. Inference runs (vocal + text)
3. Safety check flags crisis (`risk_level >= 0.7` or `is_urgent`)
4. LLM generates restricted/crisis response
5. TTS speaks crisis protocol response
6. System logs crisis event for audit

### Degraded Mode (Safety Unavailable)

1. User speaks → STT transcribes
2. Inference runs (vocal + text)
3. Safety check unavailable → Uses degraded mode (`risk_level = 0.5`)
4. LLM generates conservative response (assumes unknown risk)
5. TTS speaks conservative response

## Configuration

### Enable/Disable Safety Flow

Safety flow is enabled by default. To disable:

```python
# In unified_audio_streaming_service.py
# Remove or comment out safety_client initialization
self.safety_client = None  # Disables safety → LLM flow
```

### Safety Thresholds

```python
# Risk level threshold for "safe" (default: 0.7)
is_safe = (
    safety_assessment.get("risk_level", 0.5) < 0.7
    and not safety_assessment.get("is_urgent", False)
)
```

## Troubleshooting

### LLM Response Not Appearing

**Check**:
1. Safety check is completing successfully
2. `is_safe` logic is passing
3. Chat service is initialized
4. LLM client is available

**Solutions**:
- Check backend logs for safety assessment results
- Verify `risk_level` is below threshold (0.7)
- Ensure `is_urgent` is false
- Check LLM client initialization

### Safety Check Failing

**Check**:
1. Safety API is available
2. Network connectivity
3. Circuit breaker status

**Solutions**:
- System falls back to degraded mode (`risk_level = 0.5`)
- LLM still responds but with conservative approach
- Check safety-api service health

### High Latency

**Check**:
1. Safety check latency (<50ms expected)
2. LLM response latency (<200ms first token expected)
3. Network latency

**Solutions**:
- Safety check should be <50ms (fast gate + full assessment)
- LLM streaming should start <200ms
- Total end-to-end should be <2000ms

## Next Steps

1. ✅ Safety layer integrated into real-time pipeline
2. ✅ LLM response generation after safety check
3. ✅ Real-time flow: STT → Safety → LLM → TTS
4. ✅ Streaming LLM responses for instant feedback
5. ✅ Crisis protocol handling for flagged messages

**You're all set!** The system now flows transcribed text through the safety layer directly to the LLM for real-time responses, ensuring safety while maintaining natural conversation flow.

---

**Questions?** Check the code comments in `unified_audio_streaming_service.py` for detailed implementation notes.