1 Architecture LLM Interaction
blightbow edited this page 2025-12-08 04:53:22 +00:00

Architecture: LLM Interaction

Layer 2 - LLM Orchestration and Response Handling


Overview

The LLM interaction module orchestrates communication between the assistant and LLM providers:

  • Context type determination - Routes to appropriate prompt assembly
  • Tool schema management - Filters tools by mode and context
  • Message building - Assembles full prompt with history and memories
  • Response parsing - Extracts tool calls from native or JSON format

1. Context Type Determination

Routes to the correct prompt assembly based on current state (llm_interaction.py:34-73).

Context Types

Context Type Trigger Prompt Style
CONTEXT_TICK_EVENT Pending event from user Full prompt with tools
CONTEXT_TICK_AUTONOMOUS No events, goal pursuit Full prompt with tools
CONTEXT_REFLECTION Reflection event Minimal prompt, journal tools only
CONTEXT_REFLECTION_CONT Tool result after reflection Minimal continuation
CONTEXT_SLEEP_CONSOLIDATE Sleep mode Sleep-specific tools
from evennia.contrib.base_systems.ai.llm_interaction import determine_context_type

context = determine_context_type(script)
# Returns one of the CONTEXT_* constants

2. Tool Schema Management

Mode Filtering

Tools are filtered by operating mode (llm_interaction.py:110-133):

from evennia.contrib.base_systems.ai.llm_interaction import get_tool_schemas

# Returns all tools in awake mode
# Returns only sleep_tools in sleep mode
schemas = get_tool_schemas(script)

Context Filtering

Further filters by context configuration:

from evennia.contrib.base_systems.ai.llm_interaction import get_tools_for_context

# Gets tools allowed for specific context type
tools = get_tools_for_context(script, context_type="CONTEXT_REFLECTION")

LLM API Format

Converts to OpenAI function calling format:

from evennia.contrib.base_systems.ai.llm_interaction import get_tool_schemas_for_llm

# Returns list of {"type": "function", "function": {...}}
api_schemas = get_tool_schemas_for_llm(script)

3. Message Building

build_llm_messages()

Assembles the complete prompt (llm_interaction.py:503-763):

from evennia.contrib.base_systems.ai.llm_interaction import build_llm_messages

messages = yield build_llm_messages(script, context_type=None)

Assembly Order

  1. System prompt - Native or fallback based on provider
  2. Character context - From character.get_context_for_prompt()
  3. Entity context - O-Mem profiles and working memory
  4. Semantic memories - Retrieved from Mem0 (awake mode only)
  5. Context buffer - Recent ambient messages
  6. Goal state - Current goals with hierarchy
  7. Conversation history - Token-budgeted recent messages
  8. Pending event - User message or reflection prompt
  9. Tool result - From previous tick if any

Token Management

History is added newest-to-oldest until budget exhausted:

available_tokens = script.db.max_context_tokens - used_tokens - 1000  # Reserve for response

for message in reversed(history):
    msg_tokens = count_tokens(message["content"], model=model)
    if history_tokens + msg_tokens > available_tokens:
        break
    history_messages.insert(0, dict(message))

4. LLM API Call

call_llm()

Calls the configured LLM provider (llm_interaction.py:246-315):

from evennia.contrib.base_systems.ai.llm_interaction import call_llm

response = yield call_llm(script, messages)
# Returns LLMResponse or None on failure

Applies configuration from script:

  • llm_temperature
  • llm_top_p
  • llm_max_tokens
  • llm_reasoning_effort
  • llm_extra_params

Provider Detection

Checks if provider supports native tool calling:

from evennia.contrib.base_systems.ai.llm_interaction import provider_supports_native_tools

# True for: openai, anthropic, openrouter, ollama
supports = provider_supports_native_tools(script)

5. Response Parsing

parse_tool_call()

Extracts tool call from LLM response (llm_interaction.py:383-501):

from evennia.contrib.base_systems.ai.llm_interaction import parse_tool_call

tool_call = parse_tool_call(script, response)
# {
#     "tool": "say",
#     "parameters": {"message": "Hello"},
#     "reasoning": "Greeting the user",
#     "tool_call_id": "call_abc123"  # None for JSON fallback
# }

Native vs JSON Fallback

Provider Type Format tool_call_id
Native (OpenAI, etc.) LLMResponse.tool_calls Present
JSON Fallback {"tool": ..., "parameters": ..., "reasoning": ...} None

Fallback JSON Extraction

If response isn't pure JSON, attempts to find embedded JSON object:

# Finds JSON even with surrounding text
start = response.find("{")
# ... balanced brace matching ...
json_str = response[start:end]
tool_call = json.loads(json_str)

6. Context Compaction

generate_context_summary()

Summarizes messages before trimming (llm_interaction.py:317-381):

from evennia.contrib.base_systems.ai.llm_interaction import generate_context_summary

summary = yield generate_context_summary(script, messages_to_compact)

Uses configurable prompt (script.db.compact_prompt) and optional dedicated model (script.db.compact_model).


7. Reflection Trigger

trigger_reflection()

Initiates automatic reflection (llm_interaction.py:775-818):

from evennia.contrib.base_systems.ai.llm_interaction import trigger_reflection

trigger_reflection(script, character)
# Adds reflection event to pending_events queue

Reflection events use event_type="reflection" for minimal prompt assembly.


Key Files

File Lines Purpose
llm_interaction.py 34-73 determine_context_type()
llm_interaction.py 76-108 get_tools_for_context()
llm_interaction.py 110-226 Tool schema functions
llm_interaction.py 246-315 call_llm()
llm_interaction.py 317-381 generate_context_summary()
llm_interaction.py 383-501 Response parsing
llm_interaction.py 503-763 build_llm_messages()
llm_interaction.py 775-818 trigger_reflection()

See also: Architecture-LLM-Providers | Architecture-Prompt-System | Architecture-Context-System