Architecture: LLM Interaction

Layer 2 - LLM Orchestration and Response Handling

---

Overview

The LLM interaction module orchestrates communication between the assistant and LLM providers:

• Context type determination - Routes to appropriate prompt assembly
• Tool schema management - Filters tools by mode and context
• Message building - Assembles full prompt with history and memories
• Response parsing - Extracts tool calls from native or JSON format

---

1. Context Type Determination

Routes to the correct prompt assembly based on current state (llm_interaction.py:34-73).

Context Types

Context Type	Trigger	Prompt Style
CONTEXT_TICK_EVENT	Pending event from user	Full prompt with tools
CONTEXT_TICK_AUTONOMOUS	No events, goal pursuit	Full prompt with tools
CONTEXT_REFLECTION	Reflection event	Minimal prompt, journal tools only
CONTEXT_REFLECTION_CONT	Tool result after reflection	Minimal continuation
CONTEXT_SLEEP_CONSOLIDATE	Sleep mode	Sleep-specific tools

from evennia.contrib.base_systems.ai.llm_interaction import determine_context_type

context = determine_context_type(script)
# Returns one of the CONTEXT_* constants

---

2. Tool Schema Management

Mode Filtering

Tools are filtered by operating mode (llm_interaction.py:110-133):

from evennia.contrib.base_systems.ai.llm_interaction import get_tool_schemas

# Returns all tools in awake mode
# Returns only sleep_tools in sleep mode
schemas = get_tool_schemas(script)

Context Filtering

Further filters by context configuration:

from evennia.contrib.base_systems.ai.llm_interaction import get_tools_for_context

# Gets tools allowed for specific context type
tools = get_tools_for_context(script, context_type="CONTEXT_REFLECTION")

LLM API Format

Converts to OpenAI function calling format:

from evennia.contrib.base_systems.ai.llm_interaction import get_tool_schemas_for_llm

# Returns list of {"type": "function", "function": {...}}
api_schemas = get_tool_schemas_for_llm(script)

---

3. Message Building

build_llm_messages()

Assembles the complete prompt (llm_interaction.py:503-763):

from evennia.contrib.base_systems.ai.llm_interaction import build_llm_messages

messages = yield build_llm_messages(script, context_type=None)

Assembly Order

1. System prompt - Native or fallback based on provider
2. Character context - From character.get_context_for_prompt()
3. Entity context - O-Mem profiles and working memory
4. Semantic memories - Retrieved from Mem0 (awake mode only)
5. Context buffer - Recent ambient messages
6. Goal state - Current goals with hierarchy
7. Conversation history - Token-budgeted recent messages
8. Pending event - User message or reflection prompt
9. Tool result - From previous tick if any

Token Management

History is added newest-to-oldest until budget exhausted:

available_tokens = script.db.max_context_tokens - used_tokens - 1000  # Reserve for response

for message in reversed(history):
    msg_tokens = count_tokens(message["content"], model=model)
    if history_tokens + msg_tokens > available_tokens:
        break
    history_messages.insert(0, dict(message))

---

4. LLM API Call

call_llm()

Calls the configured LLM provider (llm_interaction.py:246-315):

from evennia.contrib.base_systems.ai.llm_interaction import call_llm

response = yield call_llm(script, messages)
# Returns LLMResponse or None on failure

Applies configuration from script:

• llm_temperature
• llm_top_p
• llm_max_tokens
• llm_reasoning_effort
• llm_extra_params

Provider Detection

Checks if provider supports native tool calling:

from evennia.contrib.base_systems.ai.llm_interaction import provider_supports_native_tools

# True for: openai, anthropic, openrouter, ollama
supports = provider_supports_native_tools(script)

---

5. Response Parsing

parse_tool_call()

Extracts tool call from LLM response (llm_interaction.py:383-501):

from evennia.contrib.base_systems.ai.llm_interaction import parse_tool_call

tool_call = parse_tool_call(script, response)
# {
#     "tool": "say",
#     "parameters": {"message": "Hello"},
#     "reasoning": "Greeting the user",
#     "tool_call_id": "call_abc123"  # None for JSON fallback
# }

Native vs JSON Fallback

Provider Type	Format	tool_call_id
Native (OpenAI, etc.)	LLMResponse.tool_calls	Present
JSON Fallback	{"tool": ..., "parameters": ..., "reasoning": ...}	None

Fallback JSON Extraction

If response isn't pure JSON, attempts to find embedded JSON object:

# Finds JSON even with surrounding text
start = response.find("{")
# ... balanced brace matching ...
json_str = response[start:end]
tool_call = json.loads(json_str)

---

6. Context Compaction

generate_context_summary()

Summarizes messages before trimming (llm_interaction.py:317-381):

from evennia.contrib.base_systems.ai.llm_interaction import generate_context_summary

summary = yield generate_context_summary(script, messages_to_compact)

Uses configurable prompt (script.db.compact_prompt) and optional dedicated model (script.db.compact_model).

---

7. Reflection Trigger

trigger_reflection()

Initiates automatic reflection (llm_interaction.py:775-818):

from evennia.contrib.base_systems.ai.llm_interaction import trigger_reflection

trigger_reflection(script, character)
# Adds reflection event to pending_events queue

Reflection events use event_type="reflection" for minimal prompt assembly.

---

Key Files

File	Lines	Purpose
llm_interaction.py	34-73	determine_context_type()
llm_interaction.py	76-108	get_tools_for_context()
llm_interaction.py	110-226	Tool schema functions
llm_interaction.py	246-315	call_llm()
llm_interaction.py	317-381	generate_context_summary()
llm_interaction.py	383-501	Response parsing
llm_interaction.py	503-763	build_llm_messages()
llm_interaction.py	775-818	trigger_reflection()

---

See also: Architecture-LLM-Providers | Architecture-Prompt-System | Architecture-Context-System