1 Architecture Self Management
blightbow edited this page 2025-12-08 03:53:47 +00:00

Architecture: Self-Management

Layer 2 - Projects, Goals, Session Memory, and Self-Monitoring


Overview

The AI assistant has a "cognitive workspace" for managing its own state across long-running operations. This includes:

  • Projects - Pageable context slots for multi-tasking
  • Goals - Hierarchical task tracking with automatic rollup
  • Session Memory - Persistent facts and patterns
  • Self-Monitoring - Token usage and system status awareness
  • Self-Modification - Behavioral guideline updates

1. Projects System

Projects enable context-switching between long-running tasks without losing state.

Data Structure

Stored on character.db.projects:

db.projects = {
    "active": "build_tavern",      # Currently loaded project key
    "slots": {
        "build_tavern": {
            "summary": "Construct a tavern in the north district",
            "context": "Progress: foundation complete, walls 50%...",
            "status": "active",     # active | paused | completed
            "created_at": "2025-12-06T10:00:00Z",
            "last_active": "2025-12-06T14:30:00Z",
        },
        "quest_design": {...},
    }
}

Project Tools

Tool Category Purpose
create_project SAFE_CHAIN Create a new project slot with summary and initial context
list_projects SAFE_CHAIN List all projects with summaries and status (cacheable)
swap_project SAFE_CHAIN Switch to a different project, saving current progress
update_project SAFE_CHAIN Update active project's context without swapping

Context Paging Pattern

Projects implement "paging out" context - similar to OS process swapping:

Before swap:
  Active: build_tavern (full context loaded)
  Paused: quest_design (context stored in slot)

swap_project("quest_design", current_project_update="Walls 60% done")

After swap:
  Active: quest_design (context restored)
  Paused: build_tavern (context = "Walls 60% done")

When to Use Projects

  • Complex building tasks spanning multiple sessions
  • Long-term monitoring objectives
  • Player-requested tasks that may be interrupted
  • Any work requiring substantial context that shouldn't be lost

2. Goals System

Goals provide hierarchical task tracking with automatic parent-child progress rollup.

Data Structure

Stored on script.db.current_goals:

db.current_goals = [
    {
        "id": "goal_42_0",              # Format: goal_{tick}_{index}
        "description": "Build a complete tavern",
        "priority": "high",              # high | medium | low
        "status": "active",              # active | completed | abandoned
        "progress": 33,                  # 0-100 percentage
        "created": "2025-12-06T10:00:00Z",
        "parent_id": None,               # Root goal
        "subtask_ids": ["goal_42_1", "goal_42_2", "goal_42_3"],
        "auto_generated": False,
    },
    {
        "id": "goal_42_1",
        "description": "Lay foundation",
        "parent_id": "goal_42_0",        # Child of tavern goal
        "subtask_ids": [],
        "progress": 100,
        "status": "completed",
        ...
    },
]

Goal Tools

Tool Category Purpose
add_goal SAFE_CHAIN Add a new goal with optional priority
update_goal SAFE_CHAIN Update status or progress (triggers parent rollup)
decompose_goal ASYNC_REQUIRED LLM breaks goal into 3-7 subtasks

Hierarchical Rollup

When a subtask is updated, parent goals automatically recalculate:

# Parent progress = average of subtask progress
parent.progress = sum(subtask.progress for subtask in subtasks) / len(subtasks)

# Parent auto-completes when ALL subtasks complete
if all(subtask.status == "completed" for subtask in subtasks):
    parent.status = "completed"

Goal Decomposition

The decompose_goal tool uses LLM to break complex goals into actionable subtasks:

Input:  goal_42_0 = "Build a complete tavern"

Output: Creates 3-7 subtasks:
  - goal_42_1: "Design floor plan and room layout"
  - goal_42_2: "Construct building structure"
  - goal_42_3: "Add furniture and decorations"
  - goal_42_4: "Create NPC bartender"
  - goal_42_5: "Write room descriptions"

3. Session Memory

Session memory stores persistent facts and behavioral patterns.

Data Structure

Stored on character.db.session_memory:

db.session_memory = {
    "key_facts": [
        "Player Alice prefers formal address",
        "The north gate closes at midnight",
    ],
    "learned_patterns": [
        "Players tend to explore before asking for help",
    ],
    "completed_tasks": [
        {"description": "...", "timestamp": "..."},
    ],
    "last_compacted": "2025-12-06T10:00:00Z",
}

Session Memory Tools

Tool Category Purpose
add_session_memory SAFE_CHAIN Add a fact or pattern (with persona protection)
compact_session_memory SAFE_CHAIN Consolidate and deduplicate entries

Persona Protection

Session memory tools reject self-referential content:

# Rejected patterns:
"I helped the player"           # References assistant action
"I should be more formal"       # References assistant behavior
"The assistant suggested..."    # Third-person self-reference

# Accepted patterns:
"Player Alice completed the quest"      # World state
"The tavern has 5 rooms"                # Factual information
"Players prefer concise responses"      # Pattern about players

Error response guides rephrasing:

{
    "success": False,
    "error": "Content appears to reference assistant actions or identity",
    "suggestion": "Session memory should contain facts about the WORLD, PLAYERS, "
                  "or EVENTS - not what you did. Rephrase in third-person."
}

4. Self-Monitoring

The get_system_status tool provides introspection for resource-aware decision making.

Tool Output

{
    "success": True,
    "timestamp": "2025-12-06T14:30:00Z",

    "token_usage": {
        "estimated_used": 45000,
        "model_limit": 128000,
        "available": 83000,
        "usage_percentage": 35.2,
    },

    "token_advisory": {
        "level": "normal",           # normal | warning | critical
        "message": "Sufficient context available",
        "threshold": 60,
    },

    "loop_state": {
        "iteration": 2,
        "max_iterations": 5,
        "tools_this_cycle": ["inspect_location", "recall_memories"],
        "tool_count_this_cycle": 2,
    },

    "session_metrics": {
        "total_tool_calls": 47,
        "successful_calls": 45,
        "failed_calls": 2,
        "session_start": "2025-12-06T10:00:00Z",
    },

    "chain_state": {
        "in_chain": True,
        "chain_depth": 1,
        "pending_tools": [],
    },
}

Token Advisory Levels

Level Threshold Meaning Recommended Action
normal < 60% Sufficient context Proceed normally
warning 60-80% Context filling Consider wrapping up
critical > 80% Near limit Respond immediately

5. Self-Modification

The update_system_prompt tool allows behavioral guideline updates.

Tool Parameters

{
    "new_prompt": "Be concise. Prefer actions over explanations...",
    "reasoning": "Players prefer direct responses without preamble",
}

Constraints

  • Modifiable: character.db.assistant_system_prompt (self-managed)
  • Protected: character.db.user_system_prompt (immutable by assistant)

The user's system prompt always takes precedence and cannot be overwritten.


6. When to Use What

Decision Table

Situation Use Reason
Complex multi-session task Project Preserves detailed context across swaps
Trackable outcome with subtasks Goal Hierarchical progress with rollup
Player preference discovered Session Memory (fact) Persistent, influences future behavior
Behavioral insight Session Memory (pattern) Guides decision-making
One-off simple request Neither No tracking overhead needed

Projects vs Goals

Aspect Projects Goals
Storage Character (db.projects) Script (db.current_goals)
Purpose Context preservation Progress tracking
Structure Flat slots Hierarchical tree
Switching Explicit swap Always visible
Use case Multi-session work Task decomposition

Memory Tier Selection

Memory Type Lifespan Use For
Session Memory Permanent (DB) Player preferences, world facts
Journal Permanent (DB) Narrative reflection, meta-learning
RAG/Mem0 Permanent (external) Long-term world knowledge
Entity Profiles Permanent (DB) Per-entity relationships
Working Memory Session (NDB) Active conversation context

Key Files

File Lines Purpose
tools/context.py 14-167 Session memory tools
tools/context.py 244-545 Project tools
tools/context.py 547-685 GetSystemStatusTool
tools/goals.py 13-159 Goal tools
assistant_script.py 450-580 Goal management methods
assistant_character.py 180-250 Attribute initialization

See also: Architecture-Overview | Architecture-Memory-and-Sleep | Architecture-Tool-System