blightbow/evennia_ai

Fork 0

Bug: Workbench component reordering is UI-only, does not affect actual LLM prompts #16

New issue

Closed

opened 2025-12-10 01:38:55 +00:00 by blightbow · 1 comment

blightbow commented

2025-12-10 01:38:55 +00:00

Owner

Summary

The workbench UI allows drag-and-drop reordering of prompt components, but this has no effect on actual LLM prompt assembly. The reordering is cosmetic only.

What Exists

script.db.context_configs[context_type]["component_order"] stores user-defined order
api/views/mixins/workbench.py:workbench_reorder() saves this order
api/serializers/workbench.py uses it to sort components for display
static/ai/component_tree.js enables drag-and-drop reordering

What Does NOT Exist

llm_interaction.py:build_llm_messages() ignores component_order

Prompt assembly uses hardcoded order:

system → character → entity → memories → buffer → goals → history → event → tool_result

Reordering in workbench changes display but NOT actual LLM prompt order

Expected Behavior

When a user reorders components in the workbench UI, the actual LLM prompt should reflect that order.

Actual Behavior

Reordering only affects workbench display. The LLM receives components in a fixed hardcoded order regardless of user settings.

Files Involved

evennia/contrib/base_systems/ai/llm_interaction.py:build_llm_messages() - needs to read and apply component_order
evennia/contrib/base_systems/ai/api/views/mixins/workbench.py - already saves order correctly
evennia/contrib/base_systems/ai/api/serializers/workbench.py - already reads order for display

Suggested Fix

Refactor build_llm_messages() to:

Read component_order from script.db.context_configs
Build messages according to user-defined order
Fall back to default order if no custom order defined

## Summary The workbench UI allows drag-and-drop reordering of prompt components, but this has **no effect on actual LLM prompt assembly**. The reordering is cosmetic only. ## What Exists - `script.db.context_configs[context_type]["component_order"]` stores user-defined order - `api/views/mixins/workbench.py:workbench_reorder()` saves this order - `api/serializers/workbench.py` uses it to sort components for display - `static/ai/component_tree.js` enables drag-and-drop reordering ## What Does NOT Exist - `llm_interaction.py:build_llm_messages()` **ignores** `component_order` - Prompt assembly uses hardcoded order: ``` system → character → entity → memories → buffer → goals → history → event → tool_result ``` - Reordering in workbench changes display but NOT actual LLM prompt order ## Expected Behavior When a user reorders components in the workbench UI, the actual LLM prompt should reflect that order. ## Actual Behavior Reordering only affects workbench display. The LLM receives components in a fixed hardcoded order regardless of user settings. ## Files Involved - `evennia/contrib/base_systems/ai/llm_interaction.py:build_llm_messages()` - needs to read and apply `component_order` - `evennia/contrib/base_systems/ai/api/views/mixins/workbench.py` - already saves order correctly - `evennia/contrib/base_systems/ai/api/serializers/workbench.py` - already reads order for display ## Suggested Fix Refactor `build_llm_messages()` to: 1. Read `component_order` from `script.db.context_configs` 2. Build messages according to user-defined order 3. Fall back to default order if no custom order defined

blightbow added this to the Prompt System Foundation: Injection Depth + Macros milestone

2025-12-10 01:38:55 +00:00

blightbow referenced this issue

2025-12-10 01:39:09 +00:00

Feature: Pluggable component support for character card fields #19

blightbow commented

2025-12-13 02:59:09 +00:00

Author

Owner

Fix Implemented

Refactored build_llm_messages() in llm_interaction.py to respect user-defined component_order from context configs.

Changes

Core fix: Components are now built into a dict by ID, then assembled according to the user-defined component_order from script.db.context_configs[context_type]["component_order"]
Fallback behavior: When no custom order is defined (or the list is empty), components are rendered in default ascending ID order (0, 1000, 1500, 2000, 3000, 4000, 5000)
New tests: Added test_component_reordering.py with 9 tests covering:
- Custom order is respected
- Default order without custom config
- Missing components in order are skipped
- Extra components not in order are excluded
- Conversation history respects ordering
- Edge cases (empty order, duplicates, etc.)
Documentation: Updated context-examples README and coverage_matrix to document component ordering feature

Files Modified

evennia/contrib/base_systems/ai/llm_interaction.py - Refactored build_llm_messages()
evennia/contrib/base_systems/ai/tests/test_component_reordering.py - New test file
evennia/contrib/base_systems/ai/docs/context-examples/README.md - Added ordering docs
evennia/contrib/base_systems/ai/docs/context-examples/coverage_matrix.md - Added ordering section

Test Results

All 27 prompt-related tests pass (18 rendering + 9 reordering)
Full test suite: 2045 passed, 1 skipped, 0 failed

## Fix Implemented Refactored `build_llm_messages()` in `llm_interaction.py` to respect user-defined `component_order` from context configs. ### Changes 1. **Core fix**: Components are now built into a dict by ID, then assembled according to the user-defined `component_order` from `script.db.context_configs[context_type]["component_order"]` 2. **Fallback behavior**: When no custom order is defined (or the list is empty), components are rendered in default ascending ID order (0, 1000, 1500, 2000, 3000, 4000, 5000) 3. **New tests**: Added `test_component_reordering.py` with 9 tests covering: - Custom order is respected - Default order without custom config - Missing components in order are skipped - Extra components not in order are excluded - Conversation history respects ordering - Edge cases (empty order, duplicates, etc.) 4. **Documentation**: Updated context-examples README and coverage_matrix to document component ordering feature ### Files Modified - `evennia/contrib/base_systems/ai/llm_interaction.py` - Refactored `build_llm_messages()` - `evennia/contrib/base_systems/ai/tests/test_component_reordering.py` - New test file - `evennia/contrib/base_systems/ai/docs/context-examples/README.md` - Added ordering docs - `evennia/contrib/base_systems/ai/docs/context-examples/coverage_matrix.md` - Added ordering section ### Test Results - All 27 prompt-related tests pass (18 rendering + 9 reordering) - Full test suite: 2045 passed, 1 skipped, 0 failed

blightbow referenced this issue

2025-12-14 06:23:07 +00:00

Add Render Preview API endpoint for prompt assembly visibility #25

blightbow referenced this issue

2025-12-14 06:23:08 +00:00

Add visual distinction for terminal components in workbench UI #26

blightbow referenced this issue from a commit

2025-12-14 09:28:39 +00:00

Separate terminal components from reorderable zone

blightbow closed this issue

2025-12-14 09:28:39 +00:00