Custom Leo Agent Mode — Own Prompt, Own Tools, Authenticated Rails API

⚠️ Cookbook example — not live code. (KEEP THIS CALLOUT.) Every code block below is an example snippet, not part of the llamapress.ai codebase, and not running on this server. This is a reference recipe for a Leo instance (an AI coding agent) to implement in its own app — read it to understand the pattern, then recreate it there.

Leo's chat has a mode dropdown (Engineer, Beginner, Database…). Each mode routes your message to a different LangGraph agent — a graph with its own system prompt, tool set, and model. This recipe adds a new mode of your own: a domain expert that knows your app, reads your data through an authenticated Rails API call, and answers as that persona. It is three files on a host mount, no image rebuild, and usually no restart.

The auth story is the important half. Your agent gets one HTTP tool, rails_api_request, which calls your own Rails app as the signed-in user using a per-user signed token. Rails verifies the token and applies its own permission gates, so the agent can never see more than the person chatting with it can.

Know who is on the other end. The people who pick a custom mode from the dropdown are almost always non-technical — a business owner, an office manager, a salesperson. They did not write the app and they do not want to hear about it. Your system prompt is what keeps engineering detail out of their faces. Write it at a 7th-grade reading level, and borrow the voice rules from built-in Beginner mode — the section Write the prompt for a non-technical reader gives you a drop-in block.

When to use: a persona that needs your app's live data (a support triage agent, an analyst, an onboarding coach, a domain expert over your records). When not to: you only want to change wording — set a custom prompt on an existing mode instead. Also not for anything needing a shell; see Gotchas.

Minimum version: llamabot 0.5.3b. Earlier images have no layered registry and cannot load a client agent. Phase 0 below detects this in one command.

The 80/20 in one breath

1. Preflight: confirm the llamabot container has the three client mounts and that app.lib.langgraph_registry imports.
2. Write the agent at langgraph/agents/<name>/nodes.py, exposing build_workflow(checkpointer=None). Its state class must extend LlamaPressAPIState or the API token is silently dropped.
3. Write the prompt for a non-technical reader — 7th-grade reading level, no engineering words, no tool names, no error codes. Paste in the voice block below.
4. Register the graph in langgraph/langgraph.local.json (the client overlay — never the platform base).
5. Declare the dropdown entry in langgraph/agent_modes.json.
6. Verify headlessly with two docker exec one-liners: the graph builds, and the mode entry survives server-side validation.

How the registry is layered

LlamaBot is the platform (FastAPI + LangGraph runtime, shipped as the kody06/llamabot image). Leonardo is the client repo on the instance, and its langgraph/ directory is mounted into the llamabot container. The agent registry is read as two tiers and deep-merged at read time:

base langgraph.json  <  langgraph.d/*.json  <  langgraph.local.json
(baked into image)      (optional drop-ins)    (yours — host-mounted, wins)

Everything you write lives in the mounted client files. That is what makes a custom mode survive an image update.

Phase 0 — preflight

# Find the Leonardo checkout — usually ~/Leonardo, sometimes ~/dev/Leonardo:
ls -d ~/Leonardo ~/dev/Leonardo 2>/dev/null

# Find the container and confirm the three client mounts exist:
docker ps --format '{{.Names}}\t{{.Image}}' | grep -i llamabot
docker inspect <llamabot-container> --format '{{json .Mounts}}' | python3 -m json.tool

# Confirm the image supports the layered registry (this is the version gate):
docker exec <llamabot-container> python -c \
  "from app.lib.langgraph_registry import load_graphs; print(sorted(load_graphs()))"

If the import fails, the image predates the layered registry — stop and update the instance first. If a mount is missing (older compose file), add the volume lines to the llamabot service, create the host files before recreating (langgraph.local.json → {"graphs": {}}, agent_modes.json → []; a missing host file gets created as a directory), then docker compose up -d --force-recreate llamabot.

⚠️ From here on, work only in the Leonardo langgraph/ directory on the host. Never edit the base /app/app/langgraph.json inside the container — it is platform-owned and gets overwritten by image updates.

Layer 1 — The agent

The contract is one function: build_workflow(checkpointer=None) returning a compiled StateGraph. Platform code imports through the app. package.

# langgraph/agents/<name>/nodes.py   (container: /app/app/user_agents/<name>/nodes.py)
from langchain_core.messages import SystemMessage
from langgraph.graph import START, StateGraph
from langgraph.prebuilt import ToolNode, tools_condition

# ALWAYS get models via the factory — never hardcode ChatOpenAI/Gemini/etc.
from app.agents.leonardo.llm_factory import get_llm

# Authenticated HTTP into the Rails app AS THE SIGNED-IN USER. Rails verifies the
# per-user token and applies its own permission gates; LlamaBot cannot forge one.
from app.lib.llamapress_api import LlamaPressAPIState, rails_api_request

# Every tool here is bounded only by what it does itself. rails_api_request is
# bounded by the user's Rails permissions; a bash/exec tool would NOT be, and
# would hand whoever can select this mode the Rails console. Add deliberately.
tools = [rails_api_request]

SYS_MSG = """You are <persona / job description here>.

You are Leo, in "<Mode Label>" mode. The person you are talking to is NOT an
engineer. Write everything at a 7th-grade reading level. (Paste the full
"WHO YOU ARE" + "HOW YOU TALK" block from the next section here — it is what
keeps Leo's name intact and engineering detail out of the reply.)

You can call this app's JSON API with the `rails_api_request` tool. It runs as
the signed-in user, with their permissions. If a request comes back denied, that
is a correct answer, not a bug — say "you don't have access to that" in plain
words and stop. Never mention status codes, tool names, or paths to the user.
Only /api/ paths are reachable.

Available endpoints:
  GET  /api/...
"""


# REQUIRED subclass: puts api_token on the state schema. LangGraph drops
# undeclared frame fields, so plain MessagesState silently disables the tool.
class AgentModeState(LlamaPressAPIState):
    agent_prompt: str


def assistant(state: AgentModeState):
    llm = get_llm("deepseek-v4-flash")   # or another key llm_factory supports
    llm_with_tools = llm.bind_tools(tools)
    extra = state.get("agent_prompt") or ""
    sys = SystemMessage(
        content=f"{SYS_MSG}\n<DEVELOPER_INSTRUCTIONS>{extra}</DEVELOPER_INSTRUCTIONS>"
    )
    return {"messages": [llm_with_tools.invoke([sys] + state["messages"])]}


def build_workflow(checkpointer=None):
    builder = StateGraph(AgentModeState)
    builder.add_node("assistant", assistant)
    builder.add_node("tools", ToolNode(tools))
    builder.add_edge(START, "assistant")
    builder.add_conditional_edges("assistant", tools_condition)
    builder.add_edge("tools", "assistant")
    return builder.compile(checkpointer=checkpointer)

Accept the standard fields the Rails frontend sends: message, thread_id, api_token, agent_prompt. The app.lib.llamapress_api import path is the stability contract — never reach for Rails-auth helpers from deeper platform paths.

Write the prompt for a non-technical reader

This is the half that decides whether the mode is usable. A custom mode is picked from a dropdown by whoever is signed in, and that is usually a business user, not an engineer. Built-in Engineer mode assumes a developer is reading. Beginner mode does not — and a custom mode should copy Beginner mode's voice rules, because it faces the same audience.

Four failure modes to design against:

1. Identity drift. A custom mode is still Leo — the same assistant the user has been talking to, now doing a specific job. If your prompt invents a new name, the user thinks they've been handed off to a different bot. Keep the name and add the mode: "I'm Leo — I'm in Order Desk mode."
2. Jargon. "I queried the endpoint and the controller returned a 403" means nothing to an office manager, and it makes them feel stupid.
3. Leaked internals. Tool names, file paths, table and column names, status codes, and stack traces are engineering detail. They are noise at best, and a hint about how to poke at your app at worst.
4. Walls of text. Long replies get skimmed, then abandoned.

Paste this block into your SYS_MSG, then edit the endpoint list for your persona:

## WHO YOU ARE

You are Leo, in "<Mode Label>" mode. That is how you introduce yourself, and it
is the name you answer to. You are not a separate assistant with its own name —
you are the same Leo the user already knows, doing a specific job right now.

If the user asks who you are, say it in one sentence: "I'm Leo — I'm in
<Mode Label> mode, so I'm here to <one-line job>." Then get on with helping.

Never ask the user who you are or what your name is. Never ask them their name.
Never mention LangGraph, agents, graphs, modes as software, or the dropdown.
"Mode" is the only word you need, and only if they ask.

If the file .leonardo/IDENTITY.md exists, it may give you a different name and
emoji. Use that name instead of "Leo", keeping the "in <Mode Label> mode" part.

## HOW YOU TALK

The person you are talking to is NOT an engineer. Treat every message as if you
are talking to a smart friend who has never written code. They are smart. They
are not technical.

Read like a 7th grader. Short words. Short sentences. No jargon.

BANNED WORDS (never use these unless the user used them first):
  model, controller, view, partial, scaffold, migration, schema, route,
  endpoint, API, JSON, query, callback, repo, branch, commit, deploy,
  backend, frontend, database, table, column, gem, dependency, function,
  method, class, parameter, variable, token, 403, 404, 500, stack trace

PLAIN-ENGLISH SWAPS:
  database / table / record   -> "your saved information"
  query / API call / endpoint -> "I looked it up"
  403 / not authorized        -> "you don't have access to that"
  404 / not found             -> "I couldn't find that"
  500 / exception             -> "something went wrong on my end"
  validation                  -> "rule"
  field / column              -> "box" or the human label the user sees

LENGTH: most replies are 2-4 short sentences. Never write a wall of text.

TONE: warm, calm, encouraging. Confusion is normal.

NEVER describe your own process. Do not say "I called the tool", "I made a
request", "the API returned", or name a file or path. Say what you FOUND, not
how you found it. Bad: "GET /api/orders returned 12 rows." Good: "You have 12
open orders right now."

If something fails, say so plainly in one sentence and say what they can do
next. Never paste an error message, a status code, or a stack trace.

Two rules that are easy to get wrong:

• Reading level is not the same as hiding problems. The agent should still say clearly when it cannot do something. It just says "you don't have access to that" instead of "403 on /api/reports".
• The agent still needs the technical facts. Keep the endpoint list, the tool constraints, and the page context in the prompt — the model reads them. The rules above govern what comes out in the reply, not what goes in.

Test it, don't assume it. Ask the mode a question that must fail — a record the user cannot see — and read the reply out loud. If a non-technical friend would need you to translate it, the prompt is not done.

Layer 2 — Register the graph

// langgraph/langgraph.local.json
{
  "graphs": {
    "<agent_name>": "./user_agents/<name>/nodes.py:build_workflow"
  }
}

The path is container-relative: ./user_agents/..., not ./langgraph/agents/.... Overlay keys win on collision, so you can deliberately shadow a platform agent — just never do it by accident.

Layer 3 — Declare the dropdown entry

// langgraph/agent_modes.json   — a JSON ARRAY
[
  {
    "key": "<mode_key>",
    "label": "My Mode",
    "agent_name": "<agent_name>",
    "description": "What this mode is for",
    "shortLabel": "MyMode"
  }
]

Rules, enforced server-side on every chat page load. A violation is silently dropped from the dropdown, so verify (Layer 5) rather than assuming:

• key, label, agent_name are required, non-empty strings.
• agent_name must exactly match a graph registered in Layer 2.
• key must not shadow a built-in mode key. Check the live set:

  docker exec <llamabot-container> python -c \
    "from app.permissions import BUILTIN_AGENT_MODE_KEYS; print(sorted(BUILTIN_AGENT_MODE_KEYS))"
  

• Duplicate keys: first wins.

Permissions: a custom mode rides along with a role's default mode grant, so an engineer-role user sees it immediately. If an admin has explicitly configured a role's mode list (the role_agent_modes SiteSetting), you must opt the new key in there or that role won't see it.

Layer 4 — Restart (usually not needed)

Layer 5 — Verify headlessly

1. The graph registers and builds (run after every nodes.py edit):

docker exec <llamabot-container> python -c "
from app.lib.langgraph_registry import load_graphs
graphs = load_graphs(); assert '<agent_name>' in graphs, graphs
import importlib.util
spec = importlib.util.spec_from_file_location('m', '/app/app/user_agents/<name>/nodes.py')
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
print('builds:', m.build_workflow() is not None)"

2. The mode entry survives validation:

docker exec <llamabot-container> python -c "
from app.routers.ui import load_custom_agent_modes
from app.lib.langgraph_registry import load_graphs
print(load_custom_agent_modes('/app/app/agent_modes.json', load_graphs()))"

Expect your entry in the list. An empty list means it was dropped — recheck the Layer 3 rules and grep the container logs for a Dropping custom agent mode warning.

3. A live authenticated Rails call (only if your agent uses rails_api_request). Mint a genuine per-user token in the Rails container and push it through the tool:

TOKEN=$(docker compose exec -T llamapress sh -c 'cd /rails && bundle exec rails runner \
  "puts Rails.application.message_verifier(:llamabot_ws).generate({session_id: SecureRandom.uuid, user_id: User.first.id}, expires_in: 30.minutes)"' | tail -1)

docker exec -e T="$TOKEN" <llamabot-container> python -c "
import os
from app.lib.llamapress_api import rails_api_request
class R: state={'api_token': os.environ['T']}; tool_call_id='t'
print(rails_api_request.func(method='GET', path='/api/users', runtime=R())[:200])"

Expect HTTP 200 + JSON. Decode table:

4. End to end: open the chat UI as an engineer-role user → the mode appears in the dropdown → send a message → it routes to your graph.

Optional — page context, current user, and browser control

Built-in modes know what page the user is on and can drive the browser. A custom mode gets the same plumbing — none of it is reserved for built-ins. You just have to opt in.

A. What page is the user on? (debug_info)

The chat frontend asks the Rails iframe for page context and attaches it to every WebSocket message, in every mode. The payload:

{
  "request_path":   "/contacts/4",
  "view_path":      "app/views/contacts/show.html.erb",
  "full_html":      "<html>…</html>",
  "page_loaded_at": "…"
}

The request handler copies frame fields into graph state only if the state schema declares them. So declare the field, then inject it into the system message each turn:

# langgraph/agents/<name>/nodes.py
from typing import Any
from typing_extensions import NotRequired


class AgentModeState(LlamaPressAPIState):
    agent_prompt: str
    debug_info: NotRequired[dict[str, Any]]   # <- without this line, it never arrives


def assistant(state: AgentModeState):
    llm_with_tools = get_llm("deepseek-v4-flash").bind_tools(tools)
    extra = state.get("agent_prompt") or ""
    debug = state.get("debug_info") or {}     # can be absent — always fall back
    page_ctx = ""
    if debug.get("request_path"):
        page_ctx = (
            f'\nThe user is currently viewing page "{debug["request_path"]}"'
            f' (rendered by "{debug.get("view_path", "unknown")}").'
        )
    sys = SystemMessage(
        content=f"{SYS_MSG}{page_ctx}\n<DEVELOPER_INSTRUCTIONS>{extra}</DEVELOPER_INSTRUCTIONS>"
    )
    return {"messages": [llm_with_tools.invoke([sys] + state["messages"])]}

Four things to know:

• This is exactly what built-in modes do — the platform's ViewPathContextMiddleware emits a <CONTEXT page="…" file="…"/> tag. A raw StateGraph just does it inline.
• debug_info can be missing entirely, so always code the or {} fallback. The two causes are the iframe not having loaded yet, and the Rails page not rendering the page-context partial.
• Declare the nested dict, not the fields inside it. The frame carries ONE key, debug_info. There are no top-level request_path / view_path keys, so a state class that declares request_path: str gets nothing — no error, just an agent that insists it cannot see your screen. (Seen 2026-07-28 on the MBC preceptors box's "Scheduling" mode. Confirm the real frame shape from the box's own logs: docker compose logs llamabot | grep 'Created state with keys'.)
• full_html is the whole rendered page, often 100KB+. Don't paste it into the prompt wholesale. Omit it, truncate it, or include it only when the user's question is about the page's content.
• view_path is a file path — it is for the model, not the user. A non-technical user should never see app/views/contacts/show.html.erb in a reply. If your prompt includes it, add one line: "Never repeat the file name back to the user. Refer to the page the way they see it, by its title."

B. Who is the signed-in user?

debug_info does not carry the user — but the agent already is the user, because rails_api_request sends their user-scoped token. Expose a minimal identity endpoint:

# app/controllers/api/me_controller.rb
module Api
  class MeController < ApplicationController
    include LlamaBotRails::AgentAuth
    llama_bot_allow :show

    def show
      render json: current_user.slice(:id, :email, :name)   # only what the mode needs
    end
  end
end

# config/routes.rb
namespace :api do
  get "me", to: "me#show"
end

Then list it in the system prompt: GET /api/me → the currently signed-in user.

C. Driving the user's live browser tab

Three platform tools act on the Rails iframe in the chat UI. Import them — don't reimplement them:

# langgraph/agents/<name>/nodes.py
from app.agents.leonardo.rails_agent.tools import (
    navigate_browser,          # navigate the iframe to a path
    get_browser_js_logs,       # fetch (and clear) captured JS console logs
    execute_browser_js,        # run arbitrary JS in the page, return the result
    live_browser_tools_enabled,
)


def _build_tools():
    t = [rails_api_request]
    if live_browser_tools_enabled():        # honor the instance gate — never bypass it
        t.extend([navigate_browser, get_browser_js_logs])
        # execute_browser_js: add ONLY if this mode truly needs arbitrary JS.
    return t


def build_workflow(checkpointer=None):
    tools = _build_tools()                  # build once, at compile time
    builder = StateGraph(AgentModeState)
    builder.add_node("assistant", make_assistant(tools))
    builder.add_node("tools", ToolNode(tools))
    ...
    return builder.compile(checkpointer=checkpointer)   # REQUIRED — see below

How they work, so you can debug them: the tool raises a LangGraph interrupt → the platform forwards it to the chat frontend as a browser_command WebSocket frame → the frontend runs it against the iframe → the result resumes the graph. That mechanism has three consequences:

• They need the checkpointer. Interrupt/resume persists state between the two halves. Always forward the checkpointer argument into .compile() — the Layer 1 template already does.
• They are gated by the enable_live_browser_tools site setting, default off. The setting is read when the tool list is built, which is workflow compile time — so after flipping it, docker compose restart llamabot.
• They only work with a live chat tab open. In a headless or background run there is no browser to answer. Say so in the system prompt, so the agent reports the failure instead of retrying forever.

Threat model: navigate_browser is tame. execute_browser_js runs arbitrary code in the signed-in user's session — treat adding it like adding a shell tool, and leave it out unless the mode's job requires it.

Optional — expose a new Rails endpoint for the agent

The tool can only reach paths under /api/. To give your agent new data, add a normal Rails controller in that namespace:

# app/controllers/api/reports_controller.rb
module Api
  class ReportsController < ApplicationController
    include LlamaBotRails::AgentAuth

    # Only allowlisted actions are reachable with an agent token; everything else 403s.
    llama_bot_allow :index, :show

    # Rails only exempts the `Bearer` scheme from CSRF, not `LlamaBot`. Re-enable
    # per controller — never blanket-skip.
    skip_before_action :verify_authenticity_token
    before_action :verify_authenticity_token,
                  unless: -> { llama_bot_request? || api_request? }

    def index
      render json: Report.where(user: current_user).limit(50)
    end
  end
end

# config/routes.rb
namespace :api do
  resources :reports, only: [:index, :show]
end

Strong params are the escalation gate on any write action — never permit :admin, :role, or an ownership foreign key.

Gotchas (the hard-won stuff)

• Your users are not engineers — the default LLM voice is. Without explicit voice rules, a model will happily say "the API returned a 403 for /api/reports". Copy the Beginner-mode block above; it is the single highest-value part of the prompt.
• A custom mode is still Leo. Don't give it a new name. Introduce it as "Leo, in <Mode Label> mode" so the user knows it's the same assistant with a different job.
• Reading level applies to failures too. The most common leak is an error path — the happy path was written carefully and the except branch pastes a stack trace. Test a request you know will be denied.
• Forgetting LlamaPressAPIState breaks auth silently. LangGraph drops fields the state schema doesn't declare, so api_token never reaches the tool and the agent reports a vague setup problem. If rails_api_request says it has no token, this is almost always why.
• Never pair rails_api_request with an unbounded tool (bash_command, raw exec, a generic SQL runner) in the same agent. The API tool is safe because Rails gates it per user; a shell tool hands everyone who can select the mode a Rails console.
• The graph path is container-relative. ./user_agents/<name>/nodes.py, not the host path ./langgraph/agents/.... A wrong path shows up as "unknown agent" on the first message, not at registration time.
• Python edits don't take without a restart. There is no hot-reload in the backend. New files are picked up on demand; edits to an already-imported module are not.
• Edits made inside the container vanish on the next update. If your custom mode disappears after an instance update, it was written to a container path instead of the mounted host files.
• A dropped mode entry fails silently. No error in the UI — the dropdown just doesn't show it. Always run verification step 2.
• Undeclared state fields are dropped, and debug_info is the usual casualty. Same root cause as the api_token bug above: if the state class doesn't declare debug_info, the agent never learns what page the user is on and has no error to show for it.
• Browser tools need the checkpointer and a live tab. They work by interrupt/resume, so a graph compiled without the checkpointer argument breaks on the first browser call, and a headless run has nothing to answer the interrupt.
• Flipping enable_live_browser_tools needs a restart. The tool list is built at workflow compile time, so the setting change doesn't reach an already-compiled graph.
• Don't route around the tool's limits. /api/-only, no redirects, no other hosts, and no path traversal are deliberate. A 403 is a correct, reportable answer.
• Always use get_llm(...). Hardcoding ChatOpenAI or a Gemini class ties the agent to a provider key that may not exist on the box, and the failure happens at compile time, so the whole backend fails to boot — not just your mode.

Troubleshooting quick table

Files this pattern touches

langgraph/agents/<name>/nodes.py          # the agent graph
langgraph/langgraph.local.json            # client graph registry (overlay)
langgraph/agent_modes.json                # chat dropdown entries
config/routes.rb                          # optional: new /api/ route
app/controllers/api/<name>_controller.rb  # optional: new /api/ endpoint

How to adapt this to your app

1. Name the job, not the assistant. The system prompt is most of the value — describe the job, the tone, and what a good answer looks like, then list the exact endpoints it may call. Keep the assistant's name as Leo; the mode label is the job title.
2. Paste the voice block before anything else. Reading level and no-leak rules are what make the mode safe to hand to a non-technical user. Add the persona on top.
3. Start read-only. Ship with GET-only endpoints allowlisted via llama_bot_allow. Add writes once the persona is behaving, one action at a time.
4. Swap the model in get_llm("...") to whatever your box has keys for. Cheaper models are usually fine for a narrow, well-prompted persona.
5. Add your own tools the same way — append to the tools list. Keep each one narrowly scoped; the safety of the whole mode is the union of its tools.
6. Give it eyes before hands. Add debug_info (page context) and a read-only GET /api/me early — they make the persona feel aware for almost no risk. Add browser tools later, and add execute_browser_js last, if ever.
7. Safe to drop: the agent_prompt field if you don't want per-instance prompt overrides, the whole Rails-endpoint section if the persona needs no app data, and the entire browser-control section if the mode only answers questions.