Developer Guide
Runtime exports
Only manifest is required. Export the rest when you need them.
manifestobjectrequiredIdentity, flags, and surface registration.getTools()() => Tool[]Tools the model may call while the extension is enabled.executeTool(name, args, ctx)=> Promise<any>Runs a tool call. Required if getTools returns anything.useExtension()() => ExtensionStateReact hook that reports live status, prompt state, and instances. Optional; the host supplies a default.getPromptContributions(ctx)=> Contribution[] | Promise<Contribution[]>Ambient instructions merged into the system prompt every turn.onEnable(personaId) / onDisable(personaId)=> Promise<void>Setup and teardown when the extension is toggled for a persona.onBeforeUserMessageSend(ctx)=> Promise<ChatMutation | void>Rewrite or block the outgoing user message.onBeforeLLMRequest(ctx)=> Promise<LLMRequestMutation | void>Per round: replace the tool list, add instructions, or attach images.onAfterToolCall(ctx) / onAfterToolResult(ctx)=> Promise<void>Observe every tool call in the turn, including other extensions' tools.onAfterAssistantChunk(ctx) / onAfterAssistantMessage(ctx)=> Promise<void>Observe streamed text and the finished reply.onEmergencyToyStop(ctx)=> Promise<void>Toy extensions: halt output when the Toy E-Stop fires.Tools
A tool is an OpenAI-style function definition. parameters is a JSON Schema object with type: 'object', a properties map, and an optional required array. Write the description for the model: when to call it and what each argument means.
export function getTools() {
return [
{
type: 'function',
function: {
name: 'send_snapshot',
description: 'Send the user a photo matching the description.',
parameters: {
type: 'object',
properties: {
description: { type: 'string', description: 'What the photo shows.' },
mood: { type: 'string', enum: ['soft', 'teasing', 'stern'] },
},
required: ['description'],
},
},
// Optional. Only the chat UI sees these; they are stripped before the model does.
displayLabels: {
pending: 'Sending photo…',
success: 'Sent you a photo',
error: "Couldn't send the photo",
},
},
];
}
export async function executeTool(name, args, ctx) {
if (name !== 'send_snapshot') throw new Error(`Unknown tool: ${name}`);
const record = await ctx.runtime.emitImage({
url: await renderPhoto(args.description, args.mood),
prompt: args.description,
sourceExtensionKey: ctx.extensionKey,
sourceToolName: name,
sourceKind: 'tool',
displayMode: 'gallery',
});
// Whatever you return is serialised and sent back to the model as the tool result.
return { ok: true, artifactId: record.id };
}Names and collisions
Name tools however reads well, but avoid the app's own names. Built-in tools claim their names first: send_photo, change_pose, change_clothing, strip_clothing, change_location, end_turn, the memory tools (add_memory, search_memory, edit_memory, delete_memory), the setup tools (setup_user_nickname, setup_pronouns, setup_genitals), and every name starting with scenario_action__ or estim_. A tool that collides with one of those, or with an earlier-installed extension's tool, is exposed to the model as ux_<key>_<name> with dashes in the key turned into underscores. If even the prefixed name collides, the tool is dropped with a console warning.
Your executeTool always receives the original name. When a hook needs the name the model actually sees, call ctx.resolveOwnToolName('name') rather than building the prefix yourself. Tools are withheld on the last permitted tool round of a turn, set by the user's Max tool rounds per turn.
Execution context
executeTool receives args already parsed from JSON and a context object:
ctx.extensionKeystringYour manifest key.ctx.personaId / ctx.sessionIdstring?Active persona and chat session.ctx.currentMessageIdstring?The assistant message being written.ctx.runtimeExtensionRuntimeApiVariables, memories, chat, persona, media, UI, artifacts, and emergency stop.ctx.llmExtensionLLMContextcomplete(messages, opts?), describeImage(base64, prompt?, opts?), and supportsVision(). Uses the user's configured provider without exposing the key.ctx.visualContextobject?The persona's effective visual context when image generation is configured.ctx.resolveOwnToolName(name)(name) => stringFinal model-facing name for one of your tools.Return a JSON-serialisable value. It becomes the tool result the model reads. Throwing produces an error result and shows the tool's error label in the chat.
extensionState, so settings are not readable inside executeTool. Lifecycle hooks do receive it. Copy ctx.extensionState.resolvedSettings into a module variable in onBeforeLLMRequest, which runs before every round that can call your tools.useExtension()
A React hook the host calls on every render. It is the only way to report live status. The host merges its own saved data over the top, so several fields you return are placeholders:
Host-ownedreplacedisInstalled, globalSettings, personaSettings, resolvedSettings, and the update functions. Return anything; the host overwrites them from saved settings.isEnabledbooleanReturn true. Marketplace installs replace it with the persona's toggle. Dev-slot and single-file loads use the value you return, so a dev build is on for every persona while installed.isActivebooleanTrue while connected or running. One of the emergency-stop readiness conditions. When you omit statusIndicator the host derives a green Active dot from it.statusIndicator{ color, label, description? }Colour is white, green, yellow, orange, or red. Shown in the top bar and settings.attention{ ping: boolean, label? }Pulses the top-bar icon to draw the user's eye.promptState{ toys?, playerInfo? }Structured state the app renders into the prompt. Shapes are on the UI surfaces page.instancesExtensionInstance[]Named connected resources, for example devices. Each has id, name, toolName, isConnected, and optional deviceId, config, metadata.invokeControl(fieldId)function?Called when the user presses a button or device_connection settings field.createInstance / updateInstance / deleteInstance / connectInstance / disconnectInstancefunctions?Instance management the host UI calls when present.const state = { connected: false, bpm: null };
export function useExtension() {
const React = globalThis.React;
const [, rerender] = React.useState(0);
React.useEffect(() => {
const id = setInterval(() => rerender((n) => n + 1), 1000);
return () => clearInterval(id);
}, []);
return {
isInstalled: true,
isAvailable: true,
isEnabled: true, // marketplace installs replace this with the persona toggle
isActive: state.connected,
statusIndicator: state.connected
? { color: 'green', label: 'Connected' }
: { color: 'yellow', label: 'Ready' },
globalSettings: {}, // replaced by the host
personaSettings: {}, // replaced by the host
resolvedSettings: {}, // replaced by the host
promptState: state.bpm == null ? undefined : {
playerInfo: [
{ kind: 'player_info', id: 'hr', label: 'Current Heart Rate', visibility: 'visible', value: state.bpm, unit: 'bpm' },
],
},
enable: async () => {},
disable: async () => {},
updateGlobalSettings: async () => {},
updatePersonaSettings: async () => {},
updateSettings: async () => {},
};
}Lifecycle hooks
Every hook receives the same base context as executeTool (extensionKey, personaId, sessionId, currentMessageId, runtime, llm, visualContext, resolveOwnToolName) plus extensionState and the fields listed for each hook. Hooks are best-effort. Throwing is logged and ignored, so catch your own errors when you need to react to them.
onEnable(personaId)hookThe user toggled you on for a persona. Only fires for marketplace installs; dev-slot and single-file loads never call it, so do not rely on it for hydration.onDisable(personaId)hookThe user toggled you off for a persona. Same limitation as onEnable.onBeforeUserMessageSendctx.userMessageReturn { content } to rewrite the message, { blocked: true, message } to stop the send (the app shows message in a browser alert), or nothing to observe.onBeforeLLMRequestctx.messages, ctx.toolsRuns before every model round. Return { tools } to replace the whole tool list, { promptContributions } to add instructions for this round, or { userMessageImageAttachments } with data: or https URLs to attach images to the last user message.onAfterToolCallctx.toolCallThe model chose a tool. toolCall has id, name, and parsed arguments. Fires for every extension's tools, not only yours.onAfterToolResultctx.toolCall, ctx.toolResultA tool finished. toolResult is { success, result?, error? }.onAfterAssistantChunkctx.assistantTextStreamed text so far for the current round. Not awaited, so keep it cheap.onAfterAssistantMessagectx.assistantTextThe reply is complete. The right place for memory writes, counters, and chat.runTurn().export async function onBeforeUserMessageSend(ctx) {
if (/\bpause\b/i.test(ctx.userMessage || '')) {
return { blocked: true, message: 'Paused. Send anything else to continue.' };
}
}
export async function onBeforeLLMRequest(ctx) {
// Hide our own tool for this round without touching anyone else's.
const mine = ctx.resolveOwnToolName('send_snapshot');
return {
tools: ctx.tools.filter((tool) => tool.function.name !== mine),
promptContributions: [
{ placement: 'system_post', text: 'Photos are unavailable this round.' },
],
};
}
export async function onAfterAssistantMessage(ctx) {
const praised = /good girl/i.test(ctx.assistantText || '');
if (!praised) return;
const current = await ctx.runtime.variables.get('praise_count', { scope: 'persona', defaultValue: 0 });
await ctx.runtime.variables.set('praise_count', Number(current.value) + 1, { scope: 'persona' });
}tools from onBeforeLLMRequest replaces the full list, and the last extension to return one wins. Filter ctx.tools rather than building a new list.Emergency stop
The Toy E-Stop button in the top bar dispatches to every ready toy extension. Ready means all of: category: 'toy', supportsEmergencyStop: true, an exported onEmergencyToyStop, enabled for the persona, and either isActive or an instance with isConnected.
onEmergencyToyStop(ctx)hookctx.source is 'top_bar' or 'extension'. Halt output immediately. Keep the connection open unless your hardware needs a full reset.runtime.emergencyStop({ reason? })v5Any extension can fire the same stop. Resolves { ok, stoppedCount, error? } after every ready toy and host participant (such as the pattern player) has run its stop. Available in hooks, tools, and component props.Integration extensions that detect a stop condition (for example an arousal-state transition) should call runtime.emergencyStop directly rather than writing a variable and hoping the model acts on it. More on the Runtime APIs page.