Setup

The AI Agent is disabled by default. To turn it on, add the following environment variables to your .env.local:

AGENT_ENABLED="true"
OPENAI_API_KEY="..."
AGENT_STORAGE_BUCKET_NAME="your-agent-bucket"
AGENT_STORAGE_PREFIX="agent/sessions"

• AGENT_ENABLED turns the feature on.
• packages/agent/models.ts chooses the conversational models and their default.
• OPENAI_API_KEY supplies the credential referenced by the shipped catalog.
• AGENT_STORAGE_BUCKET_NAME is the private bucket where conversations are saved.
• AGENT_STORAGE_PREFIX is the storage prefix for conversation sessions.

Delete a conversation

Both chat layouts display loaded conversations by last activity, newest first. Loading more history merges those conversations into the same chronological list.

In the native workspace, right-click a conversation in history and choose Delete conversation. Keyboard users can focus the conversation and press Shift+F10 or the context-menu key; touch users can long-press it. In the floating panel, open the conversation dropdown and use the trash icon beside a conversation.

Deletion permanently removes the saved conversation and its private image and text-file assets. Deleting the selected conversation opens a fresh composer; deleting another conversation preserves the current draft. Stop an active response before deleting its conversation. If storage cleanup fails, the conversation stays available for another deletion attempt, and new sends are blocked until cleanup finishes. Retrying deletion does not repeat model or tool work.

The authenticated DELETE /api/agent/sessions/{sessionId} endpoint checks the current user and organization membership. Pi's CLI deletes local session files; this integration performs deletion through its existing S3 persistence layer.

Speech input

The shared composer supports optional speech input through @repo/speech. Configure that package and its provider independently of the agent. The microphone appears beside Send when available. Speech appends to existing text; the field becomes read-only and normal Send/Enter submission is blocked while connecting, recording, or finishing. Stop finalizes the last words and sends once through normal chat admission, including selected files, images, and image intent.

Cancel restores the draft from before recording. A failure leaves recovered text editable; silence alone does not send an existing typed draft. Closing the composer or switching conversations stops recording without sending. Speech uses transient audio; normal chat history stores the submitted text.

Choose a model and provider

Use Pi's official references to find the provider and model identifiers:

• Model catalog: available providers, model IDs, input capabilities, context windows, and pricing.
• Choose a Model: model selection and custom-model configuration.
• Provider Authentication: provider credentials and additional cloud-provider settings.

Edit packages/agent/models.ts. Each application key maps to a label, Pi provider/model pair, and the name of a server environment variable containing its credential:

export const agentModels = {
	defaultModel: "standard",
	models: {
		standard: {
			label: "GPT-4o",
			provider: "openai",
			model: "gpt-4o",
			apiKeyEnv: "OPENAI_API_KEY",
		},
		reasoning: {
			label: "GPT-5 mini",
			provider: "openai",
			model: "gpt-5-mini",
			apiKeyEnv: "OPENAI_API_KEY",
		},
	},
};

Keys such as standard are stable application selections; labels are yours to edit. Model IDs, thinking support, image input, and context limits come from the installed Pi catalog. Multiple models from one provider must reference the same apiKeyEnv. Keep credential values in .env.local or your deployment secrets, never in this map.

The default remains active when the model selector is off. Only its credential is then required; enabling selection requires credentials for every offered provider. Invalid catalog entries or missing required keys produce a startup configuration error. Credentials are registered once per provider, never swapped while conversations run.

The runtime in packages/agent/lib/server.ts resolves the provider/model pair through the installed Pi catalog. It uses in-memory credentials, disables startup catalog refresh, and does not load models.json (modelsPath: null). The latest online catalog may include models newer than the installed Pi version. An unknown provider/model pair prevents agent initialization; update the Pi dependencies or explicitly extend runtime configuration when needed.

Pi's CLI /login, /model, local configuration files, and OAuth flows are not exposed by this chat. This catalog supports API-key authentication; other authentication flows require additional runtime wiring.

These entries configure the conversational model. The separate image tools use their selected adapter's config.ts, provider credentials, and optional IMAGE_MODEL override. Speech provider selection is also independent.

Popular provider examples

Use these provider/model pairs in catalog entries. Check the live catalog and your provider account for other models and access requirements; online entries can be newer than the installed SDK.

For Anthropic, for example, use provider: "anthropic", model: "claude-sonnet-4-5", and apiKeyEnv: "ANTHROPIC_API_KEY"; supply that variable in the API environment. Restart/redeploy the API and refresh chat after catalog or credential changes.

Migrating an existing deployment

Copy your existing AGENT_PROVIDER and AGENT_MODEL values into the catalog's default entry. Those variables no longer select a model. To keep an existing credential variable, set apiKeyEnv: "AGENT_API_KEY" explicitly; there is no implicit fallback. Start with modelSelector: false, verify the default, and then enable other models. Disabled agents still need no model credentials or storage.

Conversation storage

Working S3-compatible storage is required, including for text-only chat. There is no temporary, memory-only fallback. Storage can be Amazon S3 or another compatible service, such as the local SeaweedFS service included in this project.

The agent bucket must be private and separate from your public avatar bucket. For local development, the included Docker Compose setup already provides an agent bucket.

If storage is not configured

• Missing or invalid AGENT_STORAGE_BUCKET_NAME or AGENT_STORAGE_PREFIX prevents the agent runtime from initializing. The API logs a configuration error and continues running, but agent requests return service unavailable.
• If those settings are valid but S3 credentials, bucket access, or connectivity are missing, initialization can succeed. Creating a conversation then fails when its first snapshot cannot be saved, before the AI model is called. Loading saved conversations also requires working storage.
• The chat can still appear because its visibility follows AGENT_ENABLED. A failed send shows a generic request-failed message and retains the draft; the UI does not currently identify missing storage as the cause.

Configure and verify storage before enabling the agent for users. Restart the API after correcting its environment settings.

Text-file assets

Text files use the same private bucket and scoped assets/ namespace as images. Storage retains the original uploaded bytes, including byte-order marks and line endings; conversation snapshots hold filenames and integrity-checked references. No separate extracted-text copy is stored. Existing image keys and legacy inline-image readers remain compatible.

Deploy file-aware readers with the upload UI. To stop accepting new files, retain readers for saved attachments. A rollback to older code may not understand file-bearing history; preserve the stored objects and use a compatible build rather than rewriting conversations.

Image asset storage

The image storage service in packages/agent/lib/assets.ts uses the same private bucket. Its keys live under an assets/ subtree of the configured prefix, followed by the organization (when applicable), user, and conversation IDs. This subtree is separate from conversation listing prefixes.

Asset writes are immutable and require S3 conditional writes (If-None-Match). Retrying the same operation reconciles the exact saved bytes instead of overwriting an image after a lost storage response. Keep this bucket private; the service is server-only, and callers must authorize access to the conversation before resolving an asset.

Binary storage operations enforce a 16 MiB per-object ceiling. Raster validation additionally takes explicit byte and decoded-pixel limits, checks PNG/JPEG/GIF/WebP data and MIME types, and fully decodes the image before accepting it. Animation frames count toward the decoded-pixel limit.

Snapshot readers accept stable image-reference metadata as well as existing inline image blocks. References hold image IDs, MIME types, byte lengths, and dimensions; image bytes stored separately do not consume the 4 MiB conversation JSON budget. Preserve these readers when disabling image functionality or rolling back other image changes.

Uploaded assets and their references are saved before a prompt is accepted. A Pi native custom message links the references to the admission; the saved receipt also retains them if the process stops before the user entry is appended. Existing inline images are materialized on the next admission while retaining their native entry IDs and branch relationships.

Image bytes are loaded only into a cloned model-request context. They are not copied back into saved entries or live views. Vision-capable models receive native image blocks; text-only models keep the identifying reference text. The request context has a 64 MiB aggregate image-byte limit.

Admission retries compare text, thinking selection, image intent, an explicit image aspect ratio, and ordered attachment content. An unchanged retry returns the recorded receipt; changing execution inputs requires a new admission ID. Older receipts remain readable and are never replayed automatically after a restart. Image-mode guidance lives in packages/agent/prompts/image-mode.md and is applied to the current request context only, leaving the configured conversational model and its prompt in control.

Interrupted operations can leave unreferenced private objects. There is no automatic asset garbage collector. Before applying a retention policy, account for references across saved native branches and ambiguous writes; deleting an asset merely because a request failed can break a saved conversation.

Session limits and compaction

Automatic Pi compaction and retries are enabled in packages/agent/lib/session.ts with compaction: { enabled: true } and retry: { enabled: true }. Pi's native defaults control the thresholds, retry limits, and backoff. Edit those settings there when your application needs different behavior.

Pi summarizes older model context near its context-window threshold while retaining recent context and the original native history. On a recoverable context overflow, Pi can compact and retry the model request. Transient conversational-model errors use Pi's bounded automatic retries. Summarization can make an additional model request. The chat does not expose Pi's manual compaction command.

These retries do not add automatic HTTP resubmission to native image-generation/editing tools or replay previously completed application actions. The existing admission identity, persistence barriers, and image-provider failure guards remain in effect.

There are separate limits:

The UI currently shows generic failure/unsaved states, not a context-usage meter or a dedicated “conversation full” flow. If recovery fails or a storage/image-byte limit is reached, start a New conversation with the relevant context. A larger-context model can help with a model-token limit, but does not change the storage or image-byte limits.

Compacting model context and reducing the saved JSON are different operations: Pi preserves native history entries. Automatic compaction does not remove the snapshot storage limit.

Attachment size

AGENT_MAX_ATTACHMENT_SIZE sets the per-attachment size in binary megabytes and defaults to 10. A message accepts up to four images. Attachment validation, private storage, and individual image providers also have their own ceilings; changing the composer limit does not raise those ceilings.

Text-file policy is configured in packages/agent/file-handling/text/config.ts. The defaults allow four text files in addition to images and at most 128,000 decoded JavaScript string units across file bodies in an admission or active model context. This is a character budget, not a token estimate; the model's context limit still applies. Original file bytes use the shared attachment-size setting, subject to the storage layer's 16 MiB object ceiling. Supported files must be valid UTF-8; PDF and Word conversion are not included.

File bytes and references are saved before execution. Unchanged retries reuse the admission; changing file contents, names, or order requires a new admission ID. Base64 increases request size by roughly one third: configure your HTTP proxy/body limits for the combined image and file payload, not only one attachment. Application and model limits still apply even when your proxy accepts the request.

Only references in Pi's active context are expanded. Compaction or branching can exclude old files; the original remains downloadable, but full file contents are not automatically reinserted. Reattach a file when you need the model to read its full contents again. Missing, corrupted, or oversized active files fail explicitly rather than being silently omitted.

Choose where the chat appears

You can place the chat in the app sidebar or as a floating panel:

AGENT_CHAT_PLACEMENT="native"

• native adds an Agent Chat entry to the sidebar and a full page at /agent
• floating adds a launcher button in the bottom-right corner

Run the agent

The agent runs inside the API process:

pnpm --filter @repo/api-app build
pnpm --filter @repo/api-app start

Model credentials stay server-side and are never included in prompts or saved conversations.

Tool failure diagnostics

Look in the API process terminal or deployment logs when an image, web-search, knowledge-base, or application tool fails. Pi turns tool exceptions into conversation results, so these failures do not necessarily reach the HTTP API error handler or appear in the SaaS terminal.

The tool boundary logs the tool name and call ID. Native provider diagnostics carry the same call context, plus HTTP status, endpoint, and provider request ID when available. HTTP error bodies, invalid provider results, hosted-search protocol errors, and retrieval GraphQL errors are recorded before they are replaced with safe model-visible errors. Network failures include their underlying error code/cause; cancellations and deadlines are logged too.

Error-body reads are capped at 8 KiB (or the smaller response limit), and each diagnostic is limited to 8,192 characters with a truncation marker. Known credential values, common credential fields, URL credentials/query strings, and image data are redacted. Request bodies and headers are not logged. Provider error text can still contain provider-supplied context, so these are server-side diagnostics, not content to return to chat or persist in conversation snapshots.

After updating the code, restart/redeploy the API. For a new failure, match toolCallId to the failed tool activity and inspect its provider diagnostic and request ID. A generic provider failure alone does not establish whether a prompt was rejected, model access was denied, a rate limit was reached, or a network request failed. Diagnostics discarded by older code cannot be recovered retroactively. Logging does not add automatic retries or provider fallback.

Enable the thinking selector

If your model supports reasoning and you want users to control it, enable the thinking selector in packages/agent/config.ts:

export const agentConfig: AgentConfig = {
	modelSelector: false,
	thinkingSelector: true,
};

When the selector is disabled, the agent uses the model's default reasoning level.

Model selection and thinking selection are independent. Enable either or both in packages/agent/config.ts. Models with only one thinking level do not need a thinking dropdown. Supported preferences are retained when changing models; unsupported preferences return to the selected model's default. Some reasoning models do not offer an off level. Hiding the selector does not disable their reasoning, and previous conversation settings do not override the current default.

Model and thinking choices stay with the current user's personal or organization scope across navigation, conversation switches, and panel closure. Refresh starts with the configured defaults; opening old history does not import its last model preference. The draft remains editable while model capabilities load, but new submissions wait for them. If a model is removed, chat refreshes the catalog and asks you to review the current choice before resending. Failed sends retain their draft and attachments; an unchanged accepted retry keeps its original identity.

The same settings and image-compatibility gate apply to Send, Enter, and speech Stop-to-send. You can change models during dictation; finalization uses the latest valid choices. If pending images are incompatible, the finalized draft is kept for review rather than sent automatically.

Each run freezes its model and thinking level at admission. Changing the composer affects only a later message. Native history records transitions before dispatch; Stop, disconnection, and retry preserve that run's identity. Automatic compaction uses the selected model's context window. An unrecoverable context error fails the run without selecting another model or clearing stored history.

New image attachments require a vision-capable model. Existing saved images remain visible and downloadable when you continue with a text-only model, but image bytes are not sent to that conversational model. Image-edit tools can still use saved references; text files remain usable regardless of vision support.

New receipts include execution metadata that older strict snapshot readers do not understand. For operational rollback, disable model selection while retaining the updated reader; do not strip metadata or discard conversations to run an older build.

See testing for the storage and browser checks you can run before launch.