Capabilities and tools

Tools are the actions the agent can take: reading a file, running a shell command, searching the web, and so on. Every tool the model can call (whether it is built into Fermix, registered by an installed plugin, or discovered from an MCP server, a standard way for outside programs to expose tools to an AI agent) is one entry in a single shared list, the Capabilities.Registry. There is no separate set of “main agent tools” vs “voice tools” vs “scheduled-job tools.” Trust level and policy filters decide which subset each call site sees. Skills are the one exception: they are instruction packages, not provider-visible tools, and surface through a compact catalog in the prompt plus the skill_view and skill_run built-ins — see skills.

Capability shape

Each registered capability is a struct with these fields:

%Capability{
  name: "file_read",
  description: "Read a UTF-8 text file from the workspace.",
  parameters: %{           # JSON Schema object
    type: "object",
    required: ["path"],
    properties: %{
      path: %{type: "string", description: "..."}
    }
  },
  kind: :builtin,          # :builtin | :skill | :mcp
  executor: {FermixCore.Tools.FileRead, :execute, []},
  hidden_from_agent?: false,
  policy_class: :read_only,
  owner_only?: false,
  metadata: %{
    category: :file,
    when_to_use: "...",
    examples: [...],
    failure_modes: [...]
  }
}

The parameters map is a JSON Schema object. AgentLoop passes the formatted schema to the configured provider verbatim. The executor tuple is called with the parsed argument map when the model issues a tool call. Two fields carry the trust boundary: policy_class bounds what a caller may do, while owner_only? marks a tool whose return value is the owner’s own data and keeps it out of any guest surface entirely — prompt, provider wire, and dispatch by name.

Three kinds of capability

Kind	Source	Registered by
:builtin	Pure-Elixir tool modules compiled into Fermix, sandbox preset commands, and the tools of ready plugins.	BuiltinSeeder at boot; Sandbox.CommandCapabilities for preset shell commands; Plugins.Capabilities for plugin tools (tagged plugin_owned? in metadata; a plugin that runs a local MCP process contributes its tools as :mcp instead).
:skill	Reserved in the schema for skills, but the live runtime deliberately does not register one capability per skill — skills surface through the prompt’s skill catalog and the skill_view / skill_run built-ins.	SkillRegistry keeps skills in its own store and clears any stale :skill entries from the capability registry on reload.
:mcp	Outbound MCP server tools/list response, registered as mcp_<server>_<tool> (a plugin-owned MCP server uses <plugin>_<tool>).	The per-server Capabilities.MCP.Server process when its client connects; each of its tools is unregistered when the connection goes down.

unregister_kind/3 accepts a metadata match map so a whole family of capabilities can be removed atomically — for example, every plugin_owned? tool when plugins reload, or every sandbox preset command when the sandbox config changes.

For more on skills, see skills. For plugins, see plugins. For MCP integration, see MCP.

Policy classes

Every capability carries one policy_class that describes its effect category:

Class	Meaning	Example built-ins
:read_only	Reads data without modifying anything.	file_read, content_search, glob_search, git_read, memory_recall, tool_help, list_jobs, memory_sources_list, send_attachment
:read_write	Writes or mutates local state.	file_write, file_edit, git_write, memory_store, schedule_job, pause_job, resume_job, remove_job, skill_create, model_routing_config
:exec	Spawns an OS process or delegated run. shell goes through the sandbox.	shell, skill_view, skill_run
:network	Outbound HTTP, validated by Net.Guard.	web_fetch, web_search, place_search, browser
:external_api	Calls a third-party API, typically an LLM provider. All discovered MCP tools default to this class (a per-tool policy_class override exists in [mcp.servers.<server>.tools.<tool>]).	subagents, generate_image, every :mcp capability
:gui_control	Drives the host desktop by screenshot and mouse/keyboard. Operator-only; never delegated to subagents (helper agents the main agent spins up for a piece of work).	computer_use (experimental, off by default)

Subagent workers always run with the parent trust level’s classes minus :read_write and :gui_control: delegated work can read, browse, and call APIs, but never writes local state or touches the desktop.

Trust model

Trust is stamped at ingress (the point where a message enters Fermix, for example a chat channel or the CLI) and flows through MainAgent to AgentLoop to CapabilityRegistry.list_for/2, which applies the default policy filter for that trust level.

Trust	Default allow	Default deny	Where it is set
:operator	[:read_only, :read_write, :exec, :network, :external_api, :gui_control]	[]	The human owner via CLI, daemon socket (the local connection to the long-running Fermix background process), voice click-to-talk, or the channel’s owner_user_id. Also applies to scheduled jobs created by the operator and bundled or operator-installed skills.
:guest	[:read_only] minus owner_only? capabilities	[:read_write, :exec, :network, :external_api, :gui_control]	Non-owner senders in allowed_user_ids; plugin-shipped skills (under ~/.fermix/plugins/) whose code the operator has not reviewed.
nil	(treated as :guest)		Any call path that did not set :trust. A missing trust value degrades to read-only; it never silently grants full access.

A guest additionally never receives an owner_only? capability — a tool whose return value is the owner’s own data. Workspace reads (file_read, glob_search, content_search, git_read), scheduled-job reads (list_jobs, list_job_runs, get_job_run), and memory_sources_list are absent from a guest’s surface entirely: not in the prompt, not on the provider wire, and not dispatchable by name — a guest calling one gets the same “Tool not found” error as any unregistered tool. Read-only bounds what a guest may do; the owner-only filter bounds whose data comes back. This filter is code-enforced and applies only to explicit :guest trust — operators, subagent workers, and scheduled runs are unaffected.

A few tools go further and require an attended turn — one a present owner typed or spoke, rather than one machinery generated. Each gated family carries its own condition, and every one of them is code-enforced.

Tools	What the turn must be
event_store, event_update, event_remove, reminder_snooze	Operator trust, at the top level (not a delegated sub-agent), on an interactive, CLI, or live-voice surface, and not inside a synthesized coding-run continuation.
event_list	The same, or a scheduled job the owner created — so a digest job can read stored dates while the four tools that change something stay attended-only.
Every coding agents tool	An operator turn at the top level with a live reply path, or a scheduled job naming that exact tool in its own allowed_tools. There the job’s allowlist is the authorization; a broad tool policy is deliberately not enough.
request_directory_access	An operator turn at the top level on a channel that has both a live reply path and slash commands, because the owner completes the approval with /confirm. A surface without slash commands never sees it.

Absence fails closed: a turn with no trust marker is treated as a guest, and a nesting depth other than zero is treated as delegated, so a malformed context can only narrow one of these gates, never widen it. The gate runs twice for the event and reminder family — once when the tool list is built, so the model is never offered them, and again when a call is dispatched, so a call by name from an unattended context is refused rather than executed.

A list/2 call with no :trust, :policy, or :policy_classes option bypasses the default filter and returns every registered capability. Internal callers such as fermix capabilities use this primitive path.

See ingress and trust for how trust is assigned per channel and per sender.

Filter parameters

CapabilityRegistry.list_for/1,2 accepts these options:

filter() :: [
  allowed_tools:       [String.t()] | nil,
  policy:              policy_spec(),
  policy_classes:      [policy_class()] | nil,
  trust:               :operator | :guest | nil,
  kind:                :builtin | :skill | :mcp | :all,
  excluded_categories: [atom()] | nil,
  excluded_names:      [String.t()] | nil,
  include_hidden?:     boolean()
]

Filters compose in this order: kind, then policy, then the owner-only filter (applied only for :guest trust), then allowlist (the explicit list of tool names permitted), then name exclusion, then category exclusion, then hidden filter.

Key semantics:

• allowed_tools: [] means deny all. allowed_tools: nil means no narrowing by name.
• policy: [:read_only] is shorthand for [allow: [:read_only], deny: []].
• excluded_categories: [:scheduling] removes every tool whose metadata.category matches, without listing each tool name.
• include_hidden?: true surfaces capabilities flagged hidden_from_agent?: true. Default is false. Operators use this flag to keep an MCP tool configured but invisible to the model.

Registry API

Call	Notes
register/2	Insert a capability. Returns {:error, {:duplicate_name, name}} if a capability with that name already exists.
unregister/2	Remove by name.
unregister_kind/2,3	Remove every capability of a kind, optionally filtered by a metadata match map. Used to clear capability families atomically: plugin tools on reload, sandbox preset commands on config change.
find/2	ETS lookup by name. Hot path; bypasses the GenServer.
list/1,2 / list_for/1,2	List all capabilities, with optional filtering.
refresh/2	Currently a no-op for every kind. Real re-syncs go through the owning components: plugin reload, skill_reload, and MCP connect/disconnect.
resolve_policy/2	Translate (trust, policy) into the effective allow/deny list. Public API.

Built-in tools

Forty-one built-in tools ship unconditionally with Fermix, plus three bridges for deferred tools (tool_search, tool_describe, tool_call) when tool-schema deferral is enabled (on by default). Deferral, explained below, keeps rarely-used tool definitions out of each request to save tokens. Each implements the Builtin.Tool behaviour. One of them, place_search, is credential-gated: seeded with the rest but advertised to the model only while a Brave Search API key resolves, with execution re-checking the key either way. Eight more built-ins are seeded conditionally. computer_use is experimental and off by default, and registers only when computer use is enabled in config and its native helper is installed (see the table row and the note below). The seven coding agents tools — codex_run, claude_code_run, codex_cloud_run, list_coding_runs, get_coding_run, cancel_coding_run, stop_tracking_coding_run — seed only when the coding harness is enabled and a vendor CLI (Codex or Claude Code) is installed, and none is advertised to the model until the owner approves coding agents once (see the note below the table).

Name	Policy class	Category	What it does
shell	exec	system	Run a shell command via CommandRunner with sandbox-resolved working directory and env.
file_read	read_only	file	Read a UTF-8 text file, gated by the sandbox read-path check.
file_write	read_write	file	Write a file, gated by the sandbox write-path check.
file_edit	read_write	file	Replace an exact-string span in a file.
glob_search	read_only	file	Pattern-glob the workspace.
content_search	read_only	file	Regex-search file contents.
git_read	read_only	git	Run read-only git queries: status, log, diff, branch, show. branch lists only — flags that modify a ref and bare operands (which would create one) are both refused, in every spelling git accepts, so a branch flag’s value goes in the = form.
git_write	read_write	git	Run mutating git commands: add, commit, checkout, pull. Pushing is deliberately excluded. Flags that read a file of the caller’s choosing into the repository (-F, --file, --pathspec-from-file) are refused — only the repo path is sandbox-checked, not the arguments — so a commit message comes inline with -m.
web_fetch	network	web	Fetch a public URL through Net.Guard, with a body cap and IP pinning. Returns the page’s readable text, or the JSON verbatim when the endpoint serves JSON.
web_search	network	web	Web search with DuckDuckGo as the default keyless backend; six configurable backends (Brave, Exa, Tavily, Parallel, Perplexity, Firecrawl) are supported when set up. A hard error on a configured backend degrades once to DuckDuckGo, loudly.
place_search	network	web	Structured place lookup on Brave Place Search — hours, ratings, addresses, links, one thumbnail per place. Advertised only while the shared Brave API key resolves; one provider request per lookup; fails loud with named errors and never falls back to web_search or another provider.
browser	network	web	Drive a supervised local browser for JavaScript-rendered pages, forms, and interactive content. The screenshot action returns the captured page as an image the model can see, not just a saved path; the managed browser caps live tabs and auto-closes the oldest non-active tab past the cap.
memory_recall	read_only	memory	Look up a stored memory by key, list this conversation’s memories, or search stored memories and past conversation history.
memory_store	read_write	memory	Persist a memory candidate through the admission pipeline.
memory_sources_list	read_only	memory	List configured memory sources.
subagents	external_api	delegation	Run one or more temporary subagents for delegated work, concurrently up to a cap.
skill_create	read_write	skill_admin	Scaffold a new local Fermix skill with SKILL.md and starter evals.
skill_reload	read_write	skill_admin	Reload a skill from disk without restarting the daemon.
skill_view	exec	skill_admin	Show an installed skill’s instructions and metadata.
skill_run	exec	skill_admin	Run an installed skill by name.
skill_list	read_only	system	List installed skills available to the agent.
schedule_job	read_write	scheduling	Create a cron-style scheduled agent run. The effective capability policy comes from the creator’s trust level, not the model’s.
update_job	read_write	scheduling	Update a scheduled job’s schedule, prompt, or config.
list_jobs	read_only	scheduling	List existing scheduled jobs.
list_job_runs	read_only	scheduling	List past runs for a scheduled job.
get_job_run	read_only	scheduling	Fetch detail for one scheduled job run.
run_job_now	read_write	scheduling	Trigger an immediate out-of-schedule run of a job.
pause_job	read_write	scheduling	Pause a scheduled job.
resume_job	read_write	scheduling	Resume a paused scheduled job.
remove_job	read_write	scheduling	Delete a scheduled job.
event_store	read_write	scheduling	Store a personal date (one-time or yearly) and the finite reminder plan Fermix delivers for it through the configured default background channel. Attended owner turns only.
event_list	read_only	scheduling	List or search stored dates, their plans, and the last delivery result. Defaults to active dates from today onward. Attended owner turns, plus scheduled jobs the owner created.
event_update	read_write	scheduling	Edit a stored date, replace its reminder plan, or rebind it to the current default delivery target. Attended owner turns only.
event_remove	read_write	scheduling	Soft-cancel a stored date and its unsent reminders; with no id, cancels the date behind the last reminder delivered in this conversation. Attended owner turns only.
reminder_snooze	read_write	scheduling	Defer one delivered reminder to a later time without touching the stored plan. Attended owner turns only.
model_routing_config	read_write	config	Read or update model routing config (provider, model, effort).
generate_image	external_api	media	Create or edit a raster image from a text prompt. The result is written under the sandbox media/ floor and sent to the current chat automatically. Backend is openai, openai_codex, xai, or google, configured via [fermix_core.tools.generate_image]. openai_codex runs on a connected ChatGPT/Codex login instead of an image API key — billed to that subscription. It is opt-in and fails loudly if the login is missing or not entitled; it never silently falls back to another backend.
tool_help	read_only	system	Return full documentation for one registered capability.
tool_search	read_only	system	Keyword search (using BM25, a standard text-relevance ranking) over deferred plugin/MCP tool schemas by description. Registered when tool-schema deferral is enabled (default on).
tool_describe	read_only	system	Fetch the full schema for one deferred tool by name. Registered when tool-schema deferral is enabled (default on).
tool_call	read_only	system	Invoke a deferred tool by name; the loop unwraps it before policy and dispatch, so the real tool’s own policy class is what gets enforced. Registered when tool-schema deferral is enabled (default on).
request_directory_access	external_api	system	Ask the owner, in chat, to approve access to a directory the sandbox currently denies. Surfaces only on attended operator turns; the owner confirms with a single-use 60-second /confirm token, the grant persists to [sandbox] allowed_roots, and on chat channels the original request resumes automatically. Guest, scheduled, and delegated runs never see it, and unsafe roots (all of $HOME, ~/.ssh, OS roots) are refused before the owner is even prompted. See sandbox.
send_attachment	read_only	channel	Send a local file through the active channel reply port. URLs are not fetched.
react	read_only	channel	React to the user’s message with a single fitting emoji instead of a text reply, for pure acknowledgements (“ok”, “thanks”, 👍). Advertised only on turns whose channel supports message reactions (on restricted channels the emoji choice is constrained to the channel’s allowed set); a delivered reaction with no accompanying text ends the turn without a further model call.
computer_use	gui_control	computer	Drive the host desktop by screenshot and mouse/keyboard, one action per call. Experimental and off by default. Registers for the model only when computer use is enabled and the native helper is installed; operator-only and never delegated to subagents.
codex_run	exec	harness	Run a background coding task in a repository through the Codex CLI. Seeded when the coding harness is enabled and the Codex CLI is installed; advertised only once coding agents are approved.
claude_code_run	exec	harness	Run a background coding task in a repository through the Claude Code CLI. Same seeding and approval gate as codex_run.
codex_cloud_run	exec	harness	Submit a coding task to a Codex cloud environment and track it by bounded polling. Seeded only when cloud_enabled = true.
list_coding_runs	read_only	harness	List coding runs, their status, and delivery state, with dead-lettered runs grouped separately.
get_coding_run	read_only	harness	Fetch one coding run in full: status, diagnostics, usage, and delivery.
cancel_coding_run	read_write	harness	Cancel an active local coding run by id.
stop_tracking_coding_run	read_write	harness	Stop tracking a Codex cloud run (the vendor task keeps running). Seeded only when cloud_enabled = true.

The browser tool controls a supervised local browser process; web automation belongs there. The computer_use tool is a separate, experimental capability that drives the host desktop itself — the real mouse and keyboard of the logged-in session, guided by screenshots. It is off by default, and it registers for the model only when two things hold: computer use is enabled in config and the native helper is installed. The setup Plugins page (the “Computer Use” card) does both — it sets [fermix_core.computer_use] enabled = true and fetches the helper. The helper is fetched from the compux project’s own signed release and verified against a SHA-256 checksum baked into your Fermix build, then cached under ~/.fermix/plugins/compux/; fermix plugins does not manage it. You can also set the config key yourself: a daemon that has computer use enabled but is missing the matching helper downloads it on its next start (daemon boot only, bounded to about 30 seconds, and fail-soft — a failed fetch leaves computer use off until the next restart or the setup card). Supported on Apple Silicon (M-series) macOS and Linux x86_64. Config changes apply on the next daemon restart. Operating-system permissions (macOS Screen Recording and Accessibility) are a separate diagnostic, not part of that gate — fermix doctor’s computer-use check reports what is missing, so an ungranted permission informs the operator rather than silently hiding the tool. How far it may act follows the sandbox mode (strict is look-only), and a session only starts from an attended origin — an interactive chat, fermix ask, or voice — never from a scheduled job. Before any action that would move the pointer or type — clicking, dragging, scrolling, moving the mouse, typing, pressing a key, or pasting — Fermix checks how long the machine has been idle. If you are actively using it, the agent waits a few seconds for a gap, then either takes its turn or holds the action back and tells you it stepped aside. Idle detection is macOS-only. Where it is unavailable the action proceeds, so this is a courtesy that keeps the agent from fighting you for the cursor, not a safety boundary. To take the machine back yourself, /pause hands the cursor and keyboard back to you: the computer-use session stays alive and refuses further actions until you /resume — unlike /stop, which tears the session down. See computer use for the full platform, permission, action-loop, and safety model.

The seven coding-agent tools follow the same conditional-registration pattern, with an added consent gate — both gates code-enforced. The coding agents feature (internally, the coding harness) hands repository work — reviewing changes, fixing bugs, building features — to the operator’s own Codex or Claude Code CLI, run inside the target repository. Each run tool seeds at daemon boot only when its vendor CLI is on the daemon’s PATH (the history tools seed when either CLI is present); installing a CLI takes effect on the next restart. Seeding alone advertises nothing: no harness tool is offered to the model until the owner approves coding agents once, on the setup Coding Agents tab or with [fermix_core.harness] approved = true. Until then the whole family is absent from the prompt and Fermix does coding work with its own file and shell tools — there is no in-chat consent prompt, and Fermix can never grant the approval itself. The tools stay dispatchable by name while unadvertised, so a run recorded earlier can still be read or cancelled; a by-name launch on an unapproved machine is refused with a typed error naming the fix. default_vendor filters visibility, not capability: with both CLIs installed, only the preferred vendor’s run tool is advertised and the other stays callable by name. The two cloud tools, codex_cloud_run and stop_tracking_coding_run, are not seeded at all unless cloud_enabled = true. Every harness tool is additionally authorized at execute time: the caller must be a live attended operator turn, or a scheduled job that names the exact tool in its allowed_tools — guests, delegated subagents, unattended runs, and voice calls are refused with typed errors.

For per-tool parameter schemas, failure modes, and usage examples, see the tool reference.

Tool categories

Categories let a runtime context exclude a class of tools without naming each one individually (excluded_categories: [:scheduling]).

Category	Tools
:system	shell, tool_help, skill_list, tool_search, tool_describe, tool_call, request_directory_access, sandbox preset commands
:file	file_read, file_write, file_edit, glob_search, content_search
:git	git_read, git_write
:web	web_fetch, web_search, place_search, browser
:media	generate_image
:memory	memory_recall, memory_store, memory_sources_list
:scheduling	schedule_job, update_job, list_jobs, list_job_runs, get_job_run, run_job_now, pause_job, resume_job, remove_job, event_store, event_list, event_update, event_remove, reminder_snooze
:delegation	subagents
:skill_admin	skill_create, skill_reload, skill_view, skill_run
:config	model_routing_config
:channel	send_attachment, react
:computer	computer_use
:harness	codex_run, claude_code_run, codex_cloud_run, list_coding_runs, get_coding_run, cancel_coding_run, stop_tracking_coding_run
:plugin	Every tool registered by a ready plugin

How tools are presented to the model

MainAgent fetches the filtered capability list for the current trust level and conversation context, then formats each capability’s JSON Schema into the provider-specific tool-call format before passing it to AgentLoop. The loop calls the provider, parses any tool calls in the response, looks up each capability in the registry by name, and dispatches to the executor. Results are appended to the message list and the loop continues until the model issues a final text response or the iteration cap is reached (default 100 for interactive, sub-agent, and scheduled-job turns; these caps are internal system settings, not config.toml keys).

Sandbox enforcement applies to every shell call regardless of how it reaches the executor. See sandbox for path grants, env allow/deny, and command presets.

Tool-schema deferral

By default, the full definitions (schemas) for plugin and MCP tools are left out of the request sent to the provider to save tokens; only the tool names remain listed in the runtime prompt. The model fetches a full definition on demand when it actually needs that tool. Three bridge tools handle these deferred tools:

Bridge	What it does
tool_search	BM25 search over the deferred catalog to find tools by description.
tool_describe	Fetch the full schema for one deferred tool by name.
tool_call	Invoke a deferred tool by name; the loop unwraps it so traces and policy see the real tool name. Direct calls by name also work.

Deferral is on by default. To disable it, set [fermix_core.tools.tool_search] enabled = false in config.toml. With deferral off, all schemas are included inline on every call.

Discovery honors the caller’s own ceiling. tool_search, tool_describe, and tool_help only reveal tools the caller could actually dispatch: a guest cannot enumerate or read the schema of an owner-only or above-ceiling tool, and a policy-narrowed scheduled job or subagent worker sees only its own allowed classes. A capability outside the caller’s ceiling answers with the same “Unknown capability” error as one that does not exist — the answer never confirms what the caller may not see.

Skills and MCP as capability sources

Skills are not registered as provider-visible tools. They live in their own SkillRegistry; the model discovers them through a compact skill catalog in the runtime prompt and interacts with them via the skill_view and skill_run built-ins. Each skill’s SKILL.md still defines its own allowed-tool boundary (allowed_tools), so a skill’s sub-turns can be restricted to a subset of built-ins. Bundled and locally installed skills run at operator trust; plugin-shipped skills run at guest trust. A skill’s name may not collide with a registered capability name.

Ready plugins register their tools directly into the same registry (as :builtin kind, tagged plugin_owned? in metadata so they can be swapped atomically on plugin reload). A plugin that runs a local MCP process contributes its tools through MCP discovery instead, named <plugin>_<tool>.

Outbound MCP tools enter as :external_api policy class by default (overridable per tool with policy_class in [mcp.servers.<server>.tools.<tool>]) and are removed when the MCP connection drops. Operators can set hidden_from_agent = true on any MCP tool in config.toml to keep it configured but invisible to the model.

See skills, plugins, and MCP for setup and configuration details.