Computer-Use Agent with Skills

_{Last Updated:
July 23, 2026}

🚀 Part of Haystack 3.0 Launch Week: five days of new drops (July 20–24).

In this notebook, we build a fully local, fully async agent that uses a skill to change how it reports back, and a custom bash tool to actually use the machine it runs on.

Haystack’s Agent can be given skills - folders of instructions the agent reads on demand, in the same SKILL.md format used by Claude Code and Codex. A skill teaches the agent how to do something. It doesn’t let the agent do anything by itself: skills teach, tools do.

By the end you’ll have a fully local agent that:

Discovers a real skill via progressive disclosure (it only sees skill names + one-line descriptions until it decides to load one).
Uses the actual machine it’s running on (inspecting the OS, disk space, largest files) via an approved bash command.
Cuts its own output tokens by loading a caveman skill built for exactly that.

Every piece runs through async/await end to end - Agent.run_async, the bash tool’s async_function, and the human-in-the-loop confirmation - with no blocking calls in the loop.

Stack: Haystack Agent + SkillToolset + a local Ollama model + a custom async bash Tool + human-in-the-loop confirmation.

Setup

Skills ship in Haystack v3:

%pip install -q "haystack-ai>=3.0.0" "ollama-haystack>=6.8.0"

Note: you may need to restart the kernel to use updated packages.

Running a local model with Ollama

This notebook runs fully locally with Ollama - no API key required. See the Ollama integration for setup, then pull the tool-capable model this notebook uses:

ollama pull gemma4:e4b-it-qat

OLLAMA_URL = "http://localhost:11434"
OLLAMA_MODEL = "gemma4:e4b-it-qat"

What is a Skill?

A skill teaches an agent how to do something, without giving it any new capability - it’s read-only knowledge (instructions, examples, reference files), not a new tool. The agent itself decides when one applies, based on the one-line description it’s shown. Concretely, a skill is just a directory:

skills/
  my-skill-name/
    SKILL.md     # YAML frontmatter (name, description) + markdown instructions
    README.md    # bundled reference file, read on demand

SkillToolset exposes exactly two tools to the agent, regardless of how many skills are available:

load_skill(name) - returns a skill’s full instructions, plus a manifest of any bundled files.
read_skill_file(name, path) - reads one of those bundled files on demand.

This scales to a large skill library because of progressive disclosure - loading context in stages (name/description first, full instructions only on activation, bundled files only on demand) instead of all at once. That keeps the context small even with many skills: when the toolset warms up, every skill’s name and one-line description (not its full instructions) get added to load_skill’s tool description. The model sees a menu of what’s available, but it only pays the token cost of a skill’s full instructions when it decides to use one, and only reads bundled files it actually needs via read_skill_file.

Getting a real skill

Rather than write a toy skill, we use a real, community-authored one: caveman (MIT-licensed, by Julius Brussee), which makes an agent respond in an ultra-compressed, concise style while keeping technical accuracy. Its SKILL.md description says it “auto-triggers when token efficiency is requested” - we rely on this later to let the model decide for itself when to use it, instead of forcing it through a system prompt.

%%bash
if [ ! -d "skills_cache/caveman-skill-repo/.git" ]; then
    git clone --depth 1 --filter=blob:none --sparse https://github.com/JuliusBrussee/caveman.git skills_cache/caveman-skill-repo
fi
git -C skills_cache/caveman-skill-repo sparse-checkout set skills/caveman

from pathlib import Path

SKILL_NAME = "caveman"
skills_dir = Path("skills_cache/caveman-skill-repo/skills")

print("Skill files:")
for path in sorted((skills_dir / SKILL_NAME).rglob("*")):
    if path.is_file():
        print(" ", path.relative_to(skills_dir))

Skill files:
  caveman/README.md
  caveman/SKILL.md

Giving the skill to an agent

We pass this skill to an Agent and watch it decide whether to load it. There’s no bash tool yet, so we can see load_skill fire on its own, before we add the bash tool.

from haystack.skill_stores.file_system import FileSystemSkillStore
from haystack.tools import SkillToolset


print("Discovered skills:")
skills_toolset = SkillToolset(FileSystemSkillStore(skills_dir))
for info in skills_toolset.skills.values():
    print(f"  - {info.name}: {info.description}")

Discovered skills:
  - caveman: Ultra-compressed communication mode. Cuts output tokens 65% (measured) by speaking like caveman while keeping full technical accuracy. Supports intensity levels: lite, full (default), ultra, wenyan-lite, wenyan-full, wenyan-ultra. Use when user says "caveman mode", "talk like caveman", "use caveman", "less tokens", "be brief", or invokes /caveman. Also auto-triggers when token efficiency is requested.

Notice the agent hasn’t loaded anything yet - skills_toolset.skills is metadata-only, populated purely from SKILL.md’s frontmatter. We ask for something that should make the agent load the skill and show its effect directly. We run it with run_async - Jupyter lets you await directly in a cell, no asyncio.run needed.

from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.ollama import OllamaChatGenerator
from haystack.components.generators.utils import print_streaming_chunk


reader_agent = Agent(
    chat_generator=OllamaChatGenerator(model=OLLAMA_MODEL, url=OLLAMA_URL),
    system_prompt=(
        "When a task matches one of your skills, actually call the "
        "`load_skill` tool first and follow its instructions exactly - "
        "never just write out what a tool call would look like as text."
    ),
    tools=[skills_toolset],
    streaming_callback=print_streaming_chunk,
)

result = await reader_agent.run_async(
    messages=[
        ChatMessage.from_user(
            "I need my replies to be as token-efficient as possible from "
            "now on. Load whatever skill helps with that, then answer: "
            "what is Docker and why do people use containers?"
        )
    ]
)
print(result["last_message"].text)

A sync streaming callback was provided at initialization for use in an async context. It will run synchronously on the event loop and may block it.
A sync streaming callback was provided at runtime for use in an async context. It will run synchronously on the event loop and may block it.


[TOOL CALL]
Tool: load_skill 
Arguments: {"name": "caveman"}

[TOOL RESULT]
Respond terse like smart caveman. All technical substance stay. Only fluff die.

## Persistence

ACTIVE EVERY RESPONSE. No revert after many turns. No filler drift. Still active if unsure. Off only: "stop caveman" / "normal mode".

Default: **full**. Switch: `/caveman lite|full|ultra`.

## Rules

Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). No tool-call narration, no decorative tables/emoji, no dumping long raw error logs unless asked — quote shortest decisive line. Standard well-known tech acronyms OK (DB/API/HTTP); never invent new abbreviations (cfg/impl/req/res/fn) — tokenizer split them same as full word: zero token saved, reader still decode. Full word cheaper AND clearer. No causal arrows (→) either — own token, save nothing. Technical terms exact. Code blocks unchanged. Errors quoted exact.

Preserve user's dominant language. User write Portuguese → reply Portuguese caveman. User write Spanish → reply Spanish caveman. Compress the style, not the language. No forced English openings or status phrases. ALWAYS keep technical terms, code, API names, CLI commands, commit-type keywords (feat/fix/...), and exact error strings verbatim — unless user explicitly ask for translation.

No self-reference. Never name or announce the style. No "caveman mode on", "me caveman think", no third-person caveman tags. Output caveman-only — never normal answer plus "Caveman:" recap. Exception: user explicitly ask what the mode is.

Pattern: `[thing] [action] [reason]. [next step].`

Not: "Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by..."
Yes: "Bug in auth middleware. Token expiry check use `<` not `<=`. Fix:"

## Intensity

| Level | What change |
|-------|------------|
| **lite** | No filler/hedging. Keep articles + full sentences. Professional but tight |
| **full** | Drop articles, fragments OK, short synonyms. Classic caveman. No tool-call narration, no decorative tables/emoji, no long raw error-log dumps unless asked. Standard acronyms OK; no invented abbreviations |
| **ultra** | Strip conjunctions when cause-then-effect stay unambiguous. One word when one word enough. State each fact once. NO prose abbreviations (cfg/impl/req/res/fn/auth), NO arrows (X → Y) — measured zero token saving under tokenizer, cost decode clarity. Code symbols, function names, API names, error strings: never touch |
| **wenyan-lite** | Semi-classical. Drop filler/hedging but keep grammar structure, classical register |
| **wenyan-full** | Maximum classical terseness. Fully 文言文. 80-90% character reduction. Classical sentence patterns, verbs precede objects, subjects often omitted, classical particles (之/乃/為/其) |
| **wenyan-ultra** | Extreme abbreviation while keeping classical Chinese feel. Maximum compression, ultra terse |

Example — "Why React component re-render?"
- lite: "Your component re-renders because you create a new object reference each render. Wrap it in `useMemo`."
- full: "New object ref each render. Inline object prop = new ref = re-render. Wrap in `useMemo`."
- ultra: "Inline obj prop, new ref, re-render. `useMemo`."
- wenyan-lite: "組件頻重繪，以每繪新生對象參照故。以 useMemo 包之。"
- wenyan-full: "每繪新生對象參照，故重繪；以 useMemo 包之則免。"
- wenyan-ultra: "新參照則重繪。useMemo 包之。"

Example — "Explain database connection pooling."
- lite: "Connection pooling reuses open connections instead of creating new ones per request. Avoids repeated handshake overhead."
- full: "Pool reuse open DB connections. No new connection per request. Skip handshake overhead."
- ultra: "Pool reuse open DB connections. No per-request handshake."
- wenyan-full: "池蓄已開之連，不逐請而新開，省握手之費。"
- wenyan-ultra: "池蓄連，免逐請新開，省握手。"

## Auto-Clarity

Drop caveman when:
- Security warnings
- Irreversible action confirmations
- Multi-step sequences where fragment order or omitted conjunctions risk misread
- Compression itself creates technical ambiguity (e.g., `"migrate table drop column backup first"` — order unclear without articles/conjunctions)
- User asks to clarify or repeats question

Resume caveman after clear part done.

Example — destructive op:
> **Warning:** This will permanently delete all rows in the `users` table and cannot be undone.
> ```sql
> DROP TABLE users;
> ```
> Caveman resume. Verify backup exist first.

## Boundaries

Code/commits/PRs: write normal. "stop caveman" or "normal mode": revert. Level persist until changed or session end.

Bundled files (read with `read_skill_file`):
- README.md



A sync streaming callback was provided at runtime for use in an async context. It will run synchronously on the event loop and may block it.


[ASSISTANT]
Docker container technology packages application and all dependencies into a single image, which can be run consistently across different environments.

People use containers because they provide **consistency** and **isolation**. They ensure "it works on my machine" translates to "it works everywhere," regardless of host OS differences or missing library versions. Isolation prevents one app's dependency conflict from breaking another, saving resources versus full Virtual Machines (VMs).

Docker container technology packages application and all dependencies into a single image, which can be run consistently across different environments.

People use containers because they provide **consistency** and **isolation**. They ensure "it works on my machine" translates to "it works everywhere," regardless of host OS differences or missing library versions. Isolation prevents one app's dependency conflict from breaking another, saving resources versus full Virtual Machines (VMs).

Extending with a custom bash tool

A SkillToolset can only read. To let the agent act, and actually inspect the machine it’s running on, it needs a real execution tool. Haystack has no built-in shell tool by design - running arbitrary commands is inherently risky and application-specific - so we write a small one ourselves as a plain Tool subclass. This lets the agent runthings like uname -a, df -h, or find . -type f for genuine computer use, not just describing what those commands would show. Perfectly, you should swap it for a sandboxed runner (a container, or a remote executor) without touching the skills side at all.

The tool defines both a sync function and an async_function - Haystack picks whichever matches how you call it. Since this notebook runs fully async, we drive it through invoke_async, which awaits the real asyncio.create_subprocess_shell implementation below instead of the blocking subprocess.run one.

⚠️ Safety note: this tool runs whatever command string the model gives it, via shell=True. That’s fine for a local, single-user notebook where you approve every call. Never connect an unattended version of this to untrusted input.

import asyncio
import subprocess
from typing import Any

from haystack.core.serialization import generate_qualified_class_name
from haystack.tools import Tool

_BASH_DESCRIPTION = (
    "Execute a bash command and return its combined stdout/stderr and exit code. Use this to inspect the "
    "system, run scripts, or read/write files. Never guess or make up output - always run the real command."
)
_BASH_PARAMETERS = {
    "type": "object",
    "properties": {"command": {"type": "string", "description": "The bash command to execute."}},
    "required": ["command"],
}


class BashTool(Tool):
    """A standalone Tool that executes bash commands in a subprocess, optionally scoped to a working directory."""

    def __init__(self, working_dir: str | Path | None = None, timeout: int = 60) -> None:
        self._working_dir = Path(working_dir) if working_dir is not None else None
        self._timeout = timeout
        super().__init__(
            name="bash",
            description=_BASH_DESCRIPTION,
            parameters=_BASH_PARAMETERS,
            function=self._run,
            async_function=self._run_async,
        )

    def _format(self, returncode: int, stdout: str, stderr: str) -> str:
        parts = [f"exit_code: {returncode}"]
        if stdout:
            parts.append(f"stdout:\n{stdout.rstrip()}")
        if stderr:
            parts.append(f"stderr:\n{stderr.rstrip()}")
        return "\n".join(parts)

    def _run(self, command: str) -> str:
        cwd = str(self._working_dir) if self._working_dir is not None else None
        try:
            proc = subprocess.run(command, shell=True, cwd=cwd, capture_output=True, text=True, timeout=self._timeout)
        except subprocess.TimeoutExpired:
            return f"exit_code: -1\nstderr:\nCommand timed out after {self._timeout}s."
        return self._format(proc.returncode, proc.stdout, proc.stderr)

    async def _run_async(self, command: str) -> str:
        cwd = str(self._working_dir) if self._working_dir is not None else None
        proc = await asyncio.create_subprocess_shell(
            command, cwd=cwd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
        )
        try:
            stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=self._timeout)
        except asyncio.TimeoutError:
            proc.kill()
            return f"exit_code: -1\nstderr:\nCommand timed out after {self._timeout}s."
        return self._format(proc.returncode or 0, stdout.decode(), stderr.decode())

    def to_dict(self) -> dict[str, Any]:
        return {
            "type": generate_qualified_class_name(type(self)),
            "data": {
                "working_dir": str(self._working_dir) if self._working_dir is not None else None,
                "timeout": self._timeout,
            },
        }

    @classmethod
    def from_dict(cls, data: dict[str, Any]) -> "BashTool":
        return cls(**data["data"])


bash = BashTool()
output = await bash.invoke_async(command="whoami")
print(output)

exit_code: 0
stdout:
kacper.lukawski

Keeping a human in the loop

Before letting the agent actually run shell commands, we require approval for every bash call. Haystack’s human-in-the-loop support plugs in as a before_tool hook: a ConfirmationHook intercepts pending tool calls and applies a policy through a UI before the call is allowed to run.

Policy - AlwaysAskPolicy() asks every single time (there’s also AskOncePolicy, NeverAskPolicy).
Strategy - BlockingConfirmationStrategy pauses the run and waits for your answer.
UI - SimpleConsoleUI() prints the pending call and reads your y.; RichConsoleUI` is a fancier terminal-only alternative.

We scope the hook to just the "bash" tool by name - the skill-reading tools stay unattended, since they’re read-only.

from haystack.hooks.human_in_the_loop import (
    AlwaysAskPolicy, 
    BlockingConfirmationStrategy, 
    ConfirmationHook, 
    SimpleConsoleUI
)

confirmation_hook = ConfirmationHook(
    confirmation_strategies={
        "bash": BlockingConfirmationStrategy(
            confirmation_policy=AlwaysAskPolicy(),
            confirmation_ui=SimpleConsoleUI(),
        )
    }
)

Putting it together: computer use, with and without the skill

We run the exact same computer-use task twice: once with a plain agent, once with an agent that also has the caveman skill available, and compare. Both agents get the same bash tool and the same confirmation hook, so the only variable is whether the skill is available to load.

The task is real machine inspection: OS/kernel info, disk space, the biggest files around. Approve each bash call with y + Enter.

INSPECTION_QUERY = (
    "Investigate this machine: get the OS/kernel info, the Python version, "
    "disk space usage, and the 5 largest files in the current directory. "
    "Report what you find."
)

Baseline run - no skill

baseline_agent = Agent(
    chat_generator=OllamaChatGenerator(model=OLLAMA_MODEL, url=OLLAMA_URL),
    system_prompt=(
        "You investigate the user's machine using the `bash` tool. "
        "Always run the real commands - never guess or make up any part "
        "of the output. If a command fails or gives incomplete output, "
        "run a corrected command instead of filling gaps with placeholder "
        "or hypothetical text."
    ),
    tools=[bash],
    hooks={"before_tool": [confirmation_hook]},
    streaming_callback=print_streaming_chunk,
    max_agent_steps=20,
)

baseline_result = await baseline_agent.run_async(messages=[
    ChatMessage.from_user(INSPECTION_QUERY),
])

A sync streaming callback was provided at initialization for use in an async context. It will run synchronously on the event loop and may block it.
A sync streaming callback was provided at runtime for use in an async context. It will run synchronously on the event loop and may block it.


[TOOL CALL]
Tool: bash 
Arguments: {"command": "uname -a\npython3 --version\ndf -h .\nls -lSh | head -n 5"}


--- Tool Execution Request ---
Tool: bash
Description: Execute a bash command and return its combined stdout/stderr and exit code. Use this to inspect the system, run scripts, or read/write files. Never guess or make up output - always run the real command.
Arguments:
  command: uname -a
python3 --version
df -h .
ls -lSh | head -n 5
------------------------------

Skill-aware run - same task, `caveman` available

The query below adds one phrase, “make this as token-efficient as possible,” that matches caveman’s own trigger condition. We don’t tell the agent to use the skill - it decides to, because the skill’s description says that’s exactly when it applies. That’s progressive disclosure in action.

skill_agent = Agent(
    chat_generator=OllamaChatGenerator(model=OLLAMA_MODEL, url=OLLAMA_URL),
    system_prompt=(
        "You investigate the user's machine using the `bash` tool. "
        "Always run the real commands - never guess or make up any part "
        "of the output. If a command fails or gives incomplete output, "
        "run a corrected command instead of filling gaps with placeholder "
        "or hypothetical text."
    ),
    tools=[skills_toolset, bash],
    hooks={"before_tool": [confirmation_hook]},
    streaming_callback=print_streaming_chunk,
    max_agent_steps=20,
)

skill_query = INSPECTION_QUERY + " Make this as token-efficient as possible."

skill_result = await skill_agent.run_async(messages=[ChatMessage.from_user(skill_query)])

The comparison

There are two different things worth measuring here, and they can point in different directions:

Final-answer tokens (result["messages"][-1].meta["usage"]): tokens spent on just the reply you read. This isolates the skill’s effect on style.
Whole-run total tokens (result["token_usage"]): all LLM calls in the run, including every step spent deciding which tool to call next. This depends mostly on how many separate tool calls the model made, not on the skill. Don’t treat a difference here as proof the skill worked or didn’t.

baseline_final_tokens = baseline_result["messages"][-1].meta.get("usage", {}).get("completion_tokens", 0)
skill_final_tokens = skill_result["messages"][-1].meta.get("usage", {}).get("completion_tokens", 0)
baseline_total_tokens = baseline_result["token_usage"].get("completion_tokens", 0)
skill_total_tokens = skill_result["token_usage"].get("completion_tokens", 0)

metrics = {
    "final-answer tokens": (baseline_final_tokens, skill_final_tokens),
    "whole-run total tokens": (baseline_total_tokens, skill_total_tokens),
}
for label, (before, after) in metrics.items():
    change = f"{100 * (after - before) / before:+.0f}%" if before else "n/a"
    print(f"{label:<22} baseline {before:>4}   with skill {after:>4}   change {change:>5}")

What’s next?

Sandbox the bash tool. For anything beyond a local notebook, run commands in a container or a restricted shell instead of a bare subprocess.
Combine with more tools. tools=[skills_toolset, bash, *other_tools] - a SkillToolset composes with any other Tool/Toolset, as long as there’s only one SkillToolset per agent.
Try per-run tool selection. agent.run_async(tools=[...]) lets you restrict which tools are active for a given call - handy once an agent has a larger toolbox.