|
| 1 | +--- |
| 2 | +layout: integration |
| 3 | +name: Mirage |
| 4 | +description: Give a Haystack Agent a bash shell over Mirage's unified virtual filesystem, mounting S3, Google Drive, Postgres and 50+ other backends as one filesystem |
| 5 | +authors: |
| 6 | + - name: deepset |
| 7 | + socials: |
| 8 | + github: deepset-ai |
| 9 | + twitter: haystack_ai |
| 10 | + linkedin: https://www.linkedin.com/company/deepset-ai/ |
| 11 | +pypi: https://pypi.org/project/mirage-haystack |
| 12 | +repo: https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mirage |
| 13 | +type: Tool Integration |
| 14 | +report_issue: https://github.com/deepset-ai/haystack-core-integrations/issues |
| 15 | +logo: /logos/mirage.png |
| 16 | +version: Haystack 2.0 |
| 17 | +toc: true |
| 18 | +--- |
| 19 | +### **Table of Contents** |
| 20 | +- [Overview](#overview) |
| 21 | +- [Installation](#installation) |
| 22 | +- [Usage](#usage) |
| 23 | +- [Security model](#security-model) |
| 24 | +- [License](#license) |
| 25 | + |
| 26 | +## Overview |
| 27 | + |
| 28 | +[Mirage](https://github.com/strukto-ai/mirage) is a unified virtual filesystem for AI agents: it |
| 29 | +mounts heterogeneous backends — object storage (S3, GCS, R2), databases (Postgres, MongoDB, Redis), |
| 30 | +and SaaS apps (Google Drive, Gmail, Slack, GitHub, Notion) — as a single filesystem, so every service |
| 31 | +speaks the same familiar Unix semantics. An agent can `ls`, `cat`, `grep` and pipe across mounts |
| 32 | +exactly as it would on local disk, without learning a new API for each backend. |
| 33 | + |
| 34 | +The `mirage-haystack` integration wraps Mirage as a Haystack |
| 35 | +[`Tool`](https://docs.haystack.deepset.ai/docs/tool) that an |
| 36 | +[`Agent`](https://docs.haystack.deepset.ai/docs/agent) can invoke to run bash commands across the |
| 37 | +mounted filesystem. Instead of pre-loading data into a pipeline, you hand the agent one well-described |
| 38 | +tool and let it explore the mounts itself to answer a question. |
| 39 | + |
| 40 | +## Installation |
| 41 | + |
| 42 | +```bash |
| 43 | +pip install mirage-haystack |
| 44 | +``` |
| 45 | + |
| 46 | +## Usage |
| 47 | + |
| 48 | +### Components |
| 49 | + |
| 50 | +This integration introduces the following: |
| 51 | + |
| 52 | +- `MirageMount`: A declarative, serializable description of a single backend mounted into the |
| 53 | + workspace — its mount `path` (e.g. `/s3`), its Mirage `resource` name (e.g. `"s3"`, `"gdrive"`, |
| 54 | + `"postgres"`), its `config`, and whether it is `read_only`. Credentials can be passed as Haystack |
| 55 | + `Secret`s. Call `MirageMount.available_resources()` to list every backend name you can mount. |
| 56 | +- `MirageWorkspace`: Holds a list of `MirageMount`s and lazily builds the live `mirage.Workspace` on |
| 57 | + first use. It serializes cleanly (resolving `Secret`s only at build time) and can also be run |
| 58 | + directly via `run()` / `run_async()`. |
| 59 | +- `MirageShellTool`: A Haystack `Tool` that exposes the workspace's `execute` surface to an `Agent` |
| 60 | + through a single `command` parameter. Output is normalized to text and truncated before it reaches |
| 61 | + the model. It carries the security guards (`allowed_commands`, `denied_paths`) described below. |
| 62 | + |
| 63 | +### Use with a Haystack Agent |
| 64 | + |
| 65 | +Mount a directory (or any Mirage backend) read-only and give an `Agent` a `MirageShellTool` it can |
| 66 | +drive with ordinary bash — the agent explores the files itself to answer: |
| 67 | + |
| 68 | +```python |
| 69 | +from haystack.components.agents import Agent |
| 70 | +from haystack.components.generators.chat import OpenAIChatGenerator |
| 71 | +from haystack.dataclasses import ChatMessage |
| 72 | + |
| 73 | +from haystack_integrations.tools.mirage import MirageMount, MirageShellTool, MirageWorkspace |
| 74 | + |
| 75 | +workspace = MirageWorkspace( |
| 76 | + mounts=[ |
| 77 | + MirageMount(path="/data", resource="ram"), |
| 78 | + MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True), |
| 79 | + ] |
| 80 | +) |
| 81 | +tool = MirageShellTool(workspace, allowed_commands=["ls", "cat", "grep", "head", "wc", "cp"]) |
| 82 | + |
| 83 | +agent = Agent( |
| 84 | + chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"), |
| 85 | + tools=[tool], |
| 86 | + system_prompt=( |
| 87 | + "A virtual filesystem is available through the `mirage_shell` tool. Use bash commands " |
| 88 | + "(ls, cat, grep, wc, ...) to inspect the mounts before answering. Base your answer only " |
| 89 | + "on what the files actually show." |
| 90 | + ), |
| 91 | +) |
| 92 | +agent.warm_up() |
| 93 | + |
| 94 | +result = agent.run( |
| 95 | + messages=[ChatMessage.from_user("How many lines in /s3/log.txt mention 'alert'?")] |
| 96 | +) |
| 97 | +print(result["messages"][-1].text) |
| 98 | + |
| 99 | +tool.close() |
| 100 | +``` |
| 101 | + |
| 102 | +Every backend is mounted the same way — swap the `MirageMount` for the one you need. Values that are |
| 103 | +credentials should be wrapped in a `Secret`: |
| 104 | + |
| 105 | +```python |
| 106 | +from haystack.utils import Secret |
| 107 | +from haystack_integrations.tools.mirage import MirageMount |
| 108 | + |
| 109 | +MirageMount(path="/data", resource="ram") # in-memory scratch |
| 110 | +MirageMount(path="/local", resource="disk", config={"root": "/srv/data"}) # local disk |
| 111 | +MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True) |
| 112 | +MirageMount( |
| 113 | + path="/drive", |
| 114 | + resource="gdrive", |
| 115 | + config={"client_id": "...", "refresh_token": Secret.from_env_var("GDRIVE_REFRESH_TOKEN")}, |
| 116 | + read_only=True, |
| 117 | +) |
| 118 | +``` |
| 119 | + |
| 120 | +### Run the workspace directly |
| 121 | + |
| 122 | +You don't need an `Agent` to use a workspace — call `run()` to execute a command yourself. This is |
| 123 | +handy for testing mounts or composing across backends in plain Python: |
| 124 | + |
| 125 | +```python |
| 126 | +from haystack_integrations.tools.mirage import MirageMount, MirageWorkspace |
| 127 | + |
| 128 | +ws = MirageWorkspace( |
| 129 | + mounts=[ |
| 130 | + MirageMount(path="/data", resource="ram"), |
| 131 | + MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True), |
| 132 | + ] |
| 133 | +) |
| 134 | +print(ws.run("grep -r alert /s3/logs | wc -l")) |
| 135 | +ws.close() |
| 136 | +``` |
| 137 | + |
| 138 | +## Security model |
| 139 | + |
| 140 | +Mirage never shells out to the host: every command runs inside Mirage's own virtual-filesystem |
| 141 | +interpreter. Three controls shape what an Agent can do: |
| 142 | + |
| 143 | +- **Per-mount read-only mode** (`MirageMount(..., read_only=True)`) is the authoritative write |
| 144 | + boundary. Mirage refuses any write to a read-only mount regardless of the command used, so this |
| 145 | + is how you prevent modification or deletion. Mount anything the Agent should not change as read-only. |
| 146 | +- **The command allowlist** (`allowed_commands`) restricts *which* commands may run. It is enforced |
| 147 | + against every command Mirage would execute, including commands nested inside `$(...)`, backticks, |
| 148 | + `<(...)` and subshells, so `ls "$(rm x)"` is rejected unless `rm` is also allowed. Treat it as a |
| 149 | + best-effort filter to steer the Agent, not a sandbox: allowing a command that itself runs other |
| 150 | + commands (`eval`, `bash`, `sh`, `source`, `xargs`, `timeout`) effectively allows anything, so do not |
| 151 | + list those for untrusted/hosted use. |
| 152 | +- **`denied_paths`** rejects any command whose text references one of the given path substrings. |
| 153 | + |
| 154 | +## License |
| 155 | + |
| 156 | +`mirage-haystack` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license. |
0 commit comments