Skip to content

Commit 41ffad2

Browse files
authored
Add Mirage integration tile (#539)
* Add Mirage integration tile * change logo
1 parent 5a88f0b commit 41ffad2

2 files changed

Lines changed: 156 additions & 0 deletions

File tree

‎integrations/mirage.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
---
2+
layout: integration
3+
name: Mirage
4+
description: Give a Haystack Agent a bash shell over Mirage's unified virtual filesystem, mounting S3, Google Drive, Postgres and 50+ other backends as one filesystem
5+
authors:
6+
- name: deepset
7+
socials:
8+
github: deepset-ai
9+
twitter: haystack_ai
10+
linkedin: https://www.linkedin.com/company/deepset-ai/
11+
pypi: https://pypi.org/project/mirage-haystack
12+
repo: https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mirage
13+
type: Tool Integration
14+
report_issue: https://github.com/deepset-ai/haystack-core-integrations/issues
15+
logo: /logos/mirage.png
16+
version: Haystack 2.0
17+
toc: true
18+
---
19+
### **Table of Contents**
20+
- [Overview](#overview)
21+
- [Installation](#installation)
22+
- [Usage](#usage)
23+
- [Security model](#security-model)
24+
- [License](#license)
25+
26+
## Overview
27+
28+
[Mirage](https://github.com/strukto-ai/mirage) is a unified virtual filesystem for AI agents: it
29+
mounts heterogeneous backends — object storage (S3, GCS, R2), databases (Postgres, MongoDB, Redis),
30+
and SaaS apps (Google Drive, Gmail, Slack, GitHub, Notion) — as a single filesystem, so every service
31+
speaks the same familiar Unix semantics. An agent can `ls`, `cat`, `grep` and pipe across mounts
32+
exactly as it would on local disk, without learning a new API for each backend.
33+
34+
The `mirage-haystack` integration wraps Mirage as a Haystack
35+
[`Tool`](https://docs.haystack.deepset.ai/docs/tool) that an
36+
[`Agent`](https://docs.haystack.deepset.ai/docs/agent) can invoke to run bash commands across the
37+
mounted filesystem. Instead of pre-loading data into a pipeline, you hand the agent one well-described
38+
tool and let it explore the mounts itself to answer a question.
39+
40+
## Installation
41+
42+
```bash
43+
pip install mirage-haystack
44+
```
45+
46+
## Usage
47+
48+
### Components
49+
50+
This integration introduces the following:
51+
52+
- `MirageMount`: A declarative, serializable description of a single backend mounted into the
53+
workspace — its mount `path` (e.g. `/s3`), its Mirage `resource` name (e.g. `"s3"`, `"gdrive"`,
54+
`"postgres"`), its `config`, and whether it is `read_only`. Credentials can be passed as Haystack
55+
`Secret`s. Call `MirageMount.available_resources()` to list every backend name you can mount.
56+
- `MirageWorkspace`: Holds a list of `MirageMount`s and lazily builds the live `mirage.Workspace` on
57+
first use. It serializes cleanly (resolving `Secret`s only at build time) and can also be run
58+
directly via `run()` / `run_async()`.
59+
- `MirageShellTool`: A Haystack `Tool` that exposes the workspace's `execute` surface to an `Agent`
60+
through a single `command` parameter. Output is normalized to text and truncated before it reaches
61+
the model. It carries the security guards (`allowed_commands`, `denied_paths`) described below.
62+
63+
### Use with a Haystack Agent
64+
65+
Mount a directory (or any Mirage backend) read-only and give an `Agent` a `MirageShellTool` it can
66+
drive with ordinary bash — the agent explores the files itself to answer:
67+
68+
```python
69+
from haystack.components.agents import Agent
70+
from haystack.components.generators.chat import OpenAIChatGenerator
71+
from haystack.dataclasses import ChatMessage
72+
73+
from haystack_integrations.tools.mirage import MirageMount, MirageShellTool, MirageWorkspace
74+
75+
workspace = MirageWorkspace(
76+
mounts=[
77+
MirageMount(path="/data", resource="ram"),
78+
MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True),
79+
]
80+
)
81+
tool = MirageShellTool(workspace, allowed_commands=["ls", "cat", "grep", "head", "wc", "cp"])
82+
83+
agent = Agent(
84+
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
85+
tools=[tool],
86+
system_prompt=(
87+
"A virtual filesystem is available through the `mirage_shell` tool. Use bash commands "
88+
"(ls, cat, grep, wc, ...) to inspect the mounts before answering. Base your answer only "
89+
"on what the files actually show."
90+
),
91+
)
92+
agent.warm_up()
93+
94+
result = agent.run(
95+
messages=[ChatMessage.from_user("How many lines in /s3/log.txt mention 'alert'?")]
96+
)
97+
print(result["messages"][-1].text)
98+
99+
tool.close()
100+
```
101+
102+
Every backend is mounted the same way — swap the `MirageMount` for the one you need. Values that are
103+
credentials should be wrapped in a `Secret`:
104+
105+
```python
106+
from haystack.utils import Secret
107+
from haystack_integrations.tools.mirage import MirageMount
108+
109+
MirageMount(path="/data", resource="ram") # in-memory scratch
110+
MirageMount(path="/local", resource="disk", config={"root": "/srv/data"}) # local disk
111+
MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True)
112+
MirageMount(
113+
path="/drive",
114+
resource="gdrive",
115+
config={"client_id": "...", "refresh_token": Secret.from_env_var("GDRIVE_REFRESH_TOKEN")},
116+
read_only=True,
117+
)
118+
```
119+
120+
### Run the workspace directly
121+
122+
You don't need an `Agent` to use a workspace — call `run()` to execute a command yourself. This is
123+
handy for testing mounts or composing across backends in plain Python:
124+
125+
```python
126+
from haystack_integrations.tools.mirage import MirageMount, MirageWorkspace
127+
128+
ws = MirageWorkspace(
129+
mounts=[
130+
MirageMount(path="/data", resource="ram"),
131+
MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True),
132+
]
133+
)
134+
print(ws.run("grep -r alert /s3/logs | wc -l"))
135+
ws.close()
136+
```
137+
138+
## Security model
139+
140+
Mirage never shells out to the host: every command runs inside Mirage's own virtual-filesystem
141+
interpreter. Three controls shape what an Agent can do:
142+
143+
- **Per-mount read-only mode** (`MirageMount(..., read_only=True)`) is the authoritative write
144+
boundary. Mirage refuses any write to a read-only mount regardless of the command used, so this
145+
is how you prevent modification or deletion. Mount anything the Agent should not change as read-only.
146+
- **The command allowlist** (`allowed_commands`) restricts *which* commands may run. It is enforced
147+
against every command Mirage would execute, including commands nested inside `$(...)`, backticks,
148+
`<(...)` and subshells, so `ls "$(rm x)"` is rejected unless `rm` is also allowed. Treat it as a
149+
best-effort filter to steer the Agent, not a sandbox: allowing a command that itself runs other
150+
commands (`eval`, `bash`, `sh`, `source`, `xargs`, `timeout`) effectively allows anything, so do not
151+
list those for untrusted/hosted use.
152+
- **`denied_paths`** rejects any command whose text references one of the given path substrings.
153+
154+
## License
155+
156+
`mirage-haystack` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license.

‎logos/mirage.png‎

11.3 KB
Loading

0 commit comments

Comments
 (0)