Skip to content

Commit 3f8d7a4

Browse files
feat: add ddgs web search integration listing (#548)
* feat: add ddgs web search integration listing Add the catalog page for the ddgs-haystack integration (DDGSWebSearch): keyless, multi-engine web search via ddgs (Dux Distributed Global Search). Note: needs logos/ddgs.png to be added (not included in this change). Assisted-by: Claude:claude-opus-4-8 * Apply suggestions from code review Co-authored-by: Kacper Łukawski <kacperlukawski@users.noreply.github.com> --------- Co-authored-by: Kacper Łukawski <kacperlukawski@users.noreply.github.com>
1 parent 728f509 commit 3f8d7a4

1 file changed

Lines changed: 132 additions & 0 deletions

File tree

‎integrations/ddgs.md‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
layout: integration
3+
name: ddgs
4+
description: Keyless, multi-engine web search for Haystack via ddgs (Dux Distributed Global Search)
5+
authors:
6+
- name: deepset
7+
socials:
8+
github: deepset-ai
9+
twitter: deepset_ai
10+
linkedin: https://www.linkedin.com/company/deepset-ai/
11+
pypi: https://pypi.org/project/ddgs-haystack
12+
repo: https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ddgs
13+
type: Search & Extraction
14+
report_issue: https://github.com/deepset-ai/haystack-core-integrations/issues
15+
version: Haystack 2.0
16+
toc: true
17+
---
18+
19+
### Table of Contents
20+
21+
- [Overview](#overview)
22+
- [Installation](#installation)
23+
- [Usage](#usage)
24+
- [DDGSWebSearch](#ddgswebsearch)
25+
- [License](#license)
26+
27+
## Overview
28+
29+
[ddgs](https://github.com/deedy5/ddgs) ("Dux Distributed Global Search") is a free metasearch library that aggregates results from multiple backends — DuckDuckGo, Google, Bing, Brave, Yahoo, Yandex, and more.
30+
31+
This integration provides:
32+
- [`DDGSWebSearch`](https://docs.haystack.deepset.ai/docs/ddgswebsearch): Searches the web through ddgs and returns results as Haystack `Document` objects along with source URLs.
33+
34+
No API key is required.
35+
36+
## Installation
37+
38+
```bash
39+
pip install ddgs-haystack
40+
```
41+
42+
## Usage
43+
44+
### DDGSWebSearch
45+
46+
`DDGSWebSearch` queries multiple search backends through ddgs and returns results as Haystack `Document` objects containing the content snippets and metadata (title, URL). Source URLs are also returned separately.
47+
48+
#### Basic Example
49+
50+
```python
51+
from haystack_integrations.components.websearch.ddgs import DDGSWebSearch
52+
53+
web_search = DDGSWebSearch(top_k=5)
54+
55+
result = web_search.run(query="What is Haystack by deepset?")
56+
documents = result["documents"]
57+
links = result["links"]
58+
```
59+
60+
#### In a Pipeline
61+
62+
Here is an example of a RAG pipeline that uses `DDGSWebSearch` to retrieve web content and answer a question:
63+
64+
```python
65+
from haystack import Pipeline
66+
from haystack.utils import Secret
67+
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
68+
from haystack.components.generators.chat import OpenAIChatGenerator
69+
from haystack.dataclasses import ChatMessage
70+
from haystack_integrations.components.websearch.ddgs import DDGSWebSearch
71+
72+
web_search = DDGSWebSearch(top_k=3)
73+
74+
prompt_template = [
75+
ChatMessage.from_system("You are a helpful assistant."),
76+
ChatMessage.from_user(
77+
"Given the information below:\n"
78+
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
79+
"Answer the following question: {{ query }}.\nAnswer:",
80+
),
81+
]
82+
83+
prompt_builder = ChatPromptBuilder(
84+
template=prompt_template,
85+
required_variables={"query", "documents"},
86+
)
87+
88+
llm = OpenAIChatGenerator(
89+
api_key=Secret.from_env_var("OPENAI_API_KEY"),
90+
model="gpt-4o-mini",
91+
)
92+
93+
pipe = Pipeline()
94+
pipe.add_component("search", web_search)
95+
pipe.add_component("prompt_builder", prompt_builder)
96+
pipe.add_component("llm", llm)
97+
98+
pipe.connect("search.documents", "prompt_builder.documents")
99+
pipe.connect("prompt_builder.prompt", "llm.messages")
100+
101+
query = "What is Haystack by deepset?"
102+
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
103+
print(result["llm"]["replies"][0].text)
104+
```
105+
106+
#### Async Support
107+
108+
The component supports asynchronous execution via `run_async`:
109+
110+
```python
111+
import asyncio
112+
from haystack_integrations.components.websearch.ddgs import DDGSWebSearch
113+
114+
async def main():
115+
web_search = DDGSWebSearch(top_k=3)
116+
result = await web_search.run_async(query="What is Haystack by deepset?")
117+
print(f"Found {len(result['documents'])} documents")
118+
119+
asyncio.run(main())
120+
```
121+
122+
#### Parameters
123+
124+
- **`top_k`**: Maximum number of results to return. Defaults to 10.
125+
- **`backend`**: Comma-separated ddgs backends to query, or `"auto"` to let ddgs choose (e.g. `"duckduckgo, google, brave"`). Defaults to `"auto"`.
126+
- **`region`**: Region/locale for the search, e.g. `"us-en"`, `"de-de"`, or `"wt-wt"` (no region). Defaults to `"us-en"`.
127+
- **`safesearch`**: Safe-search level: `"on"`, `"moderate"`, or `"off"`. Defaults to `"moderate"`.
128+
- **`search_params`**: Additional keyword arguments forwarded to `DDGS().text()`, for example `page` or `timelimit`.
129+
130+
### License
131+
132+
`ddgs-haystack` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license.

0 commit comments

Comments
 (0)