diff --git a/docs-website/docs/pipeline-components/embedders.mdx b/docs-website/docs/pipeline-components/embedders.mdx index 2530ce8eb17..9eb96e1613f 100644 --- a/docs-website/docs/pipeline-components/embedders.mdx +++ b/docs-website/docs/pipeline-components/embedders.mdx @@ -39,6 +39,8 @@ These are the Embedders available in Haystack: | [JinaDocumentImageEmbedder](embedders/jinadocumentimageembedder.mdx) | Computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. | | [MistralTextEmbedder](embedders/mistraltextembedder.mdx) | Transforms a string into a vector using the Mistral API and models. | | [MistralDocumentEmbedder](embedders/mistraldocumentembedder.mdx) | Computes the embeddings of a list of documents using the Mistral API and models. | +| [MockTextEmbedder](embedders/mocktextembedder.mdx) | Returns deterministic embeddings for a string without calling any API — a zero-cost stand-in for real Text Embedders in tests and prototypes. | +| [MockDocumentEmbedder](embedders/mockdocumentembedder.mdx) | Returns deterministic embeddings for a list of documents without calling any API — a zero-cost stand-in for real Document Embedders in tests and prototypes. | | [NvidiaTextEmbedder](embedders/nvidiatextembedder.mdx) | Embeds a simple string (such as a query) into a vector. | | [NvidiaDocumentEmbedder](embedders/nvidiadocumentembedder.mdx) | Enriches the metadata of documents with an embedding of their content. | | [OllamaTextEmbedder](embedders/ollamatextembedder.mdx) | Computes the embeddings of a string using embedding models compatible with the Ollama Library. | diff --git a/docs-website/docs/pipeline-components/embedders/mockdocumentembedder.mdx b/docs-website/docs/pipeline-components/embedders/mockdocumentembedder.mdx new file mode 100644 index 00000000000..0224cd7e47b --- /dev/null +++ b/docs-website/docs/pipeline-components/embedders/mockdocumentembedder.mdx @@ -0,0 +1,94 @@ +--- +title: "MockDocumentEmbedder" +id: mockdocumentembedder +slug: "/mockdocumentembedder" +description: "A Document Embedder that returns deterministic embeddings without calling any API, for tests and quick prototypes." +--- + +# MockDocumentEmbedder + +A Document Embedder that returns deterministic embeddings without calling any API, for tests and quick prototypes. + +
+ +| | | +| --- | --- | +| **Most common position in a pipeline** | In place of a real Document Embedder, in tests and prototypes | +| **Mandatory init variables** | None | +| **Mandatory run variables** | `documents`: A list of documents | +| **Output variables** | `documents`: A list of documents enriched with embeddings

`meta`: A dictionary of metadata | +| **API reference** | [Embedders](/reference/embedders-api) | +| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/mock_document_embedder.py | +| **Package name** | `haystack-ai` | + +
+ +## Overview + +`MockDocumentEmbedder` is a deterministic, zero-cost drop-in replacement for real Document Embedders such as `OpenAIDocumentEmbedder`. It implements `run`, `run_async`, and serialization like any other embedder but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes. + +The embedding is selected based on how the component is configured: + +- **Deterministic (default)**: With no configuration, each document's embedding is derived from a stable hash of its prepared text. The same text always yields the same unit-length embedding, and different texts yield different embeddings, so the mock works in retrieval pipelines and is reproducible across runs and processes. +- **Fixed embedding**: Pass an `embedding` vector. The same vector is assigned to every document. +- **Dynamic embedding**: Pass an `embedding_fn` callable that receives the prepared text of a document and returns the embedding. To support serialization, pass a named function. + +`embedding` and `embedding_fn` are mutually exclusive. + +Further optional parameters: + +- `dimension`: The number of dimensions of the deterministic embedding. Defaults to `768`. Ignored when `embedding` or `embedding_fn` is provided, since their length is determined by the value or callable. +- `model`: The model name reported in the metadata. Defaults to `"mock-model"`. +- `meta`: Additional metadata merged into the output `meta`. +- `prefix` / `suffix`: Strings added to the beginning and end of each text before embedding, mirroring real embedders. +- `meta_fields_to_embed` / `embedding_separator`: Like real Document Embedders, the metadata fields listed in `meta_fields_to_embed` are concatenated with the document content before embedding, so the deterministic embedding reflects the embedded metadata. +- `progress_bar`: Accepted for interface compatibility with real Document Embedders and ignored. + +:::info +The deterministic embeddings are derived from a hash: identical texts get identical vectors and the similarity between different texts is stable but arbitrary. For exact-match retrieval in tests this is exactly what you want. Do not expect semantically similar texts to end up close together. +::: + +Use `MockDocumentEmbedder` for documents and its counterpart [`MockTextEmbedder`](mocktextembedder.mdx) for queries. With the default deterministic mode, a query whose text matches a document's content produces the same vector, so the document is retrieved as the top hit. + +## Usage + +### On its own + +```python +from haystack import Document +from haystack.components.embedders import MockDocumentEmbedder + +embedder = MockDocumentEmbedder(dimension=8) +result = embedder.run([Document(content="I love pizza!")]) +print(result["documents"][0].embedding) # a deterministic list of 8 floats +``` + +### In a pipeline + +Use it in an indexing pipeline exactly like a real Document Embedder — no API key needed: + +```python +from haystack import Document, Pipeline +from haystack.components.embedders import MockDocumentEmbedder +from haystack.components.writers import DocumentWriter +from haystack.document_stores.in_memory import InMemoryDocumentStore + +document_store = InMemoryDocumentStore() + +indexing_pipeline = Pipeline() +indexing_pipeline.add_component("embedder", MockDocumentEmbedder(dimension=8)) +indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store)) +indexing_pipeline.connect("embedder.documents", "writer.documents") + +indexing_pipeline.run( + { + "embedder": { + "documents": [ + Document(content="My name is Wolfgang and I live in Berlin"), + Document(content="I saw a black horse running"), + ], + }, + }, +) +print(document_store.count_documents()) # 2 +``` diff --git a/docs-website/docs/pipeline-components/embedders/mocktextembedder.mdx b/docs-website/docs/pipeline-components/embedders/mocktextembedder.mdx new file mode 100644 index 00000000000..d19f19e88cc --- /dev/null +++ b/docs-website/docs/pipeline-components/embedders/mocktextembedder.mdx @@ -0,0 +1,94 @@ +--- +title: "MockTextEmbedder" +id: mocktextembedder +slug: "/mocktextembedder" +description: "A Text Embedder that returns deterministic embeddings without calling any API, for tests and quick prototypes." +--- + +# MockTextEmbedder + +A Text Embedder that returns deterministic embeddings without calling any API, for tests and quick prototypes. + +
+ +| | | +| --- | --- | +| **Most common position in a pipeline** | In place of a real Text Embedder, in tests and prototypes | +| **Mandatory init variables** | None | +| **Mandatory run variables** | `text`: A string | +| **Output variables** | `embedding`: A list of float numbers

`meta`: A dictionary of metadata | +| **API reference** | [Embedders](/reference/embedders-api) | +| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/mock_text_embedder.py | +| **Package name** | `haystack-ai` | + +
+ +## Overview + +`MockTextEmbedder` is a deterministic, zero-cost drop-in replacement for real Text Embedders such as `OpenAITextEmbedder`. It implements `run`, `run_async`, and serialization like any other embedder but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes. + +The embedding is selected based on how the component is configured: + +- **Deterministic (default)**: With no configuration, the embedding is derived from a stable hash of the input text. The same text always yields the same unit-length embedding, and different texts yield different embeddings, so the mock works in retrieval pipelines and is reproducible across runs and processes. +- **Fixed embedding**: Pass an `embedding` vector. The same vector is returned for every input. +- **Dynamic embedding**: Pass an `embedding_fn` callable that receives the prepared text (after `prefix`/`suffix` are applied) and returns the embedding. To support serialization, pass a named function. + +`embedding` and `embedding_fn` are mutually exclusive. + +Further optional parameters: + +- `dimension`: The number of dimensions of the deterministic embedding. Defaults to `768`. Ignored when `embedding` or `embedding_fn` is provided, since their length is determined by the value or callable. +- `model`: The model name reported in the metadata. Defaults to `"mock-model"`. +- `meta`: Additional metadata merged into the output `meta`. +- `prefix` / `suffix`: Strings added to the beginning and end of the text before embedding, mirroring real embedders. + +:::info +The deterministic embeddings are derived from a hash: identical texts get identical vectors and the similarity between different texts is stable but arbitrary. For exact-match retrieval in tests this is exactly what you want. Do not expect semantically similar texts to end up close together. +::: + +Use `MockTextEmbedder` for queries and its counterpart [`MockDocumentEmbedder`](mockdocumentembedder.mdx) for documents. With the default deterministic mode, a query whose text matches a document's content produces the same vector, so the document is retrieved as the top hit. + +## Usage + +### On its own + +```python +from haystack.components.embedders import MockTextEmbedder + +embedder = MockTextEmbedder(dimension=8) +result = embedder.run("I love pizza!") +print(result["embedding"]) # a deterministic list of 8 floats +``` + +### In a pipeline + +A retrieval pipeline built with mock embedders runs without any API key and always returns the same result for the same input: + +```python +from haystack import Document, Pipeline +from haystack.components.embedders import MockDocumentEmbedder, MockTextEmbedder +from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever +from haystack.document_stores.in_memory import InMemoryDocumentStore + +document_store = InMemoryDocumentStore() +documents = [ + Document(content="My name is Wolfgang and I live in Berlin"), + Document(content="I saw a black horse running"), +] + +indexed = MockDocumentEmbedder(dimension=8).run(documents=documents) +document_store.write_documents(indexed["documents"]) + +query_pipeline = Pipeline() +query_pipeline.add_component("text_embedder", MockTextEmbedder(dimension=8)) +query_pipeline.add_component( + "retriever", + InMemoryEmbeddingRetriever(document_store=document_store), +) +query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding") + +result = query_pipeline.run( + {"text_embedder": {"text": "I saw a black horse running"}}, +) +print(result["retriever"]["documents"][0].content) # "I saw a black horse running" +``` diff --git a/docs-website/sidebars.js b/docs-website/sidebars.js index 0014b605e19..bf6cbd3a6d4 100644 --- a/docs-website/sidebars.js +++ b/docs-website/sidebars.js @@ -312,6 +312,8 @@ export default { 'pipeline-components/embedders/jinatextembedder', 'pipeline-components/embedders/mistraldocumentembedder', 'pipeline-components/embedders/mistraltextembedder', + 'pipeline-components/embedders/mockdocumentembedder', + 'pipeline-components/embedders/mocktextembedder', 'pipeline-components/embedders/nvidiadocumentembedder', 'pipeline-components/embedders/nvidiatextembedder', 'pipeline-components/embedders/ollamadocumentembedder',