Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 153 additions & 44 deletions docs/src/content/docs/advanced_topics/memoization_keys.mdx

Large diffs are not rendered by default.

24 changes: 17 additions & 7 deletions docs/src/content/docs/programming_guide/function.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ A memoized function:

When a memoized function's cache hits, its **body does not run** — and neither do any nested `@coco.fn` calls inside it. The cached output is replayed directly: the previous return value is returned, and any target states the function declared on its previous run are carried over.

The fragment below omits the enclosing `App` and processing component. Treat the calls as invocations of the same processing component across successive application updates.

```python
@coco.fn(memo=True)
async def inner(text: str) -> str:
Expand All @@ -88,11 +90,11 @@ async def outer(text: str) -> str:
print("outer ran")
return await inner(text) + "!"

# First call: both print, both bodies execute
# First component update: both print, both bodies execute.
result = await outer("hello")

# Second call with same inputs: nothing prints — outer's cached value is returned
# directly, and inner is not invoked at all.
# Later component update with the same input: nothing prints. The cached value
# from outer is returned directly, and inner is not invoked at all.
result = await outer("hello")
```

Expand Down Expand Up @@ -174,11 +176,19 @@ def summarize(text: str) -> str:
return call_llm(SYSTEM_PROMPT, text, model=MODEL)
```

The value is canonicalized through the [memoization-key pipeline](../advanced_topics/memoization_keys), which honors `__coco_memo_key__()`, registered memo key functions, and the standard handling for primitives, dataclasses, and Pydantic models.
The value is canonicalized through the [memoization-key pipeline](../advanced_topics/memoization_keys), including custom memo keys, registered memo key functions, stable type IDs, and built-in handling for primitives, containers, dataclasses, Pydantic models, and class objects.

:::caution[Snapshotted at decoration time]
`deps` is evaluated **once** when the decorator is applied (typically at module import), not re-evaluated per call. For per-call or per-instance values — instance attributes in a bound method, request-scoped config, anything that changes at runtime — pass them as regular function arguments instead, so the memoization layer observes each new value.
:::
:::::caution[Snapshotted at decoration time]
`deps` is evaluated **once** when the decorator is applied (typically at module import), not re-evaluated per call.

Only `deps` is snapshotted at decoration time; regular function arguments are fingerprinted when the function is called.

This has three consequences:

- Register stable type IDs before decorating any function whose `deps` include values of that type: define `__coco_memo_type_id__` on the class, or call `register_memo_type(..., stable_type_id=...)`, before any `@coco.fn(..., deps=...)` or `@coco.fn.as_async(..., deps=...)` decorator that references those values.
- A later registration can affect future argument fingerprints, but it cannot retroactively update the logic fingerprint already computed for that decorated function.
- For per-call or per-instance values — instance attributes in a bound method, request-scoped config, anything that changes at runtime — pass them as regular function arguments instead, so the memoization layer observes each new value.
:::::

`deps` requires `logic_tracking` to be enabled; combining `deps=<value>` with `logic_tracking=None` raises `ValueError`.

Expand Down
4 changes: 4 additions & 0 deletions python/cocoindex/_internal/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,9 @@

from .memo_fingerprint import (
memo_fingerprint,
prev_type_id,
register_memo_key_function,
register_memo_type,
NotMemoKeyable,
)

Expand Down Expand Up @@ -943,7 +945,9 @@ class Cursor:
"serialize_by_pickle",
# .memo_fingerprint
"memo_fingerprint",
"prev_type_id",
"register_memo_key_function",
"register_memo_type",
"NotMemoKeyable",
# .pending_marker
"MaybePendingS",
Expand Down
2 changes: 1 addition & 1 deletion python/cocoindex/_internal/context_keys.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ def provide(self, key: ContextKey[T], value: T) -> T:
state_fns: list[StateFnEntry] = []
canonical = _canonicalize(
("context_key", key._key, value),
_seen=None,
state=None,
state_methods=state_fns,
)
fp = core.fingerprint_simple_object(canonical)
Expand Down
18 changes: 14 additions & 4 deletions python/cocoindex/_internal/function.py
Original file line number Diff line number Diff line change
Expand Up @@ -2029,7 +2029,8 @@ def __call__( # type: ignore[misc]
string or model identifier. The value is canonicalized via the
memoization-key pipeline (see
:doc:`/advanced_topics/memoization_keys` for the full contract,
including ``__coco_memo_key__()`` and registered key functions)
including ``__coco_memo_key__()``, registered memo key functions,
stable type IDs, and built-in handling)
and folded into the function's logic fingerprint; when the
canonical form changes, memoized results are invalidated and
the change propagates to callers according to ``logic_tracking``
Expand All @@ -2044,6 +2045,13 @@ def __call__( # type: ignore[misc]
config, anything that changes at runtime — pass them as regular
function arguments instead.

Stable type IDs used by ``deps`` must therefore exist before the
decorator is applied: define ``__coco_memo_type_id__`` on the class
or call ``register_memo_type(..., stable_type_id=...)``
before decorating. A later registration can affect future argument
fingerprints, but it cannot retroactively update this function's
already-computed logic fingerprint.

Requires ``logic_tracking`` to be enabled; raises ``ValueError`` if
combined with ``logic_tracking=None``.

Expand Down Expand Up @@ -2157,9 +2165,11 @@ def as_async( # type: ignore[misc]
"self": Track own code only, not children.
None: No function logic tracking (incompatible with ``deps``).
deps: Additional value(s) the function logic depends on but that
aren't visible in its body. See :func:`fn` for the full
contract — the value is canonicalized through the memoization
key pipeline and folded into the function's logic fingerprint.
aren't visible in its body. See :func:`fn` for the full contract,
including stable type IDs and their decoration-time registration
order requirement for ``deps``. The value is canonicalized through
the memoization-key pipeline and folded into the function's logic
fingerprint.

Batching and runner are fully supported since the result is always async.

Expand Down
Loading
Loading