Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
build/
/build-linux-ubuntu/
/build-ci-*/
*.qws
*.dSYM/
Expand Down
1 change: 0 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,6 @@ make # portable build
make omp # multithreaded (brew install libomp first, macOS)
make test # self-contained: stubs engine + network, tiny fixtures, no 24 GB model
# covers the compiled gateway, Chutni, and the Kimi preflight
make test-all # the above plus the Gigatoken adapter (needs `cargo +nightly`)
```

- Model (24 GB, hard-linked, never copy): `~/Documents/samosa-models/qwen36_group32_i8`
Expand Down
186 changes: 134 additions & 52 deletions Makefile

Large diffs are not rendered by default.

18 changes: 17 additions & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,21 @@ contributors, licensed under the MIT License. The encoder/decoder inference
loop in src/samosa_summarizer.cpp is adapted from llama.cpp's `simple`
example under that license.

The optional VisionPsy-Nano 460M BF16 weights are the MLX conversion published
by KaedeTai at KaedeTai/VisionPsy-Nano-460M-MLX, pinned by Samosa to revision
ca7a1bad19a08fe8e474bee24275a2e377bbabba. The upstream model is
qvac/VisionPsy-Nano-460M, licensed under the Apache License, Version 2.0.
Samosa executes the conversion through its independent native MLX C++ adapter;
the Python reference runtime is not distributed.

The optional Molmo2 4B package is converted from allenai/Molmo2-4B revision
042abfa7a38879a376cec03d949eff0aefaa0600, copyright the Allen Institute for
AI and contributors, licensed under the Apache License, Version 2.0. Samosa's
native conversion keeps the vision tower and connector in BF16 and converts
large language matrices to affine Q4/group-64. The original FP32 checkpoint,
Python reference runtime, Transformers, and PyTorch are not distributed by
Samosa.

Samosa Chat is not affiliated with or endorsed by the colibrì project,
Alibaba Cloud, the Qwen team, Falcons.ai, or the llama.cpp project.
Alibaba Cloud, the Qwen team, Falcons.ai, the llama.cpp project, qvac, KaedeTai,
the VisionPsy project, AllenAI, or the Molmo project.
56 changes: 51 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,14 @@ Samosa is a local model application and HTTP gateway. Models can be downloaded,
- Starts with zero installed models.
- Includes a browser interface for downloading models from a validated catalog.
- Supports resumable and verified model downloads.
- Automatically routes chat images and PDFs through local text/OCR and an
on-demand visual specialist; oversized inputs are bounded from live
memory/thermal conditions and multi-page work has no arbitrary page cap.
- Adds optional, on-demand Molmo2 4B intelligence as the preferred installed
image/video specialist without a Python runtime or always-resident second
model; VisionPsy remains the smaller compatibility fallback.
- Provides an internet-search feature that does not require an API key.
- Integrates Chutni folder memory via a native Gigatoken adapter for document indexing.
- Integrates portable Chutni folder memory through the bundled local `chutni-mcp` service.

## Models

Expand All @@ -31,8 +37,22 @@ Samosa is a local model application and HTTP gateway. Models can be downloaded,
| [Maple](https://huggingface.co/deepanwa/Samosa-Chat-Maple-2bit-SSD) | 5.4 GB | 2-bit MoE with routed experts streamed from SSD | Samosa Metal engine | MIT |
| [Bonsai 27B 1-bit](https://huggingface.co/prism-ml/Bonsai-27B-gguf) | 3.8 GB | 1-bit GGUF reasoning model | Prism llama.cpp | Apache-2.0 |
| [Ornith 1.0 9B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B-GGUF) | 5.6 GB | Q4_K_M GGUF coding and reasoning model | Prism llama.cpp | MIT |

Only one model is loaded in memory at a time. Selecting a new model unloads the active backend before launching the next.
| [VisionPsy-Nano 460M](https://huggingface.co/KaedeTai/VisionPsy-Nano-460M-MLX) | 1.01 GB | Auxiliary vision specialist for chat attachments (macOS arm64) | Samosa MLX Metal C++ helper | Apache-2.0 |
| [Molmo2 4B](https://huggingface.co/allenai/Molmo2-4B) | ≤4.0 GiB native package | Preferred on-demand image/video specialist (macOS arm64) | Samosa MLX/AVFoundation C++ helper | Apache-2.0 |

Only one primary chat model is loaded in memory at a time. When the optional
Molmo2 package is installed it handles admitted visual image and video work;
VisionPsy-Nano 460M is the standard-image fallback when Molmo is absent. On
constrained Macs, Samosa stops the primary model, runs one bounded Molmo
operation, and unloads it. If a conversation starts with one image or video,
Molmo's own text and point coordinates are returned directly while the selected
text model restarts for later turns. Later/composite visual workflows may use a
grounded, greedy, non-thinking synthesis pass. Molmo2 is a locally built,
hash-verified package: the app never
downloads its 19.4 GB FP32 source checkpoint. Samosa contains zero runtime
Python dependencies. The real 16 GiB-machine conversion and zero-swap
image/video qualification are recorded in
[`docs/regressions/molmo2-4b-q4-2026-08-26.md`](docs/regressions/molmo2-4b-q4-2026-08-26.md).

> **Credits.** The Qwen runtime is based on [colibrì](https://github.com/JustVugg/colibri) by JustVugg and runs the model released by [Qwen](https://huggingface.co/Qwen). Bonsai is released by [PrismML](https://huggingface.co/prism-ml), and Ornith by [DeepReinforce](https://huggingface.co/deepreinforce-ai). GGUF execution uses PrismML's [llama.cpp fork](https://github.com/PrismML-Eng/llama.cpp). Samosa is an independent project.

Expand Down Expand Up @@ -94,7 +114,9 @@ Model downloads:
| Bonsai | 3.8 GB (+17 MB runtime) |
| Ornith | 5.6 GB (+17 MB runtime) |
| Qwen | 24.0 GB |
| All Models | 33.4 GB (+17 MB runtime) |
| VisionPsy (macOS arm64 only) | 1.01 GB |
| Molmo2 native Q4 package (optional, macOS arm64) | ≤4.0 GiB |
| All Models | 34.4 GB (+17 MB runtime) |

## App Usage

Expand All @@ -105,11 +127,33 @@ samosa app
Access the interface at <http://127.0.0.1:8642>. Available settings include:

- Model selection and downloading.
- Grouped Text LLM, Vision, OCR, and Voice model settings with live readiness.
- Automatic chat-attachment routing, hardware-adaptive image resizing, and
honest text/OCR-only partial answers if visual analysis fails.
- Streamed MP4/MOV/M4V attachments with bounded overview, temporal, or
exhaustive Molmo2 coverage and explicit unprocessed intervals.
- Context size configuration (`auto` or explicit limits).
- Automatic and manual conversation compaction.
- Inference parameters (thinking budget, max tokens, seed) and optional web context tools.
- Opt-in Developer mode with one correlated local trace across routing, OCR,
VisionPsy, evidence assembly, and the final model response.

State and conversation ledgers are persisted under `~/.samosa/chats`.
Developer traces are off by default because they contain full prompts and
document/model contents; see [docs/DEVELOPER_MODE.md](docs/DEVELOPER_MODE.md).

To use the Mac's models from a phone or another computer on the same Wi-Fi:

```sh
samosa app --lan
```

Samosa opens the normal local app and prints a clean `Other devices` link.
Open it on the other device and enter the LAN password (`password1234` for
now). Keep the Mac awake, allow incoming connections if macOS asks, and stop
sharing with `samosa serve --stop`. See the full
[LAN access guide](docs/LAN_ACCESS.md) for the trust boundary, password
sessions, shared-model concurrency, model switching, and troubleshooting.

### Folder memory with Chutni

Expand Down Expand Up @@ -203,7 +247,8 @@ Test logs and regression benchmarks are located in [docs/regressions](docs/regre
## Privacy & Security

- All inference and chat data remain local to the host machine.
- The gateway server binds exclusively to `127.0.0.1`.
- The gateway binds exclusively to `127.0.0.1` unless the user explicitly
starts password-protected LAN access with `--lan`.
- No telemetry or external accounts are used.
- Downloads are limited to pinned Hugging Face repository assets and Prism GitHub release binaries.
- External web search tools are opt-in.
Expand All @@ -220,6 +265,7 @@ Test logs and regression benchmarks are located in [docs/regressions](docs/regre
|---|---|
| Installation & Model Storage | [docs/INSTALL.md](docs/INSTALL.md) |
| App & Terminal Usage | [docs/USAGE.md](docs/USAGE.md) |
| Password-protected LAN access | [docs/LAN_ACCESS.md](docs/LAN_ACCESS.md) |
| Model Specifications & Network | [docs/MODELS_AND_INTERNET.md](docs/MODELS_AND_INTERNET.md) |
| Gateway API | [docs/SERVE_API.md](docs/SERVE_API.md) |
| Chutni folder memory | [docs/CHUTNI_PROTOCOL.md](docs/CHUTNI_PROTOCOL.md) |
Expand Down
Loading
Loading