Skip to content

Combine OCR and Molmo evidence for image chat - #18

Merged
deepanwadhwa merged 1 commit into
mainfrom
feature/image-ocr-molmo-evidence
Sep 16, 2026
Merged

deepanwadhwa merged 1 commit into
mainfrom
feature/image-ocr-molmo-evidence

Conversation

@deepanwadhwa

Copy link
Copy Markdown
Owner

An ordinary image upload could enter the app's bounded analysis_depth=fast path, which disabled visual analysis and left the final text model with only an OCR status message. The image OCR bridge also expected a top-level text field even though native Tesseract returns a lines array, so successful OCR was reported as unavailable.

Every image plan now gathers both literal OCR and visual evidence. The gateway joins Tesseract line objects into bounded text, invokes the installed visual specialist (preferring Molmo2), and sends separate OCR: and MOLMO IMAGE DESCRIPTION: sections to the selected final chat model. Empty OCR and failed OCR remain distinct. Fast image handling stays bounded by using an overview Molmo pass; documents retain their existing fast reader behavior.

Validation:

  • make test
  • make test-molmo2-gateway
  • make ocr-test test-pdf-ocr-runtime
  • node tests/test_composer_ui.mjs
  • node tests/test_composer_perf.mjs
  • Manual native Tesseract read of the supplied workout screenshot recovered its workout title, duration, distance, elevation, heart-rate, and pace text.

@deepanwadhwa
deepanwadhwa merged commit b1aa3a7 into main Sep 16, 2026
3 of 4 checks passed
@deepanwadhwa
deepanwadhwa deleted the feature/image-ocr-molmo-evidence branch September 16, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant