Skip to content

feat(desktop): Telegram bot media support both ways - #922

Open
kotevcode wants to merge 2 commits into
mainfrom
fix/telegram-bot-images
Open

feat(desktop): Telegram bot media support both ways#922
kotevcode wants to merge 2 commits into
mainfrom
fix/telegram-bot-images

Conversation

@kotevcode

Copy link
Copy Markdown
Contributor

Description

The Telegram bridge was text-only: generated images were dropped ("Done (no text reply)"), and inbound photos/documents hit a generic "Only text messages are supported" error. Part of this feature existed before (image sending, removed in #831's follow-up 1224a4c as an unrelated change during the discovery refactor); this PR restores it and extends it to full two-way media support.

Outgoing: /image <prompt> generates via the best openai-images seller within the buyer routing policy and delivers the result as a Telegram photo (document fallback for oversized/non-inline formats), with a live "Generating…" draft. Assistant replies that contain image file blocks now send those images instead of collapsing to a text placeholder.

Incoming: photos and documents are downloaded (Bot API 20 MB limit enforced), persisted as conversation attachments, and passed through the same multimodal attachment pipeline as the app's paper-clip flow — images reach vision-capable models, documents get text extraction. Captions become the message text; voice/video/audio get an explicit "not supported yet" notice instead of a generic error.

Also: image-only sellers are excluded from the bot's text-model picker and default-route fallback (they remain reachable via /image), and /stop now cancels in-flight image generations too. PiChatEngine.sendMessageStream gained an attachments option and a new generateImage() entry point so non-renderer surfaces can use both flows.

Release Notes

  • The desktop Telegram bot now handles images in both directions: /image <prompt> generates pictures on the network and delivers them as photos, and image attachments in agent replies are sent instead of being dropped
  • Photos and documents sent to the bot are read by the agent like in-app attachments (captions become the message text); voice notes get a clear "not supported yet" reply
  • Image-only sellers no longer clutter the bot's /model picker, and /stop also cancels image generations

Types of Changes

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Chore (maintenance tasks, refactoring, or non-functional changes)

Checklist

  • My code follows the code style of this project
  • I have added the necessary documentation (if appropriate)
  • I have added tests (if appropriate)
  • Lint and unit tests pass locally with my changes

kotevcode and others added 2 commits August 22, 2026 00:47
Image sending was dropped from main in 1224a4c as part of a discovery
refactor cleanup. Restore it and extend the bridge to handle media in
both directions:

- /image <prompt> generates via the best openai-images seller; drafts a
  Generating status and delivers the result as a photo (document
  fallback for oversized or non-inline formats)
- Assistant replies with image file blocks now send those images instead
  of collapsing to Done (no text reply)
- Incoming photos and documents are downloaded, stored as conversation
  attachments, and passed through the same multimodal pipeline as the
  app's paper-clip attachments; captions become the message text
- Voice/video/audio get an explicit unsupported notice instead of the
  generic text-only error
- Image-only sellers are excluded from the bot's text-model picker and
  default-route fallback; /stop also cancels image generations
@kotevcode

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown

Claude finished @kotevcode's task in 3m 25s —— View job


Review complete

  • Gather context (PR description, changed files)
  • Read diff for all changed files
  • Review apps/desktop/src/main/telegram/media.ts (new)
  • Review apps/desktop/src/main/telegram/bot-api.ts
  • Review apps/desktop/src/main/telegram/bridge.ts
  • Review apps/desktop/src/main/chat/engine.ts
  • Review tests
  • Post final review

Overall this is a clean, well-scoped restoration + extension of Telegram media support. media.ts is a nice pure/testable extraction, the bot-api.ts multipart upload plumbing (tgUpload, sendPhoto/sendDocument, getFile/downloadFile) is straightforward and reuses the existing response-parsing path, and the size/mime handling (photo vs. document fallback, 20 MB Bot API cap, caption truncation) is careful. media.test.ts gives solid coverage of the new pure helpers.

One correctness issue worth fixing before merge:

Missing busy-check in runUserText allows a text turn to run concurrently with an in-flight /image generation or attachment download

apps/desktop/src/main/telegram/bridge.ts:591runUserText only guards on if (activeTurn):

const runUserText = async (chatId: number, text: string, attachments?: PreparedChatAttachment[]): Promise<void> => {
  if (activeTurn) {
    void sendToOwner('Still working on the previous message — send /stop to cancel it first.');
    return;
  }
  ...

Its two siblings, runImagePrompt (~line 649) and runUserMedia (~line 787), both correctly guard with all three busy flags:

if (activeTurn || activeImageRunConversationId || mediaInFlight) { ... }

But the direct text path at bridge.ts:889 (void runUserText(message.chat.id, text)) goes through the two-flag-short check. Concretely: owner sends /image a crazy ant, which sets activeImageRunConversationId but not activeTurn; while the image is generating, the owner sends a normal text message. runUserText sees activeTurn === null and proceeds, calling engine.sendMessageStream — which starts a text streaming run on the same conversationId while engine.generateImage is still in flight for it (confirmed in apps/desktop/src/main/chat/engine.ts:913, which itself blocks a new image run if a text run or image run is active, but nothing on the text-run side blocks against an active image run). The result is two concurrent operations writing into the same conversation's message history, which can interleave/duplicate messages in the transcript sent back to Telegram.

Suggested fix: apply the same activeTurn || activeImageRunConversationId || mediaInFlight guard in runUserText (or factor it into one shared isBusy() helper used by all three entry points, so this can't drift again).

Fix this →

Minor/non-blocking notes:

  • runUserMediaInner (bridge.ts:804) has a loop reporting failed attachments that were skipped-but-not-fatal, but classifyIncomingMedia only ever produces a single RawChatAttachment per inbound message today, so that branch is currently unreachable — harmless, just future-proofing worth a one-line comment if intentional.
  • resolveImageRoute's comment calls it "the best-ranked image service within the buyer routing policy," but it's just entries.find(...) for the first openai-images entry — fine if the catalog is already rank-ordered upstream (consistent with the existing text-model fallback pattern), just flagging in case that assumption isn't guaranteed.

I wasn't able to run pnpm typecheck/pnpm test in this sandbox (Bash tool calls required approval that wasn't available non-interactively), so this review is based on manual reading of the diff against origin/main — worth confirming CI is green on the type/lint/test fronts.
· Branch: fix/telegram-bot-images

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant