Skip to content

[codex] Add native mobile composer and markdown#3101

Merged
juliusmarminge merged 13 commits into
mainfrom
codex/mobile-native-composer-markdown
Jun 16, 2026
Merged

[codex] Add native mobile composer and markdown#3101
juliusmarminge merged 13 commits into
mainfrom
codex/mobile-native-composer-markdown

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Jun 16, 2026

Copy link
Copy Markdown
Member

Summary

Move the post-architecture mobile product work into a focused, independently reviewable PR.

  • consolidate Clerk mobile auth sheet routing and upgrade Clerk dependencies
  • add the UIKit-backed native composer with atomic skill/file tokens, selection synchronization, and pasted-image handling
  • add selectable native markdown rendering with cross-node selection, Shiki code blocks, copy actions, and generated Pierre file icons
  • align mobile chat presentation, settled-turn folding, work rows, copy behavior, keyboard-aware scrolling, and composer layout
  • share composer token parsing with web without pulling in the connection/runtime rearchitecture
  • remove all vscode-icons references outside apps/web/THIRD_PARTY_NOTICES.md

Stack

  1. main
  2. Mobile native composer, markdown, and auth ← this PR
  3. [codex] Rewrite client connection architecture #2978 Rewrite client connection architecture

The client connection/runtime rearchitecture is intentionally excluded and remains in #2978, stacked on top of this PR.

Validation

  • vp check
  • vp run typecheck
  • vp run lint:mobile
  • vp test --exclude apps/server/src/assets/AssetAccess.test.ts — 449 files and 3,554 tests passed

A full vp test run only hits the existing macOS /var versus /private/var assertions in apps/server/src/assets/AssetAccess.test.ts.


Note

High Risk
Large new native surface (Expo module + Fabric codegen, CocoaPods) on the critical composer and message-rendering path; iOS-only behavior increases platform divergence risk.

Overview
Adds two iOS-only native modules and wires them through mobile chat and settings.

t3-composer-editor is a UIKit UITextView–backed composer that renders skill and file paths as atomic attachment chips, keeps source text/selection in sync with JS, and handles image paste to temp files. @t3tools/mobile-markdown-text is a Fabric attributed-text stack (Bluesky-derived, heavily customized) with SelectableMarkdownText: GFM parsing, Shiki-highlighted code blocks, file/skill chips, Pierre file icons (sync:pierre-icons), and native selection.

Theme tokens in global.css and clerk-theme.json gain inline-skill and glass-surface colors; screens move off hard-coded colors to useThemeColor.

Clerk: new /settings/auth route with AuthView / UserProfileView, ClerkSettingsSheetDetentProvider so the settings sheet expands for sign-in, and account/waitlist flows navigate there instead of alerts or a modal. @clerk/expo bumps to ^3.4.1.

Non-iOS markdown/composer paths stub or fall back; Android is excluded from the new pods/codegen.

Reviewed by Cursor Bugbot for commit 3a4a825. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add native iOS composer editor and selectable markdown text renderer to the mobile app

  • Introduces a native iOS rich text editor (T3ComposerEditor) with inline token chips for skills and file mentions, image paste support, and theme-driven styling via a new Expo module in apps/mobile/modules/t3-composer-editor.
  • Adds a native iOS markdown renderer (T3MarkdownText) with Fabric-native attributed text runs, selectable text, syntax-highlighted code blocks, file icons, and external link favicons via apps/mobile/modules/t3-markdown-text.
  • Replaces plain TextInput with ComposerEditor in ThreadComposer and NewTaskDraftScreen, wiring skill awareness, selection control, and image paste callbacks.
  • Thread feed gains per-turn folding, auto-scroll follow when near the end, copy buttons, formatted timestamps, and native selectable markdown for both user and assistant messages.
  • Extracts collectComposerInlineTokens into packages/shared and updates the web composer to delegate to the shared implementation.
  • Adds a /settings/auth route for in-app Clerk sign-in/account UI, replacing the previous alert-based flow and controlled by a new ClerkSettingsSheetDetentProvider.
  • Risk: the native T3MarkdownText and T3ComposerEditor modules are iOS-only; non-iOS platforms fall back to stub/plain-text implementations.

Macroscope summarized 3a4a825.

juliusmarminge and others added 12 commits June 16, 2026 00:22
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: fef61048-d696-43ae-b7a9-ae2fe3151a37

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/mobile-native-composer-markdown

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Jun 16, 2026
@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🚀 Expo continuous deployment is ready!

  • Project → t3-code
  • Platforms → android, ios
  • Scheme → t3code-preview
  🤖 Android 🍎 iOS
Fingerprint 971e873f513ad97675807fa4bf5eb9577ad935ce 3d9696e1c58021a1749a75e2a781a9012f956663
Build Details Build Permalink
DetailsDistribution: INTERNAL
Build profile: preview:dev
Runtime version: 971e873f513ad97675807fa4bf5eb9577ad935ce
App version: 0.1.0
Git commit: d613d218be1006404042654242d533ab1a1f2fd4
Build Permalink
DetailsDistribution: INTERNAL
Build profile: preview:dev
Runtime version: 3d9696e1c58021a1749a75e2a781a9012f956663
App version: 0.1.0
Git commit: d613d218be1006404042654242d533ab1a1f2fd4
Update Details Update Permalink
DetailsBranch: pr-3101
Runtime version: 971e873f513ad97675807fa4bf5eb9577ad935ce
Git commit: d613d218be1006404042654242d533ab1a1f2fd4
Update Permalink
DetailsBranch: pr-3101
Runtime version: 3d9696e1c58021a1749a75e2a781a9012f956663
Git commit: d613d218be1006404042654242d533ab1a1f2fd4
Update QR

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for all 3 issues found in the latest run.

  • ✅ Fixed: Caret not restored after update
    • Changed the condition from savedRange.length > 0 to savedRange.location != NSNotFound so collapsed carets (zero-length selections) are also restored after attributedText reassignment, matching the behavior in refreshDisplayedAttachments.
  • ✅ Fixed: Bold weight maps to semibold
    • Changed the mapping for font weight 700/"700" from "semibold" to "bold" so markdown bold and heading runs render at the correct weight on iOS.
  • ✅ Fixed: Theme prop forces full rebuild
    • Added a string equality guard in setThemeJson (matching the tokensJson pattern), stored the raw themeJson string for comparison, and memoized the themeJson construction on the JS side to prevent redundant document rebuilds on re-renders.

Create PR

Or push these changes by commenting:

@cursor push f7e5cbe9a3
Preview (f7e5cbe9a3)
diff --git a/apps/mobile/modules/t3-composer-editor/ios/T3ComposerEditorView.swift b/apps/mobile/modules/t3-composer-editor/ios/T3ComposerEditorView.swift
--- a/apps/mobile/modules/t3-composer-editor/ios/T3ComposerEditorView.swift
+++ b/apps/mobile/modules/t3-composer-editor/ios/T3ComposerEditorView.swift
@@ -263,6 +263,7 @@
   private var tokensJson = "[]"
   private var tokens: [ComposerTokenPayload] = []
   private var requestedSelection: ComposerSelectionPayload?
+  private var themeJson = ""
   private var theme = ComposerThemePayload(
     text: "#262626",
     placeholder: "#8e8e93",
@@ -377,6 +378,10 @@
   }
 
   func setThemeJson(_ themeJson: String) {
+    guard self.themeJson != themeJson else {
+      return
+    }
+    self.themeJson = themeJson
     guard let nextTheme = decode(ComposerThemePayload.self, from: themeJson) else {
       return
     }

diff --git a/apps/mobile/modules/t3-markdown-text/ios/T3MarkdownText.mm b/apps/mobile/modules/t3-markdown-text/ios/T3MarkdownText.mm
--- a/apps/mobile/modules/t3-markdown-text/ios/T3MarkdownText.mm
+++ b/apps/mobile/modules/t3-markdown-text/ios/T3MarkdownText.mm
@@ -427,7 +427,7 @@
     const NSRange savedRange = _textView.selectedRange;
     _suppressSelectionChange = YES;
     _textView.attributedText = convertedAttrString;
-    if (savedRange.length > 0 && NSMaxRange(savedRange) <= _textView.attributedText.length) {
+    if (savedRange.location != NSNotFound && NSMaxRange(savedRange) <= _textView.attributedText.length) {
       _textView.selectedRange = savedRange;
     }
     _suppressSelectionChange = NO;

diff --git a/apps/mobile/modules/t3-markdown-text/src/util.ts b/apps/mobile/modules/t3-markdown-text/src/util.ts
--- a/apps/mobile/modules/t3-markdown-text/src/util.ts
+++ b/apps/mobile/modules/t3-markdown-text/src/util.ts
@@ -48,7 +48,7 @@
       return "semibold";
     case 700:
     case "700":
-      return "semibold";
+      return "bold";
     case 800:
     case "800":
       return "bold";

diff --git a/apps/mobile/src/native/T3ComposerEditor.ios.tsx b/apps/mobile/src/native/T3ComposerEditor.ios.tsx
--- a/apps/mobile/src/native/T3ComposerEditor.ios.tsx
+++ b/apps/mobile/src/native/T3ComposerEditor.ios.tsx
@@ -125,17 +125,31 @@
       })),
     );
   }, [props.value, skillLabels]);
-  const themeJson = JSON.stringify({
-    text: String(textColor),
-    placeholder: String(placeholderColor),
-    chipBackground: String(chipBackground),
-    chipBorder: String(chipBorder),
-    chipText: String(chipText),
-    skillBackground: String(skillBackground),
-    skillBorder: String(skillBorder),
-    skillText: String(skillText),
-    fileTint: String(fileTint),
-  });
+  const themeJson = useMemo(
+    () =>
+      JSON.stringify({
+        text: String(textColor),
+        placeholder: String(placeholderColor),
+        chipBackground: String(chipBackground),
+        chipBorder: String(chipBorder),
+        chipText: String(chipText),
+        skillBackground: String(skillBackground),
+        skillBorder: String(skillBorder),
+        skillText: String(skillText),
+        fileTint: String(fileTint),
+      }),
+    [
+      textColor,
+      placeholderColor,
+      chipBackground,
+      chipBorder,
+      chipText,
+      skillBackground,
+      skillBorder,
+      skillText,
+      fileTint,
+    ],
+  );
   const resolvedTextStyle = StyleSheet.flatten(textStyle) ?? {};
   return (
     <NativeView

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 86731fe. Configure here.

if (savedRange.length > 0 && NSMaxRange(savedRange) <= _textView.attributedText.length) {
_textView.selectedRange = savedRange;
}
_suppressSelectionChange = NO;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caret not restored after update

Medium Severity

When markdown content is reassigned in drawRect, selection is only restored if the saved range has non-zero length, so a collapsed caret (insertion point) is dropped after streaming or other text updates. refreshDisplayedAttachments already restores both caret and range selections.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 86731fe. Configure here.

return "semibold";
case 700:
case "700":
return "semibold";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bold weight maps to semibold

Low Severity

fontWeightToNativeProp maps both "700" and numeric 700 to "semibold" instead of "bold", so markdown runs styled with fontWeight: "700" (headings and bold in NativeMarkdownSelectableText) render lighter than intended on iOS native runs.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 86731fe. Configure here.

theme = nextTheme
applyTheme()
applyControlledDocument(force: true)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theme prop forces full rebuild

Medium Severity

setThemeJson always calls applyControlledDocument(force: true) without comparing to the current theme, and the iOS wrapper builds a new themeJson string every render. Any native prop update with unchanged colors still rebuilds the full attributed document and resets selection handling, which can interrupt typing when the composer’s parent re-renders.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 86731fe. Configure here.

// Even if the uiTextView prop is set, we can still default to using
// normal selection (i.e. base RN text) if the text doesn't need to be
// selectable
if ((!props.selectable || !props.uiTextView) && !isAncestor) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium src/MarkdownTextPrimitive.tsx:98

When uiTextView={true} is passed without explicitly setting selectable, the component returns RNText instead of the UITextView-based native component. This happens because the check !props.selectable at line 98 evaluates to true when selectable is undefined, even though textDefaults sets selectable: true by default. The uiTextView prop is effectively ignored in this case, which contradicts the intended behavior of using the native UITextView when that prop is enabled.

-  if ((!props.selectable || !props.uiTextView) && !isAncestor) {
+  const isSelectable = props.selectable ?? textDefaults.selectable;
+  if ((!isSelectable || !props.uiTextView) && !isAncestor) {
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/modules/t3-markdown-text/src/MarkdownTextPrimitive.tsx around line 98:

When `uiTextView={true}` is passed without explicitly setting `selectable`, the component returns `RNText` instead of the UITextView-based native component. This happens because the check `!props.selectable` at line 98 evaluates to `true` when `selectable` is `undefined`, even though `textDefaults` sets `selectable: true` by default. The `uiTextView` prop is effectively ignored in this case, which contradicts the intended behavior of using the native UITextView when that prop is enabled.

Evidence trail:
apps/mobile/modules/t3-markdown-text/src/MarkdownTextPrimitive.tsx lines 12-15 (textDefaults with selectable: true), line 98 (condition `(!props.selectable || !props.uiTextView) && !isAncestor`), lines 74-84 (textDefaults spread only inside MarkdownTextPrimitiveChild), lines 95-97 (comment explaining intended behavior). Commit: REVIEWED_COMMIT.

import { useThemeColor } from "../lib/useThemeColor";

function AppNavigator() {
const pathname = usePathname();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low app/_layout.tsx:34

When the user navigates to /settings/auth after the app has already mounted on a different route, clerkRouteIsActive becomes true, but ClerkSettingsSheetDetentProvider ignores the prop change because its isExpanded state is initialized only via useState(initiallyExpanded). The sheet renders with [0.7] detent instead of the intended [0.92] because isExpanded remains false. Consider synchronizing the expanded state with the route using useEffect inside the provider, or moving the route detection and state into a component that can call expand() when entering the auth route.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/app/_layout.tsx around line 34:

When the user navigates to `/settings/auth` after the app has already mounted on a different route, `clerkRouteIsActive` becomes `true`, but `ClerkSettingsSheetDetentProvider` ignores the prop change because its `isExpanded` state is initialized only via `useState(initiallyExpanded)`. The sheet renders with `[0.7]` detent instead of the intended `[0.92]` because `isExpanded` remains `false`. Consider synchronizing the expanded state with the route using `useEffect` inside the provider, or moving the route detection and state into a component that can call `expand()` when entering the auth route.

Evidence trail:
apps/mobile/src/features/cloud/ClerkSettingsSheetDetent.tsx line 26: `useState(initiallyExpanded)` — only used on mount. apps/mobile/src/app/_layout.tsx lines 34-41: `clerkRouteIsActive` passed as `initiallyExpanded`. apps/mobile/src/app/_layout.tsx line 81: `sheetAllowedDetents: isExpanded ? [0.92] : [0.7]`. apps/mobile/src/app/settings/auth.tsx: no call to `expand()`. apps/mobile/src/app/settings/waitlist.tsx line 48: only `waitlist` calls `expand()`, not `auth`.

Comment on lines +49 to +51
case 700:
case "700":
return "semibold";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium src/util.ts:49

fontWeight: 700 and fontWeight: "700" are mapped to "semibold", but React Native treats 700 as equivalent to "bold". This creates an inconsistency where fontWeight: "bold" renders as bold while fontWeight: 700 renders as semibold. Consider mapping 700 and "700" to "bold" to match React Native's behavior.

-    case 700:
-    case "700":
-      return "semibold";
+    case 700:
+    case "700":
+      return "bold";
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/modules/t3-markdown-text/src/util.ts around lines 49-51:

`fontWeight: 700` and `fontWeight: "700"` are mapped to `"semibold"`, but React Native treats `700` as equivalent to `"bold"`. This creates an inconsistency where `fontWeight: "bold"` renders as bold while `fontWeight: 700` renders as semibold. Consider mapping `700` and `"700"` to `"bold"` to match React Native's behavior.

Evidence trail:
apps/mobile/modules/t3-markdown-text/src/util.ts lines 20-61 (at REVIEWED_COMMIT): fontWeightToNativeProp function showing 700/"700" → "semibold" (line 49-51) while "bold" → "bold" (line 25). React Native Font Weight Cheatsheet (https://gist.github.com/knowbody/c5cdf26073b874eae86ba96e7cf3a540) confirms 700 = Bold. React Native docs (https://reactnative.dev/docs/text-style-props) list '700' and 'bold' as font weight values.


const INLINE_HTML_TAG_PATTERN = /<\/?(?:kbd|mark|sub|sup|u)(?:\s[^>]*)?>/gi;

function decodeHtmlEntitiesOnce(value: string): string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low src/nativeMarkdownText.ts:73

decodeHtmlEntitiesOnce throws RangeError on input like or because String.fromCodePoint rejects values above 0x10FFFF. The regex accepts any digit/hex length without range validation, so malformed entities crash the parser. Consider adding bounds checks before calling fromCodePoint.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/modules/t3-markdown-text/src/nativeMarkdownText.ts around line 73:

`decodeHtmlEntitiesOnce` throws `RangeError` on input like `�` or `�` because `String.fromCodePoint` rejects values above `0x10FFFF`. The regex accepts any digit/hex length without range validation, so malformed entities crash the parser. Consider adding bounds checks before calling `fromCodePoint`.

Evidence trail:
apps/mobile/modules/t3-markdown-text/src/nativeMarkdownText.ts lines 73-82 at REVIEWED_COMMIT: regex `#(\d+)` and `#x([0-9a-f]+)` accept unbounded numeric values; `String.fromCodePoint()` called at lines 78 and 81 without range validation; ECMAScript spec: String.fromCodePoint throws RangeError for values > 0x10FFFF.

}

const tokens = [...new Set([...builtInTokens, ...Object.keys(customIcons)])].sort();
const generatedSource = `import type { ImageSourcePropType } from "react-native";\n\nexport const MARKDOWN_FILE_ICON_SOURCES = {\n${tokens

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low scripts/sync-pierre-file-icons.mjs:130

The template on line 131 generates unquoted object keys like ${token}: require(...). When builtInTokens contains values with hyphens (e.g., c-sharp from a regex match like [^"]+), the output becomes c-sharp: require(...) which TypeScript parses as c - sharp: require(...) — a syntax error. Consider wrapping the token in quotes: "${token}": require(...).

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/modules/t3-markdown-text/scripts/sync-pierre-file-icons.mjs around line 130:

The template on line 131 generates unquoted object keys like `${token}: require(...)`. When `builtInTokens` contains values with hyphens (e.g., `c-sharp` from a regex match like `[^"]+`), the output becomes `c-sharp: require(...)` which TypeScript parses as `c - sharp: require(...)` — a syntax error. Consider wrapping the token in quotes: `"${token}": require(...)`.

Evidence trail:
apps/mobile/modules/t3-markdown-text/scripts/sync-pierre-file-icons.mjs lines 114-116 (regex `[^"+]` captures tokens), lines 130-132 (template uses unquoted `${token}:`), apps/mobile/modules/t3-markdown-text/src/markdownFileIcons.generated.ts (current output with all-simple-identifier keys). Dependency: `@pierre/trees@1.0.0-beta.4` in apps/mobile/package.json line 110.

@macroscopeapp

macroscopeapp Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

2 blocking correctness issues found. Diff is too large for automated approval analysis. A human reviewer should evaluate this PR.

You can customize Macroscope's approvability policy. Learn more.

Co-authored-by: codex <codex@users.noreply.github.com>
@juliusmarminge juliusmarminge merged commit 689a882 into main Jun 16, 2026
16 checks passed
@juliusmarminge juliusmarminge deleted the codex/mobile-native-composer-markdown branch June 16, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant