Skip to content

imap: Add empty-partial-as-quoted to imap_client_workarounds - #317

Open
0nelight wants to merge 1 commit into
dovecot:mainfrom
0nelight:imap-empty-partial-as-quoted
Open

imap: Add empty-partial-as-quoted to imap_client_workarounds#317
0nelight wants to merge 1 commit into
dovecot:mainfrom
0nelight:imap-empty-partial-as-quoted

Conversation

@0nelight

@0nelight 0nelight commented Sep 8, 2026

Copy link
Copy Markdown

Mailing list thread: https://dovecot.org/mailman3/archives/list/dovecot@dovecot.org/message/DMYA4ABWVAZDOMNOL55L7B2UEDAQSL4V/

The symptom

iOS Mail stalls for ~90 seconds after a fully answered pipelined FETCH,
then drops the connection, reconnects, and shows the message instantly. The
server logs look like an idle client going away:

Disconnected: Connection closed (UID FETCH finished 93.349 secs ago)
  in=1196 out=397777 body_count=12 body_bytes=389817

From a decrypted capture of one such session — ten pipelined commands, all ten
answered and tagged within 417 ms, then silence:

18:33:15.726  C> P65..P74 UID FETCH 959 (UID BODY.PEEK[1]<0.393216>) ... x10
18:33:16.092  S> * 916 FETCH (UID 959 BODY[1]<0> {297588} ...)
18:33:16.143  S> * 916 FETCH (UID 959 BODY[1]<393216> {0} )   ... and eight more
18:33:16.143  S> P65 OK Fetch completed (0.381 secs).
18:33:16.143  S> P66..P74 OK Fetch completed (0.001 secs each).
              ---- 93 s of silence, then the client closes ----

The message is 6,928,200 bytes but section [1] is only 297,588. The client
sizes its 384 KiB slice plan from RFC822.SIZE, so nine slices land past the
end of the section and are answered with an empty string, as RFC 9051 6.4.5
requires. Dovecot writes that as a zero-length literal, {0}\r\n.

Root cause, and it is not Dovecot

The IMAP client in iOS 26.6.1 (build 23G83) is a fork of swift-nio-imap. The
legacy ObjC stack (MFIMAPConnection) is still present in Message.framework,
but it is not what serves these connections: the Swift stack's
Tag(connectionIdentifier: UInt8, commandCounter: UInt32) produces exactly the
tag shapes in the capture above (P65, and AN196/EX33 in others). Its
FramingParser mishandles the zero-length literal:

  1. the header line is emitted as a completed frame, frameLength is reset to 0,
    state becomes .insideLiteral(remaining: 0)
  2. on the next pass bytesAvailable >= remaining is trivially true for
    remaining == 0, so frameLength += 0 leaves it at 0 and the state returns
    to .normalTraversal
  3. back in parseFrame(), if (0 < frameLength) fails, and it returns
    .incomplete(byteCountNeeded: 2)
  4. parseFrames() stops at the first .incomplete

The )\r\n that is already sitting in the client's own buffer — and every
tagged OK behind it — is never parsed. Nothing re-drives the parser except
fresh network bytes, and after a completed FETCH none arrive. The client waits
for two bytes it already has, until its own watchdog fires at ~90 s.

This is a client defect. Dovecot's reply is correct and this PR does not
suggest otherwise. The workaround is for operators who cannot wait for the
client to be fixed, which is what imap_client_workarounds has always been for.

Upstream

The client-side defect is now filed in the open-source project the
iOS stack forks from, with a runnable reproducer rather than a description:

That gives this workaround a defined end of life: once the fix ships in iOS, it
is dead weight and can be removed.

The change

Behind a new imap_client_workarounds value, empty-partial-as-quoted,
get_prefix() writes an empty string as "" instead of {0}. The ABNF has
string = quoted / literal, so this changes the encoding of the reply, not the
reply.

Arguably it is also a consistency fix. Dovecot already prefers "" for an empty
string everywhere else: imap_append_nstring() cannot produce a literal for one
(its character loop never runs), and imap_append_string_for_humans() has an
explicit size == remove_count -> str_append(dest, "\"\"") case.
get_prefix() was the exception — understandably so, since it writes the header
for data that is streamed out afterwards, and a literal is the only string form
that can carry arbitrary octets. Zero is the one size where the two forms are
interchangeable.

The workaround is deliberately not restricted to partial fetches: the defect
is in the framing of any zero-length literal, so a plain FETCH of an empty
section is affected too. That does make the setting name narrower than the
behaviour — happy to rename it to empty-string-as-quoted if you prefer.

Testing

Over a full session in the shape iOS Mail sends (ID, COMPRESS DEFLATE,
SELECT, RFC822.SIZE/BODYSTRUCTURE, then ten pipelined
BODY.PEEK[1]<offset.393216>) against a 6.9 MB message whose section [1] is
285 KB:

zero-length literals in the session physical lines per empty reply
without the workaround 9 2
with it 0 1

Also checked: a plain FETCH of an empty MIME part (covered), a run with
COMPRESS=DEFLATE negotiated (unaffected), and a control run of the same
package with the workaround off (unchanged, nine zero-length literals, no
crashes).

Running on my production server since 2026-09-08. On the wire:

* 35757 FETCH (UID 98709 BODY[]<99999999> "")

Applies cleanly to 2.4.4, 2.4.5 and main.

Prior art

tb-lsub-flags (1c3e6a4, 2011) has the same shape: enum bit in
imap-settings.h, list entry in imap-settings.c, behaviour change in the
command file. The fourth file it touched, doc/example-config/conf.d/20-imap.conf,
no longer exists after 241e4b3.

A FETCH of a body section range that lies past the end of that section is
answered with an empty string, as RFC 9051 6.4.5 requires. Dovecot writes
it as a zero-length literal, "{0}\r\n": get_prefix() emits the header for
data that is streamed out afterwards, and a literal is the only string
form that can carry arbitrary octets, so one format string serves every
size.

The framing parser in swift-nio-imap mishandles exactly the zero-length
case. It emits the header line as a completed frame and enters
.insideLiteral(remaining: 0); the next pass leaves that state again
without consuming a byte, so frameLength is still 0, and parseFrame()
returns .incomplete. parseFrames() stops at the first .incomplete, so the
")\r\n" already sitting in the client's own buffer -- and every tagged
reply behind it -- is never parsed. Only fresh network bytes re-drive the
parser, and after a fully answered FETCH none arrive.

iOS Mail (ID name com.apple.email.maild) is built on a fork of that
library. The effect is a stall of about 90 seconds after a completed,
pipelined FETCH, ended only by the client's own watchdog. It reproduces
with any message whose fetched section is much smaller than RFC822.SIZE,
because the client sizes its 384 KiB slice plan from the latter and every
slice past the end of the section then gets an empty string.

"" and {0} are the same value -- the ABNF has string = quoted / literal --
so this changes the encoding of the reply, not the reply. Dovecot already
prefers "" for an empty string everywhere else: imap_append_nstring()
cannot produce a literal for one, and imap_append_string_for_humans() has
an explicit "" case. get_prefix() was the exception.

The workaround is not restricted to partial fetches. The defect is in the
framing of any zero-length literal, so a plain FETCH of an empty section
is affected just as much.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant