Goal
Degrade gracefully when a frame's source file cannot be decoded, instead of
letting the whole debugger page raise.
Supporting arbitrary source encodings is explicitly NOT the goal. A frame whose
source cannot be read should simply render without a source listing, exactly as
already happens when the file is missing.
Done when: rendering a traceback that contains a frame from an undecodable
source file produces a debug page (that frame showing no source lines) rather
than raising, and a test covers it.
Reproduction (reproduces on Python 3.9 through 3.14) - a latin-1 encoded module
executed so that the frame has no usable loader:
src = open('mod_latin1.py', 'rb').read() # contains a 0xe9 byte
ns = {}
exec(compile(src, 'mod_latin1.py', 'exec'), ns)
from backlash.tbtools import get_current_traceback
try:
ns['boom']()
except Exception:
tb = get_current_traceback()
tb.render_full()
Current behaviour:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 52:
invalid continuation byte
render_full, render_source and plaintext all raise, so the middleware
fails while trying to report the original error and the user sees nothing
useful.
Cause: Frame.sourcelines calls plain open(self.filename), which decodes
using the locale/default encoding, and only guards against IOError.
Suggested minimal fix: widen the existing guard so a decode failure is treated
like an unreadable file, for example
except (OSError, UnicodeDecodeError): return []. The IOError spelling in
that guard is also the Python 2 alias for OSError and can be modernized in
the same edit.
The same crash occurs for ordinary UTF-8 sources when the interpreter's locale
encoding is ASCII (verified with LC_ALL=C PYTHONCOERCECLOCALE=0 PYTHONUTF8=0
on 3.9 and 3.14), which makes this reachable without any unusual source
encoding.
Note that _coding_re in backlash/tbtools.py is now unused; it is the
leftover of a coding-cookie detection block that only ever ran on Python 2.
It can be removed alongside this fix.
Why
A debugger that raises while rendering an error is worse than one that shows an
incomplete frame: the original exception is lost and the middleware itself
becomes the failure. Falling back to "no source available" keeps the rest of
the page, including the traceback, the other frames, and the console.
The locale-encoding variant means this is not limited to legacy encodings, so a
container or CI environment with a minimal locale is enough to trigger it. No
special setup on the application side is required.
Verified as pre-existing rather than a regression: the pre-modernization code
carried an encoding-detection block, but on Python 3 it was already unreachable
- text-mode
open() returns str, so f.read() raised before the detection
logic could run. The identical reproduction fails on the preceding commit.
References
backlash/tbtools.py (Frame.sourcelines, Frame.current_line, the unused
_coding_re)
backlash/tbtools.py (Traceback.render_full, Traceback.plaintext) are the
callers that currently propagate the error
tests/test_tbtools.py may be a natural home for a regression test
Goal
Degrade gracefully when a frame's source file cannot be decoded, instead of
letting the whole debugger page raise.
Supporting arbitrary source encodings is explicitly NOT the goal. A frame whose
source cannot be read should simply render without a source listing, exactly as
already happens when the file is missing.
Done when: rendering a traceback that contains a frame from an undecodable
source file produces a debug page (that frame showing no source lines) rather
than raising, and a test covers it.
Reproduction (reproduces on Python 3.9 through 3.14) - a latin-1 encoded module
executed so that the frame has no usable loader:
Current behaviour:
render_full,render_sourceandplaintextall raise, so the middlewarefails while trying to report the original error and the user sees nothing
useful.
Cause:
Frame.sourcelinescalls plainopen(self.filename), which decodesusing the locale/default encoding, and only guards against
IOError.Suggested minimal fix: widen the existing guard so a decode failure is treated
like an unreadable file, for example
except (OSError, UnicodeDecodeError): return []. TheIOErrorspelling inthat guard is also the Python 2 alias for
OSErrorand can be modernized inthe same edit.
The same crash occurs for ordinary UTF-8 sources when the interpreter's locale
encoding is ASCII (verified with
LC_ALL=C PYTHONCOERCECLOCALE=0 PYTHONUTF8=0on 3.9 and 3.14), which makes this reachable without any unusual source
encoding.
Note that
_coding_reinbacklash/tbtools.pyis now unused; it is theleftover of a coding-cookie detection block that only ever ran on Python 2.
It can be removed alongside this fix.
Why
A debugger that raises while rendering an error is worse than one that shows an
incomplete frame: the original exception is lost and the middleware itself
becomes the failure. Falling back to "no source available" keeps the rest of
the page, including the traceback, the other frames, and the console.
The locale-encoding variant means this is not limited to legacy encodings, so a
container or CI environment with a minimal locale is enough to trigger it. No
special setup on the application side is required.
Verified as pre-existing rather than a regression: the pre-modernization code
carried an encoding-detection block, but on Python 3 it was already unreachable
open()returnsstr, sof.read()raised before the detectionlogic could run. The identical reproduction fails on the preceding commit.
References
backlash/tbtools.py(Frame.sourcelines,Frame.current_line, the unused_coding_re)backlash/tbtools.py(Traceback.render_full,Traceback.plaintext) are thecallers that currently propagate the error
tests/test_tbtools.pymay be a natural home for a regression test