Repository navigation
Keep FileWriter lines intact when several threads write - #102
Merged
Merged
Conversation
FileWriter.write used print(), which writes the line and the trailing newline in two separate calls. Another thread can write between them, so concurrent metric updates sent to stdout, stderr or a file produced glued lines such as "c:a:1c:b:1" followed by an empty line. Writing the line and its newline in one call keeps each protocol line whole.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Services that record metrics from several threads while spectator-py writes to
stdout,stderror afile://location get corrupted protocol lines whenever two threads write at the same moment, so those updates are lost or misread.Root cause
FileWriter.writecallsprint(line, file=self._file).print()writes the text and theendnewline as two separatewrite()calls, so another thread can write in between. The output then contains glued lines followed by an empty line:With 4 threads each incrementing their own counter 50,000 times on a
file://registry, 6 of the 200,000 lines came out malformed on Python 3.12. With this change all 200,000 lines are well formed.Fix
Write the line and its newline in a single
write()call.Test
test_file_writer_writes_each_line_in_one_callwrapswriteon the redirected stdout and checks thatFileWriter.write("foo")makes exactly onewrite("foo\n")call. Before the change it records[call('foo'), call('\n')]and fails; after the change it passes. pylint is clean on the changed files.