Skip to content

vm: keep #ungetc's bytes out of the file, and count them in #pos - #977

Open
tannevaled wants to merge 1 commit into
mainfrom
ungetc-write-position
Open

tannevaled wants to merge 1 commit into
mainfrom
ungetc-write-position

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

What

IO#ungetc / #ungetbyte on a File now behave as MRI's read buffer does: the pushed-back bytes are never written to the file, and the position counts them. Refs #974 (its ungetc item; the pread flush item and the TOCTOU item stay open).

Why

A File holds its content in buf, and #ungetc spliced the bytes into it with no record of where they were. Two defects followed.

  • A later buffered write flushed buf whole, so the pushed-back bytes reached the disk. getc; ungetc "Z"; seek(0, :END); write "x" left "0Z123…x" in the file.
  • The cursor ignored them. MRI's position is the descriptor's offset minus what the read buffer still holds (rb_io_tell: pos -= fptr->rbuf.len), so every write after an ungetc landed late. getc; ungetc "Z"; write "AB" gave "0AB23456789", where MRI 4.0.7 gives "AB23456789".

How

internal/vm/io_pushback.go records the region [ungetLo, ungetHi) the bytes occupy and implements io_unread from io.c:

  • Writes (rb_io_check_writable), #flush, #seek, #rewind, #pos=, #truncate, #syswrite and #pwrite drop the pushback. The cursor then moves to the logical position. When that position is negative, it moves to where the read-ahead stopped.
  • #pos (io_tell, discard_rbuf = false) drops the pushback only when the position is valid. Otherwise it keeps the pushback and answers the negative position, as MRI does.
  • #sysread and #sysseek raise IOError … for buffered IO (READ_DATA_BUFFERED).
  • Flushes, #pread's buffer fallback and File#size see the file without the pushback.
  • #reopen and set_encoding_by_bom replace the buffer, so they reset the region. reopen(io) adopts the file's bytes, not the other stream's pushback.

StringIO keeps its own semantics, and those are now MRI's. ext/stringio strio_unget_bytes writes the pushback into the string, over the bytes just read. rbgo had spliced it in like a file's: StringIO.new("0123"), then getc, ungetc "Z", write "AB" gave "0AB23" at pos 1 instead of "AB23" at pos 0.

Measured

  • Differential against MRI 4.0.7: rubydiff on 44 probe rows (33 File, 8 StringIO, 3 reopen/empty), all AGREE. One exception, documented in the README divergences table: on a file larger than 8192 bytes, MRI refuses pushback beyond what was read (IOError: ungetbyte failed, io_ungetbyte's capacity check). rbgo accepts it.
  • Tests: TestUngetcOnAFileIsNotTheFiles (22 rows), TestStringIOUngetcWritesIntoTheString (9), TestReopenDropsThePushback, plus two internal tests. One holds the defensive guard in compactPushback, the other asserts diskBytes does not allocate without pushback.
  • internal/vm suite: ok, 286 s.
  • Mutation sweep, io_pushback.go: mutsweep caught 7/7. The first sweep's five survivors were read one by one: two redundant guards were removed, and three got tests.
  • Mutation sweep, the two for buffered IO refusals: mutsweep does not list them as refusals (body is a raise(...) call). A hand-made false && mutant of each turned exactly its own subtest red.
  • Cross-platform: buildscan --vet: go vet passed under all 19 CI configurations.

🤖 Generated with Claude Code

A File holds its content in buf, and #ungetc spliced the pushed-back
bytes into it with no record of where they were. A later buffered
write flushed buf whole, so the bytes reached the disk, and the cursor
ignored them: MRI's position is the descriptor's offset minus what the
read buffer still holds, so every write after an ungetc landed late.

io_pushback.go records the region the bytes occupy and implements
io.c's io_unread: writes, #flush, #seek, #rewind, #pos=, #truncate,
#syswrite and #pwrite drop the pushback and move to the logical
position; #pos (io_tell) drops it when that position is valid and
answers it negative otherwise; #sysread and #sysseek refuse "for
buffered IO"; flushes, #pread's fallback and File#size see the file
without it. IO#reopen and set_encoding_by_bom, which replace the
buffer, reset the region, and reopen(io) adopts the file's bytes, not
the other stream's pushback.

StringIO keeps its own semantics, now MRI's: strio_unget_bytes writes
the pushback into the string over the bytes just read, where it was
spliced in like a file's.

Refs #974

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant