Skip to content
Merged
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -875,7 +875,7 @@ Otherwise:
| `Numeric#to_int` | not defined — `Complex(2.9, 0).to_int` raises `NoMethodError`; MRI returns `2` |
| `File::Stat#==` | `a == b` is `false` for two stats of the same file (MRI: `true`); `a.==(b)` answers `true` |
| `File#stat` on a file unlinked while open | raises `Errno::ENOENT`; MRI answers from the open descriptor. rbgo's streams are buffered by path and hold no descriptor |
| `IO#pwrite` / `IO#pread` on a file unlinked or replaced while open | both reach the file **by its path**, since streams hold no descriptor. A `#pwrite`'s bytes are on disk when it returns, and only those bytes: the stream's other pending writes stay pending, as with `pwrite(2)`. `#pread` sees another writer's bytes and this stream's own pending writes, which it flushes first, as `rb_io_pread` does ([#949](https://github.com/go-embedded-ruby/ruby/issues/949)). The difference is with a path that no longer names the opened file. Unlinked, both fall back to the stream's buffer, where MRI still reaches the old inode. Replaced, they reach the new file. Same root as the row above. One deliberate divergence: `#pread`'s length is bounded by the file's size, so `f.pread(2**40, 0)` returns at once, where MRI 4.0.7 tries to allocate the full length |
| `IO#pwrite` / `IO#pread` on a file unlinked or replaced while open | both reach the file **by its path**, since streams hold no descriptor. A `#pwrite`'s bytes are on disk when it returns, and only those bytes: the stream's other pending writes stay pending, as with `pwrite(2)`. `#pread` sees another writer's bytes and this stream's own pending writes, which it flushes first, as `rb_io_pread` does ([#949](https://github.com/go-embedded-ruby/ruby/issues/949)). The difference is with a path that no longer names the opened file. Unlinked, both fall back to the stream's buffer, where MRI still reaches the old inode. Replaced, they reach the new file. Same root as the row above. Two deliberate divergences. First, `#pread`'s length is bounded by the file's size, so `f.pread(2**40, 0)` returns at once, where MRI 4.0.7 tries to allocate the full length. Second, a hole lives in memory when it has to: a `#pwrite` that would open a hole larger than 64 MiB past the end of the stream's in-memory copy is refused with `Errno::EFBIG` once the path is gone, where MRI writes a sparse file through its descriptor. When the file is reachable, the hole goes to disk as in MRI, and a refusal from the file (EFBIG, ENOSPC, EIO) is raised with its errno ([#973](https://github.com/go-embedded-ruby/ruby/issues/973)) |
| an unflushed file whose directory is removed before the program ends | loses its bytes; MRI, holding a descriptor to an unlinked file, still writes them. Teardown re-opens the path to flush it, so it gets `ENOENT` — and reports nothing, as MRI's shutdown ignores a failing `fflush`. Same root again: streams are buffered by path, which is also [#635](https://github.com/go-embedded-ruby/ruby/issues/635) |
| `Errno` | all **158** of MRI's constant names are present, but **32** report errno `0` where MRI has a real number (`EAUTH`, `EBADRPC`, `EDEVERR`, …). Measured by set difference on darwin: 81 names are zero-valued under rbgo, 49 under MRI, and every MRI zero is also zero under rbgo. Go's portable `syscall` package does not expose the platform-only names |
| `pp` | neither `Kernel#pp` nor `require "pp"` exists (`require "prettyprint"` does work) |
Expand Down
2 changes: 1 addition & 1 deletion internal/vm/file.go
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ func (vm *VM) registerFile() {
// a leading mode/owner/time argument followed by one or more paths, returning
// the count of paths affected (MRI semantics).
def("chmod", func(vm *VM, _ object.Value, args []object.Value, _ *Proc) object.Value {
mode := os.FileMode(vm.num2mode(args[0]) & 0o7777)
mode := chmodMode(int64(vm.num2mode(args[0])))
paths := args[1:]
for _, a := range paths {
p := pathArg(vm, a)
Expand Down
60 changes: 60 additions & 0 deletions internal/vm/file_mode.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
// Copyright (c) the go-embedded-ruby/ruby authors
//
// SPDX-License-Identifier: BSD-3-Clause

package vm

import (
"io/fs"
"os"
)

// chmodBits are the fs.FileMode bits os.Chmod sets: the nine permission bits
// and the three special ones, which Go keeps far above them (fs.ModeSetuid is
// not 0o4000).
const chmodBits = fs.ModePerm | fs.ModeSetuid | fs.ModeSetgid | fs.ModeSticky

// chmodMode turns a POSIX mode, as Ruby passes it to File.chmod (0o4755,
// 0o1777…), into the fs.FileMode os.Chmod reads (#972). Passing the integer
// straight through, as `os.FileMode(m & 0o7777)` did, keeps the nine
// permission bits and silently drops setuid, setgid and sticky: Go reads
// 0o1000 as nothing at all.
func chmodMode(m int64) fs.FileMode {
fm := fs.FileMode(m & 0o777)
if m&0o4000 != 0 {
fm |= fs.ModeSetuid
}
if m&0o2000 != 0 {
fm |= fs.ModeSetgid
}
if m&0o1000 != 0 {
fm |= fs.ModeSticky
}
return fm
}

// withOwnerWrite runs write with the owner-write bit of path put back for its
// length, for the reason ioFlush gives: a stream that already holds the
// descriptor's grant must not lose it because this model reopens by path. It
// returns refused when the mode cannot be read, and otherwise what write
// returns; when the mode cannot be changed (a file this process does not own),
// write is refused for permission again, so no separate branch is needed.
//
// Afterwards only the owner-write bit goes back to what it was (#972). The
// rest is read from the file as the write left it: MRI 4.0.7 measured on
// darwin keeps sticky through a write and loses setuid and setgid, which is
// the kernel clearing them on a write, and restoring the mode read BEFORE the
// write would put a setuid bit back on content that was just changed.
func withOwnerWrite(path string, refused error, write func() error) error {
st, err := os.Stat(path)
if err != nil {
return refused
}
ownerWrite := st.Mode() & 0o200
_ = os.Chmod(path, st.Mode()&chmodBits|0o200)
werr := write()
if after, err := os.Stat(path); err == nil {
_ = os.Chmod(path, after.Mode()&chmodBits&^0o200|ownerWrite)
}
return werr
}
4 changes: 2 additions & 2 deletions internal/vm/fileutils.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ func (vm *VM) registerFileUtils() {
// every platform (chmod is largely a no-op on Windows, so the seam is the only
// way to cover the failure path there).
sdef("chmod", func(_ *VM, _ object.Value, args []object.Value, _ *Proc) object.Value {
mode := os.FileMode(intArg(args[0]) & 0o7777)
mode := chmodMode(int64(intArg(args[0])))
for _, p := range pathsOf(args[1]) {
if err := fileChmod(p, mode); err != nil {
raise("Errno::ENOENT", "No such file or directory @ fileutils_chmod - %s", p)
Expand Down Expand Up @@ -197,7 +197,7 @@ func (vm *VM) registerFileUtils() {
// chmod_R recurses chmod into directories, as Puppet's report store driver does
// (FileUtils.chmod_R(0o750, dir)).
sdef("chmod_R", func(_ *VM, _ object.Value, args []object.Value, _ *Proc) object.Value {
mode := os.FileMode(intArg(args[0]) & 0o7777)
mode := chmodMode(int64(intArg(args[0])))
for _, p := range pathsOf(args[1]) {
for _, q := range fuWalk(p) {
if err := fileChmod(q, mode); err != nil {
Expand Down
19 changes: 10 additions & 9 deletions internal/vm/io.go
Original file line number Diff line number Diff line change
Expand Up @@ -638,7 +638,7 @@ func (vm *VM) registerIO() {
cFile.define("chmod", func(_ *VM, self object.Value, args []object.Value, _ *Proc) object.Value {
o := self.(*IOObj)
ioFlush(o)
if err := fileChmod(o.path, os.FileMode(intArg(args[0])&0o7777)); err != nil {
if err := fileChmod(o.path, chmodMode(int64(intArg(args[0])))); err != nil {
raise("Errno::ENOENT", "No such file or directory @ apply2files - %s", o.path)
}
return object.IntValue(0)
Expand Down Expand Up @@ -1290,13 +1290,7 @@ func ioFlush(o *IOObj) {
return
}
if errors.Is(err, fs.ErrPermission) {
if st, sterr := os.Stat(o.path); sterr == nil {
mode := st.Mode().Perm()
if chmodErr := os.Chmod(o.path, mode|0o200); chmodErr == nil {
err = os.WriteFile(o.path, o.buf, 0o644)
_ = os.Chmod(o.path, mode)
}
}
err = withOwnerWrite(o.path, err, func() error { return os.WriteFile(o.path, o.buf, 0o644) })
}
if err != nil {
raiseOpenErrno(err, o.path)
Expand Down Expand Up @@ -3110,10 +3104,17 @@ func ioClosedRealIO(o *IOObj) {
// does: an Integer or #to_int value narrowed to the off_t (a long long) the
// position is stored in, raising RangeError for a Bignum too large to fit.
func (vm *VM) ioOfftArg(v object.Value) int {
return clampInt(vm.ioOfftArg64(v))
}

// ioOfftArg64 is ioOfftArg without the narrowing to int: an off_t is 64 bits on
// every platform MRI builds for, and a file offset passed to the OS must not be
// clamped to 2**31-1 on a 32-bit target (#973).
func (vm *VM) ioOfftArg64(v object.Value) int64 {
if _, ok := v.(*object.Bignum); ok {
raise("RangeError", "bignum too big to convert into 'long long'")
}
return clampInt(vm.toIntCoerce(v))
return int64(vm.toIntCoerce(v))
}

// ioCIntArg coerces an argument the way MRI's NUM2INT does for a value stored in
Expand Down
47 changes: 31 additions & 16 deletions internal/vm/io_descriptors.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ package vm
import (
"errors"
"io"
"io/fs"
"math/big"
"os"
"strings"
Expand Down Expand Up @@ -308,7 +309,7 @@ func defIOSeekable(cls *RClass) {
// all before the maxlen==0 early return, so a non-String buffer still
// raises and a zero-length read leaves the buffer's bytes untouched.
n := clampInt(vm.toIntCoerce(args[0]))
off := clampInt(vm.toIntCoerce(args[1]))
off := vm.ioOfftArg64(args[1]) // NUM2OFFT: a Bignum is RangeError (#973)
var buf *object.String
if len(args) > 2 {
buf = vm.ioBufferArg(args[2])
Expand Down Expand Up @@ -344,11 +345,13 @@ func defIOSeekable(cls *RClass) {
// Unreadable by path (removed between the two calls, permission):
// the buffer below is what this stream last saw.
}
if off >= len(o.buf) {
if off >= int64(len(o.buf)) {
raise("EOFError", "end of file reached")
}
data := o.buf[off:min(off+n, len(o.buf))]
return ioReadResult(data, buf)
// off+n is never formed: with n near MaxInt it overflowed into a slice
// bounds panic (#973). What is left past off bounds the read instead.
rest := o.buf[off:]
return ioReadResult(rest[:min(n, len(rest))], buf)
})
cls.define("pwrite", func(vm *VM, self object.Value, args []object.Value, _ *Proc) object.Value {
o := self.(*IOObj)
Expand All @@ -357,27 +360,39 @@ func defIOSeekable(cls *RClass) {
// stream is checked, so a missing #to_s surfaces as NoMethodError and a
// non-Integer offset as the "into Integer" TypeError.
data := []byte(vm.objAsString(args[0]))
off := clampInt(vm.toIntCoerce(args[1]))
off := vm.ioOfftArg64(args[1]) // NUM2OFFT: a Bignum is RangeError (#973)
ioCheckOpen(o)
if off < 0 {
raise("Errno::EINVAL", "Invalid argument - pwrite")
}
if end := off + len(data); end > len(o.buf) {
o.buf = append(o.buf, make([]byte, end-len(o.buf))...)
}
copy(o.buf[off:], data) // so this stream's own later reads see them
o.writable = true // a written File flushes its buffer back on flush/close
o.writable = true // a written File flushes its buffer back on flush/close
vm.trackOpenFile(o)
// #949: MRI's rb_io_pwrite is pwrite(2), so the bytes are on disk the
// moment it returns, and ONLY those bytes: the stream's other pending
// writes stay pending. pwriteFile does exactly that, so on success
// nothing new is waiting and wbufDirty is left as it was.
if p, ok := ioPositionalPath(o); !ok || pwriteFile(p, data, off) != nil {
// Door 4 of 5. No file reachable by path: the bytes are in the buffer
// only, losable like a buffered #write until a flush. Marking them
// dirty and tracking the stream is what keeps them (#643).
o.wbufDirty = true
if p, ok := ioPositionalPath(o); ok {
err := pwriteFile(p, data, off)
if err == nil {
o.mirrorPwrite(data, off) // best effort: the file has the bytes
return object.IntValue(int64(len(data)))
}
// #973: the file was reached and refused the write — EFBIG past
// the largest offset, ENOSPC, EIO. MRI raises that errno; it used to
// be swallowed here and the bytes put in the buffer instead. A
// refusal for permission is the one kept as before: the owner-write
// retry failed, so the buffer below is the stream's grant.
if !errors.Is(err, fs.ErrPermission) {
raisePwriteErr(err, o.path)
}
}
vm.trackOpenFile(o)
// Door 4 of 5. No file reachable by path: the bytes are in the buffer
// only, losable like a buffered #write until a flush. Marking them
// dirty and tracking the stream is what keeps them (#643).
if !o.mirrorPwrite(data, off) {
raise("Errno::EFBIG", "File too large @ rb_io_pwrite - %s", o.path)
}
o.wbufDirty = true
return object.IntValue(int64(len(data)))
})
cls.define("sysseek", func(vm *VM, self object.Value, args []object.Value, _ *Proc) object.Value {
Expand Down
76 changes: 54 additions & 22 deletions internal/vm/io_positional.go
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ func ioPositionalPath(o *IOObj) (string, bool) {
// pwriteFile writes data at off in the file at path and touches nothing else:
// no other byte of the file, and nothing of the stream that opened it. A write
// past the end leaves a hole, as pwrite(2) does.
func pwriteFile(path string, data []byte, off int) error {
func pwriteFile(path string, data []byte, off int64) error {
err := writeAtPath(path, data, off)
if errors.Is(err, fs.ErrPermission) {
err = writeAtWithOwnerWrite(path, data, off, err)
Expand All @@ -58,52 +58,84 @@ func pwriteFile(path string, data []byte, off int) error {
}

// writeAtWithOwnerWrite retries a write refused for permission with the
// owner-write bit put back for the length of the write, for the reason ioFlush
// gives: the stream already holds the descriptor's grant
// (`File.new(path, "w", 0444)` writes through), and reopening by path must not
// take it away. It never widens the file's mode beyond the write. When the
// mode cannot be read, the original refusal stands; when it cannot be changed
// (a file this process does not own), the retry is refused for permission
// again and that refusal is returned, so no separate branch is needed for it.
func writeAtWithOwnerWrite(path string, data []byte, off int, refused error) error {
st, err := os.Stat(path)
if err != nil {
return refused
}
mode := st.Mode().Perm()
_ = os.Chmod(path, mode|0o200)
err = writeAtPath(path, data, off)
_ = os.Chmod(path, mode)
return err
// owner-write bit put back for the length of the write (withOwnerWrite): the
// stream already holds the descriptor's grant (`File.new(path, "w", 0444)`
// writes through), and reopening by path must not take it away.
func writeAtWithOwnerWrite(path string, data []byte, off int64, refused error) error {
return withOwnerWrite(path, refused, func() error { return writeAtPath(path, data, off) })
}

func writeAtPath(path string, data []byte, off int) error {
func writeAtPath(path string, data []byte, off int64) error {
f, err := os.OpenFile(path, os.O_WRONLY, 0)
if err != nil {
return err
}
_, werr := f.WriteAt(data, int64(off))
_, werr := f.WriteAt(data, off)
cerr := f.Close()
if werr != nil {
return werr
}
return cerr
}

// ioHoleMax bounds the hole one #pwrite may open past the end of a stream's
// buffer (#973). The buffer is a byte slice, so a hole in it is memory:
// `f.pwrite("AB", 2**40)` asked for a terabyte, and 2**62 reached the
// allocator. pwrite(2) leaves a sparse hole on disk instead, and this model
// cannot. When the file was reached, the bytes are on disk and only this
// stream's in-memory copy stops short of them, the same staleness another
// writer's bytes already have. When it was not, the buffer is the only copy,
// and #pwrite raises EFBIG — the errno pwrite(2) gives for a write past the
// largest size the file may have, which this bound is for a buffer.
const ioHoleMax = 1 << 26

// mirrorPwrite copies data into the stream's buffer at off, so this stream's
// own later reads see it, growing the buffer to hold it. It reports false and
// changes nothing when that would open a hole larger than ioHoleMax.
//
// Only the hole is bounded: the bytes of data are already in memory, and the
// buffer's length is what earlier writes made it. On a 32-bit target, a
// buffer and data that together come within ioHoleMax of 2**31 could still
// overflow end; both would then have to be resident in a 4 GiB address space.
func (o *IOObj) mirrorPwrite(data []byte, off int64) bool {
if off-int64(len(o.buf)) > ioHoleMax {
return false
}
if end := int(off) + len(data); end > len(o.buf) {
o.buf = append(o.buf, make([]byte, end-len(o.buf))...)
}
copy(o.buf[off:], data)
return true
}

// raisePwriteErr reports a pwrite(2) that reached the file and was refused,
// the way rb_io_pwrite's rb_sys_fail_path does: the real errno, labelled
// with the C function. MRI 4.0.7 on darwin: `f.pwrite("AB", 2**63-1)` is
// `Errno::EFBIG: File too large @ rb_io_pwrite - <path>`.
func raisePwriteErr(err error, path string) {
if eno, ok := errnoOfError(err); ok {
if name, ok := errnoClasses[eno]; ok {
raise("Errno::"+name, "%s @ rb_io_pwrite - %s", errnoStrerror(eno), path)
}
raise("SystemCallError", "%s @ rb_io_pwrite - %s", errnoStrerror(eno), path)
}
raise("Errno::EIO", "Input/output error @ rb_io_pwrite - %s", path)
}

// preadFile reads up to n bytes at off from the file at path. It returns io.EOF
// when off is at or past the end, which IO#pread raises as EOFError.
//
// What it allocates is bounded by what the file holds past off, not by n: n
// comes from Ruby, and `f.pread(2**40, 0)` must not be an allocation of 2**40
// bytes. io.ReadAll grows with what is actually read, and the section reader
// caps that at n (it clamps off+n itself when the sum would overflow).
func preadFile(path string, n, off int) ([]byte, error) {
func preadFile(path string, n int, off int64) ([]byte, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
data, err := io.ReadAll(io.NewSectionReader(f, int64(off), int64(n)))
data, err := io.ReadAll(io.NewSectionReader(f, off, int64(n)))
if err != nil {
return nil, err
}
Expand Down
38 changes: 38 additions & 0 deletions internal/vm/io_positional_errno_internal_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// Copyright (c) the go-embedded-ruby/ruby authors
//
// SPDX-License-Identifier: BSD-3-Clause

//go:build unix

package vm

import (
"errors"
"io/fs"
"strings"
"syscall"
"testing"
)

// TestRaisePwriteErrKeepsTheErrno: a pwrite(2) the file refused is raised with
// its own errno (#973), an errno with no class as SystemCallError, and an error
// that carries no errno at all as EIO, all labelled rb_io_pwrite. ENOSPC and
// EIO used to be swallowed: #pwrite put the bytes in the buffer and returned
// their count as if they were on disk.
func TestRaisePwriteErrKeepsTheErrno(t *testing.T) {
cases := []struct {
name string
err error
want string
}{
{"a known errno", &fs.PathError{Op: "write", Path: "p", Err: syscall.ENOSPC}, "Errno::ENOSPC"},
{"an errno with no class", syscall.Errno(0x7ffe), "SystemCallError"},
{"no errno at all", errors.New("short write"), "Errno::EIO"},
}
for _, c := range cases {
re := rubyErr(t, func() { raisePwriteErr(c.err, "p") })
if re.Class != c.want || !strings.Contains(re.Message, "@ rb_io_pwrite - p") {
t.Errorf("%s: got %s %q, want %s labelled rb_io_pwrite", c.name, re.Class, re.Message, c.want)
}
}
}
Loading
Loading