The Rubinius Language Platform
Version 5.0 (2020-05-16)
Version 4.20 (2020-04-20)
Version 4.19 (2020-04-20)
Version 4.18 (2020-04-20)
Version 4.16 (2020-02-22)
🤷 (Brian Shirai)
Start updating README. (Brian Shirai)
Move more of Environment into Machine. (Brian Shirai)
Removed C++ tests. (Brian Shirai) Maintaining these tests for a decade, I can confidently state they don't add enough value to compensate for the maintenance cost. As more C++ code (eg primitives, object classes) are replaced by instructions, the value of the tests diminishes even more.
Removed configure step for machine tests. (Brian Shirai)
Moved locks to Machine. (Brian Shirai)
Removed remnants of C++ tests. (Brian Shirai)
Boot and halt on main thread. (Brian Shirai)
Update README.md (Brian Shirai)
Update README.md (Brian Shirai)
Update README.md (Brian Shirai)
Removed old concurrency utils. (Brian Shirai)
Update README.md (Brian Shirai)
Update README.md (Brian Shirai)
Some Channel and misc cleanup. (Brian Shirai)
More Thread, ThreadState cleanup. (Brian Shirai)
Finalize UnwindSite caches. (Brian Shirai)
Release last ThreadState. (Brian Shirai)
Don't discard MachineThread ThreadState on fork. (Brian Shirai)
Version 4.15 (2020-02-02)
This reverts commit be1e28c4ccc3bcbff70f08298deec981faaa69c4.
[x] 1. Add Machine to SharedState; [x] 1. Pass Machine through Environment to SharedState
[x] 1. Move SharedState items for env into Environment;
[x] 1. Move Configuration to Machine;
[x] 1. Move ThreadNexus to Machine;
Fixnum is deprecated.
[x] 1. Move MachineThreads to Machine;
[x] 1. Move Collector to Machine;
[x] 1. Move Memory to Machine;
[x] 1. Move SymbolTable into Memory;
[x] 1. Move Globals into Memory;
Make VM tests pass for now.
Add task to show CodeDB dir. (Brian Shirai)
Revert "Add task to show CodeDB dir." (Brian Shirai) This reverts commit 2b6949efac21ec7248c8574e694c709fb2f9a89e.
Locate the CodeDB for the VM tests. (Brian Shirai)
Removed SharedState, VMThreadState. (Brian Shirai)
Moved State to ThreadState. (Brian Shirai) This is a first step to eliminating VM and ManagedThread.
Removed ManagedThread. (Brian Shirai)
Removed VM. (Brian Shirai)
Back to the ol' Trusty. (Brian Shirai) There is some process or IO related issue on Bionic that is extremely puzzling.
Ok, fine, fail on Bionic then. (Brian Shirai) Revert "Back to the ol' Trusty."
This reverts commit a53fd061fcc9f4a72c3c6329ba84a1cc618bf57b.
More cleanup. (Brian Shirai) Removed MachineThreads list. Specific MachineThread's lifecycle functions are called directly. Handling around fork() moved to Machine. Some Console cleanup.
Let backtrace locations wrap. (Brian Shirai)
Clean a few spec tags. (Brian Shirai)
Version 4.14 (2020-01-26)
Added start of array-type instructions. (Brian Shirai)
Revert "Disable experimental compiler plugins." (Brian Shirai) This reverts commit 806f3be4ac96c42a377ac7804bc36f714a33e4aa.
More work on array-type instructions. (Brian Shirai)
Added --sanitize configure flag. (Brian Shirai) This doesn't work yet on macOS :sob:
clang: error: unsupported option '-fsanitize=leak' for target 'x86_64-apple-darwin19.2.0'
Fix warning on Ubuntu Bionic. (Brian Shirai) machine/ffi_util.cpp:53:10: warning: In the GNU C Library, "major" is defined by <sys/sysmacros.h>. For historical compatibility, it is currently defined by <sys/types.h> as well, but we plan to remove this soon. To use "major", include <sys/sysmacros.h> directly. If you did not intend to use a system-defined macro "major", you should undefine it after including <sys/types.h>. [-W#pragma-messages]
Switch to AddressSanitizer. (Brian Shirai)
Enhance --sanitize configure option. (Brian Shirai)
Exclude sysmacros.h on macOS. (Brian Shirai)
Clean up some Valgrind-exposed memory leaks. (Brian Shirai)
Use unsynchronized set during GC. (Brian Shirai) We only run one thread during GC, so we don't need synchronized (un)set.
Track code return value for GC. (Brian Shirai)
Rework extended header & memory handle mechanisms. (Brian Shirai)
Added ref count instructions. (Brian Shirai)
Added deprecation for Fixnum, Bignum. (Brian Shirai)
Clean up from memory header rework. (Brian Shirai)
Removed a bunch of Rake stuff. (Brian Shirai)
Start to clean up 'Machine'. (Brian Shirai)
Checkpoint for GC after saving object reference. (Brian Shirai) It turns out, surprisingly enough, it's much easier to see the object we are trying to trace if we put it where we can see it first.
Version 4.13 (2020-01-20)
Added more math functions. (Brian Shirai)
Replaced Bignum#<<, Bignum#>> primitives. (Brian Shirai)
Replaced Fixnum#size, Bignum#size primitives. (Brian Shirai)
Replaced {Fixnum, Bignum}#bit_length primitives. (Brian Shirai)
Switch Travis to use 18.04 bionic. (Brian Shirai)
Install gdbm on Linux for Travis 18.04 bionic. (Brian Shirai)
Install gdbm on Linux for Travis 18.04 bionic. (Brian Shirai)
Revert switching Travis to bionic. (Brian Shirai)
Replaced more Bignum, Fixnum primitives. (Brian Shirai)
Added more instructions. (Brian Shirai)
Replaced {Fixnum, Bignum}#{to_s, inspect} primitives. (Brian Shirai)
Replaced {Bignum, Fixnum}#to_f primitive. (Brian Shirai)
Replaced {Fixnum, Bignum}#divmod primitives. (Brian Shirai)
Enable experimental compiler plugins. (Brian Shirai)
Ignore stdlib C-exts that won't build. (Brian Shirai)
Try bionic on Travis again. (Brian Shirai)
Moved Fixnum.=== to Integer. (Brian Shirai)
Improved Process.waitpid. (Brian Shirai)
Removed unnecessary subprocess from Fiber#resume spec. (Brian Shirai)
Reset process start time in forked child. (Brian Shirai)
Back to trusty. (Brian Shirai)
Simplify Integer.===. (Brian Shirai)
Finished removing Fixnum, Bignum classes in core. (Brian Shirai) More work remains before the constants can be removed.
Reset start time after fork. (Brian Shirai)
Handle coded caches better. (Brian Shirai) This is still very messy and all this should be proper dependencies, but I'd rather build that in the new build system that doesn't use Ruby.
Added more floating point instructions. (Brian Shirai)
Replaced Float#{nan?, infinity?} primitives. (Brian Shirai)
Make a proper pointer to registers. (Brian Shirai) Previously, the registers were referenced relative to the beginning of the stack by adjusting all the register indexes when creating the MachineCode instance.
Disable experimental compiler plugins. (Brian Shirai)
Version 4.12 (2020-01-06)
Replaced more Float, Fixnum, Bignum primitives. (Brian Shirai)
Fixed spec assuming 32bit. (Brian Shirai)
Replaced more Integer bitwise primitives. (Brian Shirai)
Removed nonsense spec. (Brian Shirai) Not going to support this behavior. Rubinius returns true for all three cases:
$ ruby -v -e 'p 0x800000000000027 <= (0x800000000000027 + 0.0)' ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin19] false $ ruby -v -e 'p 1 <= (1 + 0.0)' ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin19] true $ ruby -v -e 'p (264) <= ((264) + 0.0)' ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin19] true
Replaced Bignum#~, Bignum#-@ primitives. (Brian Shirai)
Version 4.11 (2020-01-05)
Fixed building OSX packages. (Brian Shirai)
Try re-enabling Travis OSX. (Brian Shirai)
Re-disable Travis OSX. STDIN is not a TTY. (Brian Shirai) https://travis-ci.org/rubinius/rubinius/jobs/167911681#L5416
Removed obsolete compiler define RBX_PROFILER. (Brian Shirai)
Attempt to avoid breaking LLVM compiler flags. (Brian Shirai)
Updated gems. (Brian Shirai)
More instruction tests. (Brian Shirai)
More interpreter tests. (Brian Shirai)
Tests for check_frozen instruction. (Brian Shirai)
Moved builtin/ to class/. (Brian Shirai) The name 'builtin' is not very descriptive. Now that we are going to have managed objects that are both object-oriented and simple data, and both are 'built in', we need more descriptive names for the directories, so the new names are 'class' and 'data', which parallels the names used in the code.
Fixed path to website releases data. (Brian Shirai)
For Rubinius build on aarch64-linux-gnu (Raspberry Pi 3). (#3705) (medols)
Specs for rb_obj_method_arity. (Brian Shirai)
Added rb_obj_method_arity. Fixes #3708. (Brian Shirai)
Updated daedalus-core gem. (Brian Shirai)
Added C-API rb_memhash. (Brian Shirai)
Updated rubysl-openssl version. (Brian Shirai)
Updates to issues/PR templates. (Brian Shirai)
Try putting README.md under .github. (Brian Shirai)
Revert "Try putting README.md under .github." (Brian Shirai) This reverts commit 320f5045f09473125dad16091f54c15769ed2430.
Updated README. (Brian Shirai)
Don't use enum as last arg for varargs. /ht @heftig. (Brian Shirai)
Fixed Bundler, brew wars. (Brian Shirai)
Fixed setting ThreadGroup for signal handler Thread. (Brian Shirai) In Rubinius, trapped signals are handled in a dedicated Thread.
In Ruby, the Thread where a signal is processed isn't defined, but the assumption is that the signal handler runs in Thread.main (as observed in Ruby behavior).
Also undefined in Ruby is what ThreadGroup a Thread created in a signal handler should be added to. Going with the assumption that the signal handler proc is run in Thread.main, we assume the new Thread should be added to Thread.main.group.
We now set the signal handler Thread's group to the same group as Thread.main, so that threads created in a signal handler proc have a group and are added to a group.
Unfortunately, splitting up the source location makes copy-pasting the path to edit a source file very tedious.
Now, the source location is rendered as an unbroken sequence of characters but starting on a new line if the total backtrace line exceeds the terminal width. This preserves the readability of the receiver+method, while also preserving the ability to copy-paste the source location.
Unfortunately, color is problematic due to the perception of colors and the interaction of color with terminal colors, which something which Rubinius has no control over.
The goal of readability in backtraces is an important one. There are two primary pieces of information in each backtrace line: 1. receiver+method; and 2. source code location, file+line number.
The current rendering approach is to bold the receiver+method and separate the source line on a new line when the total backtrace line exceeds the terminal width. This provides contrast that improves scanning the backtrace on primary information, the receiver+method, and easily selecting secondary information, the source location, when necessary.
As such, this convention carries over without much question to rendering backtraces. However, the context of consuming a backtrace requires different conventions, but ones that are not foreign.
So, for rendering backtraces, the conventions are retained but the ordering is changed slightly to improve processing the information. There are now four basic components to a backtrace line (for an object-oriented method): 1. the method name; 2. the instance/module indicator; 3. the fully-scoped receiver name; and 4. the source code location.
The first thing that needs to be located and digested is the method name, so the method names are right aligned allowing both easily reading the method name and quick indexing for the second important piece of information, the receiver name. The source location trails and may be on a new line if the total backtrace line rendering exceeds the terminal width.
This rendering better orders information in the backtrace based on the hierarchy of important information (method name > instance/module indicator > receiver name > source location). It also balances the tensions between making an individual line of a backtrace readable and making the whole thing scannable and readable. When rendering the full method name as A::B::C#name, the variability of the A::B::C component pushed many names too far to the right, leaving a very jagged left edge that is more difficult to scan, as well as burying the method name (the primary element) embedded in the middle of a bunch of character noise.
Fixed rendering method name. (Brian Shirai)
Catch C++ exceptions sooner. (Brian Shirai) We attempt to catch C++ exceptions as close to where they occur as possible. This means at the last executed managed code (eg method) so that as much context as possible can be given.
Added extra C++ exception catch to new interpreter. (Brian Shirai)
Use llvm-config --libs instead of --libfiles. (Brian Shirai)
fix case where IO#gets blocks forever on read to char boundary (Chuck Remes) This was a tough one. WEBrick would hang when reading an HTTP header. The cause was that IO::EachReader#read_to_separator_with_limit blocked forever trying to "peek ahead" to ensure we are reading to the end of a proper character boundary. In this case the socket had no more data to recv but it also does not set EOF, therefore the read blocked indefinitely. The simple fix was to move to a non-blocking read and skip the read entirely if the string already had a valid encoding. However, this broke several specs for reading multi-byte chars. To resolve that problem required making sure that when read_nonblock returned an empty buffer that this buffer was set to ASCII_8BIT instead of whatever the default external coding was set to (usually UTF-8). The fixes are all relatively simple but it took many hours to chase this down and figure out the minimal approach to fix it.
DRY up allocation of 8-bit string buffers (Chuck Remes)
Update issue_template.md (Brian Shirai) Fixed Markdown formatting of checkbox list.
Update pull_request_template.md (Brian Shirai) Fixed Markdown formatting of checkbox list.
Enable build from network for RPi3 (macbookpr02010)
add test for instructions::send_stack_with_block (Chuck Remes)
add test for instructions::rotate (Chuck Remes)
add work in progress test for instructions::send_vcall (Chuck Remes)
better tests for instructions::send_stack (Chuck Remes) The test_send_stack_execute_returns_null tests the situation where no value is returned from the method, e.g. nullptr.
Second test test_send_stack_method_no_method_error tests code path where the method is not found and raises a NoMethodError. This had the side effect of leaving arguments on the stack. This is a problem that requires a bug fix to the instruction.
Bug 2 was more of an issue of convenience. The FFI.config facility builds FFI::Struct class layouts from data in runtime/platform.conf. Some structs have char arrays. The original code allocated the proper space but it did not associate an array with a :char_array field. Therefore trying to access the field to assign to it directly did not work. e.g. struct = RubySL::Socket::Foreign::SockaddrUn.new struct[:sun_path] = "/some/path" # failed! This fix allows the direct assignment of string fields built using the FFI.config convenience method.
Presently, if a thread is already suspended, it won't log its stack. When the GC triggering and checkpointing is reworked, the threads that were suspended when GC occurred will check a flag that GC has run when resuming to log their stacks.
Emit the kind of thread for --gc-stack-check. (Brian Shirai)
Add MarkStack abstraction for GC. (Brian Shirai) Use the --gc-stack-check configure option to enable a version of MarkStack that stores the parent object (when available) for every object added to the mark stack.
Add deprecations for version 3.x to 4.0 transition. (Brian Shirai)
Preserve C-API handle references for several GC cycles. (Brian Shirai) In the MRI C-API, there are no explicit 'retain', 'release' operations. When a managed object is created in Rubinius, a static (ie non-moving) pointer is returned. The pointer is a MemoryHandle object that references the managed memory object.
When a GC cycle occurs, if nothing has marked the managed memory object as reachable, the MemoryHeader object would be released and the managed memory object's space reclaimed.
The problem with this is that any time the C-API calls back into the VM, a GC cycle may occur. The following code illustrates this:
VALUE c_func(VALUE a, VALUE b) { VALUE new_obj = c_api_create_obj(a, b);
/* Many C-API functions are implemented as wrappers for Ruby code,
* so this call may result in a GC operation and 'new_obj' will not
* be referenced anywhere, causing the GC to assume it is
* unreachable, and therefore garbage.
*/
other_c_api_func(new_obj);
return new_obj;
}
To address this, we track how many GC cycles have occurred while the object is not marked. To make this slighty more robust (ie not retain true garbage for too long), we could use a flag that the MemoryHandle has just been created and only retain it for several cycles if it has just been created.
Add ability to search mark stack for pointers. (Brian Shirai)
Start moving to a dedicated collector thread. (Brian Shirai)
Rename CollectorThread to Collector. (Brian Shirai) Also move GCInhibit from Memory to Collector::Inhibit.
More Collector cleanup. (Brian Shirai)
Invoke GC using a new mechanism. (Brian Shirai) Nothing about GC has changed, but the sequence to invoke it uses the new mechanism being built.
Actually run the collector on the collector worker thread. (Brian Shirai)
Move more GC to Collector. (Brian Shirai)
GC cleanup round 1. (Brian Shirai)
More moving to the Collector. (Brian Shirai)
Introduce Heap, MainHeap, MemoryTracer. (Brian Shirai)
Start replacing saw_object with trace_object. (Brian Shirai)
Detangle some GC function names. (Brian Shirai)
aarch64 build (macbookpr02010)
Remove legacy GC code. (Brian Shirai)
Fix resetting Immix allocator after collection. (Brian Shirai)
Clean up a bit more old GC related code. (Brian Shirai)
Removed Address. (Brian Shirai) The Address construct was interesting, but it obscured the memory address alignment and added another (incompatible with MemoryHeader, DataHeader, ObjectHeader) concept that complicated interacting with Immix when there is no reason for Immix to be special.
Remove collect flag and directly request collection. (Brian Shirai)
Move pinned capability to only third region. (Brian Shirai)
Split MemoryHandles from References. (Brian Shirai)
Pass address of object into collector. (Brian Shirai)
reverting to rubinius commit 363b0e7874b07bd153d9e1b3a19cc8e4ae37aea7 prior to add arm64v8changes (macbookpr02010)
add arm64v8 changes (macbookpr02010)
Some write barrier cleanup. (Brian Shirai)
Add --log-concurrent-update and --raise-concurrent-update. (Brian Shirai) The first compile-time option will log (level warn) when an object created by one thread is updated by a different thread. The second compile-time option will raise an exception. The options can be used together or separately.
Added ConcurrentUpdateError. (Brian Shirai) Also standardized exception handling into MachineException::guard.
Added '-Xmachine.concurrent.update.{log, raise}' options. (Brian Shirai)
Pass Object** to MemoryVisitor functions. (Brian Shirai) This will enable Object#become and other useful capabilities.
Hold logger lock during fork(). (Brian Shirai) A thread that does not stop for fork() because it's not mutating memory may still use the logger, which causes the logger lock to be in an indeterminate state across a fork() call, possibly resulting in the lock being locked in the child. By explicitly holding the lock, we can safely unlock it in the child and the lock is in a determinate state.
Added Object#become. (Brian Shirai) This is not complete yet, but works for variables.
Fixed write_barrier. (Brian Shirai)
Make MemoryVisitor recursive. (Brian Shirai)
Make Collector recursive. (Brian Shirai)
Rework location of checkpoints. (Brian Shirai)
Remove call frame return value. (Brian Shirai) This was added to save a transient root while the collector ran, but it's not needed with the new method of checkpointing in the ret instruction and loop back branches.
Added Thread-local Heap Container & Allocator. (Brian Shirai)
Allocate using THCA. (Brian Shirai)
Rework checkpoint. (Brian Shirai)
Limit size of object allocated by THCA. (Brian Shirai)
Clean up timestamp files generated by Makefile (ljulliar)
Simplify check_stop. (Brian Shirai) With main collection always running on a dedicated thread, there is no need for each thread to check whether they may have to run the collector when stop is set.
Add function table to LexicalScope. (Brian Shirai)
Define functions. (Brian Shirai)
Rework checking stack overrun. (Brian Shirai)
Added r_load_{0,1,nil,false,true} and goto_{past,future. (Brian Shirai) The goto_past instruction is a backward branch, which allows explicitly enabling or disabling looping constructs.
Added call instruction variants. (Brian Shirai) The call_send instruction looks for a function and falls back to dispatching a method. This will replace the send_stack instruction that is used for implicit self sends in code that could have functions.
The call instruction only looks for a function.
The call_0 instruction is for 0-arity functions.
The send_vcall instruction will be modified to look for functions.
Fixed a couple deadlocks. (Brian Shirai)
Add specs and verification for new instructions. (Brian Shirai)
Add more support for tagged nil. (Brian Shirai) And fixed other stuff.
Fixed more locking issues. (Brian Shirai)
Added push_tagged_nil instruction. (Brian Shirai)
Fix nil checks. (Brian Shirai)
More fixed nil checks. (Brian Shirai)
Tag nil on return if untagged. (Brian Shirai) When nil values are created outside the instruction set (managed code), for example, in the C-API, the untagged nil value will be tagged with the code ID when the value is returned to managed code.
Clean up some MemoryHeader ToDos. (Brian Shirai)
Better exception message for NoMethodError on nil. (Brian Shirai) The current type coercion protocol, which is modeled on MRI C functions, calls and rescues many NoMethodError on nil exceptions, so this is optionally enabled until those mechanisms can be improved.
Added exception logging. (Brian Shirai) Initially, this logs exceptions raised from managed code.
Give Bignum::from a clue. (macbookpr02010)
some synch (macbookpr02010)
some sync (macbookpr02010)
Replaced primitive for Fixnum#/. (Brian Shirai)
Don't remove code caches for now. (Brian Shirai) The core library, standard library, and codetools are all external resources. During package building, a 'rake clean' is executed, which would remove these resources.
Define Fixnum#!, Fixnum#!=. (Brian Shirai)
Simplify r_load_int instruction. (Brian Shirai) Correctness checks like this can be moved into the bytecode verifier to ensure that, for example, b_if_int precedes this instruction with no mutation of the register between the check and r_load_int.
Added more integer instructions. (Brian Shirai)
Replace primitive for Fixnum#>>, Fixnum#<<. (Brian Shirai)
Replaced primitive for Fixnum.===. (Brian Shirai)
Replaced more Fixnum primitives. (Brian Shirai)
Start unifying Fixnum, Bignum under Integer. (Brian Shirai)
Added n_promote. (Brian Shirai) Given two values, this instruction will return the 'lowest common type'. The types the instruction understands are Fixnum, Bignum, and Float. If both operands are the same type, no conversion is made. Fixnum promotes to Bignum, and both Fixnum and Bignum promote to Float.
Added missing n_promote instruction files. (Brian Shirai)
Added insn stubs for extened integer and float. (Brian Shirai)
Make --without-rpath the default. (Brian Shirai)
Replace native_int with intptr_t. (Brian Shirai)
Updated contributing guide (Brian Shirai)
Start replacing Bignum, Float primitives. (Brian Shirai)
Replaced more Fixnum, Bignum primitives. (Brian Shirai)