Snmalloc Versions Save

Minor release primarily improving support on CHERI and Open Enclave

• CHERI (#587, #557, #578)
• Open Enclave provide new low memory mode that disables some thread caching (#612, #617)
• Improve security checks (emptyness #569, separate features #587))
• Improve ranges support (#540, #556, #562, #619)
• Performance (#604, #605, #606, #607)
• NetBSD support (#583)
• ARM support (#564, #601
• Debug performance improvement (#585)
• Fix C++ overrides argument order that caused a assert failure (#597)
• Improve override for RTLD_DEEPBIND (#598)
• Code tidying (#548, #554, #558, #565, #568)
• C++ support (#574, #581, #591)
• CI improvements/build ( #555, #559, #579, #580, #582, #588, #589, #590, #592, #599, #608, #613, #614, #620)
• Documentation (#602, #610, #611)

Thanks to @devnexen, @saaramar, @brvtalcake, @jumaffre and @EAirPeter for your contributions to this release.

Minor release. Largest change is bringing online Morello CHERI support.

• CHERI support (#537, #542, #532, #543)
• Improve error messages for checks (#526, #521)
• Increased checks on client (#520, #550)
• Portability fixes for Haiku and older versions of glibc (#545, #546, #533)
• Expose macOS malloc_good_size (#538)
• General code tidying (#527, #535, #547, #551, #536, #534, #529)
• Documentation (#551, #522)

Thanks to the external contributors @panekj, @devnexen and @mfelsche.

This is a major revision of the snmalloc design. The redesign has been primarily focused on adding new security features. The redesign affects all aspects of snmalloc. A more comprehensive explanation of the new features can be found in the docs

• Uses Buddy Allocators for large allocations: In 0.5.3, we never reconsolidate large allocations, so if you ask for several 1MiB allocations, return them, and then ask for several 2MiB allocations it won't be able to use the space from the 1MiB allocations.  This is not a problem in 0.6 as we have a proper buddy allocator at that level so we recombine the big bits of memory properly
• Variable sized slabs and out of band meta-data - Previously, snmalloc used fixed sized super slabs with metadata at the head. By using variable sized slabs and out of band meta-data, the slabs don't have to be grouped into superslabs which reduces the minimum memory requirement. (https://github.com/microsoft/snmalloc/blob/main/docs/security/VariableSizedChunks.md)
• Improved per thread caching - https://github.com/microsoft/snmalloc/pull/511 - This enables very small per thread usage initially that can grow to the level required. This means that low foot print scenarios on OpenEnclave can be supported with exactly the same code paths as high footprint scenarios. This brings the per thread costs down to approximately 16k per thread of metadata + 16k per thread per sizeclass used. This will grow in powers of 2 (upto 2MiB) to cover the required amount.

All minor changes

• Fix performance regression for Debug builds (#274)
• Refactoring towards sandbox support (#221, #268, #269)
• Refactoring towards CHERI support (#266)
• Bug fixes for Power (#265), SPARC (#264), ARM 32bit (#263) Solaris (#262),

Bug fixes

• Fixes issue where unused memory is zeroed by the OS, then superslab data can become corrupt. Only affected the 16MiB chunk size configuration. The bug did not affect the Linux implementation. #259
• Typo that meant 16MiB configuration was ignored in some cases. #254

Platform Support

• Improve Haiku support #255
• Added DragonFly BSD support #252

Thanks especially to @devnexen for finding and fixing many issues in this release, and for the contributions to many previous releases in expanding snmallocs platform support.

The main change in this release is a new mechanism for tracking very coarse-grained memory usage statistics (#241). The interface is only exposed in malloc-extensions.h. We recommend static linking this API, as we may refine this in future releases.

Other changes:

• PALs now fully static (#245)

Bug fixes:

• Haiku Debug build fixed (#247)
• Windows Clang CTZ was not defined on correct size for size_t (#242)
• Fix to unit tests (#239)
• FlatPageMap calculated too large size if payload was not a byte size (#236)

This version significantly improves the peak working set on Windows, and has a lower RSS on Linux systems using transparent huge pages.

Detailed changes:

• Change default chunksize to 1MiB, and add explicit -16mib tagged version of libraries (#229).
• Adds new AddressSpaceManager that improves performance with transparent huge pages (#214, #227) also improves support for open enclave (#212)
• Improve Open Enclave support with new smaller chunk size of 256KiB (#212), and add versions of libraries and tests using this chunksize with the tag -oe.
• Preliminary support for Haiku (#218) and Solaris (#226)

Changes

• Preliminary Android support (#171)
• Fixes to 32bit compile for Linux (#173)
• Preparatory work for CHERI support (#188, #191, #193, #202)
• Performance of alloc_size (#196)
• Minor changes to build defaults (#177, #182, #187, #189, #194, #197, #206)
• Improve Open Enclave support (#205, #201, #195)
• Expose cfree (#179)
• Allow different OS_PAGE_SIZES (#185)
• Improve support for VS2019 (#207)
• Add error message for failed init (#190)

Bug fixes

• realloc of large allocation may calculate incorrectly return original allocation (#178, #209)
• Fixed sized delete of nullptr (#181)
• Fixed memory ordering in unit test (#184)

Small fixes to 0.4

• Build fixes for GCC9 and GCC 10 (#172)
• Bug fix - if first call of a thread is a calloc, then it might not be zeroed. (#172)

The following are the main changes since the previous version:

• Improved Decommit on Windows
  • Removed redundant system calls #120
  • Implement lazy decommit strategy to take OS call back, so an application not performing allocation can release memory to the OS #128
  • Bug Fix: Remove infinite loop on low-memory notification #123
• Improved performance of multiple fast and slow paths
  • Improved sized deallocation code paths #115
  • Remote deallocation code path separated into fast and slow path (increased perf considerable for producer/consumer workloads) #138
  • Improve allocation slow path and change to thread local bump allocator #143
• Improved alignment code
  • Support alignment of large allocations #124
  • Efficient calculation of alignment requests (just two additional integer instructions on fast path) #113
• Support more platforms and tool chains
  • Expose API surface for Rust #113
  • Improve GCC support #110 #134 #167
  • Add support for Clang on Windows #119
  • Provide OpenEnclave support #129 #131 #135 #140 #166 #167
  • Preliminary ARM support #142 , #161
• Bug fixes based on mimalloc-bench
  • Fixed poor performance in the message queue handling due to small batch size (#158 )
  • Fixed leak of allocators during TLS teardown (#161)
  • Fixed high page fault count from using MADV_DONTNEED for zeroing (#159)