The pure asynchronous runtime for Scala
This is the forty-ninth release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release and fully source-compatible with every 3.5.x release.
[!WARNING] The 3.5.x series contains some changes that may be semantically breaking with respect to earlier 3.x releases. If you're using fs2, http4s, or other libraries from the ecosystem, make sure you've upgraded to versions of these libraries that are compatible with this release (for fs2, that's 3.7.0+, for http4s it's 0.23.19+)!
Additionally, if you're using methods like
fromFuture
, make sure you're aware of the major changes toasync
, described in these release notes.
The most significant change in this release involves the complete greenfield rewrite of Dispatcher
. While Dispatcher
is, in practice, one of the most-used components of Cats Effect apart from IO
itself, it is also one of the oldest and most "accreted" parts of the codebase. The original variant of Dispatcher
had far fewer features and was thrown together relatively quickly during the 3.0 milestone process, and every change and enhancement to its functionality has been layered on top of that foundation.
In this release, we started with an empty editor buffer and rebuilt the whole thing, taking the opportunity to correct a number of corner cases in semantics while significantly beefing up the test suite. We're very confident that the results are much more stable and reliable than the previous iteration, and we're excited to get it into everyone's hands!
IORuntimeConfig
: reject invalid auto-yield and cancelation check thresholds by @durban in https://github.com/typelevel/cats-effect/pull/3987
Dispatcher
by @djspiewak in https://github.com/typelevel/cats-effect/pull/3923
WeakList
and use in FiberMonitor
by @armanbilge in https://github.com/typelevel/cats-effect/pull/3964
Async
scaladoc by @durban in https://github.com/typelevel/cats-effect/pull/3995
WeakList
headers by @djspiewak in https://github.com/typelevel/cats-effect/pull/4027
Full Changelog: https://github.com/typelevel/cats-effect/compare/v3.5.3...v3.5.4
This is the forty-eighth release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release and fully source-compatible with every 3.5.x release.
[!WARNING] The 3.5.x series contains some changes that may be semantically breaking with respect to earlier 3.x releases. If you're using fs2, http4s, or other libraries from the ecosystem, make sure you've upgraded to versions of these libraries that are compatible with this release (for fs2, that's 3.7.0+, for http4s it's 0.23.19+)!
Additionally, if you're using methods like
fromFuture
, make sure you're aware of the major changes toasync
, described in these release notes.
Dispatcher#unsafeRunAndForget
by @kamilkloch in https://github.com/typelevel/cats-effect/pull/3878
IO.blocking
on WSTP without BlockContext
indirection by @armanbilge in https://github.com/typelevel/cats-effect/pull/3903
blocking
by @armanbilge in https://github.com/typelevel/cats-effect/pull/3938
notifyParked
when worker transitions to blocking by @armanbilge in https://github.com/typelevel/cats-effect/pull/3937
Fiber#join
via asyncCheckAttempt
by @armanbilge, @samspills in https://github.com/typelevel/cats-effect/pull/3942
fromFuture
, fromFutureCancelable
by @TimWSpence in https://github.com/typelevel/cats-effect/pull/3892
Dispatcher.parallel
by @durban in https://github.com/typelevel/cats-effect/pull/3900
IO.raiseError
by @scott-thomson239 in https://github.com/typelevel/cats-effect/pull/3811
CallbackStack#pack
by @armanbilge in https://github.com/typelevel/cats-effect/pull/3936
Hotswap#get
from acquiring a lock when there is no Resource
present by @josgarmar28 in https://github.com/typelevel/cats-effect/pull/3922
CallbackStack
based on new "pack locking" strategy by @armanbilge, @samspills, @mernst-github in https://github.com/typelevel/cats-effect/pull/3943
Hotswap
example by @armanbilge in https://github.com/typelevel/cats-effect/pull/3885
CallbackStackSpec
by @armanbilge, @samspills in https://github.com/typelevel/cats-effect/pull/3952
Full Changelog: https://github.com/typelevel/cats-effect/compare/v3.5.2...v3.5.3
This is the forty-seventh release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release and fully source-compatible with every 3.5.x release.
The 3.5.x series contains some changes that may be semantically breaking with respect to earlier 3.x releases. If you're using fs2, http4s, or other libraries from the ecosystem, make sure you've upgraded to versions of these libraries that are compatible with this release (for fs2, that's 3.7.0+, for http4s it's 0.23.19+)!
Additionally, if you're using methods like fromFuture
, make sure you're aware of the major changes to async
, described in these release notes.
liveTraces()
on JS by @armanbilge in https://github.com/typelevel/cats-effect/pull/3724
sleepInternal
by @armanbilge in https://github.com/typelevel/cats-effect/pull/3775
Thunk.asFunction0
utility by @armanbilge in https://github.com/typelevel/cats-effect/pull/3788
Dispatcher#{unsafeRunAsync,unsafeRunAndForget}
by @kamilkloch in https://github.com/typelevel/cats-effect/pull/3822
Hotswap#get
being blocked by Hotswap#swap
by @forkedcancel in https://github.com/typelevel/cats-effect/pull/3800
Hotswap
by @armanbilge in https://github.com/typelevel/cats-effect/pull/3841
**
pattern by @djspiewak in https://github.com/typelevel/cats-effect/pull/3843
Deferred#complete
type by @ag91 in https://github.com/typelevel/cats-effect/pull/3740
reportFailure
scaladoc by @kamilkloch in https://github.com/typelevel/cats-effect/pull/3769
MapRef
docs by @BalmungSan in https://github.com/typelevel/cats-effect/pull/3829
Full Changelog: https://github.com/typelevel/cats-effect/compare/v3.5.1...v3.5.2
This is the forty-sixth release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release and fully source-compatible with every 3.5.x release.
The 3.5.x series contains some changes that may be semantically breaking with respect to earlier 3.x releases. If you're using fs2, http4s, or other libraries from the ecosystem, make sure you've upgraded to versions of these libraries that are compatible with this release (for fs2, that's 3.7.0+, for http4s it's 0.23.19+)!
Additionally, if you're using methods like fromFuture
, make sure you're aware of the major changes to async
, described in these release notes.
object
for executeBatchTaskRunnable
by @armanbilge in https://github.com/typelevel/cats-effect/pull/3642
queueMicrotask
polyfill by @armanbilge in https://github.com/typelevel/cats-effect/pull/3693
now
in WorkerThread
in states 4-63 by @djspiewak in https://github.com/typelevel/cats-effect/pull/3690
alive
in favour of nullifying queues by @samspills in https://github.com/typelevel/cats-effect/pull/3701
turbo := false
in Scastie by @armanbilge in https://github.com/typelevel/cats-effect/pull/3631
Scala3
in Supervisor
by @danicheg in https://github.com/typelevel/cats-effect/pull/3650
fromCompletableFuture
scaladoc by @danicheg in https://github.com/typelevel/cats-effect/pull/3644
Full Changelog: https://github.com/typelevel/cats-effect/compare/v3.5.0...v3.5.1
This is the forty-fifth release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release.
This release contains some changes that may be semantically breaking. If you're using fs2, http4s, or other libraries from the ecosystem, make sure you've upgraded to versions of these libraries that are compatible with this release (for fs2, that's 3.7.0, for http4s it's 0.23.19)!
Additionally, if you're using methods like fromFuture
, make sure you're aware of the major changes to async
, described in these release notes.
This is an incredibly exciting release! 3.5.0 represents the very first steps towards a fully integrated runtime, with support for timers (IO.sleep
) built directly into the Cats Effect fiber runtime. This considerably increases performance for existing Cats Effect applications, but particularly those which rely more heavily on native IO
concurrency (e.g. Http4s Ember will see more benefits than Http4s Blaze).
Additionally, we've taken the opportunity presented by a minor release to fix some breaking semantic issues within some of the core IO
functionality, particularly related to async
. For most applications this should be essentially invisible, but it closes a long-standing loophole in the cancelation and backpressure model, ensuring a greater degree of safety in Cats Effect's guarantees.
Despite the deceptively short list of merged pull requests, this release contains an unusually large number of significant changes in runtime semantics. The changes in async
cancelation (and particularly the implications on async_
) are definitely expected to have user-facing impact, potentially breaking existing code in subtle ways. If you have any code which uses async_
(or async
) directly, you should read this section very carefully and potentially make the corresponding changes.
async
Cancelation SemanticsThe IO.async
(and correspondingly, Async#async
) constructor takes a function which returns a value of type IO[Option[IO[Unit]]]
, with the Some
case indicating the finalizer which should be invoked if the fiber is canceled while asynchronously suspended at this precise point, and None
indicating that there is no finalizer for the current asynchronous suspension. This mechanism is most commonly used for "unregister" functions. For example, consider the following reimplementation of the sleep
constructor:
def sleep(time: FiniteDuration, executor: ScheduledExecutorService): IO[Unit] =
IO.async[Unit] { cb =>
IO {
val f = executor.schedule(() => cb(Right(())), time.toNanos, TimeUnit.NANOSECONDS)
Some(IO(f.cancel()))
}
}
In the above, the IO
returned from sleep
will suspend for time
. If its fiber is canceled, the f.cancel()
function will be invoked (on ScheduledFuture
), which in turn removes the Runnable
from the ScheduledExecutorService
, avoiding memory leaks and such. If we had instead returned None
from the registration effect, there would have been no finalizer and no way for fiber cancelation to clean up the stray ScheduledFuture
.
The entirety of Cats Effect's design is prescriptively oriented around safe cancelation. If Cats Effect cannot guarantee that a resource is safely released, it will prevent cancelation from short-circuiting until execution proceeds to a point at which all finalization is safe. This design does have some tradeoffs (it can lead to deadlocks in poorly behaved programs), but it has the helpful outcome of strictly avoiding resource leaks, either due to incorrect finalization or circumvented backpressure.
...except in IO.async
. Prior to 3.5.0, defining an async
effect without a finalizer (i.e. producing None
) resulted in an effect which could be canceled unconditionally, without the invocation of any finalizer. This was most seriously felt in the async_
convenience constructor, which always returns None
. Unfortunately, this semantic is very much the wrong default. It makes the assumption that the normal case for async
is that the callback just cleans itself up (somehow) and no unregistration is possible or necessary. In almost all cases, the opposite is true.
It is exceptionally rare, in fact, for an async
effect to not have an obvious finalizer. By defining the default in this fashion, Cats Effect made it very easy to engineer resource leaks and backpressure loss. This loophole is now closed, both in the IO
implementation and in the laws which govern its behavior.
As of 3.5.0, the following is now considered to be uncancelable:
IO.async[A] { cb =>
IO {
// ...
None // we aren't returning a finalizer
}
}
Previously, the above was cancelable without any caveats. Notably, this applies to all uses of the async_
constructor!
In practice, we expect that usage of the async
constructor which was already well behaved will be unaffected by this change. However, any use which is (possibly unintentionally) relying on the old semantic will break, potentially resulting in deadlock as a cancelation which was previously observed will now be suppressed until the async
completes. For this reason, users are advised to carefully audit their use of async
to ensure that they always return Some(...)
with the appropriate finalizer that unregisters their callback.
In the event that you need to restore the previous semantics, they can be approximated by producing Some(IO.unit)
from the registration. This is a very rare situation, but it does arise in some cases. For example, the definition of IO.never
had to be adjusted to the following:
def never: IO[Nothing] =
IO.async(_ => IO.pure(Some(IO.unit))) // was previously IO.pure(None)
This change can result in some very subtle consequences. If you find unexpected effects in your application after upgrading to 3.5.0, you should start your investigation with this change! (note that this change also affects third-party libraries using async
, even if they have themselves not yet updated to 3.5.0 or higher!)
From the very beginning, Cats Effect and applications built on top of it have managed timers (i.e. IO.sleep
and everything built on top of it) on the JVM by using a separate thread pool. In particular, ScheduledExecutorService
. This is an extremely standard approach used prolifically by almost all JVM applications. Unfortunately, it is also fundamentally suboptimal.
The problem stems from the fact that ScheduledExecutorService
isn't magic. It works by maintaining one or more event dispatch threads which interrogate a data structure containing all active timers. If any timers have passed their expiry, the thread invokes their Runnable
. If no timers are expired, the thread blocks for the minimum time until the next timer becomes available. In its default configuration, the Cats Effect runtime provisions exactly one event dispatch thread for this purpose.
This isn't so bad when an application makes very little use of timers, since the thread in question will spend almost all of its time blocked, doing nothing. This affects timeslice granularity within the OS kernel and adds an additional GC root, but both effects are small enough that they are usually unnoticed. The bigger problem comes when an application is using a lot of timers and the thread is constantly busy reading that data structure and dispatching the next set of Runnable
(s) (all of which complete async
s and immediately shift back into the Cats Effect compute pool).
Unfortunately, this situation where a lot of timers are in use is exactly what happens in every network application, since each and every active socket must have at least one IO.sleep
associated with it to time out handling if the remote side stops responding (in most cases, such as HTTP, even more than one timer is needed). In other words, the fact that IO.sleep
is relatively inefficient when a lot of concurrent sleep
s are scheduled is particularly egregiously bad, since this is precisely the situation that describes most real-world usage of Cats Effect.
So we made this better! Cats Effect 3.5.0 introduces a new implementation of timers based on cooperative polling, which is basically the idea that timers can be dispatched and handled entirely by the same threads which handle compute work. Every time a compute worker thread runs out of work to do (and has nothing to steal), rather than just parking and waiting for more work, it first checks to see if there are any outstanding timers. If there are some which are ready to run, it runs them. Otherwise, if there are timers which aren't yet completed, the worker parks for that period of time (or until awakened by new work), ensuring the timer fires on schedule. In the event that a worker has not had the opportunity to park in some number of iterations, it proactively checks on its timers just to see if any have expired while it has been busy doing CPU-bound work.
This technique works extremely well in Cats Effect precisely because every timer had to shift back to the compute pool anyway, meaning that it was already impossible for any timer to have a granularity which was finer than that of the compute worker thread task queue. Thus, having that same task queue manage the dispatching of the timers themselves ensures that at worst those timers run with the same precision as previously, and at best we are able to avoid a considerable amount of overhead both in the form of OS kernel scheduler contention (since we are removing a whole thread from the application!) and the expense of a round-trip context shift and passage through the external work queue.
And, as mentioned, this optimization applies specifically to a scenario which is present in almost all real-world Cats Effect applications! To that end, we tested the performance of a relatively simple Http4s Ember server while under heavy load generated using the hey
benchmark tool. The result was a roughly 15-25% improvement in sustained maximum requests per second, and a roughly 15% improvement in the 99th percentile latencies (P99). In practical terms, this means that this one change makes standard microservice applications around 15% more efficient with no other adjustments.
Obviously, you should do your own benchmarking to measure the impact of this optimization, but we expect the results to be very visible in production top-line metrics.
uncancelable
would remain masked for one stage (@djspiewak)cede
s (@armanbilge)uncancelable
body (@armanbilge)Queue.synchronous
to include a two-phase commit (@djspiewak)Queue.synchronous
internals to simplify concurrent hand-off (@djspiewak)Mutex
memory leak (@BalmungSan)ioRuntimeConfig
, pass it to CPUStarvation
(@manuelcueto)AsyncMutex
implementation (@BalmungSan)blockedThreadDetectionEnabled
configurable via a system property (@chunjef)map2
optimization (@durban)AtomicCell#get
should not semantically block (@armanbilge)Console#readLine
cancelable (@armanbilge)IODeferred
(@armanbilge)HotSwap
safe to concurrent access (@armanbilge)IORuntimeBuilder
failureReporter
config on JS (@armanbilge)cancelable
(@djspiewak)timeout
(@djspiewak)fromFutureCancelable
and friends (@armanbilge)Mutex
& AtomicCell
(@BalmungSan)IOLocal#scope
, revert #3214 (@armanbilge)ConcurrentAtomicCell
(@BalmungSan)IOLocal
- generalize scope
function (@iRevive)BatchingMacrotaskExecutor
(@armanbilge)Ref
's flatModify
(@mn98)asyncCheckAttempt
in IODeferred#get
(@armanbilge)IO#supervise
, IO#toResource
, IO#metered
(@kamilkloch)IO#voidError
(@armanbilge)async_
to be uncancelable (@djspiewak)flatModify
, on Ref
(@mn98)Defer
instance for Resource
without Sync
requirement (@Odomontois)Async#asyncCheckAttempt
for #3087 (@seigert)IOLocal#scope
(@iRevive)A very special and heartfelt thanks to all of you!
This is the forty-fourth release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release. It is fully binary compatible with every 3.x release, and fully source-compatible with every 3.4.x release. Note that source compatibility has been broken with 3.3.x in some minor areas. Since those changes require active choice on the part of users to decide the best adjusted usage for their specific scenario, we have chosen to not provide scalafixes which automatically patch the affected call sites.
Thank you, Daniel!
This is the forty-third release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release. It is expected to be fully source- and binary-compatible with the final version of 3.5.0, but there are no guarantees of such.
As with all release candidates, we are not aware of any bugs or issues preventing production use, but we are making this release precisely because we know that the changes in this version are of a sufficiently significant nature as to benefit from broader testing and experimentation across the ecosystem before we incorporate them into a stable release. If you have the time, please do take a moment to try this version in your library or service and see how things work!
For the full, cumulative list of changes in the 3.5.0 release, please also see the release notes for RC1, RC2, RC3, and RC4. This list only encompasses the changes from RC4 to RC5 (not including changes already released and ported from the 3.4.x line).
AsyncMutex
implementation (@BalmungSan)blockedThreadDetectionEnabled
configurable via a system property (@chunjef)Thank you so much!
This is the forty-second release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release. It is fully binary compatible with every 3.x release, and fully source-compatible with every 3.4.x release. Note that source compatibility has been broken with 3.3.x in some minor areas. Since those changes require active choice on the part of users to decide the best adjusted usage for their specific scenario, we have chosen to not provide scalafixes which automatically patch the affected call sites.
map2
optimization (@durban)Very special thanks to all!
This is the forty-first release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release. It is expected to be fully source- and binary-compatible with the final version of 3.5.0, but there are no guarantees of such.
As with all release candidates, we are not aware of any bugs or issues preventing production use, but we are making this release precisely because we know that the changes in this version are of a sufficiently significant nature as to benefit from broader testing and experimentation across the ecosystem before we incorporate them into a stable release. If you have the time, please do take a moment to try this version in your library or service and see how things work!
Far and away the most significant change in this release is the rewriting of the timer data structure within WorkerThread
(courtesy of @durban!). This results in significantly higher sleep
performance, particularly under load when most timers are being canceled. If you previously tested an earlier RC and saw issues with these types of scenarios, we recommend you take another look using this latest RC!
For the full, cumulative list of changes in the 3.5.0 release, please also see the release notes for RC1, RC2, and RC3. This list only encompasses the changes from RC3 to RC4 (not including changes already released and ported from the 3.4.x line).
AtomicCell#get
should not semantically block (@armanbilge)Console#readLine
cancelable (@armanbilge)IODeferred
(@armanbilge)HotSwap
safe to concurrent access (@armanbilge)IORuntimeBuilder
failureReporter config on JS (@armanbilge)Thank you so much!
This is the fortieth release in the Cats Effect 3.x lineage. It is fully binary compatible with every 3.x release, and fully source-compatible with every 3.4.x release. Note that source compatibility has been broken with 3.3.x in some minor areas. Since those changes require active choice on the part of users to decide the best adjusted usage for their specific scenario, we have chosen to not provide scalafixes which automatically patch the affected call sites.
Dispatcher
: check for outstanding actions before release (@samspills)raceOutcome
to correct implementation (@Jasper-M)Dispatcher
error reporting (@samspills)IOFiber
(@djspiewak)std.Console
(@zetashift)IODeferred
specialization (@armanbilge)unsafeRunAndForget
(@samspills)IOFiber#toString
(@durban)Special thanks to each and every one of you!