Rx Player Versions Save

DASH/Smooth HTML5 Video Player

v4.0.0

2 months ago

Release v4.0.0 (2024-02-21)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

:mag: Overview

It's finally time for the official v4.0.0 release with the v4 now becoming our default focus and the default major version when installing the RxPlayer through package managers relying on the npm registry (npm / yarn / pnpm etc.).

If you relied on the 4.0.0-rc.2 before, this release is a quasi-exact copy of that version, with only a minor fix for the representationListUpdate event - which was previously never triggered.

Because previous v4 beta and release candidate release notes already listed the new features available in v4 in comparison to v3, this release note will only summarize v4 features we consider to be the most important. They already all have been presented in one of the previous release notes.

To migrate from a v3 RxPlayer to a v4 one, you can rely on our migration guide, which lists every API that changed in that major version.

:bookmark_tabs: Changelog

We decided to compile the full v4 changelog into one (instead of splitting and associating it to the various beta and release candidates they have been initially available in).

This lead to an enormous changelog for the v4. To avoid polluting this release note we will just redirect you to our CHANGELOG.md file, here.

About the v3

We will mainly only add bug fixes and small improvements from now on to the now legacy v3.x.x versions so it stays stable and usable for people not having time yet to do the switch to the v4.

As such, it should still work as expected, but new features probably won't be added to it, unless you provide the contribution and test cases for it yourself through pull requests.

However, we do recommend you to switch to v4 instead, and not hesitate to open an issue if you find an API change to be unclear or undesirable for your usage.

A more flexible track API

Note: this feature was already presented in the v4.0.0-beta.0 release note.

One of the focus of this new major release was to improve the RxPlayer API on DASH multi-Period contents - which are contents with various set of AdaptationSets (tracks) and Representations (qualities) depending on the time period.

The RxPlayer's previous track API (e.g. setAudioTrack and getAvailableAudioTracks) only allowed to get the list and update the track for the currently-playing Period.

// Example setting the first english audio track for the current Period if found
const availableAudioTracks = rxPlayer.getAvailableAudioTracks();
const englishAudioTrack = availableAudioTracks.find((track) => {
  return track.language === "eng";
});

if (englishAudioTrack !== undefined) {
  rxPlayer.setAudioTrack(englishAudioTrack.id);
}

Now, using the track API this way still works with the same result, but it is also possible to get and set the available tracks for any Period on the content.

// Get the list of Periods currently considered by the RxPlayer:
const availablePeriods = rxPlayer.getAvailablePeriods();

// Get the list of available audio tracks for a given period
const tracks = rxPlayer.getAvailableAudioTracks(availablePeriods[0].id);

// Set an audio track for that Period
rxPlayer.setAudioTrack({
  trackId: tracks[0].id,
  periodId: availablePeriods[0].id,
});

The new tracks API also let you to choose a behavior when switching from an old to any new track (e.g.: reloading, switching in place with a potential rebuffering or seamlessly) through a new switchingMode property and also allow to rewind a little (and let you set by how much) in cases where you want to give back some context (for example when switching the audio track to another language).

Last but not least, it is also possible to restrict the Representations (a.k.a. qualities) played under that new track, this will be described in the next chapter.

Some of those features have been described in our "Selecting a Track" tutorial. To have complete informations, you can also refer to our API documentation

Improved Representation selection

Note: this feature was already presented in the v4.0.0-beta.0 release note.

Previous RxPlayer versions only allowed to specify allowed Representation(s) (i.e. qualities) by using bitrate-oriented API. For example you could call setVideoBitrate, setMaxVideoBitrate and setMinVideoBitrate to either choose a Representation (the first one) or to reduce the ranges of Representations to choose from (the latter two).

In real-life, you might instead want to select Representation(s) based on other criterias. In some more complex use cases, you might only want to allow Representations with a specific codec property. Both of those were not always possible with the previous API.

We chose to remediate to those issues in the v4 by providing a new API for Representation selection: the "Representation locking" family of API. For example, the lockVideoRepresentations method allows to select which Representation for the current video track are allowed to play, the regular RxPlayer's adaptive logic then picking its choice between them, as usual. To lock a single Representation in place, you can just communicate a single Representation's id to that method:

// Example only playing the Representation with the lowest height in the
// current video track

const videoTrack = rxPlayer.getVideoTrack();
if (videoTrack !== null && videoTrack !== undefined) {
  const lowestHeight = videoTrack.representations.sort((a, b) => {
    // Put `undefined` heights at the end of the resulting array
    if (a.height === undefined) {
          return 1; // Put `a` after `b`
    } else if (b.height === undefined) {
      return -1; // Put `b` after `a`
    }
    // Sort ascending
    return a.height - b.height; // Put the higher height after
  })[0]; // Select the lowest one
  if (lowestHeight !== undefined) {
    // Only play the lowest anounced height
    rxPlayer.lockVideoRepresentations([lowestHeight.id]);
  }
}

There is a lot more to know on this API, see the lockVideoRepresentations / lockAudioRepresentations documentation page to see all that is can do.

We rely on this new API to display a better quality selection in our demo page for example:

new-bitrate-choice Screenshot: our new demo page now allows a user to select a video quality based on its height and/or the wanted bitrate, thanks to this new API.

We also chose to remove the previous bitrate-related API to simplify the general API of the RxPlayer, considering that its behavior can be completely replaced by the new "Representation locking" methods.

Information on how to make the switch is present in its own page in our migration guide

The new `MULTI_THREAD` experimental feature

Note: this feature was already presented in the v4.0.0-rc.1 release note.

This major release also brings the possibility of running most of the RxPlayer main logic in a WebWorker, letting your application to run concurrently with it. This has potentially large positive impacts on performance and adaptive streaming stability (e.g. keeping a stable high video quality).

This new behavior is totally optional and has to be enabled through specific APIs. The RxPlayer is also able to automatically detect when multithreading is not possible (very old devices), to go back in the regular monothreading mode instead.

Running the RxPlayer without a WebWorker (the default):

+-------------------------------------------------------------------------------+
| Main thread (also running the UI)                                             |
|                                                                               |
| +------------------+     +----------------------+    +----------------------+ |
| |    Application   | --> |  RxPlayer Main [1]   | -> |   RxPlayer Core [2]  | |
| +------------------+     +----------------------+    +----------------------+ |
+-------------------------------------------------------------------------------+

Running with a WebWorker:

+----------------------------------------------------+
| Main thread (also running the UI)                  |
|                                                    |
| +------------------+      +----------------------+ |
| |    Application   | ---> |  RxPlayer Main [1]   | |
| +------------------+      +----------------------+ |
+--------------------------------------|-------------+
                                       | (messages)
+--------------------------------------|-------------+
| WebWorker                            V             |
|                           +----------------------+ |
|                           |  RxPlayer Core [2]   | |
|                           +----------------------+ |
+----------------------------------------------------+


[1] RxPlayer Main: Exposes an API to the application, performs some high-level
media monitoring, handles content decryption, displays text tracks and interacts
with web API that are only usable in the main thread.

[2] RxPlayer Core; Loads and parses the Manifest as well as media segments that
will be be played. Also monitors what's being played and will be played to
ensure a smooth playback.

Schema: Here is some high level schema of how the RxPlayer would roughly work without and with a WebWorker.

There's many things to say about this new feature and we encourage you to check if it improves the experience on your application. Everything should be detailed in its API documentation, here.

More expressive decryption options

Note: this feature was already presented in the v4.0.0-beta.0 release note.

We added and updated several decryption-related options, allowing to be more flexible in expressing how to play encrypted contents.

`onKeyOutputRestricted` / `onKeyInternalError`

Two string properties imitating the already there onKeyExpiration API have been added:

onKeyOutputRestricted: allows to indicate the behavior the RxPlayer should have when encountering the "output-restricted" MediaKeyStatus, like fallbacking to another quality or stopping with an error (the default)
onKeyInternalError: allows to indicate the behavior the RxPlayer should have when encountering the "internal-error " MediaKeyStatus, like here also fallbacking to another quality or stopping with an error (the default)

Because it allows more powerful configuration than the previous fallbackOn option, the latter has been removed.

`persistentState` / `distinctiveIdentifier`

The v4.0.0 major release adds two string properties to the keySystems option of loadVideo:

persistentState: Indicate if the CDM should be able to persist state. It actually reflects the MediaKeySystemConfiguration property of the same name, taking the same possible values.
distinctiveIdentifier: Indicate if the CDM can or even should use distinctive identifiers. Here also, this property reflects the MediaKeySystemConfiguration property of the same name.

Because, they allow the easy replacement of respectively the persistentStateRequired and of the distinctiveIdentifierRequired boolean properties of keySystems, both of those have been removed.

`videoCapabilitiesConfig` / `audioCapabilitiesConfig`

Two keySystems properties videoCapabilitiesConfig and audioCapabilitiesConfig now allow to configure respectively the videoCapabilities and audioCapabilities properties of the asked MediaKeySystemConfiguration.

Relatively powerful, this option allows to configure the asked codecs, robustnesses or both. Like the other ones presented here, this option provokes the removal of the less powerful videoRobustnesses and audioRobustnesses undocumented options.

A new player state: `"FREEZING"`

Note: this chapter was first written for the v4.0.0-beta.0 release note.

A new player state (gettable either through the playerStateChange event or the getPlayerState method) has been added: "FREEZING".

It's important to note that this new new state does not characterize a new behavior, it only put a better word onto a specific problematic playback state. Previously, it was either reported as a "BUFFERING", "SEEKING", or even "PLAYING" state depending on the situation.

This state appears when playback is temporarily stuck in place, though in opposition to the more usual "BUFFERING" and "SEEKING" state, this is not due to buffer starvation but to another, generally unknown, factor. As such, it can in most cases be considered just like a "BUFFERING" state in your application (e.g. by displaying a spinner on top of the media element) but it may be logged differently to help you pinpoint playback issues.

Screenshot: A "FREEZING" state can be treated just like a "BUFFERING" one, here with a spinner on top of the video.

Under that state, which is generally much rarer than a "BUFFERING" state for example, the RxPlayer will try various tricks to try to un-freeze playback. If they become too frequent or if those tricks don't work, it might be an issue worth investigating.

v3.33.2

2 months ago

Release v3.33.2 (2024-02-21)

Quick Links:
📖 API documentation - ⏯ Demo

NOTE: we skipped the v3.33.1 release due to a minor TypeScript typing issue seen when testing that release. This is now fixed.

:mag: Overview

This release only brings minor bug fixes on top of the v3.33.0, none being regressions (those issues have been here since the corresponding features have been introduced).

This reassure us that the v3 is stable enough for it to be reliable for people not having the time yet to make the switch to a v4, as we plan to release the official v4.0.0 just after this release.

The v3 major releases should still be maintained and receive bug fixes for some time if we find them, but our main focus will now go to the v4 releases. Note that all future v3 releases will have the legacy-v3 tag on npm.

:bookmark_tabs: Changelog

Bug fixes

dash: Don't unnecessarily reload external <UTCTiming> resources at each refresh if it failed for the first request of the Manifest [#1370]
dash: The DASH_WASM feature do not rely on WebAssembly's sign-extension operators anymore as that is poorly supported on older Samsung and LG TVs [#1372]

Other improvements

build: automatically install Rust and WASM toolchain locally if unavailable when building the RxPlayer WebAssembly file
doc: Update our documentation generator and fix all invalid anchors in it
npm: prevent the publishing of unnecessary files on the npm registry [#1377, #1378]

v4.0.0-rc.2

3 months ago

Release v4.0.0-rc.2 (2024-02-07)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

Overview

This new release candidate mainly fixes issues we've seen with the WebAssembly parser for DASH MPD, that is used for both the DASH_WASM feature (which lost its "experimental" label at v4.0.0-rc.1) and the MULTI_THREAD experimental feature.

Another minor fix is on the handling of the <UTCTiming> element in a DASH MPD. If it pointed to an URL and if the resource behind it could not be fetched when first loading the MPD due to issues with the request(s) (there may be several attempts), then the RxPlayer would keep re-loading the resource each time the MPD was refreshed, instead of just stopping doing it once it loaded successfully once - which is what the RxPlayer does now.

None of these issues are regressions, and thus some also concern the v3.33.0 (the DASH_WASM and <UTCTiming> ones). As those all are very minor issues (DASH_WASM being still experimental in v3.33.0 - with an error triggering a fallback on the JS-based parser anyway - and the <UTCTiming> issue being rare and not preventing smooth playback) they will only be released as a future v3.33.1 release once the 4.0.0 is released.

Changelog

Features

MULTI_THREAD: attachWorker now returns a Promise to indicate when WebWorker attachment failed [#1374]

Bug fixes

dash: Don't unnecessarily reload external <UTCTiming> resources at each refresh if it failed for the first request of the Manifest [#1370]
dash: The DASH_WASM feature do not rely on WebAssembly's sign-extension operators anymore as that is poorly supported on older Samsung and LG TVs [#1372]
MULTI_THREAD: properly categorize forced subtitles in multithread scenarios

Other improvements

build: automatically install Rust and WASM toolchain locally if unavailable when building the RxPlayer [#1373]
doc: Update our documentation generator and fix all invalid anchors in it
npm: prevent the publishing of unnecessary files on the npm registry [#1377, #1378]

`DASH_WASM` and `MULTI_THREAD` fix

After testing the MULTI_THREAD feature on a large array of smart TVs, we noticed that some old Samsung and LG TVs had issues instantiating our WebAssembly MPD parser (on which both the MULTI_THREAD and DASH_WASM features rely). It turned out that those TVs had support for WebAssembly (if they did not, the RxPlayer would automatically have disabled "multithread" mode), yet to a very old version of it which did not have some of its early features present in the WebAssembly file produced by the Rust compiler (the compiler we're using) by default.

This was not known to us as we always assumed that the compiler targeted by default the earliest available version of WebAssembly.

We ended up doing supplementary transformations on our WebAssembly file to remove the reliance on those newer features. The new mpd-parser.wasm file delivered with this version (as well as exported through "rx-player/experimental/features/embeds") should now be compatible with those devices.

`attachWorker` now returns a Promise

The aforementioned DASH_WASM and MULTI_THREAD issue put a light into some scenarios we were not enough prepared for: how to handle cases where the WebWorker and/or WebAssembly module relied on when using the MULTI_THREAD feature fail to initialize. This should hopefully be very rare now, yet may still happen if e.g. any of those resources are behind an URL that is not accessible or if browser security settings prevents the RxPlayer from creating or relying on a WebWorker.

To better handle those cases for now, we decided that the attachWorker method now returns a Promise. That promise will either resolve if the Worker was initialized with success and reject if it did not.

Like before, you may still call loadVideo synchronously after the attachWorker call has been made (you don't have to await this Promise) but it is now advised to await that Promise in scenarios where you're both relying on the MULTI_THREAD feature and on one of the corresponding monothreaded feature (either DASH or DASH_WASM) - in which case it is the role of the RxPlayer to choose between one or the other.

For example: the following code:

import RxPlayer from "rx-player/minimal";
import { DASH } from "rx-player/features";
import { MULTI_THREAD } from "rx-player/experimental/features";
import {
    EMBEDDED_WORKER,
    EMBEDDED_DASH_WASM,
} from "rx-player/experimental/features/embeds";

RxPlayer.addFeatures([
  // Will allow to play DASH contents in main thread
  DASH,

  // Will allow to play DASH contents in multithread scenarios
  MULTI_THREAD,
]);

const player = new RxPlayer(/* your usual RxPlayer options */);

try {
  await player.attachWorker({
    workerUrl: EMBEDDED_WORKER,
    dashWasmUrl: EMBEDDED_DASH_WASM,
  })
  console.log("Worker succesfully attached!");
} catch (err) {
  console.warn("An error arised while initializing the Worker", err);
}

player.loadVideo({ /* your usual loadVideo options */ });

Will only load the content in "multithread" mode if all of the following are true:

Your browser supports the feature (which is already checked synchronously when attachWorker is called)
Your content is compatible with multithread mode
The Worker initialization went without issue (this is the case where awaiting attachWorker has an effect)

And in other cases, it will load in monothreaded mode, as the DASH feature is also added.

If you do not await attachWorker before calling loadVideo here and the Worker initialization fails, the RxPlayer might have already begun to load the content in "multithread" mode. In that case it might fail to do so and trigger an error for that content (we're also currently exploring ways of making the RxPlayer automatically reload in monothreaded mode in this exact last scenario but it isn't done for now).

Also note that if Worker initialization fails, the RxPlayer won't try to rely on it anymore for the next loadVideo and reload calls. If you want to retry Worker initialization in that unlikely scenario, you could call attachWorker again.

Project repository updates

As we're now very confident of the stability of the v4 pre-releases, we began making changes to the RxPlayer repository:

The default branch seen on GitHub and choosen as a default target branch for Pull Requests is now dev, which corresponds to the latest merged developments on the v4. Note that it is currently further than the v4.0.0-rc.2 as we're already merging improvements for future v4 releases.

Another branch, stable corresponds to the last released v4 version (so here v4.0.0-rc.2) plus optional hotfixes. dev is based on stable.

The old master branch keeps refering to the v3 but has been renamed to legacy-v3. next does not exist anymore, any new developments for the v3 now targets legacy-v3.
The default demo page exposed both by the GitHub link and in our README.md file now leads to the v4.0.0-rc.2 demo.

The last v3.33.0 demo page can still be accessed here and the list of demo pages per version can be found here.
Likewise, the default documentation pages exposed in our README.md file now leads to the v4.0.0-rc.2 documentation.

The last v3.33.0 documentation page can still be accessed here and the list of documentation pages per version can be found here.
We filtered files that were actually published on npm when we published a new version. For example, you now shouldn't be pulling the RxPlayer source files anymore when doing so.

v4.0.0-rc.1

3 months ago

Release v4.0.0-rc.1 (2024-01-24)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

:mag: Overview

After more than three years since its early drafts and one year of open beta releases, it is finally time for the first release candidate of the v4.0.0 of the RxPlayer.

The main ideas behind this new v4 major version are:

To provide a more flexible and powerful track and quality selection API, especially more adapted to multi-Period DASH contents.

For example, it is now not only possible to set a particular quality on a content, you can also allow a subset of multiple qualities from which the RxPlayer will adaptively choose from, based on any of their characteristics exposed by the API.

You can also indicate what to do if the buffer already contains media data in another quality (reload? Keep it in the buffer? Remove it?).

Likewise, track switching becomes very configurable, allowing you to directly switch a track of any Period, again with a strategy if another track was previously present in the buffer. The RxPlayer also allow to rewind a little (and let you set by how much), in cases where you want to give back some context (for example when switching the audio track to another language).

The main API documentation about quality selection can be found here. For track switching, you can refer to the API documentation of the concerned API, see for example the setAudioTrack API documentation.
To allow a more sane default behavior and more resilience in the RxPlayer.

For example, we are now automatically reloading if something has been detected to have gone horribly wrong, such as when encountering some exceptional device-related decryption issues or when obtaining a completely different Manifest after updating it.

Moreover, if the chosen track is linked to Representations (qualities) which are all impossible to decrypt, the RxPlayer will now automatically fallback to a supported track instead (after sending events to indicate that it does so).

We were previously limited on how we could do those things to stay compatible to old APIs.
To facilitate the implementation of ambitious improvements.

For example the v4.0.0-rc.1 brings the new MULTI_THREAD experimental feature, allowing to run the RxPlayer's main logic in a WebWorker for better adaptive streaming stability and performances. This is a huge work we're right now using on a multitude of devices at Canal+, intended to improve the quality of experience, especially on devices with limited resources.

Depending on return we have on our large-scale tests (basically, if no important issue is found with it), this release candidate may become an official v4.0.0 relatively fastly (in the coming weeks).

If you're still relying on a v3.x.x release, we recommend looking up our migration guide to make the switch more easily.

:bookmark_tabs: Changelog

We decided to compile the full v4.0.0 changelog for this release (instead of comparing it to our latest beta release), which makes it too big for this release note! So we're only linking you to it here: https://github.com/canalplus/rx-player/blob/bf79b9164f8d530a94e338725f795ec5a5c7b278/CHANGELOG.md

What about the v3?

The v3.33.0 release of the RxPlayer, which was published just before this one, is planned to be our final default release on npm in the v3 major version (unless there is a major issue with it).

Once the official v4.0.0 stable release is published, v4 releases will take its place as a default when installing the package through package managers such as yarn and npm.

Moreover, most incoming features and improvements will be planned for v4.x.x releases first from now on.

v3.x.x releases will still be maintained for some time (we were for example talking about providing at least 1 year of support), but it will mostly be only bug fixes and improvements simple enough to be ported to it. Of course, outside contributions are welcomed if you want to provide improvements on it yourself.

If you want to make the switch but you're afraid of relying on a release candidate right now, we recommend maintaining a branch which will depend on this v4.0.0-rc.1 release in your project and putting it in production once the v4.0.0 is actually released.

`v4` summary

The v4.0.0-rc.1 inherits from both the v4.0.0-beta.3 (and consequently any prior v4 release) and from the just-released v3.33.0 (and consequently all previous v3 releases).

The rest of this release note is going to focus ONLY on features brought in this particular release (which means we're not relisting features already announced in previous v4 and v3 release notes).

You can look at the v4 beta release notes for more information on some earlier v4 features:

And on the the v3.33.0 release note for the just-released features which are also in this one.

The new `MULTI_THREAD` experimental feature

This release brings the possibility of running most of the RxPlayer main logic in a WebWorker, letting your application to run concurrently with it. This has potentially large positive impacts on performance and adaptive streaming stability (e.g. keeping a stable high video quality).

Running the RxPlayer without a WebWorker (the default):

+-------------------------------------------------------------------------------+
| Main thread (also running the UI)                                             |
|                                                                               |
| +------------------+     +----------------------+    +----------------------+ |
| |    Application   | --> |  RxPlayer Main [1]   | -> |   RxPlayer Core [2]  | |
| +------------------+     +----------------------+    +----------------------+ |
+-------------------------------------------------------------------------------+

Running with a WebWorker:

+----------------------------------------------------+
| Main thread (also running the UI)                  |
|                                                    |
| +------------------+      +----------------------+ |
| |    Application   | ---> |  RxPlayer Main [1]   | |
| +------------------+      +----------------------+ |
+--------------------------------------|-------------+
                                       | (messages)
+--------------------------------------|-------------+
| WebWorker                            V             |
|                           +----------------------+ |
|                           |  RxPlayer Core [2]   | |
|                           +----------------------+ |
+----------------------------------------------------+


[1] RxPlayer Main: Exposes an API to the application, performs some high-level
media monitoring, handles content decryption, displays text tracks and interacts
with web API that are only usable in the main thread.

[2] RxPlayer Core; Loads and parses the Manifest as well as media segments that
will be be played. Also monitors what's being played and will be played to
ensure a smooth playback.

Schema: Here is some high level schema of how the RxPlayer would roughly work without and with a WebWorker.

There's many things to say about this new feature and we encourage you to check if it improves the experience on your application. Everything should be detailed in its API documentation, here.

Breaking change: volume APIs

We've changed the behavior of the mute / unMute / isMute methods, so they now mutate the muted property of the media element, whereas they updated the volume property before (by setting it to 0 or resetting it to its previous value).

It means that this change can also lead to issue if you were previously relying only on setVolume to un-mute (now an unMute call will also have to be made in that scenario) or on isMute / a getVolume returning 0 to know if there was no sound (both checks will have to be performed now).

video: Example scenario in a UI where we would want now to both set the volume and unMute. Before, setting a volume automatically unmuted as both relied on the HTML volume property.

Likewise the volumeChange event now announces both the muted (as a boolean) and volume (as a number) properties, instead of just the volume.

We did those changes as it better maps to the actual HTML5 API for media volume control. By aligning with them, we hope to improve the player compatibility to current and future browsers' behaviors.

A new `requestConfig` property: `connectionTimeout`

We've also added a new property to the requestConfig option you can provide to a loadVideo call: connectionTimeout.

This new feature gives developers a finer control over request management by introducing a more granular approach to handling timeouts. Previously, it was only possible to set a global request timeout for manifests and segments requests (through respectively a manifest.timeout and a segment.timeout property).

The inclusion of the manifest.connectionTimeout and segment.connectionTimeout properties now also allow to abort requests that took too much time to establish connection (i.e: receiving HTTP headers) while still allowing to wait longer for requests that are currently downloading potentially large response data.

Schema: Very simplified illustration of the difference between the "connection establishment" which is the step that connectionTimeout acts on, and the regular end of the HTTP(S) request which is the step that timeout acts on.

Using this new property should lead to a more efficient hanged requests aborting logic if you encounter them.

It has been documented in the requestConfig documentation.

Seek-back on track switch with `relativeResumingPosition`

The v4.0.0-rc.1 also adds a new option when switching the audio track (through the setAudioTrack method) or the video track (through the setVideoTrack method): relativeResumingPosition.

With it, you can provoke a seek if the track switching interruption. The main use case for example would be to seek back one second or so when changing the audio track to a new language, allowing the final user to re-hear the corresponding sentence:

video: Example usage of the relativeResumingPosition option in setAudioTrack, the value is configured at -5. This causes a seek to the position 5 seconds before the current position when performing an audio track change.

For example, if you want to exploit this feature, you may call setAudioTrack this way:

rxPlayer.setAudioTrack({
  trackId: "my-track",

  // Ensures that there's an interruption if a previous audio track
  // was playing
  switchingMode: "direct",
  
  // Seek back 1 second after switching to the new audio track.
  relativeResumingPosition: -1,
})

Note that the relativeResumingPosition option has some intelligence and will probably not actually seek back if it was not currently playing a previous audio track for that Period (either because it was not the current Period being played, because no audio track was previously playing or because we were already currently seeking or buffering), which is what we guess you should want in most of those scenarios.

As such, that option's main purpose is just to give back audio and/or video context when the content is interrupted due to a track switch,

Easier `DASH_WASM` integration with the `EMBEDDED_WASM` embed

We profited from the new MULTI_THREAD experimental feature to facilitate management of the separate WebAssembly file that was needed to enable the DASH_WASM feature.

Before, an application had to download, store and serve itself the separate mpd-parser.wasmthat comes with each new release of the RxPlayer. This meant that each time the RxPlayer was updated, this file had to be manually updated. This was a cumbersome step that we would prefer you to avoid.

Now, the RxPlayer also provides an export which embeds that file in an easy-to-import JavaScript file. You can now just import that embedded version like this in your code:

import { EMBEDDED_DASH_WASM } from "rx-player/experimental/features/embeds";

Which can then be directly provided to a DASH_WASM.initialize call:

import { DASH_WASM } from "rx-player/features";

DASH_WASM.initialize({ wasmUrl: EMBEDDED_DASH_WASM });

Doing this removes the need to have to update manually the mpd-parser.wasm, with the cost of some minor efficiency loss at initialization time. This new way is described in the updated DASH_WASM API documentation.

Breaking change: A `representationFilter`'s `codec` is now `codecs`

The representationFilter API had to be changed a little so its API stay the same even when the new experimental MULTI_THREAD feature is relied on.

The first argument given to the representationFilter, an object describing the corresponding Representation, has now its codec property renamed to codecs. That codecs property is now an array of strings, whereas it was only a string before.

For the great majority of cases, checking the first element of that array is equivalent to the content of the old codec property.

So if you were doing something like:

function representationFilter(representationInfo) {
  if (representationInfo.codec === SOME_CODEC) {
    return false;
  }
  return true;
}

You'll now need to do something like:

function representationFilter(representationInfo) {
  // Note: `codecs` can also be `undefined`
  if (
    Array.isArray(representationInfo.codecs) &&
    representationInfo.codecs[0] === SOME_CODEC
  ) {
    return false;
  }
  return true;
}

There are very rare conditions where you might have multiple codecs in that array. For the moment, it might only happen if you're:

Relying on the MULTI_THREAD experimental feature and a worker
Running on a browser without the MSE-in-Worker feature
Are playing a DASH content
Have a scte214:supplementalCodec in the MPD
The codec support check is not yet cached (meaning that it's the first time since the RxPlayer is imported that we're checking for codec compatibility). Typically, this happens for the first loaded content
the representationFilter function is declared as a string (else we won't be relying on the worker for now)

In that very specific scenario, the RxPlayer may have several candidate codecs for a given Representation (depending on device support) but not yet know which one is actually supported when the representationFilter is called. In that case the codecs array will have multiple entries, from the most wanted codec (e.g. Dolby Vision) to the more compatible (e.g. HEVC).

Samsung TV seek-back handling refactoring

Samsung TVs (we generally refer to those platforms as "Tizen", as that's what the name of their OS is) are historically the devices giving us the most trouble. This is due to a very specific behavior we ended up calling: "Tizen automatic seek-backs".

Let's say that the RxPlayer want to seek at a given position, like 51 seconds. On all platforms but Samsung TVs, the RxPlayer would tell the browser to seek at the position 51, and then lower-level decoding layers would pick-up playback from that position.

On Samsung TVs however, it may not go that way. Tizen could decide to actually re-provoke by itself a seek to another position, often earlier (let's say at 50 seconds), without telling us, most likely for what it thinks are performance reasons (e.g. because the video frame at 50 seconds is some kind of I-frame).

Yet the RxPlayer often has very good reasons to want to seek to a given precise position. It may be because it wants to go to a specific video frame, or because it wants to go precizely at the beginning of a given time range. On Samsung TVs, we found out this is impossible to expect, as Tizen will have the last say on which position to really seek to.

TizenSeekingLoop(1) Schema: Illustration of a "seeking loop" due to Tizen behavior when seeking. Here the RxPlayer wants to seek at 51 but Samsung TVs decide to bring playback back to 50 seconds.

The previous strategy was to litter our code with exceptions just for Tizen devices at each place in the code where this "seek-back" behavior led to problems. After 5 or 6 patches, we thought for many years that we had no remaining issues. Sadly, it came biting us back recently in relatively rare situations!

So, in the v4.0.0-rc.1 release, we totally changed the way that Samsung TV behavior was handled. To keep it simple, our logic now makes a difference between the position we're actually playing at the lower-level (so influenced by Tizen "seek-backs"), and the one the RxPlayer wanted to actually play (here not influenced by it) using internally one or the other depending on the situation.

This means that even if some features (like frame-to-frame seeking) should not be possible on those devices (I don't think it actually can be possible here), the RxPlayer should now avoid most of the corresponding logic issues, as it will most of the time just go on with the position it wants to play.

Of course it's a little more complicated than that, yet the gist of it is that we refactored Samsung TV handling to better handle its peculiar behavior and it should be totally transparent for the application.

`DASH_WASM`, `DEBUG_ELEMENT` and `parseBifThumbnails` are not experimental anymore

We profited from this release candidate to remove the "experimental" status of various API, meaning they are now an integral part of the v4.x.x.

This concerns the following features:

DASH_WASM (which can now be imported from rx-player/features)
DEBUG_ELEMENT (which can now be imported from rx-player/features)

And the following tool:

parseBifThumbnails (which can now be imported from rx-player/tools)

This means what was previously written this way:

import { DASH_WASM, DEBUG_ELEMENT } from "rx-player/experimental/features";
import { parseBifThumbnails } from "rx-player/experimental/tools";

Needs now to be written this way:

import { DASH_WASM, DEBUG_ELEMENT } from "rx-player/features";
import { parseBifThumbnails } from "rx-player/tools";

Their API didn't change.

New package export strategy

This v4.0.0-rc.1 made some important updates to how the RxPlayer is exported, yet it should not impact your project in most cases:

The default build exported by the RxPlayer will be an ES2017-compatible build, with corresponding ESM exports (import/export). We also alternatively provide commonJS exports if your project relies on it instead. Previously the default "rx-player" path actually exported our bundle, and other RxPlayer exports exported ES5 code with CommonJS-style exports.

This should have most likely no impact on applications relying on the RxPlayer, beside perhaps needing to transpile the RxPlayer's ES2017 code to an earlier version if you want to support older platforms.
To resolve dependency paths (e.g. "rx-player/features", "rx-player/tools" etc.), we now make use of the exports property in our package.json, whereas it previously followed the real path of our project root directory.

This should hopefully be transparent for you as this is the new popular default way for declaring complex exports and most tools now read this exports property. However, if you depend on an old (~2021 or less) bundler or TypeScript or linter version and seeing importing issues after using those, you might need to update that dependency to a more recent version.

v3.33.0

3 months ago

Release v3.33.0 (2024-01-24)

:mag: Overview

The v3.33.0 is now available on npm.

It is planned to be the last main v3 release now that the official v4.0.0 is around the corner and became our main focus (a v4.0.0-rc1 will be published just after this one), yet is still contains its share of improvements and bug fixes:

DASH: To better support retro-compatible Dolby Vision content, the scte214:supplementalCodecs is now handled. Concerned contents will be advertised under a Dolby Vision codec for compatible devices, or the retro-compatible codec for those who aren't
A new getLivePosition method and fromLivePosition startAt option have been added, mainly for advanced ad-switching usages.
subtitles: the tts:lineHeight TTML attribute was not properly applied, it is now fixed
subtitles: A rare occurence of blinking subtitles could arise, especially on low-latency contents. This also has been fixed.
Add the possibility of setting another keySystems option on the reload API.
Several fixes on issues that could arise when doing API calls after reaching the last playable position of a content.
DEBUG_ELEMENT: Undecipherable and unsupported codecs are now announced in the debug element
Multiple other minor fixes and improvements linked to ad-switching, checking of codec support, and low-memory handling.

:bookmark_tabs: Changelog

Features

Add getLivePosition RxPlayer method [#1300]
Add startAt.fromLivePosition loadVideo option [#1300]
Add the possibility to set a new keySystems option on the reload API [#1308]

Bug fixes

Fix subtitles "blinking" in some specific conditions, especially with some DASH low-latency contents [#1314]
DASH: Fix Period overlap resolution logic for when the first Period is removed [#1311]
TTML: Fix handling of the tts:lineHeight attribute [#1320]
Fix import of the LOCAL_MANIFEST experimental feature
Avoid very rarely skipping segments which initially were too big to be pushed due to memory limitations [#1323]
Fix issue arising when using track APIs at the exact last possible position of a Period with no consecutive Period [#1337]
Starting at the end (through a startAt loadVideo option) or reloading at the end led to the restart of the content [#1338]
DRM/Safari: also perform Safari DRM work-arounds when the page is launched from the dock [#1351, #1356]

Other improvements

DASH: rely on SCTE214 supplementalCodecs instead of codecs if it's supported to better support backward compatible Dolby Vision contents [#1307]
DASH: Provide better support of the availabilityTimeOffset attribute [#1300]
DEBUG_ELEMENT: Add unsupported and undecipherable bitrates to the debug element [#1321]
DEBUG_ELEMENT: update buffer graph maximum size so it becomes more readable for lengthy contents [#1316]
DEBUG_ELEMENT: always synchronize inventory of segments before rendering it [#1317]
Remove remaining RxPlayer dependency removing possibility of some application-side bundling errors [#1312]
Add exception to text Garbage collection logic to avoid unnecessarily reload text segments frequently [#1325]
Avoid logging too much the buffer's content when our debugging UI or the demo is used [#1341]
Demo: Fix reporting of live position in demo page [#1313]

About it being the last "main v3 release"

As we've reached a point where we plan to release the first official v4.0.0 release, we also plan to make v4 releases our main focus when new features and improvements are added. As such, a v4.0.0-rc1 release should be released just after this release. If our large-scale tests go well, it will become the first v4.0.0 release.

This do not mean that the v3 will become unsupported. It should still receive bug fixes yet:

It will not be the version installed by default through npm / yarn, which will be the v4 instead once the official v4.0.0 version is released. The idea will be to set the npm legacy-v3 tag for newer v3 releases, so an rx-player v3 may be installed through something like: npm install rx-player@legacy-v3.
When planning new improvements, we now will implement it on the v4 first, and only backport it to the v3 if it is simple enough with few risks to break.
As more and more applications will rely on the v4 (we already "convinced" many at Canal+ to do so), we will progressively become more aware of issues when they concern the v4 and less about v3 issues.

DASH `supplementalCodecs` handling

We recently had some issues trying to play Dolby Vision content on LG TVs where the device would not play at all, due to a discrepancy between the video codec announced by the RxPlayer (which did not make mention of Dolby Vision), and the Dolby Vision data actually pushed to the video buffer.

This was because the media stream we've tested make usage of a smart trick: as the Dolby Vision video data was backward-compatible to regular HDR for devices not supporting Dolby Vision, two codecs were actually announced in the content's MPD (the DASH's Manifest file, which list available tracks and qualities):

The much more compatible HDR codec, as a codecs attribute (e.g. in a <Representation> or <AdaptationSet> element in the MPD) like expected
The Dolby Vision codec, as a scte214:supplementalCodecs property, that we did not process until now

repsupcodec Screenshot: Content of a DASH MPD with both a codecs property and a supplementalCodecs property in the <Representation> element

The RxPlayer previously only relied on the codecs property (so here not the Dolby Vision video codec) when interacting with lower-level media API. Though the HDR codec was compatible to the video data, it was actually expected by some Dolby Vision-capable device that the Dolby Vision codec should have been relied on when creating video buffers.

However, if we did rely on the Dolby Vision codec instead through those API, devices which do not handle Dolby Vision video content would not have been able to play the content as just "regular" HDR video data, for them it would have been Dolby Vision, which they do not know how to decode.

So we chose to put a little more intelligence in the RxPlayer, when there is a scte214:supplementalCodecs property (let's call the corresponding codec property the "supplemental codec"):

If the supplemental codec is supported by the current device (we can know that through the MediaSource.isTypeSupported API, we consider it to be the actual "codec" of the corresponding video data.
If it is not supported, we rely on the codecs instead, just like we did before

suppcodec Flowchart: How the RxPlayer will decide whether to rely on a scte214:supplementalCodecs or a codecs attribute in a DASH MPD.

This also means that API communicating about video codecs, like the getVideoTrack method, may now actually indicate the "supplemental codec" through their codec property, where it always corresponded in DASH to the corresponding codecs attribute instead). This should in most case not lead to any change in your application as this is mainly a lower-level detail about a Representation (a.k.a. media quality).

`getLivePosition` method and `startAt.fromLivePosition` option

Some applications at Canal+ relying on the RxPlayer make use of "ad-switching" technologies. Basically, the idea is to replace ads present in a source stream by targeted ads.

There are multiple techniques to implement this, yet for DASH they generally rely on its concept of Periods. In the MPD (DASH's Manifest file), each of those ads would be defined in a different <Period> element, which is a time-delimited division of the content with its own media tracks and qualities.

multiperiods Schema: Exemple of a schema representing a DASH multi-Period live content, with an ad break where each Period corresponds to a single Ad.

Another effect of ad-switching is that the server may know in advance which ad should be served to a user.

Let's for example consider a user which plays a live content. That user now encounters an ad-switched ad break at 11:04 AM, which will end 5 minutes later, at 11:09 AM. Though it is still 11:04, the server already knows which ads it's going to serve to that user from 11:04 to 11:09 (those ads are targeted and already scheduled), so it may want to already announce them in the MPD even though most of those ads are meant to be played in the future. In turn, the RxPlayer may want to load in advance that new data (reducing the risks of rebuffering comparatively to just loading until the live position) while preventing the user to play further than the "live position" (which is here still 11:04).

livemaxdiff Schema: Illustration of the aforementioned example, where future ads are already announced and loaded in advance. We can see that the maximum position the RxPlayer will load is much further than the live position we usually want the user to play.

Because in that situation. there may be a difference between the maximum reachable position (here the end of the ad break at 11:09 AM) returned by the getMaximumPosition method and the intended live position (here 11:04 AM) relatively to which the user should probably be playing (note: the RxPlayer doesn't enforce that: the initial position is relative to the live position but the user could then technically seek further than it if the application lets him/her do it), we added the getLivePosition method to let applications know about the latter.

Likewise, we added the startAt.fromLivePosition loadVideo option allowing to set an initial playback position relative to the "live position". Previously that role was filled by start.fromLastPosition but that option is only relative to the maximum position, which now might be very different.

The getLivePosition method is documented here and the startAt loadVideo option here.

Note however that the RxPlayer will only make a live position "guess" when there's an explicit clock that has been communicated to it, either through an <UTCTiming> element in the MPD, or through a serverSyncInfos option. In other situations, the RxPlayer will infer the live position from various heuristics, including the last position reachable in the Manifest.

Possibility to set a new `keySystems` when calling `reload`

In the v3.23.0, we added the reload API allowing to re-load the last loaded content at the last played position. The idea was to allow an application to reload a since-stopped or failed content with the same options than it was loaded with.

Video: action of reloading on the demo page. We can see that we restart back roughly from the same position.

Previously, the reload API had an optional argument, which only allowed to update a reloading position, and to indicate whether the content should auto-play after the reload. The idea was that if you needed to set more options, you would generally prefer to rely on the more powerful loadVideo method instead.

However, it turned out that the most frequent issues we encounter in production are DRM-related, and that many of those can be avoided by setting another DRM configuration. The main example would be low-level issues we encounter with Microsoft PlayReady SL3000 (hardware-based DRM) that we do not encounter when relying on PlayReady SL2000 (software-based DRM). In many cases, it's not even possible to guess in advance that SL3000 would fail: those problems are generally linked to PlayReady low-level implementation issues.

Specifically for those cases, we decided to add a keySystems property to the reload API, allowing you to reload the last content at its last played position with an updated DRM configuration. This keySystems property follows the exact same syntax than the one for a loadVideo call, and as such is documented at the same location, here.

This new feature thus allows to "reload" right away a content when encountering unexpected but known DRM issues on a given device, improving the quality of experience for the final user.

Multiple minor `DEBUG_ELEMENT` improvements

We added in v3.30.0 the DEBUG_ELEMENT experimental feature to allow the display of a RxPlayer overlay relying on its internal metadata to provide debugging information. We found it very useful when debugging and so we made some improvements to it in this version.

The most important is that now, undecipherable video and audio qualities are signaled by adding an E! string after the corresponding bitrate in the vb (for "Video Bitrates") and ab (for "Audio Bitrates") parts of the debug element: encrypted2 Screenshot: The RxPlayer experimental DEBUG_ELEMENT as it can be seen when playing a content on https://www.canalplus.com. Here we can see that both the 3400000 and 4500000 video bitrates are linked to undecipherable video contents (they are followed by E!). Consequently we will not be able to play them.

Likewise, unsupported codecs are signaled by adding an U! string after the corresponding quality's bitrate (note that a quality can both be undecipherable and in an unsupported codec, in which case both E! and U! might be present).

We made other cosmetic changes on this debug element, such as ensuring that the "buffer graphs" are readable (by e.g. limiting their scale if the content is very long) and always up-to-date.

v4.0.0-beta.3

6 months ago

Release v4.0.0-beta.3 (2023-10-19)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

:mag: Overview

The v4.0.0-beta.3 release is now here. As usual, it is based on the last official v3 release (here the just released v3.32.1) as well as previous v4 beta releases.

The v4.0.0-beta.3 release should be the last v4 "beta" release, meaning that the next v4-linked release will probably be our first release candidate for a first official v4.0.0 release.

We already tested the code behind this beta.3 release extensively in production conditions on most environments we target at Canal+ (which helped us detect and fix an adaptive-linked issue only seen on older Edge browser versions), so we're becoming confident that this version will work on every supported devices and for a large panel of usages.

As we wanted to ensure that the v4 API is final (or very close to final), we reviewed every RxPlayer v4 API on current usages as well as future planned usages to ensure that API stability is easy to guarantee. This led to some changes, described in this release note.

:bookmark_tabs: Changelog

Changes

The MediaError's trackInfo property is now an array renamed as tracksInfo and similar MediaError are grouped in one [#1264]
The manifestUpdateUrl loadVideo option has been removed as it was unused [#1276]
The /dist directory in the project has been removed [#1270]

Bug fixes

Fix adaptive logic on some legacy Edge browsers [#1302]

Other improvements

newAvailablePeriods is now sent lazily at the time new Periods are considered to improve performance [#1265]
Implement better error messages by not repeating the Error Type in it [#1290]
All import path to the RxPlayer now depend on the same RxPlayer modular build (and not just the minimal, as before) [#1301]

`newAvailablePeriods` event now sent lazily

The newAvailablePeriods event is a new RxPlayer event central to the v4 API indicating that new "Periods" from the current content begin to be considered by the RxPlayer. As each Period brings with it its own set of audio/text/video tracks and qualities, it is the main event to listen to when you want to choose an initial track or quality.

hierarmpd Screenshot: screenshot describing the Period, AdaptationSet and Representation elements of an MPD, which are advertised in the v4 API beginning with a newAvailablePeriods event.

In previous beta and alpha releases, the RxPlayer sent this event just after parsing the content's Manifest (and after refreshing it), by communicating directly about all Periods seen in that Manifest.

We became afraid that on very large Manifest with a large amount of Periods and track choices, the current design for this event could lead to performance issues: as soon as the Manifest was parsed (and before the content started to play), the RxPlayer would send a newAvailablePeriods event - perhaps for the hundreds of Periods seen in that Manifest. The application (the software using the RxPlayer library) would then iterate through all of those Periods, as well as its inner tracks and qualities, to make its initial choice - and all that before the content is finally able to play.

supergraph Graph: Timeline of situations leading to the newAvailablePeriods event and corresponding player actions. It then continues as the setting of the audio, video and text tracks by the application has to be done for all Periods advertised through the newAvailablePeriods event.

This seems risky performance-wise as well as unnecessary. When the content starts to play, the only track and quality choices we probably want to set are just the one linked to the Period we're initially playing. For live contents for example, we don't need to set the audio track for the program that was aired 15 minutes ago. The only time where we might want to set it, is if the user ever seeks back to that program (a situation which might never occur).

The RxPlayer now "lazily" sends newAvailablePeriods events, meaning that it will only communicate about "Period(s)" that are either playing or will be played soon by the RxPlayer. For example when playing a live content, it will probably only send this event for the currently-playing program, not the one playing before or after it.

supergraph2 Graph: Updated timeline now that newAvailablePeriods only transports information on the Period(s) that matter for now. Here we can see that the application for example only set the tracks for a single Period.

If the user seeks to another Period or if playback position reaches a new Period, a new newAvailablePeriods will then be sent for it.

Normally, this doesn't break the previous v4 API so you shouldn't need to change anything if you relied on a previous v4 version. Still, as it is a considerable change, you could need to check if this doesn't break assumptions you previously had on your side.

Changes concerning Errors

Error message changes

Historically, the message property of RxPlayer errors had the peculiar following format:

<Error Type> (<Error Code>) <Error Message>

For example, for an EncryptedMediaError with an INCOMPATIBLE_KEYSYSTEMS code, we could have:

EncryptedMediaError (INCOMPATIBLE_KEYSYSTEMS) Some description message

Packing so much information into the message was unconventional, repeated information already available elsewhere and led to an ugly reporting in debuggging tools. For example, error logged in most inspectors and debuggers are already prepended by the Error Type, leading in the previous example to the following log:

EncryptedMediaError: EncryptedMediaError (INCOMPATIBLE_KEYSYSTEMS) Some description message

Although error messages are not part of our API (so we could have changed its format at any time) we were still reluctant to do so as applications might have relied on that format to extract information programatically. Now that the official v4 is around the corner, we finally updated its format to a more legible:

<Error Code>: <Error Message>

As such the previous error message would be:

INCOMPATIBLE_KEYSYSTEMS: Some description message

And it would be logged by inspectors as:

EncryptedMediaError: INCOMPATIBLE_KEYSYSTEMS: Some description message

`trackInfo` renamed as `tracksInfo`

We made another small change to RxPlayer errors, this time on the recently-added trackInfo property that could be set on some MediaError, to add precizion on the particular track concerned.

We noticed that many of them were sent in bulk, for example when the Manifest is found to contain multiple tracks in unsupported codecs, we would send a warning event dispatching a MediaError with an MANIFEST_INCOMPATIBLE_CODECS_ERROR for each of the unsupported track (and in multi-Period contents, there could be hundreds of them).

Now we decided to group such errors together when they are happening at the same time, and to allow the setting of multiple tracks' information in a tracksInfo array property (with an s, instead of the previous trackInfo). The MediaError documentation has been updated.

Changes concerning the builds

In previous RxPlayer versions, we exposed several builds of the RxPlayer in the npm repository:

The "legacy" build, which is a bundled single JS file, which was used when importing the RxPlayer "normally" through an import RxPlayer from "rx-player" line in ES6-style (or const RxPlayer = require("rx-player") ni CommonJS style).

We relied on the Webpack bundler to produce that build.
The "modular" build, targeted by most other imports (such as the minimal build: import RxPlayer from "rx-player/minimal") which uses multiple files, allowing for "tree-shaking" on the application-side.

Here we mostly rely on TypeScript and on our own scripts.

Having this double way of exporting the RxPlayer led to some complexity on our side and we weren't too comfortable of having too much complexity in such an important area for a library.

We're now only relying on the "modular" build when the RxPlayer is imported. This should be completely transparent to you (if done well!) yet it greatly simplifies on our side the way the RxPlayer is built and distributed to applications.

Legacy single-file builds are actually still done, because they might be useful when for example relying on the "older" way of depending on the RxPlayer as a sepatate <script> tag in an HTML file, but they will now only be attached to release notes such as this one.

Future work on exploiting WebWorkers

We recently put in place on top of the v4 a proof-of-concept of running most of the RxPlayer's logic in a WebWorker (which is a way of exploiting multi-threading capabilities) with the main goal being to offer better performance on some low-end devices, low-latency live contents and/or large Manifests.

Running the RxPlayer without a WebWorker (the default):

+-------------------------------------------------------------------------------+
| Main thread (also running the UI)                                             |
|                                                                               |
| +------------------+     +----------------------+    +----------------------+ |
| |    Application   | --> |  RxPlayer Main [1]   | -> |   RxPlayer Core [2]  | |
| +------------------+     +----------------------+    +----------------------+ |
+-------------------------------------------------------------------------------+

Running with a WebWorker:

+----------------------------------------------------+
| Main thread (also running the UI)                  |
|                                                    |
| +------------------+      +----------------------+ |
| |    Application   | ---> |  RxPlayer Main [1]   | |
| +------------------+      +----------------------+ |
+--------------------------------------|-------------+
                                       | (messages)
+--------------------------------------|-------------+
| WebWorker                            V             |
|                           +----------------------+ |
|                           |  RxPlayer Core [2]   | |
|                           +----------------------+ |
+----------------------------------------------------+


[1] RxPlayer Main: Exposes an API to the application, performs some high-level
media monitoring, handles content decryption, displays text tracks and interacts
with web API that are only usable in the main thread.

[2] RxPlayer Core; Loads and parses the Manifest as well as media segments that
will be be played. Also monitors what's being played and will be played to
ensure a smooth playback.

Simple explanation of what the WebWorker project is about.

The initial results have been very encouraging, with most of the time less rebuffering chances while doing heavy interactions with the user interface than before, so we continued that work with the goal of implementing that possibility in a future RxPlayer version.

We're now almost there. The main remaining issue is that there are some performance regressions when the part of the logic running in main thread is very busy yet the "worker" (the logic running in the other thread) runs at full speed. Sometimes in that situation, the worker thread seems to make it worse for the main thread by overwhelming it with new media data.

Still we're frequently doing releases of that work under the worker tag in npm (the last version being: 4.0.0-worker.2023100500). You can have more information by looking at the corresponding Pull Request.

v3.32.1

6 months ago

Release v3.32.1 (2023-10-19)

NOTE: This is the v3.32.1 and not a v3.32.0 as you could have expected due to a small mistake when publishing our TypeScript types to npm for a v3.32.0. We detected it and fixed it immediately with a new release, the v3.32.1.

:mag: Overview

The v3.32.1 mainly brings stability improvements over the v3.31.0, as we start to focus more and more on the future v4.0.0 release.

On that matter we'll probably soon release the first v4.0.0 release candidate (paving the way for the first official v4.0.0 release), after a long period of alpha and beta releases (the next beta release should come just after this one) where we were still figuring out some API details.

Note that we're however still commited to maintain the v3.x.x releases for some time, at least in terms of providing bug fixes, as we know doing the switch to the v4 may not be in your agenda for now. As such, even when the official v4.0.0 will be released, we will continue publishing some v3.x.x releases, only with a special npm tag (e.g. rx-player@v3).

:bookmark_tabs: Changelog

Features

DASH: add optional isSpatialAudio boolean property to Representation returned by getAvailableAudioTracks, getAudioTrack, corresponding events, and trackInfo optional property of MediaError objects to signal Dolby Atmos techology [#1275]
LOCAL: add isSpatialAudio property to Representation of the experiment "local" transport (used for offline playback) [#1275]
addFeatures static method is now available on all RxPlayer builds. It was previously only in the minimal (rx-player/minimal import path) [#1287]
The NATIVE_TEXT_BUFFER, HTML_TEXT_BUFFER and IMAGE_BUFFER features are now totally optional [#1287, #1293]

Bug fixes

Fix setVideoBitrate and setAudioBitrate API which may have led to a higher quality than wanted in the default "seamless" manualBitrateSwitchingMode if our buffer-based adaptive logic decided to [#1267, #1271]
On the PlayStation 5, only switch to the "LOADED" state once the HTMLMediaElement's readyState of 4 has been reached, as it seems to switch to 3 too soon there [#1257]
DASH: Fix potential track duplication if more than two AdaptationSet have an adaptation-set-switching <SupplementalProperty> between one another [#1279]
DASH-WASM: availabilityTimeOffset is actually a floating number [#1278]

Other improvements

Do not load the last text segment if the current position goes after it as it is unnecessary [#1256]
Implement better NetworkError messages [#1274]
Set a better error message for when no keySystems option is set when playing an encrypted content
Fix very small memory leak when reloading a content [#1286]
Re-check for segments to load immediately after the manifest has been refreshed [#1282]
When "fallbacking" an undecipherable Representation, now empty the whole buffer if we can't make out where content was in the buffer [#1283]
Improve segment start detection in buffer when there's unknown data buffered before it [#1284]
DRM: Selection of alternative EME API like those used on IE11 or Safari has been refactored to facilitate future developments [#1261]

Deprecated

Deprecate the manifestUpdateUrl loadVideo option as it doesn't seem used anymore [#1288]
Deprecate the NATIVE_TEXT_BUFFER, HTML_TEXT_BUFFER and IMAGE_BUFFER features as they are now unneeded [#1287, #1293]

New `isSpatialAudio` property

This v3.32.1 release brings the new isSpatialAudio property to audio tracks on the following API:

getAvailableAudioTracks
getAudioTrack
availableAudioTracksChange
audioTrackChange
The trackInfo property for audio tracks linked to a MediaError
On the representation given in the Manifest of a "local" transport (provided by the experimental LOCAL feature to play locally-stored contents).

This property is for now only set to true when an audio track relies on Dolby Atmos under the "Dolby Digital Plus Joint Object Coding" (or JOC) technology, which allows to distribute Dolby Atmos audio content in a format compatible with Dolby Digital Plus audio (most probably under the "ec-3" codec), which is Dolby's recommended way of distributing DASH contents with Dolby Atmos audio.

So in short, for now isSpatialAudio is only set to true when an audio track relies on Dolby Atmos technology. For any other cases for now, it is not defined (a value of false would semantically mean that we're sure that the audio track is not spatial audio, which is a guess we prefer not to make for now).

Due to what's allowed in DASH, the isSpatialAudio property is set on "representations" (a.k.a. qualities), not on the track itself:

function hasSpatialAudio(audioTrack) {
    if (!audioTrack) {
        return false;
    }
    return audioTrack.representations
        .some(representation => representation.isSpatialAudio === true);
}

if (hasSpatialAudio(player.getAudioTrack()) {
    console.log("We're currently playing an audio track with spatial audio");
}

The majority of the development on this feature has been provided by an external contributor, @klatoszewski-oke, which we thank again.

`addFeatures` now globally available

On previous RxPlayer releases you were forced to rely on the minimal RxPlayer build to use any of its experimental features like DASH_WASM (WebAssembly-based DASH MPD parser), LOCAL_MANIFEST (playback of offline contents), DEBUG_ELEMENT (debugging UI) or METAPLAYLIST (client-side content generation).

If instead you used the default RxPlayer build, for example through a simple import RxPlayer from "rx-player"; import in your files, you were just left with the default set of features.

Now any build of the RxPlayer can directly call the addFeatures static method to add any of those optional features. For example to add the DEBUG_ELEMENT feature (and thus be able to call the createDebugElement method), you can write:

// 1. Add `DEBUG_ELEMENT` feature
import RxPlayer from "rx-player";
import { DEBUG_ELEMENT } from "rx-player/experimental/features";

RxPlayer.addFeatures([DEBUG_ELEMENT]);

// 2. Create RxPlayer instance and call method
const player = new RxPlayer({ /* ... */ });
player.createDebugElement(myElement);

Consequently, the features documentation has been moved from the "Getting started" part to our regular "API" documentation.

Deprecation of the `NATIVE_TEXT_BUFFER` and `HTML_TEXT_BUFFER` features

Both the NATIVE_TEXT_BUFFER feature and the HTML_TEXT_BUFFER features, which are the arguments of the addFeatures static method, have been deprecated as they are now unneeded.

They're basically optional now that importing a text parser for the corresponding type of buffer is going to implicitely add the feature anyway.

This means that if you were previously doing:

import RxPlayerMinimal from "rx-player/minimal";
import { HTML_TEXT_BUFFER, HTML_TTML_PARSER } from "rx-player/features";
RxPlayerMinimal.addFeatures([HTML_TEXT_BUFFER, HTML_TTML_PARSER]);

You can now just omit the HTML_TEXT_BUFFER feature:

import RxPlayerMinimal from "rx-player/minimal";
import { HTML_TTML_PARSER } from "rx-player/features";
RxPlayerMinimal.addFeatures([HTML_TTML_PARSER]);

As you have no reason to import one of those features without at least one parser, making it optional and implicit will generally just simplify your code.

Many small fixes

This v3.32.1 brings a lot of small fixes and improvements, that you can see on the changelog on top of this release note.

A good chunk of them were found as we were extensively testing both the future v4.0.0 as well as our proof-of-concept of trying to run the RxPlayer's internal logic in a WebWorker (more details on this hopefully soon).

We also reinforced our debugging capabilities by continuing our work on RxPaired, our remote debugger specialized for the RxPlayer that we're using internally at Canal+. This helps a lot when debugging the RxPlayer on devices such as smart TVs, chromecast and game consoles which often have their own issues. It helped us on many of those fixed bugs.

v4.0.0-beta.2

10 months ago

Release v4.0.0-beta.2 (2023-06-27)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

:mag: Overview

The new v4 beta release, based on the v3.31.0, is here. It contains all improvements from previous v4 alpha and beta releases, as well as some further improvements mostly related to DRM, all described in this release note.

About v4 beta releases

As a reminder, beta v4 versions are RxPlayer pre-releases (as some minor API changes are still done, see changelog) for the future official v4 release, a successor to the current v3 releases of the RxPlayer.

We're currently testing it on several applications. As we're doing it, our team as well as people porting it can propose some small API improvements, which may then be added to the next beta releases. After enough people have made the switch and are satisfied with the new API, the first official v4 release will be published (we're also in the process to have some applications running it in production to ensure its stability with a large enough population).

We will still continue maintaining and providing improvements to the v3 for at least as long as the v4 is in beta (and we will probably continue to provide bug fixes for the v3 for some time after the official v4.0.0 is released). This process is long on purpose to be sure that we're providing a useful v4 API for applications and also to avoid alienating application developers, as the migration from the v3 might take time.

:bookmark_tabs: Changelog

Changes

If all Representations from the current track become undecipherable, automatically switch to another track (also send a trackUpdate event) instead of stopping on error [#1234]
Only send MediaError errors with the NO_PLAYABLE_REPRESENTATION error code when no Representation from all tracks of a given type can be played [#1234]

Features

Add representationListUpdate event for when the list of available Representation for a current track changes [#1240]
Add "no-playable-representation" as a reason for trackUpdate events when the track switch is due to encrypted Representations [#1234]

Other improvements

DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer to work-around rare encryption-related issues [#1236]

`NO_PLAYABLE_REPRESENTATION` behavior change

In the v3 and previous v4 beta releases, if you could not play any Representation (i.e. quality) from your chosen audio or video track due to encryption matters, you would obtain a MediaError error with the NO_PLAYABLE_REPRESENTATION error code.

Stopping on error when no quality of the chosen track can be played seemed logical at the time, but we're now encountering use cases where it would be better if the RxPlayer automatically took the decision to change the current track instead, to one that perhaps has decipherable Representation(s). The main example we encountered was cases where we had separate video tracks, each linked to another dynamic range (e.g. an HDR and a SDR track) and with different security policies (tracks with a high dynamic range would have more drastic security policies for example). Here, I would guess that an application would prefer that by default we switch to the SDR video track if no Representation in the HDR one is decipherable, instead of just stopping playback with a NO_PLAYABLE_REPRESENTATION error.

Before:
-------

+----------------+
|    Selected    |  Not decipherable
|   Video Track  |  --------------------> NO_PLAYABLE_REPRESENTATION
| (example: HDR) |                        Error
+----------------+

Now:
----

+----------------+                        +---------------------+
|    Selected    |  Not decipherable      |      Automatic      |
|   Video Track  |  --------------------> |     fallback to     |
| (example: HDR) |                        | another video Track |
+----------------+                        |   (example: SDR)    |
                                          +---------------------+

Note that the NO_PLAYABLE_REPRESENTATION error might still be thrown, only now it is when no Representation of all tracks for the given type are decipherable.

New `trackUpdate` `reason`

Because an application might still want to be notified or even stop playback by itself when the initially-chosen track has no playable Representation, we also brought added the "no-playable-representation" reason to the trackUpdate event, which indicates that the current track for any Period of the current content was updated due to this situation.

player.addEventListener("trackUpdate", (payload) => {
  if (payload.reason === "no-playable-representation") {
    console.warn(
      `A ${payload.trackType} track was just changed ` +
      "because it had no playable Representation"
    );
  }
});

New `representationListUpdate` event

This new beta version of the v4 also brings a new event: representationListUpdate.

The problem without this event

Let's consider for example an application storing information on the currently available qualities (a.k.a. Representation) for the chosen video track of the currently-playing Period (said another way: being played right now). With the v4 that application can simply get the initial list of Representation when that Period begins to be played through the periodChange event and update this list any time the video track changes, by listening to the trackUpdate event:

let currentVideoRepresentations = null;

function updateVideoRepresentations() {
  const videoTrack = player.getVideoTrack();
  if (videoTrack === undefined || videoTrack === null) {
    currentVideoRepresentations = null;
  } else {
    currentVideoRepresentations = videoTrack.representations;
  }
}

// Set it when the Period is initially played
player.addEventListener("periodChange", () => {
  updateVideoRepresentations();
});

// Set it when the video track is changed
player.addEventListener("trackUpdate", (t) => {
  // We only want to consider the currently-playing Period
  const currentPeriodId = player.getCurrentPeriod()?.id;
  if (t.trackType === "video" && t.period.id === currentPeriodId) {
    updateVideoRepresentations();
  }
});

// Remove it when no content is loaded
player.addEventListener("playerStateChange", () => {
  if (!player.isContentLoaded()) {
    currentVideoRepresentations = null;
  }
});

This seems sufficient at first.

But now let's consider that we're playing encrypted contents, and that one of the Representation of those current video tracks became un-decipherable at some point (e.g. after its license has been fetched and communicated to your browser's Content Decryption Module). Here, an application won't be able to select that Representation anymore, so it generally will want to remove it from its internal state. However, there was no event to indicate that the list of available Representations had changed when neither the video track itself nor the Period has changed

The new event

We thus decided to add the representationListUpdate event. Exactly like the trackUpdate event, it is only triggered when the Representation list changes, i.e. not initially, when the track is chosen.

So taking into consideration that point, the previous code can be written:

let currentVideoRepresentations = null;

function updateVideoRepresentations() {
  const videoTrack = player.getVideoTrack();
  if (videoTrack === undefined || videoTrack === null) {
    currentVideoRepresentations = null;
  } else {
    currentVideoRepresentations = videoTrack.representations;
  }
}

// Set it when the Period is initially played
player.addEventListener("periodChange", () => {
  updateVideoRepresentations();
});

// Set it when the video track is changed
player.addEventListener("trackUpdate", (t) => {
  // We only want to consider the currently-playing Period
  const currentPeriodId = player.getCurrentPeriod()?.id;
  if (t.trackType === "video" && t.period.id === currentPeriodId) {
    updateVideoRepresentations();
  }
});

// Remove it when no content is loaded
player.addEventListener("playerStateChange", () => {
  if (!player.isContentLoaded()) {
    currentVideoRepresentations = null;
  }
});

// What's new:
// Set it if the list of Representation ever changes during playback
player.addEventListener("representationListUpdate", (r) => {
  // We only want to consider the currently-playing Period
  const currentPeriodId = player.getCurrentPeriod()?.id;
  if (t.trackType === "video" && t.period.id === currentPeriodId) {
    updateVideoRepresentations();
  }
});

This event is documented here.

Note about its usability

While developping that feature, we thought that its usage could be simplified if the representationListUpdate event was also sent when the track was initially chosen. We would here have no need in the previous code examples to also listen to trackUpdate events, as the representationListUpdate event would also be sent during track change.

However, it appeared to us that also sending such events on the initial track choice could quickly become complex in the RxPlayer's code, due to all the side-effects an event listener can perform (for example, you could be changing the video track inside your representationListUpdate listener, which would then have to be directly considered by the RxPlayer). Handling all kinds of side-effect inside the RxPlayer was possible, but it would have brought very complex code and potentially performance inneficiencies.

We thus decided to keep this behavior of sending that event ONLY if the Representation list changes, meaning that application developpers will most of the time also need to react to at least one other event for knowing about the initial Representations, like we did in our examples with periodChange and trackUpdate listeners.

Automatic reloading when stuck on DRM issue

The need for a last-resort solution

The majority of RxPlayer issues are now DRM-related and device-specific, generally its source being platform lower-level (CDM, browser integration...) bugs. Even if we frequently exchange with partners to obtain fixes, this is not always possible (sometimes because there are too many people relying on the older logic and thus risks in changing it, sometimes because there are a lot of higher priorities on their side).

We are now encountering relatively frequently, for some contents with DRM, what we call a "playback freeze": playback does not advance despite having data in the buffer, all being known to be decipherable.

Screenshot: Example of what we call a "freeze". We have data in the buffer, yet the position is stuck in place". In v4 versions, we have there the "FREEZING" player state.

Recently we've seen this similar issue with specific contents on Microsoft Edge, UWP applications (Windows, XBOX...) and LG TV. In all of those cases, what we call "reloading" the content after the license has been pushed always fixes the issue. The action of reloading means principally to re-create the audio and video buffers, to then push again segments on it. Reloading leads to a bad experience, as we might go to a black screen in the meantime. Thankfully, the issue mostly appeared at load, where reloading is much less noticeable.

Although we prefer providing more targeted fixes or telling to platform developers to fix their implementation, this issue was so frequent that we began to wonder if we should provide some heuristic in the RxPlayer, to detect if that situation arises and reload as a last resort mechanism in that case.

Risks and how we mitigate them

What we are most afraid here is the risk of false positives: falsely considering that we are in a "decipherability freeze" situation, where it's in fact just a performance issue on the hardware side or an issue with the content being played.

To limit greatly the risk of false positives, we added a lot of rules that will lead to a reload under that heuristic. For the curious, here they are (all sentences after dashes here are mandatory):

we have a readyState set to 1 (meaning that the browser announces that it has enough metadata to be able to play, but does not seem to have any media data to decode)
we have at least 6 seconds in the buffer ahead of the current position (so compared with the previous dash, we DO have enough data to play - generally this happens when pushed media data is not being decrypted).
The playhead (current position) is not advancing and has been in this frozen situation for at least 4 seconds
We buffered at least one audio and/or video Representation with DRM.
One of the following is true:
1. There is at least one segment that is known to be undecipherable in the buffer (which should never happen at that point, this was added as a security)
2. There are ONLY segments that are known to be decipherable (licenses have been pushed AND their key-id are all "usable") in the buffer, for both audio and video.

If all those conditions are "true" we will reload. For now with no limit (meaning we could have several reloads for one content if the situation repeats). Note that at the request level, this only might influence segment requests (which will have to be reloaded after 4 seconds) and not DRM-related requests nor Manifest requests.

Results

We actually tested that logic for some time on most devices to a large population. After some initial tweaking, we seem to have "fixed" the more difficult issues with no much false positive left.

This has encouraged us to include it in this v4 beta release so it can be present in the first official v4.0.0 release. Note that we cannot simply bring this improvement also to the v3 under SEMVER rules, as "reloading" without a specific new API is a v3 breaking change (the RELOADING state didn't exist in the `v3.0.0).

v3.31.0

11 months ago

Release v3.31.0 (2023-06-14)

:mag: Overview

The v3.31.0 release is now here!

It brings:

Many new API to improve developer experience when relying on the RxPlayer: isContentLoaded, isBuffering, isPaused, play and paused events, getLastStoredContentPosition and the trackInfo property on error
A refactored adaptive logic allowing to limit the frequency of quality switches, generally seen as a poor experience and which could lead to rebuffering in some extreme situations
A fix for the "direct" audioTrackSwitchingMode which in some situations could lead to the absence of sound when changing the audio track
A fix for DASH multi-Period contents, where we could be led to an infinite rebuffering cases in-between two periods in some conditions
"forced" subtitles support in "directfile" mode on Safari
Some Playstation 5 fixes to avoid playback issues on this platform
Better error reporting when an issue arises when initializing the DASH_WASM experimental Webassembly-based MPD parser

The future v4.0.0-beta.2 release, which will be based on this one alongside v4 features, will be released soon (in the coming days).

:bookmark_tabs: Changelog

Features

Add isContentLoaded, isBuffering, isPaused, and getLastStoredContentPosition methods [#1248]
Add play and paused events [#1253]
Add trackInfo property to some MediaError to expose information on the track that caused the error [#1241]

Bug fixes

DASH: Fix issue which could lead to infinite rebuffering when switching between multiple Periods [#1232]
Return actual ending duration through the getVideoDuration method when playing dynamic contents whose future end is already known [#1235]
DASH/WASM: actually reject the DASH_WASM.initialize's Promise if it fails [#1238]
On the PlayStation 5, set Infinity MediaSource duration for live contents to prevent playback issues [#1250]

Other improvements

adaptive: Perform various adaptive tweaks to avoid switching too much between qualities in some conditions [#1237]
Directfile: Detect "forced" subtitles on Safari when playing directfile contents (such as HLS) [#1239]
Improve "direct" audioTrackSwitchingMode compatibility by re-seeking [#1246]
The DEBUG_ELEMENT feature now uses the monospace fallback font as a default for a better rendering on apple devices
doc: externalize documentation-generator code

`isContentLoaded` and `isBuffering` method

Until now, an application had to obtain the player's state (either through the getPlayerState method or through the "playerStateChange" event) to know whether a content was loaded, buffering, ended and so on. This is fine but we thought that we could do better in terms of API.

Amongst player states, we noticed that some are frequently grouped together in some situation.

Consequently, we added the two following methods to the RxPlayer API in the v3.31.0.

isContentLoaded: Most states, with the only exceptions of the "LOADING", "RELAODING" and "STOPPED" states, could all be grouped together to indicate that a a content is currently loaded.

In turn, knowing that a content is or isn't loaded is important because when none is loaded, many methods such as setAudioTrack, setVideoBitrate and so on cannot be called.

https://github.com/canalplus/rx-player/assets/8694124/ced7fda5-39ca-4c15-8f44-384c633b2b59

Video: The three cases where a video is not considered loaded are visible in this video: 1. When loading a content 2. When reloading it (here we have changed the video bitrate while the manualBitrateSwitchingMode option was set to "reload") 3. When stopped

Because detecting any of the corresponding states is important in the RxPlayer API, we chose to add the isContentLoaded method to the RxPlayer's API, which will return true when a content can be considered as loaded (which is exactly the same than checking the corresponding states, though it is now more explicit).

This method is documented here.
isBuffering: Most applications want to show a visual indicator such as a spinner for when a content is loaded or being loaded, but not enough data is buffered for playback to happen. This is most commonly called "buffering".

https://github.com/canalplus/rx-player/assets/8694124/ecbd4aab-8621-4492-973e-9300b15971db

Video: Rebuffering cases both when loading and after seeking in a content.

This corresponds to the "SEEKING" (buffering due to a seek operation), "LOADING" (initial buffering), "RELOADING" (buffering due to the necessity to re-create media buffers) and "BUFFERING" (other regular cases of buffering) states.

The v3.31.0 version of the RxPlayer now adds the isBuffering method to the RxPlayer API, which will return true when the current player's state corresponds to any of those.

Note however that you still may want to divide, depending on your applications further those states. For example the "LOADING" and "RELOADING" states may both be associated to a black screen, whereas you'll most likely have the last frame displayed when encountering the "BUFFERING" and "SEEKING" states. So, you might still want to rely on the general state depending on what you want to do in your application.

This method is documented here.

`isPaused` method

The isPaused method may initially seem similar to the isContentLoaded and isBuffering method but unlike those two, which just facilitate the exploitation of the RxPlayer's state, isPaused actually adds a functionality.

In a media player, you frequently want to know whether the RxPlayer is "technically paused" or not, for example to know whether you should display the play or pause button in your user interface.

pause_button play_button

Screenshots: Two schreenshots of a player's control bar. In each of them, we can see at the bottom left the button which may represent a "play" order or a "paused" order depending on if the playback is currently playing or not. Here the top one display a "pause" button (and thus we're probably currently playing) whereas the bottom one display a "play" button (we're probably currently paused).

Although the "PAUSED" player state exists, it isn't sufficient to determine this. For example, you could technically be seeking, and thus in the "SEEKING" state, yet paused (which means that once enough data has been loaded you won't be playing, but paused).

Based on this, the player's state is not sufficient to determine whether playback is paused or not. Until now, applications generally kept track of their last play/pause calls and of the last RxPlayer states to deduce whether playback was paused.

However, it should be easier and less error prone to obtain such an important information directly and explicitly from the RxPlayer. This can now be done through the isPaused method which will return false when playback is considered as paused, regardless of the player's state.

const isPaused = rxPlayer.isPaused();
if (isPaused) {
  console.log("playback is currently paused");
} else {
  console.log("playback is currently not paused");
}

This method is documented here.

`"play"` and `"pause"` events

Because now that you have the new isPaused method you may also want to know when the RxPlayer enters or exits a pause.

For example to properly implement a "play/pause button", you might want to switch it reactively after a play or pause call has been processed. To allow this, two new RxPlayer events have been added:

"play" (for when playback is resumed after being paused) and
"pause" (for when playback is being paused).

https://github.com/canalplus/rx-player/assets/8694124/bfef47ab-bdf8-4b3c-8a25-9664c99e422f

Video: RxPlayer's demo page with a console opened logging "play" and "pause" events. We can see it corresponds to when the play/pause button is clicked in this case.

Note that this event is only sent when exiting or entering a pause from the time the content has been loaded by the application (e.g. from the loadVideo or reload call). As such, it won't be sent for the initial isPaused value right after loading the content. Here, isPaused should return false if the autoPlay option has been set or true if not.

The play event is documented here and the pause event is documented here.

`trackInfo` property on some `MediaError`

Errors and linked tracks

There's a vast list of errors that the RxPlayer may trigger when encountering minor (through "warning" events) or major (through "error" events) issues.

Some of those errors make more sense when linked to a track.

For example, when receiving an error with the "MANIFEST_INCOMPATIBLE_CODECS_ERROR" code, indicating that a track had none of its Representation (i.e. quality) in a codec compatible to your device, it would be nice to know whether we're talking about an audio or video track, which characteristics has this track, and so on.

The `trackInfo` property

To describe characteristics of tracks linked to some errors, the v3.31.0 release adds a trackInfo property to some MediaError errors, only when its code property is set to either:

- "NO_PLAYABLE_REPRESENTATION" (unplayable track based on DRM policy non-compliance)

- "MANIFEST_INCOMPATIBLE_CODECS_ERROR" (track with unsupported codec)

- "BUFFER_APPEND_ERROR" (error while pushing a segment)

- "BUFFER_FULL_ERROR" (error while pushing a segment which seems specifically linked to that buffer not having enough memory space left)

That trackInfo property provides characteristics about the track linked to the error, for example whether it is a video or audio track, its codec(s), dynamic range information, language, accessibility features and so on.

It is documented in the Player Errors documentation page.

`getLastStoredContentPosition` method

The v3.31.0 version also adds a very specific method, getLastStoredContentPosition, which returns the last position stored by the RxPlayer for the last loaded content.

This description might seem cryptic, and its difference to the much more useful getPosition method may initially appear unclear, but there is a simple reason why both may be useful at different point in time.

getPosition is very useful when a content is loaded (i.e. not in the "STOPPED", "LOADING" or "RELOADING" player state). But when no content is loaded, it just returns 0.

However there might be cases where you want to obtain the last known position of a content the player was at just before it actually stopped (or reloaded). One frequent such cases is when encountering an error: you might want to know at which playback position the user was when encountering the error, but when the "error" event is triggered, playback is already stopped and as such, getPosition will already return 0.

https://github.com/canalplus/rx-player/assets/8694124/22d3031a-a095-4d8e-bd5c-ba036642f2f1

Video: Example of usages of both the getPosition and getLastStoredContentPosition methods. We can see that the former returns 0 when the content is stopped while the latter returns its last known played position.

The getLastStoredContentPosition method thus allows to know which position was reached the last time the last content was considered loaded. If no content was loaded since the creation of the RxPlayer instance, it will return undefined.

This method is documented here.

Adaptive tweaks

The v3.31.0 was also the occasion to improve our "adaptive logic" (also commonly called "ABR" for Adaptive BitRate).

The adaptive logic is the part of the RxPlayer which will select the right Representation (a.k.a. quality) depending on the user's playback condition, one of the most important factor being the calculated network bandwidth.

We noticed that for some rare users, the Representation selected by that logic was changing too frequently. We try to prevent frequent quality switches because it generally leads to a less enjoyable experience than just staying on a stable one. It also may lead to some buffering cases in some extreme situations explained below.

More non-urgent quality switches

When the RxPlayer's adaptive logic decides that we need to switch to another quality it can ask to do so in one of two ways:

Switch to the wanted quality immediately, aborting requests for the previously-selected quality. That's what we call an "urgent" switch.

This can be wasteful because the segment's request might have been close to be finished, but it allows a quicker quality transition.
Switch to the wanted quality only once the requests for the previous ones have finished. That's what we call a "non-urgent" switch.

Here we will consequently "wait" a little before switching the quality but requests which already have been started and soon finished will be pushed, reducing rebuffering risks.

https://github.com/canalplus/rx-player/assets/8694124/80eb270d-219c-4806-8966-d1bffa7082b6

Video: Here after loading a content, we can see that seg-4.m4f and seg-9.m4f are set to canceled in the Status column (this table is part of Google Chrome's inspector). They both have been aborted here because of an urgent quality switch.

Before, the choice of whether we should perform an urgent or non-urgent switch was based on a complex logic using many inputs:

How much buffer we have left
When the current requests started
The expected duration of requests in the previous quality
The difference between the previous and new quality

The logic seemed technically logical most of the time, but it had a risk of leading to rebuffering in some situations. For example, if for some reasons you alternated between multiple qualities where each switch is an urgent one, you might be left in a situation where requests keep being interrupted.

To limit that event from occuring, the v3.31.0 version of the RxPlayer now only performs non-urgent switches when raising up in quality. The logic being that if we raise up in quality, the request for the previous one should finish suffliciently fast and should not lead to rebuffering.

Buffer-based logic with more guards

The RxPlayer adaptive logic is mostly based on:

bandwidth calculations - through what we call a bandwidth-based algorithm - and
on observing how the buffer fills up at various quality - through what we call a buffer-based algorithm.

Each algorithm has its strengths and weaknesses, which is why the RxPlayer uses both, generally mostly relying on the bandwidth-based algorithm when there's few data in the buffer and on the buffer-based once the buffer is filled-up.

The buffer-based algorithm has a particularity that it can lead to a better quality than what the user could be expected to have based on its network-throughput. This is not a problem in itself, but it could lead to many quality transitions if the currently-chosen quality is very high relative to the user's bandwidth.

Thus, we now added guards in our buffer-based logic, ensuring that we do not go to high in quality if the user's bandwidth can't keep up, even if we have a lot of buffer. We do that by observing whether each chosen quality is "maintainable" (meaning that it can be loaded faster than it can be played). If it isn't, we will forbid further quality raise.

Updated detection for times where the bandwidth fall

We have a logic detecting sudden fall in bandwidth while a segment is still being loaded.

Previously, if we had not much data left in the buffer AND if the pending request for the next needed segment led us to think that we will rebuffer for at least 2 seconds, we re-calculated a quick-and-dirty bandwidth based on that last request's progress alone.

This led to a temporary very poor (as in: imprecise) bandwidth calculation, which was only corrected once enough new segments where loaded. This seems fine but was actually a source of a high rhythm of quality fluctuations in some unstable networks, which we want to avoid.

The bandwidth fall detection is now triggered much less often, and we will only reset to a poor bandwidth calculation if the situation appears to be desperate enough (several seconds of estimated rebuffering time and the request was pending for much more time than expected).

This may mean that the RxPlayer will realize later when the bandwidth truly has immediately fallen, but it has also the bigger advantage of not risking an oscillating quality only because a single request took more time than expected.

More pessimistic bandwidth-based algorithm in general

When producing bandwidth estimations, we actually use a "factor", which we multiply with the currently-calculated network bandwidth to obtain the quality we should play.

Previously, the value of this factor depended on the size of the current available buffer but was between 0.72 (for when the buffer is low) and 0.8 (for when the buffer is high).

The idea was too avoid rebuffering as much as possible by being sure that the currently-chosen quality could be loaded fast enough. Depending on if there's a lot of buffer left or not, the rebuffering risks are not the same and consequently, we may apply a low factor when the risk is high but still allows to switch to a slightly higher quality if there's some buffer left.

This factor variability might have led to too much quality switches if the buffer is low. Consequently, we now aligned it to 0.72 regardless of the size of the buffer.

"forced" subtitles on Safari

The last RxPlayer version, v3.30.0, brought better support of "forced" narrative subtitles (basically, those are subtitles meant to be displayed when no other subtitles are) by adding the forced boolean property to text track information, for example returned by the getAvailableTextTracks API.

forced_dash forced_hls

Images: Semantic of signaling forced subtitles in both DASH (on top), and HLS (bottom). Though the wording is different, they correspond to roughly the same concept: subtitles that should be displayed by default.

However, this addition wasn't added for "directfile" contents, which are content playable natively on the current browser, such as mp4/webm files of even HLS contents on Safari. This is because the RxPlayer rely mostly on the browser to play and parse the characteristics of those contents (unlike other "transports", such as "dash", where the RxPlayer loads and parses the various media-related files).

Though it has been a long asked browser feature (see also here), browsers don't actually specify if a given text track is a forced narrative subtitles as no real solution has been found to satisfy everyone so no specification actually exists. Consequently, those text tracks will usually be choosable through API such as getAvailableTextTracks, but you won't be able to know, through its properties, whether it is a forced one.

As most contents played through the RxPlayer in "directfile" are simple contents (more complex ones usually being packaged as DASH contents) without forced subtitles, this is generally not that big of a problem. But we have at least one situation at Canal+ where it was problematic: when playing HLS contents on Safari. There, we might have cases where we do have forced subtitles available, and we would prefer to reliably report those tracks as such to an application.

Thankfully in this case, Safari implemented that feature even if no specification exists for now. On safari, forced narrative subtitles have a kind property set to forced.

This lets us now properly advertising forced narrative subtitles on Safari when playing HLS contents in "directfile" mode, by reusing the same API than for other types of contents.

Better "direct" `audioTrackSwitchingMode`

The RxPlayer has an option called audioTrackSwitchingMode allowing to configure the behavior the RxPlayer should have when changing the audio track during playback.

https://github.com/canalplus/rx-player/assets/8694124/37796456-9fd2-43e7-b256-87b71d7d3ce2

https://github.com/canalplus/rx-player/assets/8694124/9f7fa01f-3343-4a4a-8a54-90c5258c2fab

Videos: On top, example of an audio track switch after setting audioTrackSwitchingMode to "seamless": we have here a smooth transition but some small time in the previous audio track before switching. On bottom, example of an audio track switch after setting audioTrackSwitchingMode to "direct": the track switch is effective immediately, but we may have some rebuffering before.

Some RxPlayer users had an issue of inaudible audio after an audio track switch on Chrome when the audioTrackSwitchingMode option was set to "direct". That issue seems to be at the browser-level (or may even be at a lower level), but as it can be prevented in the RxPlayer with no much disadvantage, it is still a win to work-around it at the RxPlayer-level.

However, we lately found a new technique which seems to fix this issue reliably when the following steps are ttaken:

The application switches the audio track through the player's API
The player stops loading the old audio track and begins removing all audio data previously buffered.
Once the browser validate this removal operation, the player performs a seek to a very close position, just to force the "flushing" of lower-level buffers
The audio data from the new audio track is now pushed (after it has been loaded of course) to the audio buffer.
Once that push operation is validated in the buffer, the player seeks again to force a flush again.

The first flush thus prevent from still having the lower-level buffer considering old data in the previous track, the second one fixes the muted audio issue - for some reasons. Most of the time no interruption is visible/audible and when it is, it just seems linked to the audio track switch and does not seem shocking.

For the RxPlayer, only the first flush was performed until now. This commit now adds the second one, which seems to fix all inaudible audio issues.

As the RxPlayer's is a very technical library with a large API, we try to provide a documentation that is both complete (to allow application developers to use it to its full potential) and pleasant to browse.

A recent improvement, that we already rolled out for some time on the online documentation, was to implement what is commonly called "soft navigation". The idea is to avoid reloading the page when most internal links are clicked on, and even pre-loading the future page as soon as the link is hovered (of course, this is implemented intelligently to avoid duplicate requests, memory overloading, and so on).

That way, the documentation may now appear snappier as well as more usable, keeping the state of a documentation page when clicking on an internal link. For example, opened sub-groups of pages on the left-side of the page is not resetted anymore each time a page navigation is done, which was before a known pain of documentation users.

https://github.com/canalplus/rx-player/assets/8694124/a27be83e-850e-4569-9b5c-a809285c9262

Video: example of navigating between documentation pages fast thanks to soft navigation. Notice that the selected text on the left side stay selected after navigation, whereas previously the whole page was reloaded after clicking.

We hope that this new functionality improves your experience when browsing the RxPlayer's documentation pages.

v4.0.0-beta.1

1 year ago

Release v4.0.0-beta.1 (2023-03-08)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

Overview

This is an update of the previous v4.0.0-beta.0 release.

Compared to it, the v4.0.0-beta.1:

is now based on the v3.30.0 stable release of which it inherits all features, bug fixes and improvements (and not a partial list of them like for the beta.0)
Fix a memory leak just seen on the v4.0.0-beta.0
Provide other minor fixes over the v4.0.0-beta.0 (see the Changelog below)

Changelog

Bug fixes

(v4.0.0-beta.0-only issue) Fix memory leak
(v4.0.0-beta.0-only issue) Fix MediaSource duration when the maximum buffered end is inferior to current duration but superior to calculated duration
(v4.0.0-beta.0-only issue) Fix stopAtEnd option by also switching to STOPPED state on ended
(v4.0.0-beta.0-only issue) Fix some target's support by not relying on Promise.prototype.finally anymore [#1224]
(v4.0.0-beta.0-only issue) For dynamic contents, always set a very high duration [#1220]
(v4.0.0-beta.0-only issue) DRM: Fix fallbacking for an already-played content by checking key statuses initially linked to a MediaKeySession

Other improvements

Based on the v3.30.0 of which it inherits all the features, bug fixes and other improvements

Rx Player Versions Save

v4.0.0

Release v4.0.0 (2024-02-21)

:mag: Overview

:bookmark_tabs: Changelog

About the v3

A more flexible track API

Improved Representation selection

The new MULTI_THREAD experimental feature

More expressive decryption options

onKeyOutputRestricted / onKeyInternalError

persistentState / distinctiveIdentifier

videoCapabilitiesConfig / audioCapabilitiesConfig

A new player state: "FREEZING"

v3.33.2

Release v3.33.2 (2024-02-21)

:mag: Overview

:bookmark_tabs: Changelog

Bug fixes

Other improvements

v4.0.0-rc.2

Release v4.0.0-rc.2 (2024-02-07)

Overview

Changelog

Features

Bug fixes

Other improvements

DASH_WASM and MULTI_THREAD fix

attachWorker now returns a Promise

Project repository updates

v4.0.0-rc.1

Release v4.0.0-rc.1 (2024-01-24)

:mag: Overview

:bookmark_tabs: Changelog

What about the v3?

v4 summary

The new MULTI_THREAD experimental feature

Breaking change: volume APIs

A new requestConfig property: connectionTimeout

Seek-back on track switch with relativeResumingPosition

Easier DASH_WASM integration with the EMBEDDED_WASM embed

Breaking change: A representationFilter's codec is now codecs

Samsung TV seek-back handling refactoring

DASH_WASM, DEBUG_ELEMENT and parseBifThumbnails are not experimental anymore

New package export strategy

v3.33.0

Release v3.33.0 (2024-01-24)

:mag: Overview

:bookmark_tabs: Changelog

Features

Bug fixes

Other improvements

About it being the last "main v3 release"

DASH supplementalCodecs handling

getLivePosition method and startAt.fromLivePosition option

Possibility to set a new keySystems when calling reload

Multiple minor DEBUG_ELEMENT improvements

v4.0.0-beta.3

Release v4.0.0-beta.3 (2023-10-19)

:mag: Overview

:bookmark_tabs: Changelog

Changes

Bug fixes

Other improvements

newAvailablePeriods event now sent lazily

Changes concerning Errors

Error message changes

trackInfo renamed as tracksInfo

Changes concerning the builds

Future work on exploiting WebWorkers

v3.32.1

Release v3.32.1 (2023-10-19)

:mag: Overview

:bookmark_tabs: Changelog

Features

Bug fixes

Other improvements

Deprecated

New isSpatialAudio property

addFeatures now globally available

The new `MULTI_THREAD` experimental feature

`onKeyOutputRestricted` / `onKeyInternalError`

`persistentState` / `distinctiveIdentifier`

`videoCapabilitiesConfig` / `audioCapabilitiesConfig`

A new player state: `"FREEZING"`

`DASH_WASM` and `MULTI_THREAD` fix

`attachWorker` now returns a Promise

`v4` summary

The new `MULTI_THREAD` experimental feature

A new `requestConfig` property: `connectionTimeout`

Seek-back on track switch with `relativeResumingPosition`

Easier `DASH_WASM` integration with the `EMBEDDED_WASM` embed

Breaking change: A `representationFilter`'s `codec` is now `codecs`

`DASH_WASM`, `DEBUG_ELEMENT` and `parseBifThumbnails` are not experimental anymore

DASH `supplementalCodecs` handling

`getLivePosition` method and `startAt.fromLivePosition` option

Possibility to set a new `keySystems` when calling `reload`

Multiple minor `DEBUG_ELEMENT` improvements

`newAvailablePeriods` event now sent lazily

`trackInfo` renamed as `tracksInfo`

New `isSpatialAudio` property

`addFeatures` now globally available

Deprecation of the `NATIVE_TEXT_BUFFER` and `HTML_TEXT_BUFFER` features

`NO_PLAYABLE_REPRESENTATION` behavior change

New `trackUpdate` `reason`

New `representationListUpdate` event

`isContentLoaded` and `isBuffering` method

`isPaused` method

`"play"` and `"pause"` events

`trackInfo` property on some `MediaError`

The `trackInfo` property

`getLastStoredContentPosition` method

Better "direct" `audioTrackSwitchingMode`