Breaking changes in getStats

Using the getStats API is not as simple as it seems to be. Not because of the code to write, but because you need to take the time to understand the statistics shown, to interpret them correctly and to follow the changes over the versions of Chrome and the other browsers.

And here, the underlying complexity for the Web developers mainly come from the following aspects:

• Chrome has brought in many statistics that have never been standardized in this way. You can read this excellent article in BlogGeekMe to better understand where this is coming from Can I trust WebRTC getStats accuracy.

• The need for improving the statistics due to the change introduced to the WebRTC stack.

• The limited documentation that exists on this topic. Statistics are not described nor explained to developers except in the W3C specifications which address stack’s vendors.

• (For Fun) The impossibility for Web developers and their users to stick to a browser’s version (if we except the ESR versions). Browsers’ updates are like a steamroller that comes every month.

All this together makes the use of the statistics complicated.

This article focus on the recent (breaking) changes that need to be handled shortly as well as some changes that will come in the next versions of the browsers.

Update March, 19th

Following discussions in the WEBRTCWG and on this issue raised on GitHub #742, it seems that it is not urgent to use the new RTCAudioPlayoutStats report.

Detecting breaking changes in WebRTC JS API ?

I like this post from Eric Elliott about software versioning: Software versions are broken.

For developers who use a library versioned according to this scheme, things are clear: Every time the first number changes, there is at least one breaking change. And it means that upgrading to this version is sweaty but if the owner has done his job, there will be a cook-book to successfully overcome any breakable change.

Here, knowing that there is a breaking change is more complicated:

• First you need to know where to find the information: Discuss-WebRTC seems to be the place where official announcements are made. Why all breaking changes are not in the Chrome Roadmap ? Why the complete changelog is not part of the Chrome releases blog or in the Chromium Blog ?

• Then and once in the right place, you need to know how to detect breaking changes from all messages posted in the group. A first filter can be done by selecting messages starting by PSA. PSA stands for Public Service Announcement. It is used to draw the attention of the developers on a particular point and most of the time (100% of the time ?), these messages come from the Google team. Unfortunately, not necessarily to announce a breaking change…

• Finally, from the PSA posts, it is up to you to understand if it concerns the libWebRTC API or the JavaScript Web API.

So, definitively, for a Web developer, there is a lack of a real changelog and explanation to overcome these recent changes.

I never understood why the JavaScript API documentation is not provided by the browsers’ vendors who prefer to rely on MSN and the W3C instead of offering a documentation that reflects their level of implementation, behavior and choices…

From that, you can understand why some are late for applying the recent breaking changes. Do you remember all the negotiations about the migration to the Unified Plan ?

That’s said, here is what I understood from the recent changes done.

Note: For every new Chrome version, there is an associated WebRTC Release Notes that can be found in the WebRTC group. An example here. But most of the points targets libWebRTC. Not clear at first look which points address Web developers…

Recent breaking changes in getStats

Here is the list of the recent breaking changes.

Breaking change for statistics identifier

This change has been announced in Sept, 06th in the post Discuss-WebRTC and introduced in Chrome M107.

Before this change, the statistics identifiers of the report were contained a string that distinguished the type of the report. Eg: RTCOutboundRTPAudioStream_3138056031. In this example, we can clearly identify that the statistics are associated to an outbound audio report.

But, it was decided to have ID more concise such as with random identifiers.

The problem is that nobody explained to developers that even if it was a convenience way to identify the type of the report, it should not be used for this cause. And developers never thought that their cherished identifiers could change one day…

Thus, all applications that used identifiers as a “type” or that did some parsing with the identifier had to overcome a breaking change.

Note: In the same way, prefixing the identifier with “DEPRECATED” was also a breaking changes…

Chrome 107 that handled this change, was released in Oct, 25th; 50 days after the announcement.

If your application relies on the statistics ID and you didn’t make the change, you should have an issue accessing the statistics.

Breaking change for codecs reports

In the same period, Chrome released a change that concerns the codecs reports and that could be seen as minor but before the version 107 in some configuration, a simple call to getStats could generate several hundreds of codecs reports (as mentioned in this issue).

So a fix has been provided. And recently, a new change has been added to limit the codecs reports to the one used in the call (observed in Chrome Canary M113).

If for any reasons, a developer was using these codecs reports to do some live or post-mortem analysis such as trying to list the different ones, even if it is not the right way to do that, this can lead to a breaking change in his code.

If your application relies on the codecs list from these reports, you should see a very short list now and for sure not the complete supported list of codecs.

Breaking change for Stream and Track reports

The announcement of the deprecated track report has been done in July 2022 in this post Discuss-WebRTC and in September, the same announcement was done for the stream report Discuss-WebRTC.

It was originally planned to introduce this change in Chrome M109 but since it caused the Twilio SDK to break (see the bug here), a rollback was made.

The new plan is to land this change in Chrome M112 and for those who can’t make the change before, to register an Origin Trial which will allow access to these reports up to and including Chrome M115 (September, 24th).

To summarize:

• Reports of type track and stream will be removed

• Property trackId will be removed from all reports of type inbound-rtp and outbound-rtp.

Here is the new map of the reports that should be available in M112

The original RTCMediaStreamStats and RTCMediaStreamTrackStats reports will so join the long list of the obsolete statistics which appears more and more like a vast cemetery.

If your application relies on the track and stream reports, it is urgent to update your code before M112. It should not be complicated to map your existing code to the statistics accessible from the other reports.

Other updates in getStats

And this cleanup is not finished. Here are some browser updates that should arrive soon.

Chrome: New breaking change for the legacy based getStats

In Chrome, there are two versions of getStats(), one that is spec-compliant and that returns a promise, and the one that is non-standard and that uses a callback as the first argument.

The deprecation and the removal concern the non-standard version of getStats(). The announcement was made in February 2023 in this post Intent to Deprecate and Remove: Legacy callback-based RTCPeerConnection.getStats() API

After several discussions, the final countdown should be like the following (from the post):

• Add deprecation warning in M113 (Release: May 2, 2023) with the possibility to opt-in to the Deprecation Trial.

• Throw an exception if the trial is not used in Canary/Dev/Beta in M114

• Throw an exception if the trial is not used in Stable in M119 (Release: October 31, 2023).

• Let M121 be the last version where the Trial is available (Release: January 9, 2024). In other words the first version where the trial and legacy getStats API is entirely removed is M122 (Release: February 6, 2024).

Chrome: New Media-Playout report

Recently, the new Media-Playout report appeared in Chrome M111.

This report gives statistics about the audio playout path.

Here is some interesting new metrics that can be computed from this report:

• The delta duration of synthetized playout: In case of big increase in a short period, the perceived quality should be impacted

• The average playout delay: As for the previous one, detecting peaks in this delay could be a sign of brief audio degradation

• The percent of synthetized playout: It could give you a good indication if the playout path is able or not to produce audio.

The report of type inbound-rtp contains a new playoutId property to link this report.

These new statistics as well as some new events to alert the application should be landed soon in WebRTCMetrics.

Firefox: No real progress…

I would have liked to have some improvements to list about Firefox, but I checked the last statistics got from Firefox 110 and from the nightly version 112 and didn’t see any major progress.

Firefox still suffers from the lack of statistics and reports (around 33% less statistics than in Chrome).

For example, there is still no report of type media-source and transport. And so, I still have to use the property selected from the candidate-pair report to identify the pair used. This property does not conform to the specification.

But on other hand, the remote-outbound report for video is implemented. While I never managed to get it with Chrome (only for the audio part).

Safari: Wait for the next libWebRTC synchro

I don’t know if I’ wrong’ but as Safari (Webkit) is synchronized with libWebRTC version M106, the next big update in terms of statistics improvement should be done at the next synchronization.

I compared the latest Safari Technology Preview (165) with the current release (16.3) and as expected, I saw no progress.

Conclusion

The getStats API hides a complex topic that cannot be summarized in a blog post nor in the MSN Web Docs.

Improving the communication and the documentation around this API should help browsers vendors to move forward and prevent developers from always finding themselves on the edge.