WebRTC API update Q2 2024

This article provides a quick update on the modified webRTC API for Q2 2024. If you’re a developer, take a look at these changes and see how they may impact your application.

Some of the APIs discussed here are not defined by the webRTC working group, but are part of the webRTC ecosystem.

The information here has been collected from the available release notes taken from Chrome, Safari and Firefox.

WebRTC Legacy API

Here are the changes that browsers have made to complete their implementation of the WebRTC API.

RTCIceTransport (Firefox)

Starting Firefox 125, the properties RTCIceTransport.state and RTCIceTransport.gatheringState, and their associated events statechange and gatheringstatechange, are now supported, along with the RTCDtlsTransport.iceTransport.

“Time makes everything happen when you know how to be patient” - Clement Marot

RTCIceCandidate (Firefox)

Starting Firefox 126, all RTCIceCandidate properties and methods are now supported and match the specification, except the unimplemented relayProtocol and url properties.

New properties supported are: foundation, component, priority, address, protocol, port, type, tcpType, relatedAddress, relatedPort, and usernameFragment.

Firefox fills in the gap in their legacy API implementation.

RTCIceCandidate (Chrome)

Starting Chrome 124, the RTCIceCandidate.relayProtocoland RTCIceCandidate.url properties are now supported.

You don’t need anymore to deduce the relayed protocol from the priority property. This is now directly available thanks to the relayProtocol property.

RTCRemoteOutboundRtpStreamStats (Chrome)

Starting Chrome 125 (I didn’t find the WebRTC release notes for Chrome 125), the getStats API now returns the missing remote-outbound-rtp report for video.

Note: The remote-outbound-rtp report was already available for an audio stream. This time we have the same for video. This new report can be correlated with the inbound-rtp video report to have all statistics for the associated inbound stream (mainly RTT).

RTCPeerConnectionStats, RTCMediaSourceStats (Firefox)

As I didn’t check since Firefox 107, the following reports RTCPeerConnectionStats and RTCMediaSourceStats are now available in Firefox. I saw it in Firefox 124.

This is perhaps not new, but I didn’t find any information about them in the release notes.

Note: roundTripTime, roundTripTimeMeasurements and totalRoundTripTime are still missing in report RTCRemoteOutboundRtpStreamStats.

WebRTC Extensions API

Here, we can find additional APIs that are defined to extend the WebRTC existing API surface.

MediaStreamTrackAudioStats (Chrome)

Starting Chrome 125 (?), the audio track now exposes a property stats to access the interface MediaStreamTrackAudioStats.

This interface comes from the specification document WebRTC Media Capture and Streams Extensions.

The following counters are available: deliveredFrames, totalFrames, deliveredFramesDuration, totalFramesDuration, latency, and some other for the average and min/max latency.

I used this interface in a P2P call. Strangely, I didn’t succeed to block the deliveredFrames counter when the microphone was muted.

I tried by pressing the physical mute button of my microphone or by using the built-in Audio Midi Setup application. And even if the track was marked as muted, the counter still increased, although the specification states that “if no sound is flowing, for example if the track is muted or disabled, the counters do not increase”.

It is like frames are still flooding… Perhaps I missed something or the implementation is not yet complete.

For a remote track, I’m not sure on how to interpret the latency got…

ImageCapture (Safari)

This interface comes from the specification document MediaStream Image Capture.

ImageCapture interface is available in Safari Preview 194. All APIs are available except the grabFrame() method which must return an ImageBitmap object. You can only use the takePhoto() to get the Blob object.

Note: ImageCapture is still under a flag in Firefox. The flag dom.imagecapture.enabled needs to be set in about:config

Unbundled WebRTC API

Here, this is more complex. These APIs allow you to build your own machinery to handle the media streams and the necessary transport layer.

WebTransport (Safari)

WebTransport interface is coming in latest Safari Preview 194. I didn’t see it before in version 17.4. WebTransportBidirectionalStream and WebTransportDatagramDuplexStream, both are available.

WebCodecs (Safari)

A lot of new interfaces have been implemented in latest Safari Preview 194: EncodedAudioChunk, AudioData, AudioDecoder, AudioEncoder interfaces are now available.

Others interesting API

These APIs are not directly related to WebRTC but can be useful in the context of a WebRTC application.

Pressure API

Starting Chrome 125, you can now use the new Pressure API to get the CPU pressure of the system. This is useful to adapt your application to the current system load and avoid performance and/or battery issues.

Today, by using the getStats API, we can know if something limits the media I’m sending to someone else. The current most ‘limiter’ of the system can be retrieved thanks to the qualityLimitationReason property. And this indicator is per track sent. From this indicator, we try to deduce if the system has a CPU, a bandwidth or an other issue that affects the experience.

Here, the proposal is to focus on the CPU and to get 4 levels of pressure:

• Nominal: The pressure, temperature and/or energy are at an acceptable level. No effect on the battery and or the performance of the system.
• Fair: The system is doing tasks resulting in pressure, temperature and/or energy usage that are slightly elevated. Fans can be heard and the battery is impacted.
• Serious: The system is overloaded and is the pressure, temperature and/or energy are consistently highly elevated. Maintaining a such level drastically reduces the battery life of a laptop, tablet or mobile device.
• Critical: It affects mainly the temperature of the system that increases. Cooling may be inefficient and the system may throttle down to avoid overheating. Performance of the system is heavily impacted.

As you understand, unlike a CLI load command, the Pressure API doesn’t return the average load of the system during the last N seconds. This API gives you a level of pressure. This is more a ‘health’ indicator that your application can track and use to adapt its behavior.

Here is an example of how to use the Pressure API:

const pressureObserver = new PressureObserver((change) => {
    //do something when the pressure changes
});

// Observe the CPU pressure every 3 seconds
pressureObserver.observe('cpu', {sampleInterval: 3000});

// Stop observing the CPU pressure
pressureObserver.unobserve('cpu');

That’s it. Each time the pressure level changes, the callback function is called with the new pressure level.

{
  "source":"cpu",
  "state": "nominal",
  "time": 1716218985845.574
}

At this time of writing, the cpu is the only source available. Other sources may be added in the future.

Unlike the qualityLimitationReason indicator, the Pressure API exposes a global indicator of the system. Up to the application to adapt its behavior or not.

Availability: Chrome only at this time.

WebSocketStream API

The WebSocket API is a powerful tool to communicate between a client and a server. But it is not always easy to use, especially when you need to send a lot of data.

Starting Chrome 124, you can now use the new WebSocketStream API to send and receive data in a more efficient way.

The WebSocket API has 2 main problems:

• When sending messages: You need to rely on the bufferedAmount property to know if you can send more data

• When receiving messages: The onmessage event handler can be overwhelmed by a large number of messages without giving your application asynchronous control. Your application may not respond, or the browser may run out of memory and freeze.

The WebSocketStream API manages the backpressure thanks to the ReadableStream and WritableStream usage.

const streamSocket = new WebSocketStream('wss://example.com');

const {readable, writable} = await streamSocket.opened;
const reader = readable.getReader();
const writer = writable.getWriter();

while (true) {
  const {value, done} = await reader.read();
  if (done) {
    break;
  }
  const result = await process(value);
  await writer.write(result);
}

In this example, the application reads some data, then processes it and only after, reads the next chunk. So your pipeline is under control.

Note: WebTransport also manages the backpressure and can be an alternative to the WebSocketStream on network that supports QUIC.

Availability: Chrome only at this time.

This is very interesting to see these APIs coming in Chrome: Detecting the CPU pressure in one hand and managing the backpressure in the other hand. They will for sure open new possibilities for WebRTC applications.