Coturn, the fragile colossus

Last Update, August, 22nd 2022 with good news around Coturn: Starting the end of July, Gustavo Garcia took the lead and merged a lot of PR. So it seems that a new dynamic is being set up.

Coturn is the most famous open source Turn server available today and the most used in WebRTC solutions. It is available for many Linux distributions and for macOS and there are many Docker images ready to be configured.

Deploying and configuring an instance of Coturn is accessible to any developer with some networking skills and is therefore not limited to the Ops team.

Available since late 2012 (first release), Coturn is primarily used in many WebRTC solutions today as it allows to make WebRTC calls when network equipments block peers-to-peers connections.

With its 8k stars in GitHub, Coturn could be considered as a highly maintained project: With 76 contributors, this open-source project is a success. But if you look closely at the numbers, the status is not so good: The original contributor is no longer active and the last one who was active the last three years seems to have no time to maintain the project.

So, at that time of writing, it seems that there is no active maintainer able to merge the existing PRs. What alerted me was the ticket Is the project dead? posted on GitHub: The community have started to detect anomalous signals from this open source…

So what’s next? For those who have a complete WebRTC solution based on Coturn, are we in danger? What does it mean if the project is no longer maintained? And, are there any alternatives?

A piece of the puzzle

Having a TURN server (properly configured) is mandatory today to enable WebRTC calls in a complex network environment and to give peers their public addresses (STUN). So, without a STUN+TURN server today, WebRTC calls are not possible.

Fortunately, the STUN/TURN server is only one piece of this puzzle and a piece that is not strongly coupled to the others: One TURN server can easily be replaced by another. There is no dependency between the front-end, the back-end and the TURN server, except, the URL and credentials.

The complexity comes for the Ops team to operate and monitor this server: This server should be easily integrated with existing Ops tools.

Current Status

The last version 4.5.2 was released in January 2021 and the last actual commit was done in September 2021. A version 4.5.3 is still in progress.

Since that last change, a few docker builds have been released thanks to tyranron but today, they are 23 pull requests ready to be merged that are still waiting for someone and around 300 open issues.

Coturn is very popular today and has been with WebRTC since the beginning. Here are some numbers from GitHub:

What makes Coturn indispensable is the support of a large number of RFC. Today, Coturn allows to have a call in more than 99,9% of the cases (not sure of cases not supported). So, for enterprise networks, this is the best choice.

But the question is how long it will continue to work perfectly if there is no new version.

The Threats

The problem is not necessary with the current level of functionalities or supported protocols as seen, because Coturn does them.

Rather, it’s more about 3 things:

• Standards evolutions: RFC8489 (STUN), RFC8656 (TURN) are the evolutions of current standards. They are not implemented in Coturn today.

• Security: The support of DTLS 1.3 and the support of OpenSSL 3.0.1 are mandatory in terms of evolution and will need to be developed progressively to offer the best level of security. A significant SSL/TLS Renegotiation DOS Vulnerability has been found and must be fixed quickly. This issue was opened in February 2022.

• Current user-reported crashes: There are not many opened issues but Coturn definitively needs someone or a team that has the time to analyze the open issues and fix them if they are relevant.

This is not the time to panic, just to wonder about next year if nothing changes in Coturn. And the opportunity to discover alternatives just in case.

Believe in open-source

Of course, at any time, someone could decide to take over. The current maintainers are looking forward to that. With a few people, Coturn can rise again. It is not too late.

It is strange to think that such a project is maintained by one person at a time… To most of us, Coturn is like a commodity, but it is not… Coturn has just started exposing Prometheus metrics and has to implement recent RFCs, this takes time.

But, what is the benefit to the maintainer to implement a new RFC now? For fun? Visibility? Personally, I’m betting on an intermediate model that offers free software and a paid solution/service or paid support offering. This should be a clear win-win. I understand that after a long time, free maintainers change their priority…

DIY

Certainly, one of the solution is to fork Coturn and add any specific requirements.

I have no doubt that some major CPaaS players who use Coturn already have their private branch derived. But not all companies have the ability to re-internalize such development.

For the others, let’s see the alternatives

The challengers

I was really surprised to discover that challengers exist and can offer real alternatives to Coturn. I was never interested in the past to test an alternative solution as everyone advocates for Coturn.

For those who just want the takeaway, here is the result of my tests. I didn’t spend a lot of time on each, so it should be interpreted as “a first impression”!

I tested using 6 criteria:

• Configuration: What possibilities you have to tune the TURN server to fit your needs. The more possibilities, the better the score
• RFC: What functionalities is offered by the TURN server. The more RFC implemented, the higher the score
• Binary: What systems are supported ? What kind of packages are provided ? The more possibilities, the higher the score
• Serviceability: How easy it is to monitor and understand what happened in term of usage or problems. The more possibilities, the higher the score
• Community: Is there an active community where to share knowledge, experiences and to share problems. The more the community is active, the higher the score
• Activity: Is there development in progress, a planned road-map. The more commits, the higher the score.

The final score is the sum of all individual scores.

It is difficult to be a rival of Coturn but Eturnal and Pion Turn are two great challengers today.

Eturnal

Eturnal is a STUN and TURN server developed by ProcessOne, the company behind ejabberd. First used internally, it was released publicly in July 2020 as mentioned in that article.

Eturnal can be installed using apt on Ubuntu. A binary is installed and can be launched using the command eturnalctl. For those who want, docker images are available as well as executables for Windows.

The first beta version was released in June 2020.

(* and ProcessOne)

Discovering and using Eturnal was really a pleasure. I was not expecting that. I used the same way for testing both ICE and Media parts as for the other Turn servers.

Once configured the /etc/eturnal.yml file with a relay_ipv4_addr and an authentication secret, I was able to connect in TCP/UDP and have a relayed call.

What was interesting is the fact that the binary can be called “as an API” using Operations: A ping operation is available to check if the process is still alive, an info operation gives the uptime, active turn sessions, processes as well as some information about the memory and CPU usage.

And as for Ejabberd, Eturnal comes with modules that add extra features on top of the TURN server. For example, the module mod_stats_influx allows to log events and statistics into an InfluxDB database.

A detailed Changelog is available.

In short:

PROs

• Linux, Windows, Docker support
• UDP/TCP/TLS support for IPv4/IPv6
• Authentication secret
• Logs, metrics, status, a lot of operations are available to know the current state
• Modules: mod_stats_prometheus (Prometheus exporter)
• ProcessOne

CONs

• Not available on MacOS for local dev.
• No active community, no feedbacks found

Eturnal is an alternative to Coturn. With ProcessOne behind, this Turn server has a promising future. Currently, this is the most active one in term of commits.

Note: There is a dedicated STUN & TURN library for Erlang & Elixir, for Erlang’s fans :-)

Pion Turn

Pion Turn as described in GitHub, is a Go toolkit for building TURN servers and clients. There is no real binary to download but samples that can be used and extended depending on your needs.

Pion, the team behind that project is working on an open source and cross-platform stack for Real-time Media and data communication called Pion WebRTC. This is an alternative WebRTC stack to libWebRTC.

For Pion Turn, the first available version was released in May 2018.

I never used Go before, so I was not sure to be able to test this TURN server. I installed Go on my Ubuntu EC2 host and try to follow the guides.

In order to test in live, I followed the Quick Start. I was able to test with the last available release v2.0.2.

Here are the commands I used (adapted from the Quick Start).

$> wget -q https://github.com/pion/turn/releases/download/v2.0.2/turn_2.0.2_Linux_i386.tar.gz
$> tar -xvf turn_2.0.2_Linux_i386.tar.gz
$> ./turn-server-simple --public-ip xxx.xxx.xxx.xxx --users username=password

Then, I made two tests:

• On Ice, by using the turn-server-simple and the turn-server-tcp samples to be able to test the STUN part and the possibility to have UDP and TCP routes.
• On Media, by using a sample I have to make a P2P relayed call through the Pion TURN server.

All tests worked fine! Additionally, I used the turn-server-log sample which logs every requests-answers and allow to understand what happened (the protocol).

In short:

PROs

• Go language if you are a Go developer
• Growing and active community: Slack #Pion Channel has more than 2,466 members
• Ready to use and extendable samples
• Documentation
• Sean DuBois, the creator of Pion who is very active on Slack to help anyone on Pion integration.

CONs

• Go language if you are not a Go developer
• This is a toolkit and not a deployable binary nor a Docker Image as explained in that issue, even if developers continue to ask for binaries, docker images and out of the box solutions…
• Need to have strong skills on how TURN works to adapt the samples

Pion Turn and all the Pion ecosystem based on GO is very promising. Here, this is a real alternative to Coturn but dedicated to Ops or developers who want to put their hands in the engine.

Violet

Violet is a lightweight STUN/TURN server written in C without dependencies, based on libjuice.

This project has been started in January 2020 and there is only a first version available (0.4.0).

As mentioned in GitHub, this project has no dependencies so it was very easy to install in on my EC2 Ubuntu host following the documentation. Here there is no binary, you need to clone the repository and submodule (libJuice) and then to compile the project to obtain the binary.

Once done, the Violet Turn server can be launched.

I used the same protocol as for Pion and Node-Turn:

• A first test for validating the Stun part as well as the relay candidates.
• A second test by establishing a video call through the Violet Turn server.

The executable can be used with the option -h to display the list of available options. Logs level, daemon mode, credentials, range, addresses as well as the possibility to limit the allocation and to work in STUN only, all these options can be configured directly without using a config file.

I successfully got a srflx and a relay/udp candidates and the video call worked great.

To summarize:

PROs

• Simple, light and easy to use Turn server
• Last version of standards implemented (8489, 8656)
• Logs
• STUN only mode

CONs

• Only UDP support, no TCP, TLS
• No really community (yet), no feedbacks on usage on production
• No metrics

Cubuntu@ip-xxx-xxx-xxx-xxx:~/violet/build$ ./violet --help
Usage: ./violet [options]

  -h, --help                Display this message
  -v, --version             Display the version
  -f, --file=FILE           Read configuration from FILE
  -o, --log=FILE            Output log to FILE (default stdout)
  -l, --log-level=LEVEL     Set log level to LEVEL: fatal, error, warn, info (default), debug, or verbose
  -d, --daemon              Detach from terminal and run as daemon
  -p, --port=PORT           UDP port to listen on (default 3478)
  -r, --range=BEGIN:END     UDP port range for relay (default automatic)
  -b, --bind=ADDRESS        Bind only on ADDRESS (default any address)
  -e, --external=ADDRESS    Avertise relay on ADDRESS (default local address)
  -c, --credentials=USER:PASS   Add TURN credentials (may be called multiple times)
  -q, --quota=ALLOCATIONS   Set an allocations quota for the last credentials (default none)
  -m, --max=ALLOCATIONS     Set the maximum number of allocations (default 1000)
  -s, --stun-only           Disable TURN support

Violet is a great STUN/TURN server that can be used if you need one for prototyping. The lack of TCP/TLS support is a showstopper for production. Not (yet) an alternative to Coturn.

Node-Turn

Node-Turn is a STUN/TURN server for Node.JS created in 2018.

To test it, I create a little Node.JS application in JavaScript in add this library. I deployed this server on a EC2 machine using Ubuntu.

Then, I made two tests:

• On Ice by using the Trickle Ice test to check the candidates returned.
• On Media by using a sample I have to make a P2P relayed call through the TURN server

Globally, it was working in some minutes which was good but this open-source seems to be in the same situation as for Coturn. No commit since the last 2 years…

PROs

• Node.JS library

CONs

• Vulnerability detected with Log4JS (Incorrect default permissions in log4js)
• Limited protocols supported, no TCP/TLS candidates gathered
• No real documentation on advanced usage
• No metrics exported
• No REST Admin API
• I didn’t succeed to have logs
• No commit the last 2 years, no community, no other experience or feedback found

var Turn = require('node-turn');
var server = new Turn({
  // set options
  authMech: 'long-term',
  credentials: {
    username: "password"
  }
});
server.start();

Node-Turn is an easy-to-use library for playing with a Turn server and discovering how it works. But as this project seems to be discontinued and not really complete in terms of protocols supported or documentation. This can’t be a rival to Coturn.

Open Relay (UFO…)

Open Relay is a free TURN server provided by Metered Video.

There is nothing to install here as Metered Video offers direct access to his (their?) TURN server(s).

I added it to the list just for information about this initiative as the tests couldn’t be performed correctly: I tested the ICE connection that worked fine, but I never succeeded to have a relayed call using the Open Relay Turn server.

Note: It is not a surprise as relaying traffic has a cost. I’m not sure where the Turn server(s) is(are) deployed but it seems to be only in Canada…

Rust Turn

It seems that there are at least 2 Rust implementations of Turn:

• Rusturn which seems to be a complete rewrite of a Turn server using Rust. Not tested because there is no binary, and it seems that Rust skills are needed. Only the RFC5766 seems to be implemented. The current version is the 0.0.4 (…).

• WebRTC.rs/turn which is a rewrite of the Pion Turn in Rust. It is part of the WebRTC.rs implementation which is a rewrite of the Pion stack in Rust.

These solutions are for those who know Rust!

Return (Resiprocate)

Return is a C++ implementation of a Turn Server included in the Resiprocate framework (SIP stack + other tools).

This project is still active and the ReTurn server continues to evolve (Only UDP relay’s is implemented today).

Resiprocate is about 18 years old. Even all these years, reTurn implementation is still young.

Abandoned

Here are some others that has been abandoned since a while:

• Jitsi Turn Server: No commit since November 2015

• GoRTC: These initiatives have been abandoned in favor of Pion.

• GO Stun: Go implementation of STUN, TURN and ICE Protocols

• WebRTC-Stun: A Stun server written in TypeScript. No commit since January 2019

• Turn.Server: A C# implementation

• StunLib: A STUN and TURN library written in C

• Mongoose ICE: STUN/TURN server written in Elixir developed by Erlang Solutions

• XTurn: Another TURN server in Elixir developed by Xirsys who now sells a TURN Cloud platform.

• Restund: Modular STUN/TURN server written in C by Creytiv.com

• StunMan: Open source STUN server and client available for Linux, MacOs and Windows.

• JSTUN: A very old STUN server (RFC 3489)

• Other projects exist in GitHub if you do some archaeology…

Tools around Turn

When searching alternatives to Coturn, I found some interesting tools. Here are some links.

Stunner

Stunner is a tool to test and exploit STUN, TURN and TURN over TCP servers.

It allows to target your Turn server and check for some vulnerabilities. I tested the info basic command that gives me an overview on my Turn and the range-scan command to detect any traffic to internal system. I didn’t try the advanced commands but it seems to very powerful.

For testing, Stunner, you can download binaries (MacOs, Windows and Linux) or to clone the repository, install Go and compile the project using go build. The result is a binary that you can use to test your server.

505 stars on GitHub already collected for that tool in a few months only!

Stun

Stun is a library to implement a STUN server using Node.Js as well as a STUN client to exchange messages to any STUN servers.

Stun implements RFC5389 with partial support of RFC5766, RFC5245 and RFC5780.

Unfortunately, last commit has been done in June 2020. Nothing new since that date.

Other abandoned Node.JS STUN projects like STUN-JS, Node-Stun or MiniStun can be explored…

Stun reached 133 stars on GitHub…

Flutter WebRTC Server

Flutter WebRTC Server is an “all in one” solution based on Go & Flutter for providing a WebRTC signaling server including the Turn Part.

The Turn server is based on Pion Turn. The last version is more than one year old.

Conclusion

As described in that article, alternatives to Coturn exist, so whatever happens for Coturn, there will be a solution.

My preference goes to Eturnal.