LiveSwitch has full support for simulcast as of version 1.9.0. Partial support was available in earlier releases, starting with version 1.4.0.
Simulcast is a shortened word for "simultaneous broadcast". It is a technique used in WebRTC to improve the overall quality of a group conference.
Instead of sending just one video stream to the server, each participant can send multiple copies ("encodings") of the same video, but encoded to different quality targets. For example, a high-quality (full bitrate) and low-quality (reduced bitrate) stream would be typical.
To illustrate why this is useful, consider a 4-way group call with Alice, Bob, Carol, and Dan:
- Alice is sending video to LiveSwitch using a bitrate of 2mbps.
- Bob and Dan have good Internet connections and can receive Alice's video from LiveSwitch at 2mbps without issue.
- Carol, however, has a limited Internet connection and can only sustain inbound video from LiveSwitch at about 1mbps.
In this scenario, in order to keep Carol from dropping the call, LiveSwitch must coordinate with Alice to reduce her outbound bitrate to about 1mbps. This has an unfortunate side effect for Bob and Dan, who must now also see Alice at the reduced bitrate. This negative side effect is amplified and becomes more and more noticeable as you add more participants, which is where simulcast comes in.
With simulcast, LiveSwitch can send Carol a low-quality encoding, while allowing Bob and Dan to continue receiving the high-quality encoding.
Note that simulcast does not provide any benefit for 2-way calls. In a 2-way call, the bitrate simply adapts to each receiver. There are no side effects because there are no other receivers to impact.
For the same reason, simulcast is not supported for peer connections. Only SFU and MCU connections can take advantage of this feature.
The simulcast implementation in LiveSwitch is compatible with the draft IETF proposal (https://tools.ietf.org/html/draft-ietf-mmusic-sdp-simulcast-14) which uses RIDs to identify simulcast streams. Non-RID-based simulcast (using SSRCs only) is available for native platforms, but will likely be deprecated in a future release once the draft simulcast specification is finalized.
How does Simulcast Improve Stream Quality?
Simulcast is a powerful tool for providing optimal video stream quality given current network conditions. It works by simultaneously sending multiple feeds of different quality, dynamically enabling and disabling them according to who is subscribed to them, and on the receiver side, switching quickly between them as network conditions change.
So, will simulcast, by itself, solve every possible quality issue you may encounter? It depends, but we've given you the tools needed for most situations. There are other mechanisms at play which also factor into overall stream quality. For example, within the bounds of a particular encoding layer, simulcast relies on Bandwidth Estimation & Adaptation to adapt stream bandwidth as needed. Another mechanism which improves overall stream quality is Adaptive Video Resolution, where we adapt the resolution of the video stream constraining it to the actual resolution of the receivers' view. For example, if we are sending HD video at 1280 X 720, but all the receivers are displaying the downstream in a smaller view (perhaps because the application layout dictates demoting participant views during a screen share, or similar use case), then the sender’s resolution can be adjusted as needed so that bandwidth is not unnecessarily wasted.
The reality is that when considering overall stream quality simulcast is an important tool in our tool belt, but it is not the only one, and by itself cannot solve all quality issues for all network conditions. LiveSwitch 1.9.0 with simulcast enabled, has shown significant improvement in overall bandwidth handling under many scenarios, such as most broadcast use cases (which typically have less than ten senders), or most huddle use cases (where typically everyone is a sender, but most huddles have between 3 to 8 participants). For the typical broadcast or huddle use case, simulcast works very well to improve overall stream quality.
We are constantly looking to improve our bandwidth management and adaptation policies and capabilities. If you have a use-case that you believe could be improved, we want to hear about it - please contact us and let us know!
Simulcast is fully supported in all native platforms and partially supported in web browsers. See the feature matrix at the end of this page for specific details of what is and is not available.
To use simulcast, simply set the
SimulcastMode for your
That's it! Make sure this video stream is used when creating your SFU or MCU upstream connection, and you're done.
Simulcast is most useful when LiveSwitch can use bandwidth estimation to determine which simulcast encoding to send to which participant.
For this reason, it is strongly recommended to set your
Enabled in your client application (
VideoStream owns the policy) and in your administration console (
Channel configuration owns the policy). This is the default behaviour, so unless you have explicitly disabled it, there is no action required on your part.
Simulcast has a sensible default configuration out of the box, but it can be tuned for custom use cases.
Some key points to note:
- All simulcast configuration is done on your
- In native apps, configuration changes must be applied prior to calling
- In web apps, configuration changes must be applied prior to calling
VideoSimulcastEncodingCount property lets you set the number of encodings to initialize. The default value is
A maximum of 4 encodings will be accepted by the LiveSwitch Media Server.
Bits Per Pixel
VideoSimulcastBitsPerPixel property lets you control the bitrates used by the video encoders for each encoding. The default value is
The number of bits per pixel (bpp) describes the desired quality of a video stream independent of the frame size (width/height) and frame rate. Given a video source that raises frames of a particular size and a particular rate, we can determine the target bitrate for a video encoder (e.g. VP8, H.264, VP9) using a simple formula:
Because it optimizes bandwidth usage based on the actual characteristics of the video stream and ensures that quality is consistent across devices and platforms, it is strongly recommended to use bpp for custom simulcast configurations.
For example, a full HD video camera producing 1920x1080 images at 60 frames per second (fps) using the default bpp of 0.05 would imply that the target bitrate for the video encoder should be:
On the other hand, a camera producing 640x480 images at 30 fps using the same default bpp of 0.05 would imply that the target bitrate for the video encoder should be:
VideoSimulcastDegradationPreference property lets you control the trade-offs that lower-quality encodings must make to achieve lower bitrates. The default value is
There are four values allowed:
Resolution will reduce the number of pixels in half for each secondary encoding. For example, a video source raising 1280x720 frames with 3 encodings will result in resolutions of 1280x720, 905x509, and 640x360.
FrameRate will reduce the number of frames in half for each secondary encoding. For example, a video source raising frames at 30 frames per second with 3 encodings will result in frame rates of 30, 15, and 7.5.
Balanced will reduce both the number of pixels and the number of frames for each secondary encoding, but less than the reduction that would take place with either
FrameRate. For example, a video source raising 1280x720 frames at 30 frames per second with 3 encodings will result in resolution and frame rate combinations of 1280x720@30fps, 1076x605@21fps, and 905x509@15fps.
Automatic will select the degradation preference from the above 3 options based on the
VideoType of the video source. There are three possible video types, and each type results in a different degradation preference:
Cameratranslates to a degradation preference of
Screentranslates to a degradation preference of
Unknowntranslates to a degradation preference of
Your application can take explicit control of simulcast behaviour at runtime. In particular, you can add code to:
- Disable and re-enable send encodings
- Selecting an initial receive encoding
- Switching the current receive encoding
Enabling and Disabling Send Encodings
Enabling and disabling send encodings is a 3-step process:
- Get the array of
- Change the
- Set the array of
This round-trip process is ensures that the number of encodings does not change and that the correct encodings are updated.
Selecting an Initial Receive Encoding
RemoteEncoding property of the
VideoStream can be set before opening the connection to request a specific encoding from LiveSwitch. By default, LiveSwitch will attempt the highest quality encoding and downgrade if necessary.
Switching the Current Receive Encoding
Update method can be used after opening a connection to request a different encoding from LiveSwitch. This is a 3-step process:
- Get the connection
- Change the
- Update the connection
If you simply want to set an upper bitrate constraint, e.g. based on the size of a view in your application, then you can set the
MaxReceiveBitrate of your
LiveSwitch will automatically switch the inbound stream to a more suitable encoding if a better option is available to satisfy that constraint.
Client Feature Matrix
|Sending multiple encodings||Yes||Yes||No||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Deactivating a send encoding||Yes||No||No||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Updating a send encoding bitrate||Yes||Yes||No||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Updating a send encoding frame rate||No1||No1||No||No||Yes||Yes||Yes||Yes||Yes||Yes|
|Updating a send encoding scale (size)||Yes||Yes||No||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Receiving an encoding||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Switching receive encoding||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Multiple VP8 encodings||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Multiple H.264 encodings||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Multiple VP9 encodings||No||No||No||No||No||Yes||Yes||Yes||Yes||Yes|
Server Feature Matrix
|Simulcast on forwarded streams (SFU)||Yes||Shipped|
|Simulcast on transcoded streams (SFU)||Yes||Shipped|
|Simulcast on mixed streams (MCU)||Yes||Shipped|
|Client-driven switching to a new receive encoding||Yes||Shipped|
|Server-driven switching to a new receive encoding when the current send encoding is deactivated||Yes||Shipped|
|Server-driven switching to a new receive encoding based on bandwidth estimation||Yes||Shipped|
|Server-driven activation and deactivation of unused send encodings||Yes||Shipped|