Migrating to IceLink 3.2 from 3.0 and 3.1

With IceLink 3.2.0, there are several minor breaking changes, which are overviewed briefly in the release notes. This guide goes into more detail and explains how each breaking change affects your application and whether or not you will need to make any changes. If you are experiencing any build or runtime issues when migrating to IceLink 3.2, this guide is the right place to be.

Deprecation of ImageScaler

In previous versions of the SDK, to create a LocalMedia implementation, one of the methods that had to be implemented was the CreateImageScaler method. This was an abstract method off the RtcLocalMedia class, which was expected to return an instance of the ImageScaler class. The ImageScaler class is now deprecated - its functionality has been merged into the ImageConverter class.

Because of this, the CreateImageScaler method has been removed. You should remove this method from all of your LocalMedia implementations. If you were using a scale value other than 1.0, you can instead set this using the Scale property of the ImageConverter instance you return from your CreateImageConverter method. For example, if your LocalMedia class previously had the following two methods:

public class LocalCameraMedia : FM.IceLink.RtcLocalMedia<...>
{
    public override FM.IceLink.VideoPipe CreateImageConverter(FM.IceLink.VideoFormat outputFormat)
    {
        return new FM.IceLink.Vuv.ImageConverter(outputFormat);
    }

    public override FM.IceLink.VideoPipe CreateImageScaler()
    {
        return new FM.IceLink.Yuv.ImageScaler(1.5);
    }
}
public class LocalCameraMedia extends fm.icelink.RtcLocalMedia<...> {
    @Override
    public fm.icelink.VideoPipe createImageConverter(fm.icelink.VideoFormat outputFormat) {
        return fm.icelink.yuv.ImageConverter(outputFormat);
    }

    @Override
    public fm.icelink.VideoPipe createImageScaler() {
        return fm.icelink.yuv.ImageScaler(1.5);
    }
}

@implementation LocalCameraMedia
- (FMIceLinkVideoPipe *)createImageConverterWithOutputFormat: (FMIceLinkVideoFormat *)outputFormat {
    return [FMIceLinkYuvImageConverter imageConverterWithOutputFormat:outputFormat];
}

- (FMIceLinkVideoPipe *)createImageScaler {
    return [FMIceLinkYuvImageScaler imageScalerWithScale:1.5f];
}
@end
public class LocalCameraMedia : FMIceLinkRtcLocalMedia {
  override func createImageConverter(outputFormat:FMIceLinkVideoFormat) -> FMIceLinkVideoPipe {
      return FMIceLinkYuvImageConverter(outputFormat)
  }

  override func createImageScaler() -> FMIceLinkVideoPipe {
      return FMIceLinkYuvImageScaler(scale: 1.5)
  }
}

In 3.2.0, you instead set the Scale property on the ImageConverter instance, and then return it instead. An example follows.

public class LocalCameraMedia : FM.IceLink.RtcLocalMedia<...>
{
    public override FM.IceLink.VideoPipe CreateImageConverter(FM.IceLink.VideoFormat outputFormat)
    {
        return new FM.IceLink.Vuv.ImageConverter(outputFormat)
        {
            Scale = 1.5
        };
    }
}
public class LocalCameraMedia extends fm.icelink.RtcLocalMedia<...> {
    @Override
    public fm.icelink.VideoPipe createImageConverter(fm.icelink.VideoFormat outputFormat) {
        fm.icelink.yuv.ImageConverter converter = fm.icelink.yuv.ImageConverter(outputFormat);
        converter.setScale(1.5);

        return converter;
    }
}

@implementation LocalCameraMedia
- (FMIceLinkVideoPipe *)createImageConverterWithOutputFormat: (FMIceLinkVideoFormat *)outputFormat {
    FMIceLinkYuvImageConverter* converter = [FMIceLinkYuvImageConverter imageConverterWithOutputFormat:outputFormat];
    [converter setScale: 1.0f];

    return converter;
}
@end
public class LocalCameraMedia : FMIceLinkRtcLocalMedia {
  override func createImageConverter(outputFormat:FMIceLinkVideoFormat) -> FMIceLinkVideoPipe {
      var converter = FMIceLinkYuvImageConverter(outputFormat)
      converter.setScale(1.5)

      return converter
  }
}

Migration

  • Remove any CreateImageScaler method implementations from all LocalMedia implementations.
  • If you were using a value other than 1.0 for the ImageScaler, set this value using the Scale property of your ImageConverter instance.

Versioning of OpenH264 Native Binaries

The licensing terms for Cisco's OpenH264 library prevent us from bundling the native binary with our SDK. Because of this, we have always included code in our examples to demonstrate how to download the OpenH264 binary from Cisco as part of an application's bootstrapping process. In the latest version, the process for doing this has changed slightly. The OpenH264 version number is now appended to the filename of the downloaded native binary. This removes ambiguity as to which version is installed on disk, and also simplifies upgrading when a new OpenH264 version is available. The expected filename can be retrieved by invoking the GetLibaryName method of the new Utility class. An example of this follows.

Note that this section only has code snippets for C# and Java. For Objective-C, you should use the built-in H264 library and for JavaScript, codec support is handled by the browser.


var name = FM.IceLink.OpenH264.Utility.GetLibraryName();
String name = fm.icelink.openh264.Utility.getLibraryName();

As an added bonus, the new version includes an additional helper method that automatically downloads the OpenH264 file. For .NET and JRE environments, the file will automatically be downloaded into an appropriate directory and loaded. For Android, you will have to choose where to place the downloaded file and explicitly load it by invoking the System.load method. This is shown below.

Going forward, this is the recommended method for downloading the OpenH264 binary.

Note: UWP does not support automatic download of the OpenH264 native library.

FM.IceLink.OpenH264.Utility.DownloadOpenH264().Then((o) =>
{
    ...
});
// android
android.view.Context context;
final String filesPath = context.getFilesDir().getPath();

fm.icelink.openh264.Utility.downloadOpenH264(filesPath).then((o) -> {
    String libPath = Paths.get(filesPath, fm.icelink.openh264.Utility.getLoadLibraryName());
    System.load(libPath);
});

// java
fm.icelink.openh264.Utility.downloadOpenH264().then((0) -> {
    ...
});

Migration

  • Remove any application-level code that downloads OpenH264 binaries, and replace it with the Utility.DownloadOpenH264 method.

Enforcing Mandatory RTP/RTCP Multiplexing

Without multiplexing, each audio and video stream requires two distinct connections. One connection handles RTP packets, which hold the actual audio and video payload data, and one connection handles RTCP packets, which controls how and when payload data is sent. In a video conference, this means that you would need four connections for each user - two for audio and two for video. Multiplexing allows you to combine these RTP and RTCP connections, using one connection to send both RTP and RTCP packets. This means that half as many connections are required.

Although RTP/RTCP multiplexing is well supported, there are some older clients that do not support it. Up until now, because of these older clients, the default behavior of the Frozen Mountain SDK has been to "negotiate" with each peer. What this means is that before we commit to multiplexing, we check that this is supported by each peer. This negotiation is done during the offer/answer exchange, which adds extra overhead becuase of the need to generate and send both RTP and RTCP candidates. It also adds extra complexity during the conference itself, as both RTP and RTCP streams require connectivity checks and must be managed independently.

Going forward, the default behavior for the Frozen Mountain SDK will be to require multiplexing. As of Chrome 57, this is the default behavior in Chrome and we expect that more vendors will follow suit. This will slightly reduce the time
it takes to establish a connection and will also reduce the overhead bandwidth that is used to maintain audio and video connections.

If your application needs to work with systems that do not support RTC/RTCP multiplexing, you can enable the old behavior by setting the MultiplexPolicy property of any Connection instance. You can enable this on a per-connection
basis. An example of enabling the old "negotiated" behavior is shown below.


var connection = new FM.IceLink.Connection();
connection.MultiplexPolicy = FM.IceLink.MultiplexPolicy.Negotiated;
fm.icelink.Connection connection = new fm.icelink.Connection();
connection.setMultiplexPolicy(fm.icelink.MultiplexPolicy.Negotiated);
FMIceLinkConnection* connection = [FMIceLinkConnection connection];
[connection setMultiplexPolicy: FMIceLinkMultiplexPolicyNegotiated];
var connection = FMIceLinkConnection()
connection.setMultiplexPolicy(FMIcelinkMultiplexPolicyNegotiated)

Migration

  • If your application connects to endpoints that do not support RTP/RTCP multiplexing, set the MultiplexPolicy property of each Connection instance that you create.

Modifications to Examples' Signalling

The IceLink 3 example applications have been updated to allow interop with the IceLink 2 examples. As a side effect, the IceLink 3.2.0 examples require additional steps to interop with examples from the 3.1.x and 3.0.x version of the IceLink SDK.

The reason for this difference is that the IceLink 2 examples would always join a channel with the pattern "/{channelId}", regardless of whether or not they were using automatic WebSync signalling or manual WebSync signalling. In the 3.0.x and 3.1.x versions of IceLink 3, the examples would use different channel patterns, depending on whether or not they used automatic or manual signalling - "/auto-signalling/{channelId}" and "/manual-signalling/{channelId}", respectively.

With IceLink 3.2.0, the channel name is once again "/{channelId}". To interop with older examples, the older examples must be updated to use the new format. Note that this will only affect a minority of clients, and if you don't specifically need older examples to connect to newer examples, then you can safely ignore this.

Migration:

  • If you wish to connect an example from IceLink 3.0.x and IceLink 3.1.x to an example from IceLink 3.2.x, remove all instances of "/auto-signalling" from the old examples.

Minor Changes

There were also a few minor breaking changes in this release that don't warrant their own section.

  • The RtpSequenceNumber property of the AudioFrame and VideoFrame classes is now of type int, rather than long because the RTP packer header only supports 16-bit sequence numbers. If you have using the value of this property, you may have to modify your type casts slightly.
  • The Duration property has been removed from The VideoFrame class. Audio frames represent data over a fixed duration, but there is no such concept with video frames. No migration should be required, as the behavior of this property was never well-defined.
  • You can no longer modify the FrameDuration property of an AudioSource instance after instantiation. This property is required to initialize codecs, and cannot be reliably changed after initialization. No migration should be required, as the behavior of this property was never well-defined.
  • The duration properties of all audio-related classes are now fixed-point instead of floating-point. Multiplying several floating point numbers together caused subtle errors during timestamp calculations. If you were using the values of these properties, you may have to modify your type casts slightly.
  • The CurrentRoundTripTime and TotalRoundTripTrim properties of the CandidatePairStats class have no been updated to conform more closely to the webrtc stats specification. Both properties are now floating-point numbers, rather than integers. If you were using the values of these properties, you may have to modify your type casts slightly.

Additionally, the behavior of the CurrentRoundTripTime property has changed slightly. This property now measures instantaneous round trip time, rather than smoothed round trip time. If you were relying on the old behavior, you will have to modify your application logic to accomodate this.


If you run into any other issues or have questions about migrating to IceLink 3.2, feel free to open a discussion in the comments below and we'll be happy to assist!