Enabling Acoustic Echo Cancellation

If you ever hear yourself speaking a few hundred milliseconds after you have spoken, this is most likely because the remote peer does not have Acoustic Echo Cancellation (AEC). The remote peer is playing your audio stream through the speakers, then picking it up with the microphone and streaming it back to you. Acoustic Echo Cancellation solves this by keeping track of the audio data you send. If someone retransmits data that is similar enough to the data you have just sent, it is recognized as an echo and is removed from the audio data before playback.

IceLink provides an acoustic echo cancellation implementation. It is included with the standard IceLink distribution as a separate library. Read on for information on how to add this to your projects.

Including the Library

Here we cover the inclusion of the audio processing libraries specifically, full instructions on setting up project structure are covered in the Starting a New Project section. The acoustic echo cancellation implementation can be found in the FM.IceLink.AudioProcessing library and associated native libraries. There are no libraries for iOS and macOS because Apple implements their own echo cancellation so there is no need for IceLink to re-invent the wheel. Similarly, this is not needed on web platforms as each browser has its own implementation.

For a .NET Project 

You will need to include FM.IceLink.AudioProcessing.dll, and the architecture-specific build for the libaudioprocessingfm.dll native library. This can be found in the NET/DotNetXX/Libraries/lib/win_x86 and NET/DotNetXX/Libraries/lib/win_x64 folders of the standard IceLink distribution, where XX is the .NET version, ie: DotNet20, DotNet35, etc.

For an Android or Java Project 

You will need to include fm.icelink.audioprocessing.jarand the architecture-specific build for the libaudioprocessingfmJNI.so native library. This can be found in the Android/Libraries/jniLibs/arm64-v8a, Android/Libraries/jniLibs/armebi-v7a and Android/Libraries/jniLibs/x86 folders. If you are working with proguard you will also need to specify the following proguard rule: 

-keep public class fm.icelink.audioprocessing.** { *; }

For a Xamarin Android Project 

You will need to include FM.IceLink.AudioProcessing.dll, and the architecture-specific build for the libaudioprocessingfm.dll native library. Do not include the Android JNIs. Instead, include the shared object library that can be found in the Xamarin/Libraries/Android/lib/arm64-v8a, Xamarin/Libraries/Android/lib/armebi-v7a and Xamarin/Libraries/Android/lib/x86 folders.


Implementing AecContext

The FM.IceLink.AecContext class is an abstract class that provides baseline acoustic echo cancellation functionality. You will need to implement a platform-specific version of this for each of your projects. There are two methods to implement, CreateOutputMixerSink and CreateProcessor.

You must first implement the CreateOutputMixerSink method. This method creates a sink where the mixed audio output can be played. Generally, this will correspond to your system speakers. In your implementation, instantiate a sink appropriate for your platform, as shown in the examples below.

The next method you must implement is CreateProcessor. For this, you should create and return an instance of FM.IceLink.AudioProcessing.AecProcessor. Its constructor requires you to specify two things: the audio settings and the tail length to use for echo cancellation. The tail length tells the library how long to keep track of audio data for echo cancellation. Generally, you want this to be the combined delay between the sender and receiver. The code below demonstrates how to calculate this for each platform. For .NET, if supported, you can simply provide an instance of FM.IceLink.Dmo.AecProcessor. This processor uses DirectX Media Objects and will generally provide better echo cancellation.

Note that due to the platform limitations mentioned above, there are only code snippets for C# and Android. The snippets assume you are using the FM.IceLink.Dmo library in .NET, and the FM.IceLink.NAudio library as a fallback. For Android, they assume you are using the fm.icelink.android.

DMO and VMs

Note that in virtual environments DMO can still produce echo. For improved results disable all enhancements to your virtual audio devices, but for best results you must not run in a VM.


public class AecContext : FM.IceLink.AecContext
{
    protected override FM.IceLink.AecPipe CreateProcessor()
    {
        if (Dmo.AecProcessor.IsSupported())
        { 
            return new Dmo.AecProcessor();
        }
        else
        {
            var config = new FM.IceLink.AudioConfig(16000, 1);
            var tailLength = FM.IceLink.NAudio.Sink.GetBufferDelay(config) + FM.IceLink.NAudio.Source.GetBufferDelay(config);

            return new FM.IceLink.AudioProcessing.AecProcessor(config, tailLength);
        }
    }

    protected override FM.IceLink.AudioSink CreateOutputMixerSink(FM.IceLink.AudioConfig config)
    {
        return new FM.IceLink.NAudio.Sink(config);
    }
}
public class AecContext extends fm.icelink.AecContext {
    @Override
    protected fm.icelink.AecPipe createProcessor() {
        fm.icelink.AudioConfig config = new fm.icelink.AudioConfig(16000, 1);
        int tailLength = fm.icelink.android.AudioTrackSink.getBufferDelay(config) + fm.icelink.android.AudioRecordSource.getBufferDelay(config);

        return new fm.icelink.audioProcessing.AecProcessor(config, tailLength);
    }

    @Override
    protected fm.icelink.AudioSink createOutputMixerSink(fm.icelink.AudioConfig config) {
        return new fm.icelink.android.AudioTrackSink(config);
    }
}

Now that you've created an AecContext implementation, you have to add it to your local and remote media objects.

Using AecContext with Local and Remote Media

In the previous sections on local and remote media, you were shown how to implement the RtcLocalMedia and RtcRemoteMedia classes. At the time, we glanced over the constructors that required an AecContext parameter. Now that you know how to implement AecContext, you can implement these constructors as well.

You only need to create one AecContext instance for all of your local and remote media objects. The context wraps a set of factory methods that create objects needed for echo cancellation, so you can safely reuse the context instance itself. The following is an example of how to integrate AecContext into your local and remote media implementation.

public class LocalMedia : FM.IceLink.RtcLocalMedia<...>
{
	/// <summary>
    /// Initializes a new instance of the LocalMedia class.
    /// </summary>
    /// <param name="disableAudio">Whether to disable audio.</param>
    /// <param name="disableVideo">Whether to disable video.</param>
    /// <param name="aecContext">Your singleton AEC context, if using software echo cancellation.</param>
    public LocalMedia(bool disableAudio, bool disableVideo, AecContext aecContext)
        : base(disableAudio, disableVideo, aecContext)
    {
    }
}

public class RemoteMedia : FM.IceLink.RtcRemoteMedia<...>
{	
	/// <summary>
    /// Initializes a new instance of the RemoteMedia class.
    /// </summary>
    /// <param name="disableAudio">Whether to disable audio.</param>
    /// <param name="disableVideo">Whether to disable video.</param>
    /// <param name="aecContext">Your singleton AEC context, if using software echo cancellation.</param>
    public RemoteMedia(bool disableAudio, bool disableVideo, AecContext aecContext)
        : base(disableAudio, disableVideo, aecContext)
    {
    }
}
public class LocalMedia : fm.icelink.icelink.RtcLocalMedia<...> {
	/**
 	* @param disableAudio Whether to disable audio.
 	* @param disableVideo Whether to disable video.
 	* @param aecContext Your singleton AEC context, if using software echo cancellation.
 	*/
    public LocalMedia(bool disableAudio, bool disableVideo, AecContext aecContext) {
        super(disableAudio, disableVideo, aecContext);
    }
}

public class RemoteMedia : fm.icelink.icelink.RtcRemoteMedia<...> {	
	/**
 	* @param disableAudio Whether to disable audio.
 	* @param disableVideo Whether to disable video.
 	* @param aecContext Your singleton AEC context, if using software echo cancellation.
 	*/
    public LocalMedia(bool disableAudio, bool disableVideo, AecContext aecContext) {
        super(disableAudio, disableVideo, aecContext);
    }
}

Wrapping Up

That's it for the guide. The next section contains a few helpful references and ideas on where you can go to get help. You can also browser our Advanced Topics guide for more information.