Remote Media

You've learned how to handle local media. In this tutorial, you'll learn how to handle remote media to capture a remote user's audio and video.

Note

For web, we don't implement remote media because LiveSwitch provides one that works with the major browsers.

Prerequisites

This tutorial requires the Local Media app you have created earlier.

Create Remote Media

To create remote media, do the following:

  1. Define Remote Media: Similar to a user's local media, we implement other participants' remote media by extending the RtcRemoteMedia<T> class. The generic type T represents the type of object used for displaying the video preview of these remote participants. We'll create a class named RemoteMedia in a separate file.

  2. Enable AEC in Remote Media: We've already implemented AEC when we created the local media. We now need to enable it in our remote media so that the remote user's echo can be removed for the local user.

  3. Play Remote Audio: To play audio from a remote video feed, we implement the following:

    • The CreateAudioRecorder method that records a user's audio.
    • The CreateAudioSink method that returns an audio sink for audio playback.
    • The CreateOpusDecoder method that decodes Opus encoder.

  4. Play Remote Video: To play remote video, we implement the following:

    • The CreateVideoRecorder method that records a user's video.
    • The CreateViewSink method that creates a view object to playback a remote video feed.
    • The CreateVp8Decoder, CreateVp9Decoder, and CreateH264Decoder methods that decode the VP8, VP9, and H.264 encoders.
    • The CreateImageConverter method that provides a minor image formatting utility.

Paste the following code inside the HelloWorld namespace of the RemoteMedia.cs file.

public class RemoteMedia : RtcRemoteMedia<System.Windows.Controls.Image>
{
    // Enable AEC
    public RemoteMedia(bool disableAudio, bool disableVideo, AecContext aecContext)
    : base(disableAudio, disableVideo, aecContext)
    {
        Initialize();
    }

    // Remote Audio
    protected override AudioSink CreateAudioRecorder(AudioFormat inputFormat)
    {
        return new FM.LiveSwitch.Matroska.AudioSink(Id + "-remote-audio-" + inputFormat.Name.ToLower() + ".mkv");
    }

    protected override AudioSink CreateAudioSink(AudioConfig config)
    {
        return new FM.LiveSwitch.NAudio.Sink(config);
    }

    protected override AudioDecoder CreateOpusDecoder(AudioConfig config)
    {
        return new FM.LiveSwitch.Opus.Decoder(config);
    }

    // Remote Video
    protected override VideoSink CreateVideoRecorder(VideoFormat inputFormat)
    {
        return new FM.LiveSwitch.Matroska.VideoSink(Id + "-remote-video-" + inputFormat.Name.ToLower() + ".mkv");
    }

    protected override ViewSink<System.Windows.Controls.Image> CreateViewSink()
    {
        return new FM.LiveSwitch.Wpf.ImageSink()
        {
            ViewScale = LayoutScale.Contain
        };
    }

    protected override VideoDecoder CreateVp8Decoder()
    {
        return new FM.LiveSwitch.Vp8.Decoder();
    }

    protected override VideoDecoder CreateVp9Decoder()
    {
        return new FM.LiveSwitch.Vp9.Decoder();
    }

    protected override VideoDecoder CreateH264Decoder()
    {
        return null;
    }

    protected override VideoPipe CreateImageConverter(VideoFormat outputFormat)
    {
        return new FM.LiveSwitch.Yuv.ImageConverter(outputFormat);
    }
}

Congratulations, you've built a LiveSwitch app to handle remote media!