Trying to use Azure Communications' VideoStreamRenderer in VueJs

[photocurio]

I am building a web application for joining Teams meetings, using Azure Communications javascript libraries, and VueJs. I am having trouble rendering the video streams. The Azure library renders an HTML5 video element with a listener for a video stream. In Microsoft’s documents this is done with vanilla JS like this:

const rendererLocal = new VideoStreamRenderer(this.localVideoStream)
const videoView = await rendererLocal.createView({ 'scalingMode': 'Crop' })
document.getElementById('localVideo').appendChild(videoView.target)

Obviously, I don’t want to be adding elements to the DOM this way. My first thought was to add the videoView to data, and then render it with v-html like this:

<div v-html="videoView.target.outerHtml"></div>

This does not work. The videoView object created by the VideoStreamRenderer has event listeners atached to the video element, which don’t get passed in with the v-html directive. There is no simple attribute, such as src, that can be added to the video element. Does anyone know how to properly use VideoStreamRenderer with Vue?

[sai0909]

@photocurio - This may not be the answer. I am also trying to build web app fro joining teams meeting ACS with Vue Js. It would be helpful,If you please share some repo’s for example reference.

[photocurio]

@sai0909 I can’t share the code for the app I built, because it is in a company repo, behind a firewall. I will try to put together a ACS-Vue boilerplate soon, for sharing. I can give you some tips now though.

First, when rendering video streams you do have to do it the Microsoft way. Don’t try to add the stream to Vue’s data. You have to use getElementById and appendChild.

Second, when making ACS objects, like the VideoStreamRenderer, don’t put those into Vue’s data. Instead, add them to the Vue instance as top level properties. Vue won’t be aware of these data objects, and you can’t use data from them in the templates, but your component methods can use them. I’ve pasted an example of my RemoteStream component.

Note that although the videoStream itself has been passed to the component from the parent, the rendererRemote object is just added as a top level property. Then the renderer can be used by other methods on the component.

Side note: you might ask how I got the videoStream passed to the component without adding it to the parent component’s data. Actually I did add that to the parent, and it works. But for the most part I solved problems by keeping ACS objects out of component data.

Also note the listener subscribeToParticipantVideoStream. Listeners are the key to reactivity. Microsoft has built in all sort of events that get emitted by the ACS objects (such as the Call object, or the VideoStream object, etc). You can extract data from the ACS object when the events fire, and then add that data to Vue’s state. This is how I got the isSpeaking property into the component:

1. listen for the event from the ACS object
2. extract the relevant data from the ACS object
3. add that data (usually a boolean or a string) to the Vue component
4. use the data in the template.

<template>
	<div v-if="videoStream" :id="'remote-stream-' + videoStream._tsParticipant.id" 
		class="video-stream remote-stream"
		:class="[isVideoShowing ? 'showing-video' : 'no-video', isSpeaking && 'is-speaking']">
		<span class="display-name">{{ streamName }} <img class="fa fa-muted" v-if="isMuted"
			:src="icons['microphoneSlash']" alt="Muted" /></span>
		<div class="speaker"></div>
	</div>
</template>
<script>
import { VideoStreamRenderer } from '@azure/communication-calling'
import microphoneSlash from '../../img/fontawesome/microphone-slash.svg'
export default {
	name: 'RemoteStream',
	data() {
		return {
			isVideoShowing: false,
			isSpeaking: null,
			isMuted: null,
			noVideoBg: 'no-video-white',
			icons: {
				microphoneSlash
			}
		}
	},
	rendererRemote: null,
	props: {
		videoStream: Object,
		streamName: String
	},
	async mounted() {
		this.subscribeToParticipantVideoStream(this.videoStream)
	},
	methods: {
		/**
		 * Subscribes to remote video stream events 
		 */
		async subscribeToParticipantVideoStream(remoteParticipant) {
			this.isSpeaking = await remoteParticipant.isSpeaking
			this.isMuted = await remoteParticipant.isMuted
			remoteParticipant.on('isSpeakingChanged', async () => {
				this.isSpeaking = await remoteParticipant.isSpeaking
			})
			remoteParticipant.on('isMutedChanged', async () => {
				this.isMuted = await remoteParticipant.isMuted
			})
			remoteParticipant.on('videoStreamsUpdated', (e) => {
				e.added.forEach((v) => {
					this.handleVideoStream(v)
				})
			})
			remoteParticipant.videoStreams.forEach((v) => {
				this.handleVideoStream(v)
			})
		},
		/**
		 * Appends remote video stream to state.
		 */
		async remoteVideoView(remoteVideoStream) {
			this.rendererRemote = new VideoStreamRenderer(remoteVideoStream)
			const videoView = await this.rendererRemote.createView({ 'scalingMode': 'Crop' })
			document.getElementById('remote-stream-' + this.videoStream._tsParticipant.id).appendChild(videoView.target)
		},

		/**
		 * Handler for ACS isAvailableChanged event
		 * This adds or drops video streams.
		 */
		handleVideoStream(remoteVideoStream) {
			remoteVideoStream.on('isAvailableChanged', () => {
				if (
					remoteVideoStream.isAvailable &&
					remoteVideoStream.mediaStreamType == 'Video'
				) {
					this.remoteVideoView(remoteVideoStream)
					this.isVideoShowing = true
				} else {
					this.rendererRemote.dispose()
					this.isVideoShowing = false
				}
			})
			if (
				remoteVideoStream.isAvailable &&
				remoteVideoStream.mediaStreamType == 'Video'
			) {
				this.remoteVideoView(remoteVideoStream)
				this.isVideoShowing = true
			}
		}
	}
}
</script>

[sai0909]

@photocurio Thanks. How did you manage the ACS objects like call agents. Let me share the code which i am trying to do via Store. I am trying to store callAgent in the state but when i am accessing i am not able to get methods from the state.

How and where do you have callAgent in app. @photocurio

    import { createStore } from 'vuex';
import { CallClient, LocalVideoStream, Renderer } from '@azure/communication-calling';
import { AzureCommunicationTokenCredential } from '@azure/communication-common';
/* eslint-disable */

export default createStore({
  state: {
    meetingLinkInput: 'link',
    callAgent: {},
    deviceManager: {},
    outgoing: {
      userToCall: '',
      canCall: false,
      canHangUp: false,
    },
    incoming: {
      callerId: '',
      canAccept: true,
      canDecline: false,
    },
    call: {
      current: {},
      incoming: false,
      outgoing: true,
      showVideo: false,
      inProgress: false,
    },
  },

  getters: {
    getOutgoing: (state) => state.outgoing,
  },

  mutations: {
    async setPrerequisites(state, payload) {
      state.callAgent = payload.callAgent;
      state.deviceManager = payload.deviceManager;
      state.outgoing.canCall = true;
      console.log('payload',await payload.deviceManager.getCameras())
    },
    userToCall(state, payload) {
      state.outgoing.userToCall = payload;
    },
    setIncomingContext(state, payload) {
      state.call.incoming = true;
      state.call.outgoing = false;
      state.incoming.callerId = payload.callerIdentity.communicationUserId;
      state.call.current = payload;
    },
    setOutgoingContext(state) {
      state.call.outgoing = true;
      state.call.incoming = false;
      state.call.showVideo = false;
      state.call.inProgress = true;
    },
    canCall(state, payload) {
      state.outgoing.canCall = payload;
      state.outgoing.canHangUp = !payload;
    },
    callInProgress(state, payload) {
      state.call.inProgress = payload;
    },
    setCall(state, payload) {
      state.call.current = payload;
    },
    setVideo(state, payload) {
      state.call.showVideo = payload;
    },
    hideVideo(state) {
      const videoElement = document.getElementById('video');
      videoElement.innerHTML = '';
      state.call.showVideo = false;
    },
  },

  actions: {
    async initPrerequisites({ commit }) {
      const callClient = new CallClient();
      console.log('in callClient',callClient );
      const tokenCredential = new AzureCommunicationTokenCredential('token');
      const callAgent = await callClient.createCallAgent(tokenCredential);
      console.log(callAgent);
      callAgent.on('incomingCall', async function({incomingCall}) {
          commit('setIncomingContext', incomingCall);
      });
      const deviceManager = await callClient.getDeviceManager();
      console.log(deviceManager);
      commit('setPrerequisites', {callAgent: callAgent, deviceManager: deviceManager});
    },
    async startCall({ commit, state, dispatch }) {
      const camera = await state.deviceManager.getCameras();
      var localVideoStream = []
      if (camera[0]) {
        console.log('camera',camera[0]);
        localVideoStream = new LocalVideoStream(camera[0]);
       // return localvideostream;
    } else {
        console.error(`No camera device found on the system`);
    }
    console.log('localVideoStream',localVideoStream)
      // const localVideoStream =return new LocalVideoStream(videoDeviceInfo[0]);
      const videoOptions = localVideoStream ? { localVideoStreams: [localVideoStream] } : undefined;
      console.log('sai',await state.callAgent.join());
      const call = state.callAgent.join(
        { meetingLink: state.meetingLinkInput.value },
        {videoOptions},
      );
      call.on('callEnded', async () => {
        commit('setOutgoingContext');
        commit('canCall', true);
        commit('callInProgress', false);
        commit('hideVideo');
      });
      commit('callInProgress', true);
      commit('setCall', call);
      commit('setOutgoingContext');
      commit('canCall', false);
      dispatch('showVideo');
    },
    hangUp({ commit, state }) {
      const call = state.call.current.hangUp({ forEveryone: true });
      commit('setCall', call);
      commit('setOutgoingContext');
      commit('hideVideo');
    },
    async showVideo({ commit, state }) {
      const stream = state.call.current.remoteParticipants[0].videoStreams[0];
      const renderer = new Renderer(stream);
      const view = await renderer.createView();

      commit('setVideo', true);
      const videoElement = document.getElementById('video');
      videoElement.appendChild(view.target);
    },
  },
  modules: {
  },
});

[photocurio]

@sai0909

const call = state.callAgent.join(
        { meetingLink: state.meetingLinkInput.value },
        {videoOptions},
      );

Hey Sai,
In my app, the callAgent is in the parent component. It is not added to the state.

I have not gone carefully through your code, and I have not used Vuex. But it looks like you are adding ACS objects to the Vuex state. That was a source of problems for me.

The only reason to add something to state is:

• you have to output data in the template, such as string, or use a boolean to show or hide something
• you need to pass some data to a child component

I don’t think the call object needs to be held in state. The call, an ACS object, has to manage its own state. So, assign the call as a top-level property of your javascript app, as in app.call.

The way you interact with it is to subscribe to events emitted by the app.call, and assign handlers to them. For example, there is an event called stateChanged that is emitted when the caller is out of the lobby and connected to the call or meeting. If you want your UI to change when the caller connects to the remote participant, listen for this event, and then check the value of app.state. Your event handler can setState on the Vuex instance, and change the data and booleans that the UI requires.

You can see examples of such handlers in Microsoft’s boilerplate examples of how to use ACS. See the line

call.on('stateChanged', () => {
    callStateElement.innerText = call.state;
})

I hope this helps!