Share your screen over Web(Screen sharing with HTML5)

A whole new concept on how a user can share his desktop or browser screens to other users is “Screen sharing” .

Is it possible?

While webRTC has not yet been supported by IE and Safari and these browsers  donot support screensharing  as well,there are several webapps that allow you and me to share the screens without the need of any plug-ins.The only one thing that is required to allow screen sharing is a browser called Chrome.Some of the webapps that support this screen sharing is same.io,talky.io and appear.in

Want to know how to do this in chrome?

Then follow the below steps-

1.          Type chrome://flags in the address bar
2.          You will find a setting like  “Enable screen capture support in getUserMedia()”
3.          Click the enable button
4.          Restart your browser.

Webapps like same.io and talky.io allows a user to share their screens. By allowing your chrome browser to access your web camera and microphone and by clicking on the “Share my Screen” button the user can share his screens.

Once the user hits that button the user will be provided with a unique web url and if anyone gets to know this url can access your screen through the chrome browser.

The other webapp talky.io allows private sessions wherein a user can protect his screens using a password.

Appear.in is a web app for video meetings that allows the user to share the desktop screen to other participants.

However a tech savvy guy named Rafael Weinstein has proposed a technique which uses Mutation observers and a websocket.

Let’s see how this works:

1. The presenter shares his screen to the viewer using a websocket
2. As the user performs any scroll operation the observer sends the changes and the report back to the viewer using Rafael mutation summary library.

Another method on how screensharing is done is:

1. Rewrite all the pages on the URL to absolute to prevent static images and CSS elements from broken links
2. Clone the page’s document
3. Make the clone readonly, hidden and non-selectable and prevent scrolling.
4. Capture the current position of the screen and add them as data-* attributes as duplicate
5. Create a new BLOB page as duplicate.

The code for this is as follows-

 function screenshot(){
    urlsToAbsolute(document.images);  
    urlsToAbsolute(document.querySelectorAll("link[rel='stylesheet']"));
    urlsToAbsolute(document.scripts);

    var screen= document.documentElement.cloneNode(true);

    screen.style.pointerEvents = 'none';
    screen.style.overflow = 'hidden';
    screen.style.userSelect = 'none';

    var blob1 = new Blob([screen.outerHTML], {type: 'text/html'});

    window.open(window.URL.createObjectURL(blob1));
 }

   

Before the final BLOB is created this java script code should be written so that  when then presenter scrolls the page the user follows.

Set the scrollX and scrollY positions

 addOnPageLoad().
 screen.dataset.scrollX = window.scrollX;
 screen.dataset.scrollY = window.scrollY;

and finally write this below code –

 var script = document.createElement('script');
 script.textContent = '(' + addOnPageLoad_.toString() + ')();';.
 screenshot.querySelector('body').appendChild(script);
 function addOnPageLoad() {
    window.addEventListener('DOMContentLoaded', function(e) {
        var scrollX = document.documentElement.dataset.scrollX || 0;
        var scrollY = document.documentElement.dataset.scrollY || 0;
        window.scrollTo(scrollX, scrollY);
    });
 }

Another method is make the screenshot as a png file and share it using a websocket.

 var img_mimetype = 'images/jpeg';
 var IMG_QUALITY = 90;
 var SEND_INTERVAL = 280;
 var wes = new WebSocket('ws://...', 'sample-protocol');
 wes.binaryType = 'blob';
 function captureAndSendTab() {
    var opts = {format: img_mimetype, quality: IMG_QUALITY};
    chrome.tabs.captureVisibleTab(null, opts, function(dataUrl){
       wes.send(convertDataURIToBlob(dataUrl, IMG_MIMETYPE));
    });
  }
 var intervalId = setInterval(function() {
     if (ws.bufferedAmount == 0) {
         captureAndSendTab();
      }
    }, SEND_INTERVAL);

Conclusion:

Though without the use of WebRTC we have several technologies that help us to make screen sharing possible we still have to wait for the WebRTC to take a leap in making this screen sharing possible with HTML5 .

So guys wait to see the revolution that WebRTC is going to bring.

Plugin less video call using webRTC

Introduction to WebRTC

WebRTC enables peer to peer communication using simple Javascript APIs. It supports to build rich, high quality, RTC applications using Javascript and HTML5. There Three APIs yet to develop RTC applications.

1. Media Stream
2. RTC Peer Connection
3. RTC Data Channel

Getting started using getUserMedia

getUserMedia generates  MediaStream which receives Audio and Video from device. navigator.getUserMedia() gives mediastream having audio and video tracks. To get audio tracks from stream use stream.getAudioTracks() and to get stream.getVideoTracks(). 

getUserMedia takes three parameters constraints object, success callback and failure callback. The out received from the getUserMedia can be passed to RTCPeerConnection.

navigator.getUserMedia({ "audio": true, "video": true });

RTC Peer connection to exchanges streams

RTC peer connection API is to establish a persistent connection between peers. Rest of the process similar to video calling using WebRTC peer connection.

var signalChannel = createSignalingChannel();
var peerConn;
var config = ...;

// execute makeCall make a call
function makeCall(isCaller) {
    peerConn = new RTCPeerConnection(config);

    // sending ice candidates to the other peer
    peerConn.onicecandidate = function (evt) {
        signalChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // set remote stream to the remote video element
    peerConn.onaddstream = function (evt) {
        remoteView.src = URL.createObjectURL(evt.stream);
    };

    // get the local stream and send
    navigator.getUserMedia({ "audio": true, "video": true }, function (stream) {
        selfView.src = URL.createObjectURL(stream);
        peerConn.addStream(stream);

        if (isCaller)
            peerConn.createOffer(gotDescription);
        else
            peerConn.createAnswer(peerConn.remoteDescription, gotDescription);

        function gotDescription(desc) {
            peerConn.setLocalDescription(desc);
            signalChannel.send(JSON.stringify({ "sdp": desc }));
        }
    });
}

signalChannel.onmessage = function (evt) {
    if (!peerConn)
        makeCall(false);

    var signalData = JSON.parse(evt.data);
    if (signalData.sdp)
        peerConn.setRemoteDescription(new RTCSessionDescription(signalData.sdp));
    else
        peerConn.addIceCandidate(new RTCIceCandidate(signalData.candidate));
};

How RTC Peer connection  descriptions exchange works?

1. Peer1 runs the RTCPeerConnection createOffer() method. 
2. The callback argument of this is passed an RTCSessionDescription: Peer1's local session description.
3. In the callback, Peer1 sets the local description using setLocalDescription() and then sends this session description to Peer2 via their signaling channel. Note that RTCPeerConnection won't start gathering candidates until setLocalDescription() is called: this is codified in JSEP IETF draft.
4. Peer2 sets the description Peer1 sent him as the remote description using setRemoteDescription().
5. Peer2 runs the RTCPeerConnection createAnswer() method, passing it the remote description he got from Peer1, so a local session can be generated that is compatible with hers. 
6. The createAnswer() callback is passed an RTCSessionDescription: Peer2 sets that as the local description and sends it to Peer1.
7. When Peer1 gets Peer2's session description,  sets that as the remote description with setRemoteDescription.
8. Both peers now having others streams.

Web RTC audio only Call

Introduction to WebRTC

WebRTC enables peer to peer communication using simple Javascript APIs. It supports to build rich, high quality, RTC applications using Javascript and HTML5. There Three APIs yet to develop RTC applications.

1. Media Stream
2. RTC Peer Connection
3. RTC Data Channel

Getting started using getUserMedia

getUserMedia generates  MediaStream which receives Audio and Video from device. navigator.getUserMedia() gives mediastream having audio and video tracks. To get audio tracks from stream use stream.getAudioTracks() and to get stream.getVideoTracks(). 

getUserMedia takes three parameters constraints object, success callback and failure callback. The out received from the getUserMedia can be passed to RTCPeerConnection.

navigator.getUserMedia({ "audio": true, "video": true });

For audio call you just need to put false to video constraint.

navigator.getUserMedia({ "audio": true, "video": false });

RTC Peer connection to exchanges streams

RTC peer connection API is to establish a persistent connection between peers. Rest of the process similar to video calling using WebRTC peer connection.

var signalChannel = createSignalingChannel();
var peerConn;
var config = ...;

// execute makeCall make a call
function makeCall(isCaller) {
    peerConn = new RTCPeerConnection(config);

    // sending ice candidates to the other peer
    peerConn.onicecandidate = function (evt) {
        signalChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // set remote stream to the remote video element
    peerConn.onaddstream = function (evt) {
        remoteView.src = URL.createObjectURL(evt.stream);
    };

    // get the local stream and send
    navigator.getUserMedia({ "audio": true, "video": false }, function (stream) {
        selfView.src = URL.createObjectURL(stream);
        peerConn.addStream(stream);

        if (isCaller)
            peerConn.createOffer(gotDescription);
        else
            peerConn.createAnswer(peerConn.remoteDescription, gotDescription);

        function gotDescription(desc) {
            peerConn.setLocalDescription(desc);
            signalChannel.send(JSON.stringify({ "sdp": desc }));
        }
    });
}

signalChannel.onmessage = function (evt) {
    if (!peerConn)
        makeCall(false);

    var signalData = JSON.parse(evt.data);
    if (signalData.sdp)
        peerConn.setRemoteDescription(new RTCSessionDescription(signalData.sdp));
    else
        peerConn.addIceCandidate(new RTCIceCandidate(signalData.candidate));
};

How RTC Peerconnection  descriptions exchange works?

1. Peer1 runs the RTCPeerConnection createOffer() method. 
2. The callback argument of this is passed an RTCSessionDescription: Peer1's local session description.
3. In the callback, Peer1 sets the local description using setLocalDescription() and then sends this session description to Peer2 via their signaling channel. Note that RTCPeerConnection won't start gathering candidates until setLocalDescription() is called: this is codified in JSEP IETF draft.
4. Peer2 sets the description Peer1 sent him as the remote description using setRemoteDescription().
5. Peer2 runs the RTCPeerConnection createAnswer() method, passing it the remote description he got from Peer1, so a local session can be generated that is compatible with hers. 
6. The createAnswer() callback is passed an RTCSessionDescription: Peer2 sets that as the local description and sends it to Peer1.
7. When Peer1 gets Peer2's session description,  sets that as the remote description with setRemoteDescription.
8. Both peers now having others streams.

HTML5: Recording Audio,Video Streams

MediaStreamRecorder API is webRTC API that supports recording streams with getUserMedia or remote streams passed over peerconnection session.

 
    <script language="javascript" type="text/javascript">
    function onVideoFail(e) {
        console.log('webcam fail!', e);
      };
 
    function hasGetUserMedia() {
      // Note: Opera is unprefixed.
      return !!(navigator.getUserMedia || navigator.webkitGetUserMedia ||
                navigator.mozGetUserMedia || navigator.msGetUserMedia);
    }
 
    if (hasGetUserMedia()) {
      // Good to go!
    } else {
      alert('getUserMedia() is not supported in your browser');
    }
 
    window.URL = window.URL || window.webkitURL;
    navigator.getUserMedia  = navigator.getUserMedia ||
                             navigator.webkitGetUserMedia ||
                              navigator.mozGetUserMedia ||
                               navigator.msGetUserMedia;
 
    var video = document.querySelector('video');
    var streamRecorder;
    var webcamstream;
 
    if (navigator.getUserMedia) {
      navigator.getUserMedia({audio: true, video: true}, function(stream) {
        video.src = window.URL.createObjectURL(stream);
        webcamstream = stream;
    //  streamrecorder = webcamstream.record();
      }, onVideoFail);
    } else {
        alert ('failed');
    }
 
    function startRecording() {
        streamRecorder = webcamstream.record();
        setTimeout(stopRecording, 10000);
    }
    function stopRecording() {
        streamRecorder.getRecordedData(postVideoToServer);
    }
    function postVideoToServer(videoblob) {
 
        var data = {};
        data.video = videoblob;
        data.metadata = 'test metadata';
        data.action = "upload_video";
        jQuery.post("uploadvideo.php", data, onUploadSuccess);
    }
    function onUploadSuccess() {
        alert ('video uploaded');
    }
 
    </script>
 

    <video autoplay></video>
    <div id="webcamcontrols">
        <button class="recordbutton" onclick="startRecording();">RECORD</button>
    </div>

"getRecorderdedData" return  blob that was recorded.

HTML5 Video Streaming: Media Source API

Streaming Video using HTML5 is now very simple, Media Source API gives streaming video as chunks. playing video with chunks of data is now possible.

This API allows to generate media streams, using  javascript streaming is enabled by extending HMTLElement.

<video id="video" src="" height="300" width="300">

      Video element is not supported in your browser

</video>

window.MediaSource = window.MediaSource || window.WebKitMediaSource;

var mediasource = new MediaSource();

var video = document.getElementById('video');
video.src = window.URL.createObjectURL(ms);

mediasource.addEventListener('webkitsourceopen', function(e) {
  ...
  var sourceBuffer = mediasource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');
  sourceBuffer.append(oneVideoWebMChunk);
  ...

//start playing after video header recieved(i.e; first chunk)

//call media.endOfStream(); before appending last chunk

}, false);

playing large volume media files is a bit hard, just make them as chunks and stream with video element. Remember that Video file header contains vital information to play video, this should be necessary to seek and play stream video. if you want to play video from a segment you should first load video fie header then load the segment data to stream your data.

you start playing the stream after receiving header only.

Media Source API now works only chrome( set --enable-media-source flag ).
Media Source Extensions specification changed, changes expected to be made in next release.