I'm trying to write a proof of concept to show that a 3rd party hardware device with "Videophone" capabilities can call a web application on the same LAN. At the recommendation of the manufacturer of the device, I'm using linphone.
Firstly, Linphone for Windows works perfectly (After enabling H263 under video codecs) So to clarify, the 3rd party device makes use of the H263 video codec.
Linphone for the browser using my proof of concept, shows video when called from linphone for windows but NOT when called from the 3rd part device.
First Problem
After some digging through https://www.linphone.org/docs/linphone-web/external-LinphoneCore.html and looking at the results of "core.videoCodecs" I noticed that the only video codec returned was VP8.
After more digging, the x-linphone-web plugin does not include all codecs by default. So I recompiled the plugin with support for "H263" enabled. "core.videoCodecs" now returned with "H263" as one of the codecs. At this point, there is still no video when I call from the 3rd party device to my proof of concept.
Other things I've tried:
I noticed that linphone for windows exports its configuration to a linphonerc file. Tried to initialise the linphone core using the same parameters an linphone for windows
3.1 Tried using core.init('URI to file using the format file:///C:/test.linphonerc'); This caused an invalid URI error.
3.2 Tried using core.init('URI to file using the format 'local:///test.linphonerc'); This didn't cause an error but created new file C:\Users***\AppData\Local\Temp\linphone-web\8d89-653a-c164-9d79\test.linphonerc
3.3 Tried creating the file in code using newLpConfig('local:///test.linphonerc') then initialising using core.init('local:///test.linphonerc'). newLpConfig writes to a new folder in C:\Users***\AppData\Local\Temp\linphone-web\ and init tries to read from yet another new folder in C:\Users***\AppData\Local\Temp\linphone-web\
Looking at wireshark logs to see the difference between calling linphone for windows and calling proof of concept with linphone browser from 3rd party device.
4.1 Linphone(Windows) replies to the SIP Invite with a "101 Dialog Establishment" while Linphone(Browser) replies with a "100 Trying"
4.2 When the "answer call" button is pressed, Linphone(Windows) replies with "200 OK" that includes a SDP media description "Media Description, name and address (m): video 9078 RTP/AVP 34" while Linphone(Browser) replies with "200 OK" with a media description "Media Description, name and address (m): video 0 RTP/AVP 0"
4.3 On answer, Linphone(Windows) replies with "200 OK" that includes a SDP media attribute "Media Attribute (a): rtpmap:34 H263/90000" while Linphone(Browser) replies with "200 OK" without a video media attribute.
Incase someone would like to see the code.
linphone.html
<html>
<head>
<script type="text/javascript" src="linphone.js"></script>
</head>
<body onload="initLinphoneCore()" style="background:#000000; font-family : Arial; font-size: 12;">
<object id="core" type="application/x-linphone-web" width="0" height="0">
<param name="onload" value='initLinphoneCore()'>
</object>
<div style="width : 640px; height : 480px; position : absolute; top : 30%; left : 30%; right : 30%; background : #EEEEEE;">
<div id="status" style="position:relative; height:30px; font-size:18px;top:0px; background:#888888; color:#FFFFFF;text-align: center;"></div>
<div id="videoContainer1" style="position:relative; top: 25%; left:25%;" ></div>
<div style="position:absolute; width : 100%; height:30px; font-size:18px; bottom:0px; text-align: right; background : #333333;">
<input type="button" OnClick="answerCall()" value="Answer Call">
</div>
</div>
</body>
</html>
linphone.js
var CallStatusIdle = 0;
var CallStatusIncomingReceived = 1;
var CallStatusConnected = 6;
var CallStatusStreamsRunning = 7;
var CallStatusError = 12;
var CallStatusEnd = 13;
var currentCall;
var currentCallStatus = CallStatusIdle;
//-------------------------------------------------------------------------------------------------------
// Name : addEvent
// Desc : Register to be notified of an event.
// param obj :
// param name :
// param func :
//-------------------------------------------------------------------------------------------------------
function addEvent(obj, name, func)
{
if (obj.attachEvent) {
obj.attachEvent("on"+name, func);
} else {
obj.addEventListener(name, func, false);
}
}
//-------------------------------------------------------------------------------------------------------
// Name : updateStatus
// Desc : Function that display some text in a html element.
// param id :
// param value :
// param bgColour:
// param fgColour:
//-------------------------------------------------------------------------------------------------------
function updateStatus(id, value, bgColour, fgColour)
{
document.getElementById(id).innerHTML= value;
document.getElementById(id).style.background = bgColour;
document.getElementById(id).style.color = fgColour;
}
//-------------------------------------------------------------------------------------------------------
// Name : getCore
// Desc : Get a reference to linphone core.
//-------------------------------------------------------------------------------------------------------
function getCore()
{
return document.getElementById('core');
}
//-------------------------------------------------------------------------------------------------------
// Title : initLinphoneCore.
// Desc : Initialise the linphone core.
//-------------------------------------------------------------------------------------------------------
function initLinphoneCore()
{
var core = getCore();
addEvent(core, "callStateChanged", onCallStateChanged);
/*
Initial attempt was to initialise using core.init();
Haven't managed to get that working so attempt "n" is to specify the configuration as exported using
the windows desktop version of linphone.
*/
var config = core.newLpConfig("local:///config.linphonerc");
config.cleanSection("rtp");
config.setString('rtp', 'download_ptime', "0");
config.setString('rtp', 'audio_rtp_port', "7078");
config.setString('rtp', 'video_rtp_port', "9078");
config.setString('rtp', 'audio_jitt_comp', "60");
config.setString('rtp', 'video_jitt_comp', "60");
config.setString('rtp', 'nortp_timeout', "30");
config.setString('rtp', 'audio_adaptive_jitt_comp_enabled', "1");
config.setString('rtp', 'video_adaptive_jitt_comp_enabled', "1");
config.setString('rtp', 'audio_dscp', "0x2e");
config.setString('rtp', 'video_dscp', "0x2e");
config.cleanSection("sip");
config.setString('sip', 'media_encryption', "none");
config.setString('sip', 'default_proxy', "-1");
config.setString('sip', 'sip_port', "5062");
config.setString('sip', 'sip_tcp_port', "0");
config.setString('sip', 'sip_tls_port', "0");
config.setString('sip', 'use_info', "1");
config.setString('sip', 'guess_hostname', "1");
config.setString('sip', 'inc_timeout', "30");
config.setString('sip', 'in_call_timeout', "0");
config.setString('sip', 'delayed_timeout', "4");
config.setString('sip', 'use_ipv6', "0");
config.setString('sip', 'register_only_when_network_is_up', "1");
config.setString('sip', 'register_only_when_upnp_is_ok', "1");
config.setString('sip', 'dscp', "0x1a");
config.cleanSection("video");
config.setString('video', 'display', "1");
config.setString('video', 'capture', "1");
config.setString('video', 'automatically_initiate', "1");
config.setString('video', 'automatically_accept', "1");
config.setString('video', 'show_local', "0");
config.setString('video', 'self_view', "0");
config.setString('video', 'size', "svga");
// config.setString('video', 'device', "Removed for SO Question");
config.cleanSection("net");
config.setString('net', 'download_bw', "0");
config.setString('net', 'upload_bw', "0");
config.setString('net', 'adaptive_rate_control', "1");
config.setString('net', 'firewall_policy', "0");
config.setString('net', 'mtu', "1300");
config.cleanSection("sound");
// config.setString('sound', 'playback_dev_id', "Removed for SO Question");
// config.setString('sound', 'ringer_dev_id', "Removed for SO Question");
// config.setString('sound', 'capture_dev_id', "Removed for SO Question");
config.setString('sound', 'echocancellation', "0");
config.setString('sound', 'mic_gain_db', "0.000000");
config.setString('sound', 'local_ring', "C:\Program Files (x86)\Linphone\share\sounds\linphone\rings\oldphone.wav");
config.setString('sound', 'playback_gain_db', "0.000000");
config.cleanSection("video_codec_0");
config.setString('video_codec_0', 'mime', "H263");
config.setString('video_codec_0', 'rate', "90000");
config.setString('video_codec_0', 'enabled', "1");
config.setString('video_codec_0', 'recv_fmtp', "");
config.cleanSection("video_codec_1");
config.setString('video_codec_1', 'mime', "VP8");
config.setString('video_codec_1', 'rate', "90000");
config.setString('video_codec_1', 'enabled', "1");
config.cleanSection("video_codec_2");
config.setString('video_codec_2', 'mime', "H263-1998");
config.setString('video_codec_2', 'rate', "90000");
config.setString('video_codec_2', 'enabled', "1");
config.setString('video_codec_2', 'recv_fmtp', "1;QCIF=1");
// Write configuration file:
var configFileSynch = config.sync();
if ( configFileSynch !== 0 ) {
console.log( 'Failed to write configuration file.' );
return;
}
// Initialise core:
var coreInit = core.init( 'local:///config.linphonerc' );
if ( coreInit !== 0 )
{
console.log( 'Failed to initialise core.' );
return;
}
// Setup core logging.
core.logHandler = function( level, message )
{
window.console.log(message);
}
// Start main loop:
core.iterateEnabled = true;
updateStatus ( 'status', 'Linphone Core Initialised', "#888888", '#FFFFFF');
}
//-------------------------------------------------------------------------------------------------------
// Name : onCallStateChanged
// Desc : On notification of call status, decide on the logic to perform.
// param event : The event that triggered all of this.
// param call : The call object that had its state changed.
// param state : The new state of the call.
// param message : A description of the call state.
//-------------------------------------------------------------------------------------------------------
function onCallStateChanged(event, call, state, message)
{
try {
// Keep a reference for later use:
currentCall = call;
currentCallStatus = state;
// Log new call state:
window.console.log('Call state changed : ' + state);
if ( CallStatusIncomingReceived === state )
{
updateStatus('status', message, "#0000AA", '#FFFFFF');
document.getElementById('videoContainer1').innerHTML= "<object id='remoteVideo1' type='application/x-linphone-web-video' style='display: inline; width: 320px; height: 240px; position:absolute'><param name='onload' value='onLoadVideo1' /><param name='magic' value='1' /></object>";
}
else if ( CallStatusConnected === state )
{
updateStatus('status', message, "#000000", '#FFFFFF');
}
else if( CallStatusStreamsRunning === state )
{
updateStatus('status', message, "#00AA00", '#FFFFFF');
}
else if( CallStatusEnd === state )
{
updateStatus('status', message, "#000000", '#FFFFFF');
}
else if( CallStatusError === state )
{
updateStatus('status', message, "#440000", '#FFFFFF');
}
}
catch (err)
{
var msg = "There was an error during call status change : " + err.message;
console.log ( msg );
updateStatus ( 'status', msg, "#000000", '#FFFFFF');
}
}
//-------------------------------------------------------------------------------------------------------
// Name : answerCall
// Desc : User clicked button to answer call.
//-------------------------------------------------------------------------------------------------------
function answerCall()
{
try {
core.acceptCall ( currentCall) ;
}
catch (err)
{
var msg = "There was an error while answering call : " + err.message;
console.log ( msg );
updateStatus ( 'status', msg, "#000000", '#FFFFFF');
}
}
//-------------------------------------------------------------------------------------------------------
// Name : onLoadVideo1
// Desc : Initialise video parameters after x-linphone-web-video injected and loaded.
//-------------------------------------------------------------------------------------------------------
function onLoadVideo1() {
var v = document.getElementById('remoteVideo1');
core.videoPolicy.automaticallyAccept = true;
core.videoDisplayEnabled = true;
core.nativeVideoWindowId = v.window;
}
Any idea why the video is displaying when calling Linphone for windows but not when calling my proof of concept that makes use of Linphone for browser?
This is my first question on https://stackoverflow.com/ If you have any pointers on improving this question please let me know.
This morning I posted the same question with the linphone developers mailing list. I received a reply, which when implemented solved my problem.
Link to the mailing list with the reply. http://lists.gnu.org/archive/html/linphone-developers/2015-05/msg00086.html
Basically, now that the plugin had support for H263, I had to enable the codec.
Enabling the H263 codec in my proof of concept.
core.enablePayloadType ( core.videoCodecs[2], true );