WebRtc是google开源的视频通话技术，Kurento是Kurento公司开源的媒体服务器。两者结合起来可以达到多人视频通话的效果。目前在git上Android端webrtc+Kurento的demo几乎没有，本文主要介绍一下如何将两者结合以及一些需要注意的地方。
-
需要的库
KurentoRoomAndroid: 官方地址为 https://github.com/nubomedia-vtt/kurento-room-client-android我们仅使用其中的除了libjinglepeerconnection的其他jar包。
libjinglepeerconnection: 根据上面的kurentoRoom地址引用下来的库中是有libjinglepeerconnection的，但编译版本较早，新版的webrtc已经有些不同。自行编译webrtc难度较大，读者可以先使用这个版本:https://github.com/BaeBae33/webrtc_android(将相关类放到自己工程下可能会报错，把buildToolsVersion提升到25.0.0及以上即可)
-
权限
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
-
视频通道建立流程
webrtc的流程网上很多，相信很多人都能大致看得懂。这里我就谈谈具体在android下如何实施。
场景为用户A进入Kurento房间并接收B发起的视频流(B已经在房间，并且发布视频,假设我们就是用户A):
- 创建PeerConnectionFactory
在多人视频通话中,我们只需要实例化一个factory。代码如下:
//初始化PeerConnectionFactory，以后用于生产PeerConnection
PeerConnectionFactory.initializeInternalTracer();
PeerConnectionFactory.initializeFieldTrials("");
if (!PeerConnectionFactory.initializeAndroidGlobals(
activity.this, true, true, true)) {
Log.e(TAG, "Failed to initializeAndroidGlobals");
}
options = new PeerConnectionFactory.Options();
options.networkIgnoreMask = 0;
factory = new PeerConnectionFactory(options);
Log.d(TAG, "Peer connection factory created.");
// Set default WebRTC tracing and INFO libjingle logging.
// NOTE: this _must_ happen while |factory| is alive!
Logging.enableTracing("logcat:", EnumSet.of(Logging.TraceLevel.TRACE_DEFAULT));
Logging.enableLogToDebugOutput(Logging.Severity.LS_INFO);
- 连接websocket并加入房间，代码很简单，就不贴了。
- 加入房间成功后，在RoomListener.onRoomResponse内我们接收到类似这样的回调:
{"value":[{"id":"1","streams":[{"id":"webcam"}]}],"sessionId":"il67bnrkjeduve2q1jd2klik8d"}
字符串中存在"webcam"，说明id为1的用户正在房间内发布视频。我们可以从这里初始化相关信息并createOffer。我封装了一个 initpeer方法，大家可以参考一下(用户B的type为OTHER)。
public void initPeer() {
peer = new PeerConnectionClient();
if (type.equals(Type.SELF)) {
sparam = new PeerConnectionClient.SignalingParameters(getServers(), true, id, null, null);
param = new PeerConnectionClient.
PeerConnectionParameters(true, false, true, 640, 480, 0, 0, "VP8",
false, false, 0, "opus", false, false, false, false, false, false, false, null);
} else if (type.equals(Type.OTHER)) {
sparam = new PeerConnectionClient.SignalingParameters(getServers(), false, id, null, null);
param = new PeerConnectionClient.
PeerConnectionParameters(false, false, true, 640, 480, 0, 0, "VP8",
false, false, 0, "opus", false, false, false, false, false, false, false, null);
}
events = new PeerConnectionClient.PeerConnectionEvents() {
@Override
public void onLocalDescription(SessionDescription sdp) {
// TODO Auto-generated method stub
LogCat.i("onLocalDescription1:" + sdp.description);
LogCat.i(type.toString());
if (type.equals(Type.SELF)) {
roomApi.sendPublishVideo(sdp.description, false, 1);
} else if (type.equals(Type.OTHER)) {
LogCat.i(type.toString());
roomApi.sendReceiveVideoFrom(id + "_webcam", sdp.description, 2);
}
}
@Override
public void onIceCandidate(IceCandidate candidate) {
// TODO Auto-generated method stub
LogCat.i("onIceCandidate:" + candidate.toString());
LogCat.i("onIceCandidate:detail1:" + candidate.sdp + "," + candidate.sdpMid + "," + String.valueOf(candidate.sdpMLineIndex));
roomApi.sendOnIceCandidate(id, candidate.sdp, candidate.sdpMid,
String.valueOf(candidate.sdpMLineIndex), 3);
}
@Override
public void onIceCandidatesRemoved(IceCandidate[] candidates) {
// TODO Auto-generated method stub
}
@Override
public void onIceConnected() {
// TODO Auto-generated method stub
}
@Override
public void onIceDisconnected() {
// TODO Auto-generated method stub
}
@Override
public void onPeerConnectionClosed() {
// TODO Auto-generated method stub
}
@Override
public void onPeerConnectionStatsReady(StatsReport[] reports) {
// TODO Auto-generated method stub
}
@Override
public void onPeerConnectionError(String description) {
// TODO Auto-generated method stub
}
};
peer.createPeerConnectionFactory(factory, eglBase.getEglBaseContext(), param, events);
if (type.equals(Type.SELF)) {
VideoRenderer.Callbacks remoteRender = new VideoRenderer.Callbacks() {
@Override
public void renderFrame(VideoRenderer.I420Frame i420Frame) {
LogCat.i(i420Frame.toString());
}
}; peer.createPeerConnection(eglBase.getEglBaseContext(), proxyRenderer, remoteRender, sparam);
peer.createOffer();
} else if (type.equals(Type.OTHER)) {
peer.createPeerConnection(eglBase.getEglBaseContext(), null, proxyRenderer, sparam);
peer.createOffer();
}
}
需要注意的是PeerConnectionEvents中一些回调如何处理，以及自己的peer与其他人的peer的一些参数区别。
- 在RoomListener.onRoomResponse回调中set sdpAnswer:
SessionDescription sdpAnswer = new SessionDescription(SessionDescription.Type.ANSWER,(String) response.getObj().get("sdpAnswer"));
peer.setRemoteDescription(sdpAnswer);
- 在RoomListener.onRoomNotification回调中addRemoteIceCandidate:
if(notification.getMethod().equals(
RoomListener.METHOD_ICE_CANDIDATE)){
// TODO
peer.addRemoteIceCandidate(iceCandidate);
}
如果代码没有错，到此视频通道应该就打通成功，并且能看到B的实时视频了。
-
显示视频画面
SurfaceViewRenderer与ProxyRenderer:
SurfaceViewRenderer是显示的控件,ProxyRenderer是实现了VideoRenderer.Callbacks的一个类:
private class ProxyRenderer implements VideoRenderer.Callbacks {
private VideoRenderer.Callbacks target;
synchronized public void renderFrame(VideoRenderer.I420Frame frame) {
if (target == null) {
Logging.d(TAG, "Dropping frame in proxy because target is null.");
VideoRenderer.renderFrameDone(frame);
return;
}
target.renderFrame(frame);
}
synchronized public void setTarget(VideoRenderer.Callbacks target) {
this.target = target;
}
}
我们可以通过setTarget方法将视频流显示到一个SurfaceViewRenderer上，也可以随时更换到另一个SurfaceViewRenderer上：
proxyRenderer.setTarget(renderer);
-
免费可用的STUN
List<PeerConnection.IceServer> iceServers = new ArrayList<>();
iceServers.add(new PeerConnection.IceServer("stun:stun.xten.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voipbuster.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voxgratia.org:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.sipgate.net:10000"));
iceServers.add(new PeerConnection.IceServer("stun:stun.ekiga.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.ideasip.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.schlund.de:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voiparound.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voipbuster.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voipstunt.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:numb.viagenie.ca:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.counterpath.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.1und1.de:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.gmx.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.bcs2005.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.callwithus.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.counterpath.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.internetcalls.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voip.aebc.com:3478"));
-
可能遇到的问题
- 为什么IceConnectionState一直停在checking?
打洞不通 :( - 为什么只有声音没有画面?
有声音说明打通了，但视频流没有放到控件上。确保这几点正确:- proxyRenderer设置到SurfaceViewRenderer上，并且传递到了PeerConncetionClient内部。
- PCObserver.onAddStream中对stream做了处理:
@Override
public void onAddStream(final MediaStream stream) {
Log.d(TAG, "onAddStream");
executor.execute(new Runnable() {
@Override
public void run() {
if (peerConnection == null || isError) {
return;
}
if (stream.audioTracks.size() > 1 || stream.videoTracks.size() > 1) {
reportError("Weird-looking stream: " + stream);
return;
}
if (stream.videoTracks.size() == 1) {
Log.i(TAG, "onAddStream Success");
remoteVideoTrack = stream.videoTracks.get(0);
remoteVideoTrack.setEnabled(renderVideo);
remoteVideoTrack.addRenderer(new VideoRenderer(remoteRender));
}
}
});
}
- 如果是自己的画面出不来，看看是不是没有createVideoTrack:
if (videoCallEnabled) {
mediaStream = factory.createLocalMediaStream("ARDAMS");
if (videoCapturer == null) {
reportError("Failed to open camera");
return;
}
mediaStream.addTrack(createVideoTrack(videoCapturer));
mediaStream.addTrack(createAudioTrack());
peerConnection.addStream(mediaStream);
findVideoSender();
}
- 如果是其他人的画面出不来，看看sdpMediaConstraints添加OfferToReceiveVideo了没:
if (videoCallEnabled) {//videoCallEnabled || peerConnectionParameters.loopback
sdpMediaConstraints.mandatory.add(
new MediaConstraints.KeyValuePair("OfferToReceiveVideo", "false"));
sdpMediaConstraints.mandatory.add(
new MediaConstraints.KeyValuePair("OfferToReceiveAudio", "false"));
} else {
sdpMediaConstraints.mandatory.add(
new MediaConstraints.KeyValuePair("OfferToReceiveVideo", "true"));
sdpMediaConstraints.mandatory.add(
new MediaConstraints.KeyValuePair("OfferToReceiveAudio", "true"));
}
-
总结
kurento android之路不易，但更多的坑在webrtc中。比如我遇到的nativeFreeFactory(nativeFactory)崩溃，根据日志原因在native层没有分离线程。看样子要改源码了。
