開篇導讀，工程目錄在 https://github.com/MrZhaozhirong/AppWebRTC 自行拾取，工程環境是Gradle4.0.x+Androidx，是手動重新fork整個WebRTCdemo，官方原始碼在這里，
前言
正式開始Android-WebRTC的內容，網上搜索到的不外乎就是WebRTC-Codelab的搬運教程，學習demo也是代碼片段；要不然就是老司機直接Nignx+coturn+webrtc.js.api搭載一套，這些內容我還是感覺不全面，沒有一個很清晰的整體架構認識，所以就決定去由淺到深，一步步的去挖掘，到現在我都不敢說已經完全了解WebRTC里面的所有內容，但起碼是有一個清晰的認知構成圖，WebRTC不單單是一套API或理解為一個SDK，它是一套基于瀏覽器web上針對RTC（Real-Time Communications）實時通信的一套開發協標準（或者說是解決方案？），既然能基于瀏覽器web上的，那其他平臺上肯定就有其對應的實作，
至于RTC與WebRTC有什么區別？實際上，二者不能劃等號，RTC從功能流程上來說，包含采集、編碼、前后處理、傳輸、解碼、緩沖、渲染等很多環節，每一個細分環節，還有更細分的技術模塊，比如，前后處理環節有美顏、濾鏡、回聲消除、噪聲抑制等，采集有麥克風陣列等，編解碼有VP8、VP9、H.264、H.265等等，WebRTC是RTC的一部分，是Google的一個專門針對網頁實時通信的標準及開源專案，只提供了基礎的前端功能實作，包括編碼解碼和抖動緩沖等，開發者若要基于WebRTC開發商用專案，那么需要自行做服務端實作和部署，信令前后端選型實作部署，以及手機適配等一系列具體作業；在此之外還要在可用性和高質量方面，進行大量的改進和打磨，對自身開發能力的門檻要求非常高，一個專業的RTC技術服務系統，需要除了涵蓋上述的通信環節外，實際上還需要有解決互聯網不穩定性的專用通信網路，以及針對互聯網信道的高容忍度的音視頻信號處理演算法，當然常規云服務的高可用、服務質量的保障和監控維護工具等都只能算是一個專業服務商的基本模塊，所以，WebRTC僅是RTC技術堆疊中的幾個小細分的技術組合，并不是一個全堆疊解決方案，
搞清楚這些關系之后，理論基礎是必不可少的，還是強烈建議初學小白先去了解WebRTC的理論知識（之前搬運的兩篇長篇理論文章就很不錯，趕緊淦），再然后才是找代碼閱讀，理論—實踐—再理論—再優化，唯有這條路才是走向技術專家的唯一捷徑，
如何開始？
按照RTC的技術方向劃分，可以簡單分類以下幾個大領域：
- 端對端鏈接 (信令signal、stun打洞、turn轉發)；
- 視頻采集 / 編解碼 / 濾鏡處理；
- 音頻采集 / 編解碼 / 回聲消除降噪處理；
- 實時傳輸 / QoE質量保障；
本篇內容從第一部分（端對端鏈接）開始，分析Android WebRTCdemo的組成，簡單的延伸到官方套件api的解讀分析，
Android的WebRTC demo工程有兩個依賴，一個是官方包，一個是libs/autobanh.jar此包主要是負責websocket的通信，
dependencies {
implementation fileTree(dir: "libs", include: ["*.jar"]) // libs/autobanh.jar
implementation 'androidx.appcompat:appcompat:1.1.0'
implementation 'org.webrtc:google-webrtc:1.0.32006'
}檔案組成及其主要功能，按繼承關系可以如下劃分，（暫不顯示org.webrtc:google-webrtc包里面的類）
Demo的核心基本就是兩大塊，AppRTCClient 和 PeerConnectClient，CallActivity作為一個載體承接邏輯互動，其余的都是圍繞這幾部分進行額外的引數設定，
接下來介紹1v1視頻通話測驗的邏輯，以及如何理解 信令Signal？
什么是信令Signal？
先說說如何利用demo進行測驗：
1. Go to https://appr.tc from any browser and create any room number 先在瀏覽器打開https://appr.tc生成room id，然后進入房間，
2. Start the Android app. Enter the room number and press call. Video call should start. 輸入room id然后鍵入呼叫，
經過步驟1~2之后，ConnectActivity就會跳轉到CallActivity.onCreate，有如下代碼片段：
peerConnectionParameters =
new PeerConnectionClient.PeerConnectionParameters(
intent.getBooleanExtra(EXTRA_VIDEO_CALL, true),
loopback, tracing, videoWidth, videoHeight,
intent.getIntExtra(EXTRA_VIDEO_FPS, 0),
intent.getIntExtra(EXTRA_VIDEO_BITRATE, 0),
intent.getStringExtra(EXTRA_VIDEOCODEC),
intent.getBooleanExtra(EXTRA_HWCODEC_ENABLED, true),
intent.getBooleanExtra(EXTRA_FLEXFEC_ENABLED, false),
intent.getIntExtra(EXTRA_AUDIO_BITRATE, 0),
intent.getStringExtra(EXTRA_AUDIOCODEC),
intent.getBooleanExtra(EXTRA_NOAUDIOPROCESSING_ENABLED, false),
intent.getBooleanExtra(EXTRA_AECDUMP_ENABLED, false),
intent.getBooleanExtra(EXTRA_SAVE_INPUT_AUDIO_TO_FILE_ENABLED, false),
intent.getBooleanExtra(EXTRA_OPENSLES_ENABLED, false),
intent.getBooleanExtra(EXTRA_DISABLE_BUILT_IN_AEC, false),
intent.getBooleanExtra(EXTRA_DISABLE_BUILT_IN_AGC, false),
intent.getBooleanExtra(EXTRA_DISABLE_BUILT_IN_NS, false),
intent.getBooleanExtra(EXTRA_DISABLE_WEBRTC_AGC_AND_HPF, false),
intent.getBooleanExtra(EXTRA_ENABLE_RTCEVENTLOG, false),
dataChannelParameters);
Uri roomUri = intent.getData(); // 默認為https://appr.tc
String roomId = intent.getStringExtra(EXTRA_ROOMID); // 房間rumber
String urlParameters = intent.getStringExtra(EXTRA_URLPARAMETERS);
roomConnectionParameters = new AppRTCClient.RoomConnectionParameters(
roomUri.toString(), roomId, loopback, urlParameters);從名字和引數串列，可以大概知道PeerConnectionParameters是一些與音視頻模塊的相關配置引數；另外一個RoomConnectionParameters也很簡單，就是房間URL和房間ID，接下來就是實體化AppRtcClient和PeerConnectionClient，
// Create connection client. Use DirectRTCClient if room name is an IP
// otherwise use the standard WebSocketRTCClient.
if (loopback || !DirectRTCClient.IP_PATTERN.matcher(roomId).matches()) {
appRtcClient = new WebSocketRTCClient(this); // AppRTCClient.SignalingEvents Callback
} else {
Log.i(TAG, "Using DirectRTCClient because room name looks like an IP.");
appRtcClient = new DirectRTCClient(this);
}
// Create peer connection client.
peerConnectionClient = new PeerConnectionClient(getApplicationContext(), eglBase, peerConnectionParameters, CallActivity.this);
PeerConnectionFactory.Options options = new PeerConnectionFactory.Options();
if (loopback) {
options.networkIgnoreMask = 0;
}
peerConnectionClient.createPeerConnectionFactory(options);
if (screencaptureEnabled) {
startScreenCapture();
} else {
startCall();
}1、備注清楚的描述了，測驗使用ip的房間名字就實體DirectRTCClient，其余情況都使用WebSocketRTCClient，
2、PeerConnectClient是封裝PeerConnection的類，掌握WebRTC理論基礎的同學可以知道，WebRTC實作了以下三套API：
- MediaStream (也可以叫作 getUserMedia)
- RTCPeerConnection
- RTCDataChannel
其中PeerConnection是WebRTC進行網路連接的核心，到Android版本的API使用了Factory工廠模式配置創建PeerConnection，這個留著下文展開分析，
3、第三個細節還想聊聊的就是WebRTC也用到EGL環境，證明底層視頻采集渲染處理都用到OpenGL，這也是留著以后慢慢分析，
4、最后就是Android版本的WebRTC也支持Android L以上的螢屏投影（哎喲~不錯喔）；接下來分析startCall
private void startCall() {
if (appRtcClient == null) {
Log.e(TAG, "AppRTC client is not allocated for a call.");
return;
}
callStartedTimeMs = System.currentTimeMillis();
// Start room connection.
logAndToast(getString(R.string.connecting_to, roomConnectionParameters.roomUrl));
appRtcClient.connectToRoom(roomConnectionParameters); // <- 這句是重點，
// Create and audio manager that will take care of audio routing,
// audio modes, audio device enumeration etc.
audioManager = AppRTCAudioManager.create(getApplicationContext());
// Store existing audio settings and change audio mode to
// MODE_IN_COMMUNICATION for best possible VoIP performance.
// This method will be called each time the number of available audio devices has changed.
audioManager.start((device, availableDevices) -> {
//onAudioManagerDevicesChanged(device, availableDevices);
Log.d(TAG, "onAudioManagerDevicesChanged: " + availableDevices + ", " + "selected: " + device);
// TODO: add callback handler.
});
}看著還蠻簡單的，其中AppRTCAudioManager是創建和管理音頻設備，將負責音頻路由，音頻模式，音頻設備列舉等，功能非常豐富包含了：無線近場傳感器設備，藍牙耳機，傳統有線耳機等，但是這不是本篇文章分析的重點，有興趣的同學自行插眼，以后再學習，
重點是 AppRtcClient.connectToRoom(roomConnectionParameters); 具上分析知道此時AppRTCClient的實體物件是WebSocketRTCClient，里面跳轉到對應的代碼進行分析，
// Connects to room - function runs on a local looper thread.
private void connectToRoomInternal() {
String connectionUrl = getConnectionUrl(connectionParameters);
// connectionUrl = https://appr.tc/join/roomId;
wsClient = new WebSocketChannelClient(handler, this);
RoomParametersFetcher.RoomParametersFetcherEvents callbacks =
new RoomParametersFetcher.RoomParametersFetcherEvents() {
@Override
public void onSignalingParametersReady(final SignalingParameters params) {
WebSocketRTCClient.this.handler.post(new Runnable() {
@Override
public void run() {
WebSocketRTCClient.this.signalingParametersReady(params);
}
});
}
@Override
public void onSignalingParametersError(String description) {
WebSocketRTCClient.this.reportError(description);
}
};
new RoomParametersFetcher(connectionUrl, null, callbacks).makeRequest();
}
// Callback issued when room parameters are extracted. Runs on local looper thread.
private void signalingParametersReady(final SignalingParameters signalingParameters) {
Log.d(TAG, "Room connection completed.");
if (!signalingParameters.initiator || signalingParameters.offerSdp != null) {
reportError("Loopback room is busy.");
return;
}
if (!signalingParameters.initiator && signalingParameters.offerSdp == null) {
Log.w(TAG, "No offer SDP in room response.");
}
initiator = signalingParameters.initiator;
messageUrl = getMessageUrl(connectionParameters, signalingParameters);
// https://appr.tc/message/roomId/clientId
leaveUrl = getLeaveUrl(connectionParameters, signalingParameters);
// https://appr.tc/leave/roomId/clientId
events.onConnectedToRoom(signalingParameters);
wsClient.connect(signalingParameters.wssUrl, signalingParameters.wssPostUrl);
wsClient.register(connectionParameters.roomId, signalingParameters.clientId);
}1、請求連接 https://appr.tc/join/roomId，加入到roomId對應的房間，并回傳當前房間的信令引數signalingParameters，
2、通過回傳的信令引數signalingParameters，獲取兩個房間事件的url（message，leave）還有兩個WebSocket的url，可以看到類似日志列印如下：
com.zzrblog.appwebrtc D/RoomRTCClient: RoomId: 028711912. ClientId: 74648260
com.zzrblog.appwebrtc D/RoomRTCClient: Initiator: false
com.zzrblog.appwebrtc D/RoomRTCClient: WSS url: wss://apprtc-ws.webrtc.org:443/ws
com.zzrblog.appwebrtc D/RoomRTCClient: WSS POST url: https://apprtc-ws.webrtc.org:443
com.zzrblog.appwebrtc D/RoomRTCClient: Request TURN from: https://appr.tc/v1alpha/iceconfig?key=
com.zzrblog.appwebrtc D/WebSocketRTCClient: Room connection completed.
com.zzrblog.appwebrtc D/WebSocketRTCClient: Message URL: https://appr.tc/message/028711912/74648260
com.zzrblog.appwebrtc D/WebSocketRTCClient: Leave URL: https://appr.tc/leave/028711912/74648260
四個url只有一個signalingParameters.wssUrl是真的websocket的url，其他都是https是的請求，可以猜想wssUrl是負責房間內部訪客之間的信令互動的地址，其他的都是用于維護房間狀態，
所以到這里知道，信令其實就是一系列用于維護當前 終端/房間 進行端對端通話預連接的邏輯引數，所以WebRTC不可能定義或者實作 與業務強相關的api，因為不同業務需求，信令引數的型別就會改變，很難實作協議化的統一，
4、再分析wsClient（WebSocketChannelClient）的connect 和 register進一步肯定：wssUrl是用于send發送功能命令進行信令互動，wssPostUrl只有在退出房間的時候進行請求，
5、這部分邏輯分析完之后，不要忘了還有 events.onConnectedToRoom(signalingParameters); 把信令引數回呼宿主CallActivity，
private void onConnectedToRoomInternal(final AppRTCClient.SignalingParameters params) {
signalingParameters = params;
VideoCapturer videoCapturer = null;
if (peerConnectionParameters.videoCallEnabled) {
videoCapturer = createVideoCapturer();
}
peerConnectionClient.createPeerConnection(
localProxyVideoSink, remoteSinks, videoCapturer, signalingParameters);
if (signalingParameters.initiator) {
// 房間創建第一人走這里
peerConnectionClient.createOffer();
} else {
if (params.offerSdp != null) {
peerConnectionClient.setRemoteDescription(params.offerSdp);
// 如果不是房間創建第一人，那就判斷信令是否拿到offersdp，
// 如果有有offersdp就證明有人進入房間并發出了offer
// 設定remote sdp 并 answer
peerConnectionClient.createAnswer();
}
if (params.iceCandidates != null) {
// Add remote ICE candidates from room.
for (IceCandidate iceCandidate : params.iceCandidates) {
peerConnectionClient.addRemoteIceCandidate(iceCandidate);
}
}
}
}分析這一段CallActivity.onConnectedToRoom，配合注釋邏輯應該不難理解，正常測驗流程createPeerConnection之后一般都是走createAnswer的分支，因為用 https://appr.tc 進入房間后便成為創建房間的第一人，按照流程步驟先分析PeerConnection的創建程序，
PeerConnection的創建
現在開始分析PeerConnectionClient是如何創建PeerConnection物件，到現在為止PeerConnectionClient一共進行了三個步驟：1、PeerConnectionClient建構式；2、createPeerConnectionFactory；3、createPeerConnection；按照這個流程貼出詳細代碼，
1、PeerConnectionClient建構式
public PeerConnectionClient(Context appContext, EglBase eglBase,
PeerConnectionParameters peerConnectionParameters,
PeerConnectionEvents events)
{
this.rootEglBase = eglBase; this.appContext = appContext; this.events = events;
this.peerConnectionParameters = peerConnectionParameters;
this.dataChannelEnabled = peerConnectionParameters.dataChannelParameters != null;
final String fieldTrials = getFieldTrials(peerConnectionParameters);
executor.execute(() -> {
Log.d(TAG, "Initialize WebRTC. Field trials: " + fieldTrials);
PeerConnectionFactory.initialize(
PeerConnectionFactory.InitializationOptions.builder(appContext)
.setFieldTrials(fieldTrials)
.setEnableInternalTracer(true)
.createInitializationOptions());
});
}
WebRTC API代碼//
public static class InitializationOptions.Builder {
private final Context applicationContext;
private String fieldTrials = "";
private boolean enableInternalTracer;
private NativeLibraryLoader nativeLibraryLoader = new DefaultLoader();
private String nativeLibraryName = "jingle_peerconnection_so";
@Nullable private Loggable loggable;
@Nullable private Severity loggableSeverity;
... ...
}使用Factory的配置模式初始化PeerConnectionFactory，其中有兩個我稍微留意的地方：fieldTrials 和 nativeLibraryName = "jingle_peerconnection_so"; 這里買個關子，放到后面深入原始碼分析的再詳解，
2、createPeerConnectionFactory
private void createPeerConnectionFactoryInternal(PeerConnectionFactory.Options options) {
// Check if ISAC is used by default.
preferIsac = peerConnectionParameters.audioCodec!=null
&& peerConnectionParameters.audioCodec.equals(AUDIO_CODEC_ISAC);
// Create peer connection factory.
final boolean enableH264HighProfile =
VIDEO_CODEC_H264_HIGH.equals(peerConnectionParameters.videoCodec);
final VideoEncoderFactory encoderFactory;
final VideoDecoderFactory decoderFactory;
if (peerConnectionParameters.videoCodecHwAcceleration) {
encoderFactory = new DefaultVideoEncoderFactory(
rootEglBase.getEglBaseContext(), true, enableH264HighProfile);
decoderFactory = new DefaultVideoDecoderFactory(rootEglBase.getEglBaseContext());
} else {
encoderFactory = new SoftwareVideoEncoderFactory();
decoderFactory = new SoftwareVideoDecoderFactory();
}
final AudioDeviceModule adm = createJavaAudioDevice();
factory = PeerConnectionFactory.builder()
.setOptions(options)
.setAudioDeviceModule(adm)
.setVideoEncoderFactory(encoderFactory)
.setVideoDecoderFactory(decoderFactory)
.createPeerConnectionFactory();
Log.d(TAG, "Peer connection factory created.");
adm.release();
// 篇幅關系，只顯示關鍵代碼，
}
AudioDeviceModule createJavaAudioDevice() {
if (!peerConnectionParameters.useOpenSLES) {
Log.w(TAG, "External OpenSLES ADM not implemented yet.");
// TODO: Add support for external OpenSLES ADM.
}
// Set audio record error callbacks.
JavaAudioDeviceModule.AudioRecordErrorCallback audioRecordErrorCallback;
// Set audio track error callbacks.
JavaAudioDeviceModule.AudioTrackErrorCallback audioTrackErrorCallback;
// Set audio record state callbacks.
JavaAudioDeviceModule.AudioRecordStateCallback audioRecordStateCallback;
// Set audio track state callbacks.
JavaAudioDeviceModule.AudioTrackStateCallback audioTrackStateCallback;
// 篇幅關系就不把代碼貼全了，
return JavaAudioDeviceModule.builder(appContext)
.setSamplesReadyCallback(saveRecordedAudioToFile)
.setUseHardwareAcousticEchoCanceler(!peerConnectionParameters.disableBuiltInAEC)
.setUseHardwareNoiseSuppressor(!peerConnectionParameters.disableBuiltInNS)
.setAudioRecordErrorCallback(audioRecordErrorCallback)
.setAudioTrackErrorCallback(audioTrackErrorCallback)
.setAudioRecordStateCallback(audioRecordStateCallback)
.setAudioTrackStateCallback(audioTrackStateCallback)
.createAudioDeviceModule();
}這里有幾個點需要標記，根據是否使用硬體加速初始化VideoEncode/DecoderFactory；然后就是AudioDeviceModule，是WebRTC-Java層代表音頻模塊的介面定義類，當前的音頻模塊只支持OpenSLES，然后有兩個我非常關心的設定setUseHardwareAcousticEchoCanceler / setUseHardwareNoiseSuppressor，眾所周知WebRTC被google收購前，最聞名的技術點就是音頻處理這一塊，以后必須深挖出文章，
用一張圖簡單描述PeerConnectionFactory的構成：
3、createPeerConnection
終于到了真正創建PeerConnection的地方了，廢話就不說了，看代碼吧，
public void createPeerConnection(final VideoSink localRender,
final List<VideoSink> remoteSinks,
final VideoCapturer videoCapturer,
final AppRTCClient.SignalingParameters signalingParameters)
{
this.localRender = localRender; // 本地視頻渲染載體VideoSink
this.remoteSinks = remoteSinks; // 遠程端視頻渲染載體VideoSink，可能多個，所以是List
this.videoCapturer = videoCapturer; // 本地視頻源
this.signalingParameters = signalingParameters;
executor.execute(() -> {
createMediaConstraintsInternal();
createPeerConnectionInternal();
maybeCreateAndStartRtcEventLog();
});
}
private void createMediaConstraintsInternal() {
// Create video constraints if video call is enabled.
if (isVideoCallEnabled()) {
videoWidth = peerConnectionParameters.videoWidth;
videoHeight = peerConnectionParameters.videoHeight;
videoFps = peerConnectionParameters.videoFps;
}
// Create audio constraints.
audioConstraints = new MediaConstraints();
// added for audio performance measurements
if (peerConnectionParameters.noAudioProcessing) {
Log.d(TAG, "Audio constraints disable audio processing");
audioConstraints.mandatory.add(
new MediaConstraints.KeyValuePair(AUDIO_ECHO_CANCELLATION_CONSTRAINT, "false"));
audioConstraints.mandatory.add(
new MediaConstraints.KeyValuePair(AUDIO_AUTO_GAIN_CONTROL_CONSTRAINT, "false"));
audioConstraints.mandatory.add(
new MediaConstraints.KeyValuePair(AUDIO_HIGH_PASS_FILTER_CONSTRAINT, "false"));
audioConstraints.mandatory.add(
new MediaConstraints.KeyValuePair(AUDIO_NOISE_SUPPRESSION_CONSTRAINT, "false"));
}
// Create SDP constraints.
sdpMediaConstraints = new MediaConstraints();
sdpMediaConstraints.mandatory.add(
new MediaConstraints.KeyValuePair("OfferToReceiveAudio", "true"));
sdpMediaConstraints.mandatory.add(new MediaConstraints.KeyValuePair(
"OfferToReceiveVideo", Boolean.toString(isVideoCallEnabled())));
}
private void createPeerConnectionInternal() {
PeerConnection.RTCConfiguration rtcConfig =
new PeerConnection.RTCConfiguration(signalingParameters.iceServers);
// TCP candidates are only useful when connecting to a server that supports ICE-TCP.
rtcConfig.tcpCandidatePolicy = PeerConnection.TcpCandidatePolicy.DISABLED;
rtcConfig.bundlePolicy = PeerConnection.BundlePolicy.MAXBUNDLE;
rtcConfig.rtcpMuxPolicy = PeerConnection.RtcpMuxPolicy.REQUIRE;
rtcConfig.continualGatheringPolicy = PeerConnection.ContinualGatheringPolicy.GATHER_CONTINUALLY;
rtcConfig.keyType = PeerConnection.KeyType.ECDSA; //Use ECDSA encryption.
// Enable DTLS for normal calls and disable for loopback calls.
rtcConfig.enableDtlsSrtp = !peerConnectionParameters.loopback;
rtcConfig.sdpSemantics = PeerConnection.SdpSemantics.UNIFIED_PLAN;
// 請關注這里
peerConnection = factory.createPeerConnection(rtcConfig, pcObserver);
List<String> mediaStreamLabels = Collections.singletonList("ARDAMS");
if (isVideoCallEnabled()) {
peerConnection.addTrack(createVideoTrack(videoCapturer), mediaStreamLabels);
// We can add the renderers right away because we don't need to wait for an
// answer to get the remote track.
remoteVideoTrack = getRemoteVideoTrack();
if (remoteVideoTrack != null) {
remoteVideoTrack.setEnabled(renderVideo);
for (VideoSink remoteSink : remoteSinks) {
remoteVideoTrack.addSink(remoteSink);
}
}
}
peerConnection.addTrack(createAudioTrack(), mediaStreamLabels);
if (isVideoCallEnabled()) {
for (RtpSender sender : peerConnection.getSenders()) {
if (sender.track() != null) {
String trackType = sender.track().kind();
if (trackType.equals(VIDEO_TRACK_TYPE)) {
Log.d(TAG, "Found video sender.");
localVideoSender = sender;
}
}
}
}
if (peerConnectionParameters.aecDump) {
try {
ParcelFileDescriptor aecDumpFileDescriptor =
ParcelFileDescriptor.open("Download/audio.aecdump"), ...);
factory.startAecDump(aecDumpFileDescriptor.detachFd(), -1);
} catch (IOException e) {
Log.e(TAG, "Can not open aecdump file", e);
}
}
}
private void maybeCreateAndStartRtcEventLog() {
rtcEventLog = new RtcEventLog(peerConnection);
rtcEventLog.start(createRtcEventLogOutputFile());
}代碼編幅有點長，已經是壓縮保留有用的部分，有WebRTC基礎的同學應該可以理解 函式createMediaConstraintsInternal 的邏輯，對應getUserMedia（mediaStreamConstraints）的約束條件設定，
再認真看看 函式createPeerConnectionInternal的邏輯，設定PeerConnection.RTCConfiguration，重點是信令引數SignalingParameters中的iceServers，記錄著業務服務器（就是我們程式員）所提供的打洞服務器stun的url 和 轉發服務器的turn的url，然后使用 PeerConnectionFactory.createPeerConnection(rtcConfig, pcObserver);創建PeerConnection，并通過pcObserver回呼各種狀態處理，由于篇幅關系就不貼代碼了，同學可以自行跟進代碼，記住那幾個IceCandidateConnection的回呼就可以了，
接著就是PeerConnection的兩個addTrack，一起來解讀一下：
List<String> mediaStreamLabels = Collections.singletonList("ARDAMS");
peerConnection.addTrack(createVideoTrack(videoCapturer), mediaStreamLabels);
peerConnection.addTrack(createAudioTrack(), mediaStreamLabels);
private @Nullable AudioTrack createAudioTrack() {
audioSource = factory.createAudioSource(audioConstraints);
localAudioTrack = factory.createAudioTrack(AUDIO_TRACK_ID, audioSource);
localAudioTrack.setEnabled(enableAudio);
return localAudioTrack;
}
private @Nullable VideoTrack createVideoTrack(VideoCapturer capturer) {
surfaceTextureHelper =
SurfaceTextureHelper.create("CaptureThread", rootEglBase.getEglBaseContext());
videoSource = factory.createVideoSource(capturer.isScreencast());
capturer.initialize(surfaceTextureHelper, appContext, videoSource.getCapturerObserver());
capturer.startCapture(videoWidth, videoHeight, videoFps);
localVideoTrack = factory.createVideoTrack(VIDEO_TRACK_ID, videoSource);
localVideoTrack.setEnabled(renderVideo);
localVideoTrack.addSink(localRender);
return localVideoTrack;
}
/// WebRTC.PeerConnection.內部代碼
public RtpSender addTrack(MediaStreamTrack track, List<String> streamIds) {
if (track != null && streamIds != null) {
RtpSender newSender = this.nativeAddTrack(track.getNativeMediaStreamTrack(), streamIds);
if (newSender == null) {
throw new IllegalStateException("C++ addTrack failed.");
} else {
this.senders.add(newSender);
return newSender;
}
} else {
throw new NullPointerException("No MediaStreamTrack specified in addTrack.");
}
}第一個重點，在PeerConnection.addTrack的內部，通過nativeAddTrack之后回傳一個RtpSender的物件，并在java層上維護起來；
好奇細心的同學可能還會發現，除了RtpSender，還有RtpReceiver and RtpTransceiver，RtpTransceiver = RtpSender + RtpReceiver ；看起來有點凌亂，現在先有個認識（挖坑）以后再來深入分析（填坑）
public class PeerConnection {
private final List<MediaStream> localStreams;
private final long nativePeerConnection;
private List<RtpSender> senders;
private List<RtpReceiver> receivers;
private List<RtpTransceiver> transceivers;
... ...
}
public class RtpTransceiver {
private long nativeRtpTransceiver;
private RtpSender cachedSender;
private RtpReceiver cachedReceiver;
... ...
}
public class RtpReceiver {
private long nativeRtpReceiver;
private long nativeObserver;
@Nullable private MediaStreamTrack cachedTrack;
... ...
}
public class RtpSender {
private long nativeRtpSender;
@Nullable private MediaStreamTrack cachedTrack;
@Nullable private final DtmfSender dtmfSender;
... ...
}
第二個重點，把本地的videoTrack和audioTrack添加到PeerConnection之后，接著嘗試從PeerConnection的RtpTransceiver中獲取 遠端的remoteVideoTrack，并把當前對應渲染遠程視頻的VideoSink加入到remoteVideoTrack；同時再獲取對應的，貼出對應的代碼片段：
remoteVideoTrack = getRemoteVideoTrack();
if (remoteVideoTrack != null) {
for (VideoSink remoteSink : remoteSinks) {
remoteVideoTrack.addSink(remoteSink);
}
}
// Returns the remote VideoTrack, assuming there is only one.
private @Nullable VideoTrack getRemoteVideoTrack() {
for (RtpTransceiver transceiver : peerConnection.getTransceivers()) {
MediaStreamTrack track = transceiver.getReceiver().track();
if (track instanceof VideoTrack) {
return (VideoTrack) track;
}
}
return null;
}到這里基本已經解讀完 createPeerConnection的邏輯，這里給出兩個總結思路點：
1、怎樣理解Track，Source，Sink的三者關系，它們是如何連接上的？
答：其實從字面意思可以理解：首先source就是資料來源或者資料輸入的封裝，對于視頻來源一般都是攝像頭物件或者檔案物件，source注入到track輸送軌道，成為一個source與sink的溝通橋梁，sink就相當于輸送軌道的終點水槽，一個source可以流經多個track，一個track最終也可以流到多個sink，并通過不同的sink做處理之后進行輸出，至于在WebRTC的代碼中，VideoSink無外乎就是surfaceview等android系統的渲染載體，也可以是本地檔案寫入，抽象理解這些物件之間的關系有助于以后深入分析代碼，
2、PeerConnection的構成如下所示：（緊接著上方Factory的圖）
onConnectedToRoom
還記得PeerConnection是從哪里觸發創建的嗎？（參考onConnectedToRoomInternal代碼片段）就是在連接房間前訪問https://appr.tc/join/roomid，獲取到信令引數之后的SignalingEvents.onConnectedToRoom回呼，那么現在回歸到這里，正常測驗用瀏覽器訪問 https://appr.tc 隨機生成roomid后進入房間便成為創建房間的第一人，所以在createPeerConnection之后一般都是走createAnswer的分支，從代碼看到createOffer / createAnswer / setRemoteDescription，唯獨缺了setLocalDescription，帶著這些疑問我們繼續分析PeerConnectionClient的邏輯流程，
public void createAnswer() {
executor.execute(() -> {
if (peerConnection != null && !isError) {
isInitiator = false;
peerConnection.createAnswer(sdpObserver, sdpMediaConstraints);
}
});
}
public void createOffer() {
executor.execute(() -> {
if (peerConnection != null && !isError) {
isInitiator = true;
peerConnection.createOffer(sdpObserver, sdpMediaConstraints);
}
});
}
public void setRemoteDescription(final SessionDescription sdp) {
executor.execute(() -> {
// 按需修改sdp設定
SessionDescription sdpRemote = new SessionDescription(sdp.type, sdpDescription);
peerConnection.setRemoteDescription(sdpObserver, sdpRemote);
});
}
private class SDPObserver implements SdpObserver {
@Override
public void onCreateSuccess(SessionDescription sdp) {
if (localSdp != null) {
reportError("LocalSdp has created.");
return;
}
String sdpDescription = sdp.description;
if (preferIsac) {
sdpDescription = preferCodec(sdpDescription, AUDIO_CODEC_ISAC, true);
}
if (isVideoCallEnabled()) {
sdpDescription =
preferCodec(sdpDescription, getSdpVideoCodecName(peerConnectionParameters), false);
}
final SessionDescription renewSdp = new SessionDescription(sdp.type, sdpDescription);
localSdp = renewSdp;
executor.execute(() -> {
if (peerConnection != null && !isError) {
Log.d(TAG, "Set local SDP from " + sdp.type);
peerConnection.setLocalDescription(sdpObserver, sdp);
}
});
}
@Override
public void onSetSuccess() {
executor.execute(() -> {
if (peerConnection == null || isError) {
return;
}
if (isInitiator) {
// For offering peer connection we first create offer and set
// local SDP, then after receiving answer set remote SDP.
if (peerConnection.getRemoteDescription() == null) {
// We've just set our local SDP so time to send it.
Log.d(TAG, "Local SDP set successfully");
events.onLocalDescription(localSdp);
} else {
// We've just set remote description,
// so drain remote and send local ICE candidates.
Log.d(TAG, "Remote SDP set successfully");
drainCandidates();
}
} else {
// For answering peer connection we set remote SDP and then
// create answer and set local SDP.
if (peerConnection.getLocalDescription() != null) {
// We've just set our local SDP so time to send it, drain
// remote and send local ICE candidates.
Log.d(TAG, "Local SDP set successfully");
events.onLocalDescription(localSdp);
drainCandidates();
} else {
Log.d(TAG, "Remote SDP set succesfully");
}
}
});
}
@Override
public void onCreateFailure(String error) {
reportError("createSDP error: " + error);
}
@Override
public void onSetFailure(String error) {
reportError("setSDP error: " + error);
}
}從實作代碼可以看到 PeerConnection的createOffer/Answer的事件都由一個SDPObserver接管，其中有兩個回呼函式onCreateSuccess/onSetSuccess，看名字就可以猜測到onCreateSuccess是createOffer/Answer成功之后的回呼，onSetSuccess是setLocal/RemoteDescription的回呼，
按照正常流程進行解讀：
1、在createPeerConnection之后呼叫createAnswer，觸發回呼SDPObserver.onCreateSuccess，此時全域變數localSdp==null，會跟著創建localSdp并呼叫setLocalDescription
2、創建localSdp呼叫setLocalDescription后，觸發SDPObserver.onSetSuccess，因為是非創建第一人，走isInitiator==false的分支；因為在第一步setLocalDescription，所以PeerConnection.getLocalDescription() != null，回呼PeerConnectionEvents.onLocalDescription(localSdp)到CallActivity；
implements PeerConnectionClient.PeerConnectionEvents
@Override
public void onLocalDescription(SessionDescription sdp) {
runOnUiThread(() -> {
if (appRtcClient != null) {
if (signalingParameters!=null && signalingParameters.initiator) {
appRtcClient.sendOfferSdp(sdp);
} else {
appRtcClient.sendAnswerSdp(sdp);
}
// ... ...
}
});
}3、因為是非創建房間第一人，走isInitiator==false的分支，觸發AppRTCClient實體WebSocketRTCClient.sendAnswerSdp
@Override
public void sendAnswerSdp(SessionDescription sdp) {
handler.post(new Runnable() {
@Override
public void run() {
JSONObject json = new JSONObject();
jsonPut(json, "sdp", sdp.description);
jsonPut(json, "type", "answer");
wsClient.send(json.toString());
}
});
}
public void onWebSocketMessage(String message) {
JSONObject json = new JSONObject(message);
String msgText = json.getString("msg");
String errorText = json.optString("error");
if (msgText.length() > 0) {
json = new JSONObject(msgText);
String type = json.optString("type");
if (type.equals("candidate")) {
events.onRemoteIceCandidate(toJavaCandidate(json));
}else if (type.equals("remove-candidates")) {
JSONArray candidateArray = json.getJSONArray("candidates");
IceCandidate[] candidates = new IceCandidate[candidateArray.length()];
for (int i = 0; i < candidateArray.length(); ++i) {
candidates[i] = toJavaCandidate(candidateArray.getJSONObject(i));
}
events.onRemoteIceCandidatesRemoved(candidates);
} else if (type.equals("answer")) {
if (initiator) {
SessionDescription sdp = new SessionDescription(
SessionDescription.Type.fromCanonicalForm(type), json.getString("sdp"));
events.onRemoteDescription(sdp);
} else {
reportError("Received answer for call initiator: " + message);
}
} else if (type.equals("offer")) {
if (!initiator) {
SessionDescription sdp = new SessionDescription(
SessionDescription.Type.fromCanonicalForm(type), json.getString("sdp"));
events.onRemoteDescription(sdp);
} else {
reportError("Received offer for call receiver: " + message);
}
} else if (type.equals("bye")) {
events.onChannelClose();
} else {
reportError("Unexpected WebSocket message: " + message);
}
} else {
if (errorText.length() > 0) {
reportError("WebSocket error message: " + errorText);
} else {
reportError("Unexpected WebSocket message: " + message);
}
}
}4、wsClient是利用WebSocket建立連接wssUrl的通信物件，之前就分析過wssUrl對應的服務是用于負責房間內部訪客之間交換信令引數，在onWebSocketMessage處理各種型別的訊息并進行回呼，但是這里并不是回呼 type.equals("answer") 型別的資訊，有興趣的同學把這個回呼的資訊全列印出來，這對于了解整個信令引數互動的程序是非常有幫助的，
我這里直接給出答案：可能是 "candidate" / "offer"型別的訊息，也可能是什么資訊都沒收到了， "candidate"型別的訊息可能性比較大，因為"offer"在wssUrl創建鏈接成功之后一般就會立刻收到此類資訊，進而回呼CallActivity.onRemoteDescription，進而呼叫PeerConnectionClient.createAnswer()，有同學可能就懵逼了，第1步不是已經createAnswer了嚒？是的，但是這次回呼SDPObserver.onCreateSuccess后，localSdp！=null，就不再往外發送任何型別的資訊了，到此local/remote的sdp都已經設定成功了，
結束了嗎？
到此本篇文章算是結束了，但是Android-WebRTC還有很多很多內容值得深挖，后續會有一系列文章，記錄自己的學習程序，重點是放在網路連接傳輸，視頻編解碼，音頻處理(回聲消除降噪)等模塊，
下一篇內容多數會是涉及WebRTC Java的API分析，以及打開jingle_peerconnection_so的原始碼之門，
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/275554.html
標籤：其他
下一篇：C++跳躍游戲之能否跳到某個位置