How Connect handles group calls: WebRTC, SFU, and active-speaker detection
By DeskTrust Engineering
This post is for engineers evaluating Connect for their team, or curious about how a small WebRTC add-on scales past 2 participants without becoming an SFU-vendor customer.
The naïve approach: mesh WebRTC
For 1:1 calls, WebRTC peers connect directly and exchange media. Simple. For 3+ people, the naïve approach is a full mesh: every peer connects to every other peer. Each peer sends N-1 upload streams and consumes N-1 downloads.
Doesn\'t scale. At 6 participants, each device is uploading 5x its own bitrate. Bandwidth crushes home connections; CPU crushes laptops.
The SFU approach: mediasoup
Connect uses an SFU (Selective Forwarding Unit). Each peer sends ONE upload to the server. The server forwards it to N-1 other peers as N-1 downloads. Peers only upload their own bitrate, and the server does the fanout.
We chose mediasoup because:
- It\'s a Node.js library, not a heavy dependency. Runs in-process with our Socket.io server.
- It\'s battle-tested in production at scale (used by CoScreen, others).
- Simulcast and SVC are supported (v1 doesn\'t use them, but the path exists).
- Router + Transport + Producer + Consumer abstractions map cleanly to our data model.
What one call looks like server-side
Per session:
- 1 mediasoup Router (created lazily on first call:accept, closed on last hangup)
- N × 2 WebRtcTransports (each participant has a send and receive transport)
- N × M Producers (each participant produces N audio + video + optional screen tracks; M ≈ 2-3)
- N × (N-1) × M Consumers (each participant consumes everyone else\'s producers)
- 1 AudioLevelObserver for active-speaker detection (throttled at 500ms)
Signaling: Socket.io
All signaling — invite, accept, decline, ICE candidates, DTLS parameters, produce, consume, hangup — flows over Socket.io. The client-server request-response pattern for produce and consume uses Socket.io ack callbacks:
socket.emit('call:consume', {
sessionId, transportId, producerId, rtpCapabilities
}, (result) => {
// result contains { consumerId, kind, rtpParameters, source, principalKey }
});For cross-process fanout (HTTP API triggers a socket event), we use a Redis pub/sub channel called connect:emit. The Next.js API publishes JSON messages; the standalone Socket.io process subscribes and re-emits to socket rooms.
Active-speaker detection
Router.createAudioLevelObserver was the biggest single UI-quality win. Threshold set to -70 dBov, interval 500ms. Emits volumes events with the loudest speaker.
Server broadcasts call:active-speaker { sessionId, principalKey } to all participants. Client applies an emerald ring to the matching tile. Making 6+ person calls readable that wouldn\'t be otherwise.
Why we cap at 25 participants
Two reasons:
- Server CPU. mediasoup workers fanout consumers on the CPU. With N participants each producing 2 tracks (audio + video), the server manages N × 2 × (N-1) consumers. At N=25, that\'s ~1,200 concurrent forwarding operations per session.
- UI readability. A 25-tile grid is unreadable. We cap the display to 9 simultaneous visible tiles; participants past that stay audible but not gridded.
The plan-based limit (2/4/10/25 depending on tier) protects the shared VPS from a single 24-person free-tier call.
What happens when someone drops mid-call
- Their Socket disconnects (browser close, tab kill, network loss).
- Server marks that participant\'s
CallParticipant.leftAt = now. - All their producers close automatically (mediasoup cleans up).
- Server broadcasts
call:participant-leftto remaining participants. - Their tile disappears from everyone\'s grid.
- If remaining participants = 0, session ends. Otherwise call continues.
No hung sessions, no phantom tiles. The 8-second grace period on socket disconnect only applies to standalone screen broadcasts (where losing the presenter should tear down the session, but a brief reload shouldn\'t).
What we punted on
- Simulcast (adaptive layers per receiver bandwidth) — v1.1
- SVC (in-track scalable video coding) — v2
- Cross-region SFU sharding — v2, for the KOC-scale deployments
- Bandwidth-based track pause (turn off remote video when receiver is bandwidth-starved) — v1.1
For a 2-25 person call on decent networks, the simple SFU-forwarding path works fine. Those optimizations kick in when you push the extremes.