Mediasoup: Building Scalable WebRTC Video Conferencing Applications

What Is Mediasoup?

Mediasoup is an open-source SFU (Selective Forwarding Unit) implementation written in C++ with Node.js bindings. It’s production-ready and used by companies building real-time communication platforms.

Unlike a full conferencing solution (like Jitsi or Kurento), mediasoup is a lower-level building block. You use it as the media engine, then build the rest (signaling, UI, authentication) yourself.

1
Your Application
2
    ↓
3
mediasoup (Media Routing)
4
    ↓
5
WebRTC ↔ Clients

This gives you complete control and flexibility.

Part 1: Understanding SFU Through Mediasoup

What Mediasoup Does

Mediasoup receives audio and video streams from endpoints and selectively forwards them to other endpoints.

1
Client A sends stream → mediasoup → forwards to B, C, D
2
Client B sends stream → mediasoup → forwards to A, C, D
3
Client C sends stream → mediasoup → forwards to A, B, D
4
Client D sends stream → mediasoup → forwards to A, B, C

Each client:

Sends one stream (audio + video)
Receives multiple streams (from all other clients)
Can choose which streams to receive (selective)
Can choose spatial and temporal layers (quality/bandwidth control)

This is the SFU model: receive multiple streams, process them client-side, display as you wish.

Why Mediasoup Over MCU?

MCU (Multipoint Control Unit) would:

Mix all streams into one
Cause high latency (processing)
Require powerful servers

Mediasoup just routes. Minimal processing. Low latency.

1
MCU:  Client A + B + C → Mix → Compose → Single output → All clients
2
SFU:  Client A → Forward → All clients (separately)
3
      Client B → Forward → All clients (separately)
4
      Client C → Forward → All clients (separately)

Part 2: The Architecture — Workers and Routers

Worker Model

Mediasoup uses a worker pool model:

1
┌─────────────────────────────────────────┐
2
│ Node.js Application (Single-threaded)   │
3
└────────────┬────────────────────────────┘
4
             │
5
    ┌────────┴────────┬──────────┬──────────┐
6
    │                 │          │          │
7
   ▼                 ▼          ▼          ▼
8
[Worker 0]      [Worker 1]  [Worker 2]  [Worker 3]
9
(CPU Core 0)    (CPU Core 1)(CPU Core 2)(CPU Core 3)
10
    │                 │          │          │
11
    └─────────────────┴──────────┴──────────┘
12
              (Each handles routers)

Key concept: Each worker consumes one CPU core.

If your server has 8 cores, keep 8 workers ready. Each handles media routing independently.

1
import mediasoup from 'mediasoup';
2

3
const workers: mediasoup.Worker[] = [];
4

5
async function createWorkers(numWorkers: number) {
6
  for (let i = 0; i < numWorkers; i++) {
7
    const worker = await mediasoup.createWorker({
8
      logLevel: 'warn',
9
      logTags: ['rtp', 'rtcp'],
10
      rtcMinPort: 40000,
11
      rtcMaxPort: 40100,
12
    });
13

14
    worker.on('died', () => {
15
      console.error(`Worker ${i} died, restart it`);
16
      // Restart logic here
17
    });
18

19
    workers.push(worker);
20
  }
21
  return workers;
22
}

Router: The Media Hub

When a meeting starts, you create a router on a worker:

1
const router = await selectedWorker.createRouter({
2
  mediaCodecs: [
3
    {
4
      kind: 'audio',
5
      mimeType: 'audio/opus',
6
      clockRate: 48000,
7
    },
8
    {
9
      kind: 'video',
10
      mimeType: 'video/VP8',
11
      clockRate: 90000,
12
    },
13
  ],
14
});

The router is responsible for:

Receiving streams from producers
Forwarding to consumers
Managing transports
Handling media codec negotiation

One router per meeting. All participants in a meeting use the same router.

Note

Important: A router is tied to a worker, which is tied to a CPU core. If you have 1000 concurrent meetings and 8 workers, you’ll distribute ~125 meetings per worker. This is why worker pool design is critical for scaling.

Part 3: Peers, Transports, Producers, and Consumers

What Is a “Peer”?

Tricky part: Mediasoup doesn’t have a native “peer” concept. A peer is an application-level abstraction.

Correct way to think about it:

1
interface Peer {
2
  id: string;                          // Unique peer ID
3
  userId: string;                      // Associated user account
4
  websocket: WebSocket;                // WebSocket connection (for signaling)
5
  sendTransport?: mediasoup.Transport; // For sending media
6
  recvTransport?: mediasoup.Transport; // For receiving media
7
  producers: Map<string, mediasoup.Producer>;  // Audio, video producers
8
  consumers: Map<string, mediasoup.Consumer>;  // Consuming from others
9
  dataProducer?: mediasoup.DataProducer;       // DataChannel producer
10
  dataConsumers: Map<string, mediasoup.DataConsumer>;
11
}

A peer is metadata that ties together:

User identity
WebSocket connection (signaling)
Mediasoup transports, producers, consumers

1
const peers = new Map<string, Peer>();
2

3
// When user connects via WebSocket
4
websocket.on('message', async (msg) => {
5
  const { peerId, action, data } = JSON.parse(msg);
6
  let peer = peers.get(peerId);
7

8
  if (!peer) {
9
    peer = {
10
      id: peerId,
11
      userId: data.userId,
12
      websocket,
13
      producers: new Map(),
14
      consumers: new Map(),
15
      dataConsumers: new Map(),
16
    };
17
    peers.set(peerId, peer);
18
  }
19

20
  // Handle actions (join, produce, consume, etc.)
21
});

Part 4: The Critical Transport Concept

Why Two Separate Transports?

Tricky part: Many developers think one transport handles both sending and receiving.

Wrong:

1
const transport = await router.createWebRtcTransport();
2
// Try to both produce and consume on the same transport
3
// This doesn't work as expected!

Correct:

Mediasoup requires separate transports:

Send Transport: For producing media (audio/video)
Receive Transport: For consuming media

1
// Create send transport on server
2
const sendTransport = await router.createWebRtcTransport({
3
  listenInfos: [{ ip: '0.0.0.0', announcedAddress: 'example.com' }],
4
  enableSctp: true,  // For DataChannel
5
  numSctpStreams: { OS: 1024, MIS: 1024 },
6
});
7

8
// Create recv transport on server
9
const recvTransport = await router.createWebRtcTransport({
10
  listenInfos: [{ ip: '0.0.0.0', announcedAddress: 'example.com' }],
11
  enableSctp: true,
12
  numSctpStreams: { OS: 1024, MIS: 1024 },
13
});
14

15
peer.sendTransport = sendTransport;
16
peer.recvTransport = recvTransport;

Why two transports?

Asymmetry in WebRTC: Sending and receiving use different SDP negotiations
Independent control: You can pause sending without affecting receiving
Better resource management: Clearer separation of concerns

Creating Transports on the Client

The client-side must mirror the server-side transports:

1
// Client-side TypeScript
2
import * as mediasoupClient from 'mediasoup-client';
3

4
const device = new mediasoupClient.Device();
5

6
// After loading device RTP capabilities
7
const sendTransport = device.createSendTransport({
8
  id: serverSendTransport.id,
9
  iceParameters: serverSendTransport.iceParameters,
10
  iceCandidates: serverSendTransport.iceCandidates,
11
  dtlsParameters: serverSendTransport.dtlsParameters,
12
});
13

14
const recvTransport = device.createRecvTransport({
15
  id: serverRecvTransport.id,
16
  iceParameters: serverRecvTransport.iceParameters,
17
  iceCandidates: serverRecvTransport.iceCandidates,
18
  dtlsParameters: serverRecvTransport.dtlsParameters,
19
});
20

21
// Handle connect event (DTLS handshake)
22
sendTransport.on('connect', async ({ dtlsParameters }, callback, errback) => {
23
  try {
24
    // Signal to server to complete DTLS handshake
25
    await signalServer('connectTransport', {
26
      transportId: sendTransport.id,
27
      dtlsParameters,
28
    });
29
    callback();  // Confirm to mediasoup-client
30
  } catch (error) {
31
    errback(error);
32
  }
33
});

Note

Common mistake: Not handling the “connect” event properly. The client won’t complete the DTLS handshake unless you signal the server from the connect event handler.

Part 5: Producing Media (Sending Video/Audio)

The Flow: Client → Server

Step 1: Client gets media stream

1
const stream = await navigator.mediaDevices.getUserMedia({
2
  audio: true,
3
  video: { width: { ideal: 1280 }, height: { ideal: 720 } },
4
});
5

6
const audioTrack = stream.getAudioTracks()[0];
7
const videoTrack = stream.getVideoTracks()[0];

Step 2: Client produces on send transport

1
// Produce audio
2
const audioProducer = await sendTransport.produce({
3
  track: audioTrack,
4
  encodings: [
5
    {
6
      maxBitrate: 128000,  // 128 kbps for audio
7
    },
8
  ],
9
  codecOptions: {
10
    opusNoiseSuppression: true,
11
  },
12
});
13

14
// Produce video
15
const videoProducer = await sendTransport.produce({
16
  track: videoTrack,
17
  encodings: [
18
    {
19
      maxBitrate: 5000000,  // 5 Mbps for video
20
      dtx: false,
21
    },
22
  ],
23
  codecOptions: {
24
    videoGoogleStartBitrate: 1000,
25
  },
26
});
27

28
// Store for later
29
peer.producers.set('audio', audioProducer);
30
peer.producers.set('video', videoProducer);

Step 3: Transport emits “produce” event

The transport emits this event when media is first sent:

1
sendTransport.on('produce', async ({ kind, rtpParameters }, callback, errback) => {
2
  try {
3
    // Signal server to create server-side Producer
4
    const producerId = await signalServer('produce', {
5
      transportId: sendTransport.id,
6
      kind,
7
      rtpParameters,
8
    });
9
    callback({ id: producerId });  // Confirm to mediasoup-client
10
  } catch (error) {
11
    errback(error);
12
  }
13
});

Step 4: Server creates Producer

1
// On server, handle produce signal
2
const producer = await peer.sendTransport.produce({
3
  kind,
4
  rtpParameters,
5
});
6

7
peer.producers.set(kind, producer);
8

9
// Notify all other peers that this peer produced media
10
broadcast('newProducer', {
11
  peerId: peer.id,
12
  producerId: producer.id,
13
  kind,
14
});

Note

Timing is important: The “produce” event fires after transport.produce() is called. You must signal the server before returning from the event. The client waits for your callback.

Part 6: Consuming Media (Receiving Video/Audio)

The Flow: Server → Client

Important: Consuming is reverse order. Server initiates, client responds.

Step 1: Other peers receive “newProducer” notification

When peer A produces, all other peers are notified about peer A’s producer.

Step 2: Server creates Consumer

Peer B’s server-side code:

1
async function subscribeToProducer(peerB: Peer, producerId: string) {
2
  // Check if peer B can consume this producer
3
  if (!router.canConsume({
4
    producerId,
5
    rtpCapabilities: peerB.device.rtpCapabilities,
6
  })) {
7
    throw new Error('Cannot consume this producer');
8
  }
9

10
  // Create consumer on peer B's recv transport
11
  const consumer = await peerB.recvTransport.consume({
12
    producerId,
13
    rtpCapabilities: peerB.device.rtpCapabilities,
14
    paused: true,  // Start paused!
15
  });
16

17
  peerB.consumers.set(consumer.id, consumer);
18

19
  return {
20
    id: consumer.id,
21
    producerId,
22
    kind: consumer.kind,
23
    rtpParameters: consumer.rtpParameters,
24
  };
25
}
26

27
// Signal peer B about the new consumer
28
await signalClient(peerB, 'newConsumer', {
29
  id: consumer.id,
30
  producerId,
31
  kind: consumer.kind,
32
  rtpParameters: consumer.rtpParameters,
33
});

Step 3: Client creates Consumer

1
recvTransport.on('connect', async ({ dtlsParameters }, callback, errback) => {
2
  // Same as send transport
3
  await signalServer('connectTransport', {
4
    transportId: recvTransport.id,
5
    dtlsParameters,
6
  });
7
  callback();
8
});
9

10
// Handle newConsumer notification
11
websocket.on('newConsumer', async (msg) => {
12
  const { id, producerId, kind, rtpParameters } = msg;
13

14
  const consumer = await recvTransport.consume({
15
    id,
16
    producerId,
17
    kind,
18
    rtpParameters,
19
  });
20

21
  // Store the consumer
22
  peer.consumers.set(id, consumer);
23

24
  // Get the track and attach to video element
25
  const track = consumer.track;
26
  const videoElement = document.createElement('video');
27
  videoElement.srcObject = new MediaStream([track]);
28
  videoElement.play();
29

30
  document.body.appendChild(videoElement);
31

32
  // Resume the consumer (server paused it)
33
  await signalServer('resumeConsumer', { consumerId: id });
34
  await consumer.resume();
35
});

Step 4: Server resumes Consumer

1
// Handle resume signal
2
const consumer = peer.consumers.get(consumerId);
3
await consumer.resume();

Note

Critical mistake: Starting consumers in “paused” state is recommended. Resume them after client confirms they’re created. This prevents video freeze or audio glitches. The docs recommend it, but many developers skip this step.

Part 7: Spatial and Temporal Layers (Quality Control)

What Are Layers?

When a client sends video, it can use simulcast or SVC (Scalable Video Coding):

1
Simulcast: Send multiple resolutions simultaneously
2
  - High: 1280x720 @ 30fps
3
  - Medium: 640x360 @ 30fps
4
  - Low: 320x180 @ 30fps
5

6
SVC: Single stream with nested quality layers
7
  - Temporal: 30fps, 15fps, 7.5fps
8
  - Spatial: 1280x720, 640x360, 320x180

The producer sends all layers. The consumer (on server) can select which layers to forward:

1
// Producer sends with simulcast
2
const videoProducer = await sendTransport.produce({
3
  track: videoTrack,
4
  encodings: [
5
    { maxBitrate: 5000000, rid: 'h' },     // High
6
    { maxBitrate: 1000000, rid: 'm' },     // Medium
7
    { maxBitrate: 300000, rid: 'l' },      // Low
8
  ],
9
  codecOptions: {
10
    videoGoogleStartBitrate: 1000,
11
  },
12
});
13

14
// Server-side consumer can select preferred layers
15
const consumer = await peer.recvTransport.consume({
16
  producerId,
17
  rtpCapabilities: peer.device.rtpCapabilities,
18
});
19

20
// Set preferred layers (e.g., medium resolution)
21
consumer.setPreferredLayers({
22
  spatialLayer: 1,      // Medium (0=low, 1=medium, 2=high)
23
  temporalLayer: 1,     // 15fps
24
});
25

26
// Listen for actual layer changes
27
consumer.on('layerschange', (layers) => {
28
  console.log('Effective layers:', layers);
29
  // Notify client about actual layers being sent
30
});

This is how Zoom handles bandwidth optimization: it downgrades layers for slow clients.

Part 8: Data Channels (Non-Media Communication)

Producing Data (Sending)

For things like chat messages, you can use DataChannels:

1
// Client produces data
2
const dataProducer = await sendTransport.produceData({
3
  ordered: true,
4
  maxRetransmits: 3,
5
});
6

7
peer.dataProducer = dataProducer;
8

9
// Send data through it
10
dataProducer.send(JSON.stringify({
11
  type: 'chat',
12
  message: 'Hello!',
13
  timestamp: Date.now(),
14
}));

Consuming Data (Receiving)

1
// Server creates data consumer
2
const dataConsumer = await peer.recvTransport.consumeData({
3
  dataProducerId,
4
  paused: true,
5
});
6

7
peer.dataConsumers.set(dataConsumer.id, dataConsumer);
8

9
// Handle incoming data
10
dataConsumer.on('message', (msg) => {
11
  const data = JSON.parse(msg);
12
  console.log('Received:', data);
13
});
14

15
dataConsumer.on('open', () => {
16
  console.log('DataChannel opened');
17
});

Note

Use cases for DataChannels:

Chat messages
Typing indicators
Pointer/cursor sharing
Game state sync
Metadata exchange

Anything that’s not audio/video but needs low-latency delivery.

Part 9: The Tricky Part — Signaling Synchronization

Common Mistake: Forgetting to Signal Both Sides

Many developers create mediasoup objects but forget to tell the other side:

Wrong:

1
// Server creates consumer
2
const consumer = await transport.consume({
3
  producerId,
4
  rtpCapabilities,
5
});
6
// Oops, forgot to signal the client!
7
// Client has no idea consumer was created
8
// Video won't appear

Correct:

1
// Server creates consumer
2
const consumer = await transport.consume({
3
  producerId,
4
  rtpCapabilities,
5
  paused: true,
6
});
7

8
// Signal client about new consumer
9
await sendSignal(peerId, 'newConsumer', {
10
  id: consumer.id,
11
  producerId,
12
  kind: consumer.kind,
13
  rtpParameters: consumer.rtpParameters,
14
});
15

16
// Client receives signal and creates matching consumer
17
// Client calls resume when ready

Another Mistake: Not Handling Close Events

Wrong:

1
// Server closes a producer
2
await producer.close();
3
// Client doesn't know producer closed
4
// Client's consumer becomes stale

Correct:

1
// Listen for close events and notify client
2
producer.on('transportclose', () => {
3
  // Transport closed, all producers/consumers on it are closed
4
  broadcast('producerClosed', { producerId: producer.id });
5
});
6

7
// Client receives close signal
8
websocket.on('producerClosed', async ({ producerId }) => {
9
  const consumer = findConsumerByProducerId(producerId);
10
  await consumer.close();  // Clean up client-side
11
});

The core principle: Mediasoup doesn’t auto-sync between client and server. You must signal every action.

Part 10: Troubleshooting Common Issues

Issue 1: “Producer is closed” Error

Cause: Producer closed on server, but client tried to produce again.

Solution: Listen to producer events and handle closure:

1
audioProducer.on('transportclose', () => {
2
  console.log('Transport closed, producer is dead');
3
  setProducerActive(false);
4
});
5

6
audioProducer.on('score', (score) => {
7
  // Monitor producer quality
8
  if (score.score < 5) {
9
    console.warn('Poor producer quality');
10
  }
11
});

Issue 2: “No suitable codec” Error

Cause: Client’s RTP capabilities don’t match router’s codecs.

Solution: Ensure device is loaded properly:

1
const device = new mediasoupClient.Device();
2

3
// Load device with router's RTP capabilities
4
await device.load({
5
  routerRtpCapabilities: routerCapabilities,
6
});
7

8
// Now device knows what codecs are supported
9
const sendTransport = device.createSendTransport({
10
  id: serverTransport.id,
11
  // ...
12
});

Issue 3: Video Freezes or Choppy Audio

Cause: Consumer’s preferred layers don’t match producer’s capabilities.

Solution: Check consumer state:

1
// Monitor consumer stats
2
consumer.on('statschange', (stats) => {
3
  console.log('Consumer stats:', stats);
4
  // Check jitter, packet loss, bitrate
5
  if (stats.jitter > 100) {
6
    console.warn('High jitter');
7
  }
8
});
9

10
// Check if consumer is paused
11
if (consumer.paused) {
12
  console.log('Consumer is paused');
13
  // Might need to resume
14
}

Part 11: The Connection Flow (End-to-End)

Let’s trace a complete flow:

1
// 1. User A joins meeting
2
websocket.emit('joinMeeting', { meetingId, userId: 'A' });
3

4
// 2. Server creates router and transports for A
5
const router = await worker.createRouter({ mediaCodecs });
6
const sendTransportA = await router.createWebRtcTransport();
7
const recvTransportA = await router.createWebRtcTransport();
8

9
// 3. Server sends transports to client A
10
websocket.emit('joinedMeeting', {
11
  sendTransport: {
12
    id: sendTransportA.id,
13
    iceParameters: sendTransportA.iceParameters,
14
    // ...
15
  },
16
  recvTransport: {
17
    // ...
18
  },
19
});
20

21
// 4. Client A creates local transports
22
const sendTransport = device.createSendTransport(/* server data */);
23
const recvTransport = device.createRecvTransport(/* server data */);
24

25
// 5. Client A connects sendTransport (DTLS handshake)
26
sendTransport.on('connect', async ({ dtlsParameters }, callback) => {
27
  await signalServer('connectTransport', {
28
    transportId: sendTransport.id,
29
    dtlsParameters,
30
  });
31
  callback();
32
});
33

34
// 6. User B joins meeting
35
websocket.emit('joinMeeting', { meetingId, userId: 'B' });
36

37
// 7. Server creates transports for B
38
const sendTransportB = await router.createWebRtcTransport();
39
const recvTransportB = await router.createWebRtcTransport();
40

41
// 8. Server notifies A about new peer B
42
websocket.emit('newPeer', { peerId: 'B' });
43

44
// 9. User A produces video
45
const videoProducer = await sendTransport.produce({
46
  track: videoTrack,
47
  encodings: [{ maxBitrate: 5000000 }],
48
});
49

50
// 10. sendTransport emits "produce" event
51
sendTransport.on('produce', async ({ kind, rtpParameters }, callback) => {
52
  const producerId = await signalServer('produce', {
53
    transportId: sendTransport.id,
54
    kind,
55
    rtpParameters,
56
  });
57
  callback({ id: producerId });
58
});
59

60
// 11. Server creates producer on A's sendTransport
61
const producerA = await sendTransportA.produce({
62
  kind: 'video',
63
  rtpParameters,
64
});
65

66
// 12. Server notifies all peers (including B) about A's producer
67
broadcast('newProducer', {
68
  peerId: 'A',
69
  producerId: producerA.id,
70
  kind: 'video',
71
});
72

73
// 13. Server creates consumer on B's recvTransport
74
const consumerA_onB = await recvTransportB.consume({
75
  producerId: producerA.id,
76
  rtpCapabilities: rtpCapabilitiesB,
77
  paused: true,
78
});
79

80
// 14. Server signals B about A's video
81
websocket.emit('newConsumer', {
82
  id: consumerA_onB.id,
83
  producerId: producerA.id,
84
  kind: 'video',
85
  rtpParameters: consumerA_onB.rtpParameters,
86
});
87

88
// 15. Client B creates consumer on recvTransport
89
const consumer = await recvTransport.consume({
90
  id: consumerA_onB.id,
91
  producerId: producerA.id,
92
  kind: 'video',
93
  rtpParameters,
94
});
95

96
// 16. B's track is ready, attach to video element
97
const videoElement = document.createElement('video');
98
videoElement.srcObject = new MediaStream([consumer.track]);
99
videoElement.play();
100

101
// 17. Client B resumes consumer
102
await consumer.resume();
103
websocket.emit('resumeConsumer', { consumerId: consumerA_onB.id });
104

105
// 18. Server resumes consumer
106
const consumer = peersB.consumers.get(consumerId);
107
await consumer.resume();
108

109
// 19. Video flows: A → mediasoup router → B
110
// Success!

That’s the complete flow.

Part 12: Production Considerations

Monitoring

Monitor these metrics:

1
// Worker CPU usage
2
worker.on('died', () => {
3
  console.error('Worker crashed');
4
  // Restart logic
5
});
6

7
// Producer quality
8
producer.on('score', (score) => {
9
  metrics.recordProducerScore(producer.id, score.score);
10
});
11

12
// Consumer stats
13
setInterval(async () => {
14
  const stats = await consumer.getStats();
15
  metrics.recordConsumerStats(consumer.id, stats);
16
}, 5000);

Scaling

For 1000+ concurrent users:

Multiple workers: 1 per CPU core
Load balancing: Distribute users across workers
Router pooling: Reuse routers across meetings
Memory management: Close transports/consumers cleanly
Database: Track user → router → worker mappings

Security

Authenticate WebSocket connections
Validate user permissions before allowing consume
Rate limit signaling messages
Use TLS for all connections

Conclusion: Mediasoup Is Powerful But Requires Care

Mediasoup is an excellent choice for building video conferencing. It’s:

Low-level (you control everything)
High-performance (forwarding, not mixing)
Scalable (worker model)
Flexible (you build the app layer)

But it requires understanding:

The SFU model
Producer/Consumer flows
Transport mechanics
Signaling synchronization
Event handling

The most common mistakes:

Forgetting to signal both client and server
Not handling close events
Not starting consumers in paused state
Misunderstanding separate send/recv transports
Not monitoring producer/consumer quality

Master these, and you can build production-grade video conferencing systems.

References

Now go build something amazing.