We have a little Nodejs app, which starts a stream process, with a child_process.spawn. On the client-side, we have an HTML5-canvas element, which records the video data new MediaRecorder(canvas.captureStream(30), config), then this client sends its data to our Nodejs server over a WebSocket connection. We using FFmpeg for video encoding and decoding, then we send the data to our 3-rd party service (MUX), which accepts the stream and broadcasts them. Sadly the process continuously loses its fps, and after in general 1 minute, stops with an interesting error code. (when we save the video result locally instead of streaming via rtmps, it works perfectly.
*The whole system is in docker.
The error:
stderr: [tls @ 0x7f998e7bca40] Error in the pull function.
Our_app_logs: | av_interleaved_write_frame(): I/O error
Our_app_logs: | [flv @ 0x7f998eeb1680] Failed to update header with correct duration.
Our_app_logs: | [flv @ 0x7f998eeb1680] Failed to update header with correct filesize.
Our_app_logs: | Error writing trailer of rtmps://global-live.mux.com/app/94e85197-78a3-f092-3437-03d93aba74e0: I/O error
Our_app_logs: | <Buffer 5b 74 6c 73 20 40 20 30 78 37 66 39 39 38 65 37 62 63 61 34 30 5d 20 45 72 72 6f 72 20 69 6e 20 74 68 65 20 70 75 6c 6c 20 66 75 6e 63 74 69 6f 6e 2e ... >
Our_app_logs: | stderr: frame= 1478 fps= 25 q=23.0 Lsize=
402kB time=00:01:02.89 bitrate= 52.4kbits/s speed=1.05x
Our_app_logs: | video:369kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.034639%
Our_app_logs: | <Buffer 66 72 61 6d 65 3d 20 31 34 37 38 20 66 70 73 3d 20 32 35 20 71 3d 32 33 2e 30 20 4c 73 69 7a 65 3d 20 20 20 20 20 34 30 32 6b 42 20 74 69 6d 65 3d 30 ... >
Our_app_logs: | stderr: [tls @ 0x7f998e7bca40] <Buffer 5b
74 6c 73 20 40 20 30 78 37 66 39 39 38 65 37 62 63 61 34 30 5d 20>
Our_app_logs: | stderr: The specified session has been invalidated for some reason.
Our_app_logs: | <Buffer 54 68 65 20 73 70 65 63 69 66 69 65 64 20 73 65 73 73 69 6f 6e 20 68 61 73 20 62 65 65 6e 20 69 6e 76 61 6c 69 64 61 74 65 64 20 66 6f 72 20 73 6f 6d ... >
Our_app_logs: | stderr: Last message repeated 1 times
Our_app_logs: | <Buffer 20 20 20 20 4c 61 73 74 20 6d 65 73 73 61 67 65 20 72 65 70 65 61 74 65 64 20 31 20 74 69 6d 65 73 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: frame I:13 Avg QP: 5.39 size:
2478
Our_app_logs: | <Buffer 66 72 61 6d 65 20 49 3a 31 33 20 20 20 20 41 76 67 20 51 50 3a 20 35 2e 33 39 20 20 73 69 7a 65 3a 20 20 32 34 37 38 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: frame P:1465 Avg QP:13.51 size:
235
Our_app_logs: | <Buffer 66 72 61 6d 65 20 50 3a 31 34 36 35 20 20 41 76 67 20 51 50 3a 31 33 2e 35 31 20 20 73 69 7a 65 3a 20 20 20 32 33 35 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: mb I I16..4: 99.2% 0.1% 0.7%
Our_app_logs: | <Buffer 6d 62 20 49 20 20 49 31 36 2e 2e 34 3a 20 39 39 2e 32 25 20 20 30 2e 31 25 20 20 30 2e 37 25 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: mb P I16..4: 0.3% 0.0% 0.0% P16..4: 0.1% 0.0% 0.0% 0.0% 0.0% skip:99.6%
Our_app_logs: | <Buffer 6d 62 20 50 20 20 49 31 36 2e 2e 34 3a 20 20 30 2e 33 25 20 20 30 2e 30 25 20 20 30 2e 30 25 20 20 50 31 36 2e 2e 34 3a 20 20 30 2e 31 25 20 20 30 2e ... >
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: 8x8 transform intra:0.3% inter:17.3%
Our_app_logs: | <Buffer 38 78 38 20 74 72 61 6e 73 66 6f 72 6d 20 69 6e 74 72 61 3a 30 2e 33 25 20 69 6e 74 65 72 3a 31 37 2e 33 25 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: coded y,uvDC,uvAC intra: 1.4% 6.9%
4.7% inter: 0.0% 0.0% 0.0%
Our_app_logs: | <Buffer 63 6f 64 65 64 20 79 2c 75 76 44 43 2c 75 76 41 43 20 69 6e 74 72 61 3a 20 31 2e 34 25 20 36 2e 39 25 20 34 2e 37 25 20 69 6e 74 65 72 3a 20 30 2e 30 ... >
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i16 v,h,dc,p: 90% 5% 5% 0%
Our_app_logs: | <Buffer 69 31 36 20 76 2c 68 2c 64 63 2c 70 3a 20 39 30 25 20 20 35 25 20 20 35 25 20 20 30 25 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23%
18% 51% 6% 0% 0% 0% 0% 3%
Our_app_logs: | <Buffer 69 38 20 76 2c 68 2c 64 63 2c 64 64 6c 2c 64 64 72 2c 76 72 2c 68 64 2c 76 6c 2c 68 75 3a 20 32 33 25 20 31 38 25 20 35 31 25 20 20 36 25 20 20 30 25 ... >
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33%
25% 40% 0% 0% 0% 0% 0% 0%
Our_app_logs: | <Buffer 69 34 20 76 2c 68 2c 64 63 2c 64 64 6c 2c 64 64 72 2c 76 72 2c 68 64 2c 76 6c 2c 68 75 3a 20 33 33 25 20 32 35 25 20 34 30 25 20 20 30 25 20 20 30 25 ... >
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: i8c dc,h,v,p: 86% 7% 6% 0%
Our_app_logs: | <Buffer 69 38 63 20 64 63 2c 68 2c 76 2c 70 3a 20 38 36 25 20 20 37 25 20 20 36 25 20 20 30 25 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: Weighted P-Frames: Y:0.1% UV:0.1%
Our_app_logs: | <Buffer 57 65 69 67 68 74 65 64 20 50 2d 46 72 61 6d 65 73 3a 20 59 3a 30 2e 31 25 20 55 56 3a 30 2e 31 25 0a>
Our_app_logs: | stderr: [libx264 @ 0x7f998e790080] <Buffer 5b 6c 69 62 78 32 36 34 20 40 20 30 78 37 66 39 39 38 65 37 39 30 30
38 30 5d 20>
Our_app_logs: | stderr: kb/s:2041.23
Our_app_logs: | <Buffer 6b 62 2f 73 3a 32 30 34 31 2e 32 33 0a>
Our_app_logs: | stderr: Conversion failed!
Our_app_logs: | <Buffer 43 6f 6e 76 65 72 73 69 6f 6e 20 66 61 69 6c 65 64 21 0a>
Our_app_logs: | close, code: 1, signal: null
Our_app_logs: | from react application: 14203
Our_app_logs: | Status ok...
Our_app_logs: | Data ok...
Our_app_logs: | FFmpeg ok...
Our_app_logs: | Writeable ok... <Buffer c4 81 0e 11 00 00 00 00 01 61 c7 80 5b 00 b6 72 03 bc 00 b7 03 de 59 7f 3c 27 80 01 b3 87 bc b2 e6 84 d0 f0 02 2d c0 00 00 00 01 61 00 08 70 c7 80 5b ... > undefined
Our_app_logs: | stderr: ffmpeg version 4.2.4 Copyright (c)
2000-2020 the FFmpeg developers
Our_app_logs: | built with gcc 9.2.0 (Alpine 9.2.0)
Our_app_logs: | configuration: --prefix=/usr --enable-avresample --enable-avfilter --enable-gnutls --enable-gpl --enable-libass --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-libtheora --enable-libv4l2 --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-libxcb --disable-stripping --disable-static --disable-librtmp --enable-vaapi --enable-vdpau --enable-libopus --disable-debug
Our_app_logs: | <Buffer 66 66 6d 70 65 67 20 76 65 72 73 69 6f 6e 20 34 2e 32 2e 34 20 43 6f 70 79 72 69 67 68 74 20 28 63 29 20 32 30 30 30 2d 32 30 32 30 20 74 68 65 20 46 ... >
Our_app_logs: | stderr: libavutil 56. 31.100 / 56. 31.100
Our_app_logs: | libavcodec 58. 54.100 / 58. 54.100
Our_app_logs: | libavformat 58. 29.100 / 58. 29.100
Our_app_logs: | libavdevice 58. 8.100 / 58. 8.100
Our_app_logs: | libavfilter 7. 57.100 / 7. 57.100
Our_app_logs: | libavresample 4. 0. 0 / 4. 0. 0
Our_app_logs: | libswscale 5. 5.100 / 5. 5.100
Our_app_logs: | libswresample 3. 5.100 / 3. 5.100
Our_app_logs: | libpostproc 55. 5.100 / 55. 5.100
Our_app_logs: | <Buffer 20 20 6c 69 62 61 76 75 74 69 6c 20 20 20 20 20 20 35 36 2e 20 33 31 2e 31 30 30 20 2f 20 35 36 2e 20 33 31 2e 31 30 30 0a 20 20 6c 69 62 61 76 63 6f ... >
Our_app_logs: | stderr: [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | Last message repeated 4 times
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | <Buffer 5b 68 32 36 34 20 40 20 30 78 37 66 32 39 39 66 34 34 66 36 30 30 5d 20 6e 6f 6e 2d 65 78 69 73 74 69 6e 67 20 50 50 53 20 31 34 20 72 65 66 65 72 65 ... >
Our_app_logs: | stderr: [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | Last message repeated 5 times
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_headerOur_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] non-existing PPS 14 referenced
Our_app_logs: | [h264 @ 0x7f299f44f600] decode_slice_header error
Our_app_logs: | [h264 @ 0x7f299f44f600] no frame!
The FFmpeg config:
const FFMPEG_CONFIG = [
'-i',
'-',
// video codec config: low latency, adaptive bitrate
// '-vcodec',
// 'copy',
'-c:v',
'libx264',
'-preset',
'veryfast',
'-tune',
'zerolatency',
// audio codec config: sampling frequency (11025, 22050, 44100), bitrate 64 kbits
'-c:a',
'aac',
'-ar',
'44100',
'-b:a',
'64k',
//force to overwrite
'-y',
// used for audio sync
'-use_wallclock_as_timestamps',
'1',
'-async',
'1',
//'-filter_complex', 'aresample=44100', // resample audio to 44100Hz, needed if input is not 44100
//'-strict', 'experimental',
'-bufsize',
'1000',
'-f',
'flv',
];
The process:
const process = child_process.spawn('ffmpeg', [
...FFMPEG_CONFIG,
// 'local.bin',
url,
]);
process.stderr.on('data', data => {
console.log(`stderr: ${data}`, data);
});
process.stdin.on('error', e => {
console.log('FFmpeg STDIN Error', e);
});
process.on('error', err => console.log(err));
process.on('close', (code, signal) => {
console.log(`close, code: ${code}, signal: ${signal}`);
});
The writing:
if (!Buffer.isBuffer(data)) return;
if (!process.stdin.writable) return;
process.stdin.write(data);
Im found another FFmpeg config that works perfectly.
-f lavfi -re -i anullsrc -f h264 -thread_queue_size 1024 -framerate 10 -probesize 100 -i - -vcodec copy -acodec aac -g 20 -f flv
Update: Another issue with this, if you use docker, the built-in docker network is limited. So you have to build a network manually between the services. Why? The data transfer between the services is huge and with this limitation, the FFmpeg not receives enough data.