Adding FFMPEG Layer to HLS streaming causes video playback issues

Question

I have been searching a lot about HLS streaming and have succeeded to create a simple HLS streaming server with nodejs, the problem now is I need to add a layer of ffmpeg encoding to the .ts chunks before streaming to the user, without this layer everything works fine and on my server only 3 requests are seen:

manifest.m3u8
output_000.ts
output_000.ts
output_001.ts
output_002.ts

But then when I add a simple ffmpeg layer that literally copies everything from the ts file and output the stream (I will add of course dynamic filters to each request, thats why I need this ffmpeg layer), the player goes insane and request the whole video in just 5 seconds or something:

manifest.m3u8
output_000.ts
output_000.ts
output_001.ts
output_002.ts
output_001.ts
output_003.ts
output_002.ts
...
output_095.ts

I have also notices that the numbers aren't increasing uniformly and suspect this is part of the issue, I have tried adding more ffmpeg options to not do anything to the .ts files that are being fed to it as they are a part of a bigger video.

Here's my NodeJS server (NextJS API route):

const fs = require(`fs`);
const path = require(`path`);
const {exec, spawn} = require(`child_process`);
const pathToFfmpeg = require(`ffmpeg-static`);

export default function handler(req, res) {
  
    const { filename } = req.query;
    console.log(filename);
    const filePath = path.join(process.cwd(), 'public', 'stream', `${filename}`);
    const inputStream = fs.createReadStream(filePath);

    // first check if that is ts file..
    if(filename.indexOf(`.ts`) != -1){
  
      const ffmpegProcess = spawn(pathToFfmpeg, [
        '-f', `mpegts`,
        '-i', 'pipe:0', // specify input as pipe
        '-c', 'copy', 
        '-avoid_negative_ts', '0',
        `-map_metadata`, `0`,  // copy without re-encoding
        '-f', 'mpegts', // output format
        'pipe:1'        // specify output as pipe
      ], {
        stdio: ['pipe', 'pipe', 'pipe'] // enable logging by redirecting stderr to stdout
      });
      res.status(200);
      res.setHeader('Content-Type', 'application/vnd.apple.mpegurl');
      res.setHeader('Cache-Control', 'no-cache');
      res.setHeader('Access-Control-Allow-Origin', '*');
 

      // ffmpegProcess.stderr.pipe(process.stdout); // log stderr to stdout
  
      inputStream.pipe(ffmpegProcess.stdin);
      ffmpegProcess.stdout.pipe(res);
  
      ffmpegProcess.on('exit', (code) => {
        if (code !== 0) {
          console.error(`ffmpeg process exited with code ${code}`);
        }
      });
    }else{
      // if not then stream whatever file as it is
      res.status(200);
      res.setHeader('Content-Type', 'application/vnd.apple.mpegurl');
      inputStream.pipe(res);
    }
  }

I have tried to feed the request's player appropriate headers but that didn't work, I have also tried to add the '-re' option to the ffmpeg encoder itself and hoped for minimal performance hits, but that also caused playback issue due to being too slow.

Answer 1

Update:

So with the help of ChatGPT which is definitely not going to replace me xD it finally gave me these options:

-copyts -copytb 0

Which I believe are used to copy the timestambs of the stream segemented files to literally simulate that this segement output is actually the one that was generated when segmenting the original mp4 file.

so the full command that works is as follows:

   const ffmpegProcess = spawn(pathToFfmpeg, [
        '-f', `mpegts`,
        '-i', 'pipe:0', // specify input as pipe
        '-c:v', 'libx264', // re-encode with libx264
        '-preset', 'ultrafast', // set the encoding preset to ultrafast
        '-crf', '30', // set the Constant Rate Factor (CRF) to 18 (lower CRF = higher quality)
        '-map_metadata', '0',
        '-movflags', 'frag_keyframe+empty_moov+default_base_moof+faststart',
        '-filter:v', `drawtext=fontfile=./pdfs/fonts/Rubik-Regular.ttf: text='© yoursite.COM | ${userData.email} | ${userData.name}': x=${randomX}:y=${randomY}: fontsize=18: [email protected]: box=1: [email protected]`,
        '-profile:v', 'baseline', // encoding profile
        '-maxrate', '4000k',
        `-copyts`, `-copytb`, `0`,
        `-bsf:v`, `dump_extra`,
        `-err_detect` , `ignore_err`,
        '-f', 'mpegts', // output format
        'pipe:1'        // specify output as pipe
      ], {
        stdio: ['pipe', 'pipe', 'pipe'] // enable logging by redirecting stderr to stdout
 });

For anyone wondering how this exactly works, basically I first took a regular input mp4 video then I use an ffmpeg command to convert it to HLS stream, which is the easy part.

After this I created my own serverless streaming api endopoint to serve the requests myself instead of just inputing the file name which gave me more control on who can access the stream and more importantly enabled me to add ffmpeg layer to segments of the video before being streamed to the user to add whatever variable text or filters needed.