Search code examples
flutterdartaudioudpstreaming

How to stream microphone audio over UDP with dart/flutter?


With my program, I need to be able to speak into 1 device and my voice play out another device. I was going to originally use TCP to do this, but then it was recommended to me that I should use UDP to reduce latency and make the audio sound better when it comes through. But as the title says, how can I take a dart Stream and send it through UDP? I know there is a function on a TCP Socket object from the dart:io library called addStream() that makes this easy, not 100% sure how it works, but I was wondering if there was something similar to this for UDP? I'm using this library for recording audio, by the way. I'm not really sure what I'm doing and was hoping that someone could explain how to do this in dart, and hopefully this could help a wandering and desperate Googler. This is what I had for my program that used TCP:

void handleClient(Socket client) {
    print("Connected a client");
    client.addStream(_audioStream);
  }
  _audioStream = _recorder.audioStream.listen((data) { 
    if (_isRecording) {
      ServerSocket.bind(InternetAddress.anyIPv4, 4444)
      .then((ServerSocket server) {
        print("Server socket init");
        server.listen(handleClient);
      });
    }
  });

Again, this was the version that uses TCP, and of course this didn't work. I'm not really sure what I'm doing, but hopefully that should give you a good idea of what I'm trying to do. How do I send an audio stream over UDP with dart?


Solution

  • Ended up figuring it out almost 2+ years later. The latency isn't great and I'm still working on that. But to any lost and wandering Googlers, here you go:

    You'll need the mic_stream and flutter_sound packages off pub.dev.

    Sender side:

    Stream<Uint8List>? stream = await MicStream.microphone(
          sampleRate: 32000,
          audioFormat: AudioFormat.ENCODING_PCM_8BIT,
        );
        int? bufferSize = await MicStream.bufferSize;
        RawDatagramSocket.bind(InternetAddress.anyIPv4, port).then((socket) {
          StreamSubscription<List<int>> listener = stream!.listen((sample) async {
            socket.send(sample, InternetAddress(sendToAddress), port);
          });
        });
    

    and the receiving side:

    FlutterSoundPlayer player = FlutterSoundPlayer();
    await player.openPlayer(enableVoiceProcessing: false);
        await player.startPlayerFromStream(
            codec: Codec.pcm16, numChannels: 1, sampleRate: 32000);
        var receiver = UDP.bind(
            Endpoint.unicast(InternetAddress(thisDeviceIPAddress), port: const Port(37069)));
        final udpSocket = await RawDatagramSocket.bind(a5atHome, port);
        udpSocket.listen((RawSocketEvent event) {
          Uint8List? data = udpSocket.receive()?.data;
          if (data != null) {
            player.foodSink!.add(FoodData(data)); // this plays the audio
          } else {}
        });