Search code examples
androidwebrtcandroid-mediaprojection

Can I get MediaStreamTrack from AudioRecord?


I want audio share with webrtc in android. I try with MediaProjection, video share is OK. but, audio record have not Audio Track. How can I get Audio Track from Audio Record?

AudioPlaybackCaptureConfiguration config = new AudioPlaybackCaptureConfiguration.Builder(sMediaProjection)
                            .addMatchingUsage(AudioAttributes.USAGE_MEDIA)
                            .addMatchingUsage(AudioAttributes.USAGE_VOICE_COMMUNICATION)
                            .addMatchingUsage(AudioAttributes.USAGE_VOICE_COMMUNICATION_SIGNALLING)
                            .addMatchingUsage(AudioAttributes.USAGE_GAME)
                            .build();

AudioFormat audioFormat = new AudioFormat.Builder()
                            .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
                            .setSampleRate(44100)
                            .setChannelMask(AudioFormat.CHANNEL_IN_MONO)
                            .build();

audioRecord = new AudioRecord.Builder()
                            .setAudioFormat(audioFormat)
                            .setBufferSizeInBytes(BUFFER_SIZE_IN_BYTES)
                            .setAudioPlaybackCaptureConfig(config)
                            .build();

audioRecord.startRecording();

//Other code

String audioTrackId = stateProvider.getNextTrackUUID();

AudioSource as = new AudioSource(audioRecord.getAudioSource());

tracks[0] = pcFactory.createAudioTrack(audioTrackId, as);  // Not Working

Solution

  • I think you have to checks below

    https://developer.android.com/reference/android/media/AudioPlaybackCaptureConfiguration

    1. the usage value MUST be AudioAttributes#USAGE_UNKNOWN or AudioAttributes#USAGE_GAME or AudioAttributes#USAGE_MEDIA. All other usages CAN NOT be captured.
    2. the capture policy set by their app (with AudioManager#setAllowedCapturePolicy) or on each player (with AudioAttributes.Builder#setAllowedCapturePolicy) is AudioAttributes#ALLOW_CAPTURE_BY_ALL, whichever is the most strict.

    You can it with below adb command

    • Run webrtc and enter next command
    $ adb shell dumpsys audio
    ...
    ...
      players:
      AudioPlaybackConfiguration piid:15 type:android.media.SoundPool u/pid:1000/1619 state:idle attr:AudioAttributes: usage=USAGE_ASSISTANCE_SONIFICATION content=CONTENT_TYPE_SONIFICATION flags=0x800 tags= bundle=null
      AudioPlaybackConfiguration piid:23 type:android.media.SoundPool u/pid:10222/2040 state:idle attr:AudioAttributes: usage=USAGE_ASSISTANCE_SONIFICATION content=CONTENT_TYPE_SONIFICATION flags=0x800 tags= bundle=null
    ...
    ...
    allowed capture policies:
    ...
    ...
    

    players: means
    These are audio attribute information which is being used now. You could see the usage using by webrc with pid

    flags=0x800: means
    Shows what value the app has setAllowedCapturePolicy. Refer to below link for the information of flags https://cs.android.com/android/platform/superproject/+/master:frameworks/base/media/java/android/media/AudioAttributes.java;drc=master;l=1532?q=capturePolicyToFlags&ss=android%2Fplatform%2Fsuperproject&hl=ko

    allowed capture policies: means
    It shows which app has set setAllowedCapturePolicy.

    1. their app attribute allowAudioPlaybackCapture in their manifest MUST either be set to true

    If webrc doesn't set this value, you could see below warning log.

            ALOGW("%s: Playback capture is denied for uid %u as the manifest property could not be "
                  "retrieved from the package manager: %s", __func__, uid, status.toString8().c_str());