I am trying to extract and play audio samples from a video file in Android 11. However, I have followed some tutorials from the official documentation, but it still doesn't work. All I get is noise for a shorter duration of the audio. Please check the code I am using to achieve this:
// This is how I get the audio samples
public ArrayList<byte[]> getAudio() {
int index = 0;
ArrayList<byte[]> audio = new ArrayList<>();
ByteBuffer buffer = ByteBuffer.allocateDirect(format.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE));
for (;;) {
int size = extractor.readSampleData(buffer, 0);
total_size += size;
if (size < 0) { // no more data to read
break;
}
for (int i = 0; i < size / 2; i++) {
byte[] sample = new byte[2];
for (int j = 0; j < sample.length; j++) {
sample[j] = buffer.get();
}
audio.add(sample);
}
extractor.advance();
}
return audio;
}
Log.d("AudioTest", "total audio data size in bytes: " + total_size);
// This is how I play the audio
int SIZE = AudioTrack.getMinBufferSize(48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, 48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, SIZE, AudioTrack.MODE_STREAM);
ArrayList<byte[]> audio = getAudio();
long start = System.currentTimeMillis();
track.play();
byte[] data = new byte[audio.size() * 2];
for (int i = 0, k = 0; i < audio.size(); i++) {
byte[] sample = audio.get(i);
for (int j = 0; j < sample.length; j++, k++) {
data[k] = sample[j];
}
}
track.write(data, 0, data.length);
long duration = System.currentTimeMillis() - start;
Log.d("AudioTest", "that took: " + (duration / 1000) + "s");
I am facing an issue where my audio plays random noise for only 5 seconds. The getAudio() method retrieves only 487,831 bytes from the 15-second video file. Here are the details of the video file:
Frames:
Samples:
Any help would be highly appreciated.
Thank you.
The audio is compressed using the Advanced Audio Coding (AAC) compression. You can see it from this line:
mime: audio/mp4a-latm.
But you are not decoding it. The MediaExtractor reads compressed samples while the AudioTrack expects decoded data (AudioFormat.ENCODING_PCM_16BIT in your case).
You can use the MediaCodec API to decode the audio buffers that MediaExtractor reads.
By the way, you will not need to copy the decoded buffers into an array, just pass them to the AudioTrack as is.
Alternatively, you could detect the correct encoding and specify it when you set the AudioTrack up. See here and here.