I need to record audio in an Android application that needs to be imported into Python (ndarray) for plotting and signal processing. It seemed like such a simple idea.
I started with a simple bit of code for AAC/MPEG4 recording. Recording worked great. I can play it on the Android phone (Nexus 5X) and on a Mac (Quicktime). No problem! Right?!? But finding codec/formats that match between Android and Python seems to not be trivial. I'm wondering if the file/codec format written by Android is non-standard and FFMPEG can't read it.
If so, what is good audio format/codec that can be written simply in Android and read into an array in Python (2.7.x). Thanks.
Details: Here is an abbreviated form of the android code:
private final int AUDIO_SAMPLE_RATE = 16000;
private final String FILE_EXTENSION = "m4a"; // Audio file extension
mAbsolutePathFile = workingDir + "/" + mFilename + FILE_EXTENSION;
mMediaRecording = new MediaRecorder();
mMediaRecording.setAudioSource (MediaRecorder.AudioSource.MIC);
mMediaRecording.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
mMediaRecording.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
mMediaRecording.setAudioSamplingRate(AUDIO_SAMPLE_RATE);
mMediaRecording.setOutputFile(mAbsolutePathFile);
mMediaRecording.prepare();
mMediaRecording.start();
As I mentioned, the result audio plays nicely in both Android and MacOS, so all seemed well. I did a bit of searching to find a python package for AAC audio and pydub looked like the simplest (I tried audiotools, but couldn't find sample code). To install pydub, I followed the instructions:
pip install pydub
and
brew install libav --with-libvorbis --with-sdl --with-theora
and
brew install ffmpeg --with-libvorbis --with-ffplay --with-theora
Following the discussions (here), i tested ffmpeg and it does execute from the command line:
$ffmpeg
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
But when I try to read the file using Python:
import pydub
pydub.AudioSegment.from_file("sensorlog_2017-02-03_12-50-25-345_Dev26c5_Loc27_TypeAUDIO.m4a", "aac")
I get:
Traceback (most recent call last):
File "...anaconda2/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-8-607e3f2a62c3>", line 1, in <module>
pydub.AudioSegment.from_file("sensorlog_2017-02-03_12-50-25-345_Dev26c5_Loc27_TypeAUDIO.m4a", "aac")
File ".../anaconda2/lib/python2.7/site-packages/pydub/audio_segment.py", line 472, in from_file
raise CouldntDecodeError("Decoding failed. ffmpeg returned error code: {0}\n\nOutput from ffmpeg/avlib:\n\n{1}".format(p.returncode, p_err))
CouldntDecodeError: Decoding failed. ffmpeg returned error code: 1
Output from ffmpeg/avlib:
avconv version 11.4, Copyright (c) 2000-2014 the Libav developers
built on Feb 3 2017 12:09:15 with Apple LLVM version 8.0.0 (clang-800.0.42.1)
[aac @ 0x7ff30001cc00] get_buffer() failed
[aac @ 0x7ff30001cc00] channel element 3.14 is not allocated
[aac @ 0x7ff30001cc00] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7ff30001cc00] Input buffer exhausted before END element found
[aac @ 0x7ff30001cc00] More than one AAC RDB per ADTS frame is not implemented. Update your Libav version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7ff30001cc00] Error decoding AAC frame header.
[aac @ 0x7ff300001000] Could not find codec parameters (Audio: aac, 4.0, fltp, 213 kb/s)
[aac @ 0x7ff300001000] Estimating duration from bitrate, this may be inaccurate
/var/folders/cm/1r6x6rbj7hn_51qvfjzj7nx80000gn/T/tmps7b8Gf: could not find codec parameters
In summary, is this a conflict between Android and FFMPEG? Should I use a different codec/format? Should I use a different Python Audio library? Thanks.
There was some additional discussion over in pydub's GitHub issues, but for future wanderers, I believe the solution was to use "mp4" as the format
from pydub import AudioSegment
sound = AudioSegment.from_file("./sensorlog.m4a", format="mp4")
Side note: if you can't figure out the right format, you can leave out the format argument altogether and pydub/ffmpeg/avconv/whoever will try to figure it out.
sound = AudioSegment.from_file("./sensorlog.m4a")
You may also find it useful to try pydub.utils.mediainfo() though depending on your ffmpeg/avconv installation it may or may not be available (it uses ffprobe internally)
>>> from pydub.utils import mediainfo
>>> mediainfo("./sensorlog.m4a")
{u'DISPOSITION': {u'attached_pic': u'0',
u'clean_effects': u'0',
u'comment': u'0',
u'default': u'1',
u'dub': u'0',
u'forced': u'0',
u'hearing_impaired': u'0',
u'karaoke': u'0',
u'lyrics': u'0',
u'original': u'0',
u'timed_thumbnails': u'0',
u'visual_impaired': u'0'},
u'TAG': {u'com.android.version': u'7.0',
u'compatible_brands': u'isommp42',
u'creation_time': u'2017-02-03T20:50:33.000000Z',
u'handler_name': u'SoundHandle',
u'language': u'eng',
u'major_brand': u'mp42',
u'minor_version': u'0'},
u'avg_frame_rate': u'0/0',
u'bit_rate': u'15539',
u'bits_per_raw_sample': u'N/A',
u'bits_per_sample': u'0',
u'channel_layout': u'mono',
u'channels': u'1',
u'codec_long_name': u'AAC (Advanced Audio Coding)',
u'codec_name': u'aac',
u'codec_tag': u'0x6134706d',
u'codec_tag_string': u'mp4a',
u'codec_time_base': u'1/8000',
u'codec_type': u'audio',
u'duration': u'7.808000',
u'duration_ts': u'62464',
u'filename': u'/Users/jiaaro/Downloads/sensorlog_2017-02-03_12-50-25-345_Dev26c5_Loc27_ActvAppTesting_TypeAUDIO.m4a',
u'format_long_name': u'QuickTime / MOV',
u'format_name': u'mov,mp4,m4a,3gp,3g2,mj2',
u'id': u'N/A',
u'index': u'0',
u'max_bit_rate': u'12200',
u'nb_frames': u'61',
u'nb_programs': u'0',
u'nb_read_frames': u'N/A',
u'nb_read_packets': u'N/A',
u'nb_streams': u'1',
u'probe_score': u'100',
u'profile': u'LC',
u'r_frame_rate': u'0/0',
u'sample_fmt': u'fltp',
u'sample_rate': u'8000',
u'size': u'15167',
u'start_pts': u'0',
u'start_time': u'0.000000',
u'time_base': u'1/8000'}