Running a non-blocking, high-performance activity in nativescript/javascript

Question

This question is about running a non-blocking, high-performance activity in nativescript that is needed for the simple task of reading and saving raw audio from the microphone by directly accessing the hardware through the native Android API. I believe I have brought the nativescript framework to the edge of its capabilities, and I need experts' help.

I'm building a WAV audio recorder in Nativescript Android. Native implementation is described here (relevant code below).

In short, this can be done by reading audio steam from an android.media.AudioRecord buffer, and then writing the buffer to a file in a separate thread, as described:

Native Android implementation

startRecording() is triggered by a button press, and starts a new Thread that runs writeAudioDataToFile():

private void startRecording() {
    // ... init Recorder

    recorder.startRecording();

    isRecording = true;

    recordingThread = new Thread(new Runnable() {
        @Override
        public void run() {
            writeAudioDataToFile();
        }
     }, "AudioRecorder Thread");

    recordingThread.start();
}

Recording is stopped by setting isRecording to false (stopRecording() is triggered by a button press):

private void stopRecording() {
   isRecording = false;

   recorder.stop();
   recorder.release();

   recordingThread = null;
}

Reading and saving buffer is stopped if isRecording = false:

private void writeAudioDataToFile() {
    // ... init file and buffer
    ByteArrayOutputStream recData = new ByteArrayOutputStream(); 
    DataOutputStream dos = new DataOutputStream(recData);

    int read = 0;

    while(isRecording) {
        read = recorder.read(data, 0, bufferSize);

        for(int i = 0; i < bufferReadResult; i++) {
            dos.writeShort(buffer[i]);
        }
    }
}

My Nativescript javascript implementation:

I wrote a nativescript typescript code that does the same as the native Android code above. The problem #1 I faced was that I can't run while(isRecording) because the javascript thread would be busy running inside the while loop, and would never be able to catch button clicks to run stopRecording().

I tried to solve problem #1 by using setInterval for asynchronous execution, like this:

startRecording() is triggered by a button press, and sets a time interval of 10ms that executes writeAudioDataToFile():

startRecording() {
    this.audioRecord.startRecording();

    this.audioBufferSavingTimer = setInterval(() => this.writeAudioDataToFile(), 10);
}

writeAudioDataToFile() callbacks are queued up every 10ms:

writeAudioDataToFile() {
    let bufferReadResult = this.audioRecord.read(
        this.buffer,
        0,
        this.minBufferSize / 4
    );

    for (let i = 0; i < bufferReadResult; i++) {
        dos.writeShort(buffer[i]);
    }

}

Recording is stopped by clearing the time interval (stopRecording() is triggered by button press):

stopRecording() {
    clearInterval(this.audioBufferSavingTimer);

    this.audioRecord.stop();
    this.audioRecord.release();
}

Problem #2: While this works well, in many cases it makes the UI freeze for 1-10 seconds (for example after clicking a button to stop recording).

I tried to change the time interval that executes writeAudioDataToFile() from 10ms to 0ms and up to 1000ms (while having a very big buffer), but then the UI freezes were longer and, and I experienced loss in the saved data (buffered data that was not saved to the file).

I tried to offload this operation to a separate Thread by using a nativescript worker thread as described here, where startRecording() and stopRecording() are called by messages sent to the thread like this:

global.onmessage = function(msg) {

    if (msg.data === 'startRecording') {
        startRecording();
    } else if (msg.data === 'stopRecording') {
       stopRecording();
    } 

}

This solved the UI problem, but created problem #3: The recorder stop was not executed on time (i.e. recording stops 10 to 50 seconds after the 'stopRecording' msg.data is received by the worker thread). I tried to use different time intervals in the setInterval inside the worker thread (0ms to 1000ms) but that didn't solve the problem and even made stopRecording() be executed with greater delays.

Does anyone have an idea of how to perform such a non-blocking high-performance recording activity in nativescript/javascript? Is there a better approach to solve problem #1 (javascript asynchronous execution) that I described above?

Thanks

Answer 1

I would keep the complete Java implementation in actual Java, you can do this by creating a java file in your plugin folder: platforms/android/java, so maybe something like: platforms/android/java/org/nativescript/AudioRecord.java

In there you can do everything threaded, so you won't be troubled by the UI being blocked. You can call the Java methods directly from NativeScript for starting and stopping the recording. When you build your project, the Java file will automatically be compiled and included.

You can generate typings from your Java class by grabbing classes.jar from the generated .aar file of your plugin ({plugin_name}.aar) and generate type declarations for it: https://docs.nativescript.org/core-concepts/android-runtime/metadata/generating-typescript-declarations

This way you have all the method/class/type information available in your editor.