Search code examples
androidaudiojava-native-interfacemidijavax.sound.midi

Virtual synthesizer in Android (based on jar) - native jni?

I am trying to load & use virtual (soft) synthesizer of java in Android. I read Gervill jar is to be used for this. Other building block is javax-sound present in my project as an aar file (as is it not present in trimmed Java package of Android)

Gradle file inclusions are

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
apply plugin: 'kotlin-kapt'

android {
    compileSdkVersion 29
    buildToolsVersion "29.0.0"
    dataBinding {
        enabled = true
    }
    defaultConfig {
        applicationId "com.mmm.ttt"
        minSdkVersion 18
        targetSdkVersion 29
        multiDexEnabled true
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation project(path: ':javax-sound')
    implementation project(path: ':gervilljar')         // This is folder name at project root folder level containing gervill.jar
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation 'com.android.support:multidex:1.0.3'
}

Problem areas of code

Code 1

MidiSystem.getMidiDeviceInfo().size    //results in an empty array in all cases

Some answers suggest only inclusion of gervill jar is enough for auto detection. But since that hasnt worked for me, I written this

Code 2

var synth = SoftSynthesizer() as Synthesizer
        MidiSystem.addSynthesizer(synth)

I get compilation error for this

Supertypes of the following classes cannot be resolved. Please make sure you have the required dependencies in the classpath:
    class com.sun.media.sound.SoftSynthesizer, unresolved supertypes: com.sun.media.sound.ReferenceCountingDevice
    class com.sun.media.sound.AudioSynthesizer, unresolved supertypes: javax.sound.midi.Synthesizer

Code 3

var synth = MidiSystem.getSynthesizer()
synth?.open()

With inclusion to gradle as above, this is the run time error in absence of code 2

Caused by: jp.kshoji.javax.sound.midi.MidiUnavailableException: Synthesizer not found
        at jp.kshoji.javax.sound.midi.MidiSystem.getSynthesizer(MidiSystem.java:323)

I followed https://stackoverflow.com/a/38749847/1029110 for adding aar/jar files
But, havent linked each other in the module dependency gradle (Should/How to do that?) although gervill needs javax.sound

Final aim is to load & play soundfont files. Avoiding native jni code if possible.

EDIT

1 problem identified. Gervill.jar is not good enough to work error free. So I a picked few other classes from com.sun.media.sound.* and compiled into new jar. Now that error in code 2 (unresolved supertypes) has gone away.

Next trial was to include Android midi service

val midiManager = context.getSystemService(Context.MIDI_SERVICE) as android.media.midi.MidiManager
midiManager.getDevices()

This gave 2 services depending on the external applications installed on mobile. But they are not linked to the softsynthesizer in anyway.  
So, Code 1 still has empty array for MidiDeviceInfo.  
Part of Code 3: synth?.open() still failing badly, but with newer error. (Currently stuck here)

Caused by: jp.kshoji.javax.sound.midi.MidiUnavailableException: Can not open line
        at com.sun.media.sound.SoftSynthesizer.open(SoftSynthesizer.java:1132)
        at com.sun.media.sound.SoftSynthesizer.open(SoftSynthesizer.java:1036)
Caused by: java.lang.IllegalArgumentException: No line matching interface jp.kshoji.SourceDataLine supporting format PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian is supported.
        at jp.kshoji.javax.sound.sampled.AudioSystem.getLine(AudioSystem.java:475)
        at jp.kshoji.javax.sound.sampled.AudioSystem.getSourceDataLine(AudioSystem.java:602)
        at com.sun.media.sound.SoftSynthesizer.open(SoftSynthesizer.java:1066)
        at com.sun.media.sound.SoftSynthesizer.open(SoftSynthesizer.java:1036) 

Code 4:

jp.kshoji.javax.sound.midi.MidiSystem.getSoundbank(File("soundfont.sf2"))

Fails with

Caused by: java.lang.UnsupportedOperationException: not implemented.
    at jp.kshoji.javax.sound.midi.MidiSystem.getSoundbank(MidiSystem.java:277)

I have checked the Java for windows version source code. There is no override or implementation of the file to use getSoundbank.

Fixes & future steps?

Solution

  • Solutions in my understanding

    Code 3.

    github.com/KyoSherlock/MidiDriver is the android ready code which has same contents of com.sun.media.sound & jp\kshoji\javax\sound\midi put together. Also trimmed to exclude unwanted classes.

    Other midi drivers like

    are based on native C/C++ code, which I avoided all along.

    This answer helped https://stackoverflow.com/a/56700883/1029110

    Code 4.

    Call to load soundbank should be done via SF2SoundBank & not via MidiSystem.getSoundbank(soundFontFile) 

    val sf = SF2Soundbank(assets.open("SmallTimGM6mb.sf2"))

    Code 1.

    Might not be important for producing sounds via phone speaker, as even the KyoSherlock running sample returns 0

    In depth reading can be done here http://jsresources.sourceforge.net/faq_midi.html