Search code examples
iosswifttensorflowkerastensorflow-lite

Using TensorFlow libraries on IOS

For feature extraction of my Keras trained model, I was looking for a signal processing library to compute log mel spectograms on IOS, using Swift. During development I used scikit.signal library in Python for feature extraction.

Then I noticed Tensorflow has support for signal processing:

https://www.tensorflow.org/api_guides/python/contrib.signal

I also know that we can import our tf models to be used by core ml during IOS development. I wonder but couldn't find evidence that if I can also take adventage of these tf signal processing libraries on IOS. Maybe by making them a part of my model, use them instead of scikit.signal library on desktop. Then when I import it, it is part of my model, or something like that? I also see something called tf-lite, but dont know if it includes these libraries.

Solution

  • Regarding TensorFlow Lite, there are a few things you can look into:

    • TensorFlow Lite now implements Mfcc and AudioSpectrogram as custom ops. You can try to follow the tutorial to convert a TensorFlow model to TensorFlow Lite model, and add --allow_custom_ops argument when running tflite_convert tool. See if it works.
    • If the previous approach doesn't work, TensorFlow Lite has an experimental feature Using TensorFlow Lite with select TensorFlow ops. It supports Mfcc and AudioSpectrogram operations.