I've built a small machine learning project that classifies music based on genre for my project, and I'm serving the feature extraction function through a REST API - flask-restful hosted on Heroku. The function takes in an audio file and returns extracted features such as MFCC. I've tested the local server and the one on Heroku using cURL(POST) and it works just fine, but when I make the same POST request from and Android device it returns a HTTP 500. Now, I've tried two different libraries to make the POST request. Fuel HTTP and the one from loopj.
Both of them yield the same result, the logs say that the audio file uploaded through POST is of type None(At least that is what the librosa.load() function claims), which isn't possible. I don't understand what I'm doing wrong, possibly something fundamentally wrong. I'm quite new to networking libraries and the whole concept. Any help is highly appreciated. I'm open to any suggestions or alternate approaches!
I've attached all the necessary logs and code snippets below, please take a look. Thanks!
from flask import Flask, jsonify, request
from flask_restful import Resource, Api, reqparse
import werkzeug
import scipy.optimize
import os,pickle
import numpy as np
from sklearn.preprocessing import StandardScaler
import librosa
app = Flask(__name__)
api = Api(app)
class receiveWav(Resource):
def post(self):
parse = reqparse.RequestParser()
parse.add_argument('file', type=werkzeug.datastructures.FileStorage, location='files')
args = parse.parse_args()
audioFile = args['file']
scaler = pickle.load(open("scaler.ok","rb"))
x , sr = librosa.load(audioFile,mono=True,duration=5)
y=x
#Extract the features
chroma_stft = librosa.feature.chroma_stft(y=y, sr=sr)
spec_cent = librosa.feature.spectral_centroid(y=y, sr=sr)
spec_bw = librosa.feature.spectral_bandwidth(y=y, sr=sr)
rolloff = librosa.feature.spectral_rolloff(y=y, sr=sr)
zcr = librosa.feature.zero_crossing_rate(y)
rmse = librosa.feature.rms(y=y)
mfcc = librosa.feature.mfcc(y=y, sr=sr)
features = f'{np.mean(chroma_stft)} {np.mean(rmse)} {np.mean(spec_cent)} {np.mean(spec_bw)} {np.mean(rolloff)} {np.mean(zcr)}'
for e in mfcc:
features += f' {np.mean(e)}'
input_data2 = np.array([float(i) for i in features.split(" ")]).reshape(1,-1)
input_data2 = scaler.transform(input_data2)
return jsonify(input_data2.tolist())
def get(self):
return ("Try POST!")
api.add_resource(receiveWav, '/receiveWav/sav')
# driver function
if __name__ == '__main__':
app.run(debug = True)
IN:
curl -X POST -F 'file=@iam.wav' https://audient.herokuapp.com/receiveWav/sav
OUT:
[[-0.5121340988129909,-0.019333716494365955,-0.5550177910353545,-0.4930899684337351,-0.48935258100758594,-0.4062782248006835,0.3213519031236203,0.7112413195038644,-0.5840644603304033,0.2721270179974677,-0.4193148299872463,0.3797072443498176,-0.9736845762509901,0.6999042763966654,-0.5770378663005774,0.49129379959372144,-0.36939016178222783,0.5953006858517871,-0.07876657112131476,0.8944914190287592,-0.7771445271993948,-0.6157818858980287,0.05354594954508769,0.04207617031311767,-0.08474674018805041,0.7518709362787497]]
Herou Log(cURL):
2020-03-09T12:42:33.177419+00:00 heroku[router]: at=info method=POST path="/receiveWav/sav" host=audient.herokuapp.com request_id=80bc4fa5-8d7b-419a-942b-f5c1c658f972 fwd="183.82.146.189" dyno=web.1 connect=1ms service=5615ms status=200 bytes=675 protocol=https
2020-03-09T12:42:33.175908+00:00 app[web.1]: 10.71.199.143 - - [09/Mar/2020:12:42:33 +0000] "POST /receiveWav/sav HTTP/1.1" 200 522 "-" "curl/7.55.1"
Fuel.upload("http://d6ea6102.ngrok.io/receiveWav/sav")
.add(FileDataPart(File(output!!),filename = "iam.wav"))
.also { println(it) }
.response { result ->
Log.d("HUSKY", "${result.get()}")
}
val client = AsyncHttpClient()
val params = RequestParams()
params.put("iam.wav", File(output!!))
client.post("https://audient.herokuapp.com/receiveWav/sav", params,object : AsyncHttpResponseHandler() {
override fun onStart() {
// called before request is started
}
override fun onSuccess(
statusCode: Int,
headers: Array<Header?>?,
response: ByteArray?
) {
println("HTTP STATUS 200 OK")
}
override fun onFailure(
statusCode: Int,
headers: Array<Header?>?,
errorResponse: ByteArray?,
e: Throwable?
) {
// called when response HTTP status is "4XX" (eg. 401, 403, 404)
println("HTTP STATUS 400/500 NOT OK")
}
override fun onRetry(retryNo: Int) {
// called when request is retried
}
})
2020-03-09T12:44:33.130446+00:00 app[web.1]: [2020-03-09 12:44:33,129] ERROR in app: Exception on /receiveWav/sav [POST]
2020-03-09T12:44:33.130463+00:00 app[web.1]: Traceback (most recent call last):
2020-03-09T12:44:33.130464+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
2020-03-09T12:44:33.130464+00:00 app[web.1]: rv = self.dispatch_request()
2020-03-09T12:44:33.130465+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
2020-03-09T12:44:33.130466+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2020-03-09T12:44:33.130466+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 468, in wrapper
2020-03-09T12:44:33.130466+00:00 app[web.1]: resp = resource(*args, **kwargs)
2020-03-09T12:44:33.130467+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/views.py", line 89, in view
2020-03-09T12:44:33.130468+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2020-03-09T12:44:33.130469+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
2020-03-09T12:44:33.130469+00:00 app[web.1]: resp = meth(*args, **kwargs)
2020-03-09T12:44:33.130469+00:00 app[web.1]: File "/app/app.py", line 20, in post
2020-03-09T12:44:33.130470+00:00 app[web.1]: x , sr = librosa.load(audioFile,mono=True,duration=5)
2020-03-09T12:44:33.130471+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/librosa/core/audio.py", line 129, in load
2020-03-09T12:44:33.130471+00:00 app[web.1]: with sf.SoundFile(path) as sf_desc:
2020-03-09T12:44:33.130471+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 629, in __init__
2020-03-09T12:44:33.130472+00:00 app[web.1]: self._file = self._open(file, mode_int, closefd)
2020-03-09T12:44:33.130472+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 1182, in _open
2020-03-09T12:44:33.130472+00:00 app[web.1]: raise TypeError("Invalid file: {0!r}".format(self.name))
2020-03-09T12:44:33.130481+00:00 app[web.1]: TypeError: Invalid file: None
2020-03-09T12:44:33.132158+00:00 app[web.1]: 10.33.187.161 - - [09/Mar/2020:12:44:33 +0000] "POST /receiveWav/sav HTTP/1.1" 500 37 "-" "-"
2020-03-09T12:44:33.227673+00:00 app[web.1]: /app/.heroku/python/lib/python3.6/site-packages/sklearn/base.py:318: UserWarning: Trying to unpickle estimator StandardScaler from version 0.21.3 when using version 0.22.2.post1. This might lead to breaking code or invalid results. Use at your own risk.
2020-03-09T12:44:33.227682+00:00 app[web.1]: UserWarning)
2020-03-09T12:44:33.229363+00:00 app[web.1]: [2020-03-09 12:44:33,228] ERROR in app: Exception on /receiveWav/sav [POST]
2020-03-09T12:44:33.229365+00:00 app[web.1]: Traceback (most recent call last):
2020-03-09T12:44:33.229365+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
2020-03-09T12:44:33.229366+00:00 app[web.1]: rv = self.dispatch_request()
2020-03-09T12:44:33.229366+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
2020-03-09T12:44:33.229367+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2020-03-09T12:44:33.229367+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 468, in wrapper
2020-03-09T12:44:33.229368+00:00 app[web.1]: resp = resource(*args, **kwargs)
2020-03-09T12:44:33.229368+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/views.py", line 89, in view
2020-03-09T12:44:33.229369+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2020-03-09T12:44:33.229369+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
2020-03-09T12:44:33.229370+00:00 app[web.1]: resp = meth(*args, **kwargs)
2020-03-09T12:44:33.229370+00:00 app[web.1]: File "/app/app.py", line 20, in post
2020-03-09T12:44:33.229370+00:00 app[web.1]: x , sr = librosa.load(audioFile,mono=True,duration=5)
2020-03-09T12:44:33.229371+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/librosa/core/audio.py", line 129, in load
2020-03-09T12:44:33.229371+00:00 app[web.1]: with sf.SoundFile(path) as sf_desc:
2020-03-09T12:44:33.229372+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 629, in __init__
2020-03-09T12:44:33.229372+00:00 app[web.1]: self._file = self._open(file, mode_int, closefd)
2020-03-09T12:44:33.229373+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 1182, in _open
2020-03-09T12:44:33.229373+00:00 app[web.1]: raise TypeError("Invalid file: {0!r}".format(self.name))
2020-03-09T12:44:33.229468+00:00 app[web.1]: TypeError: Invalid file: None
2020-03-09T12:44:33.231741+00:00 app[web.1]: 10.31.120.51 - - [09/Mar/2020:12:44:33 +0000] "POST /receiveWav/sav HTTP/1.1" 500 37 "-" "Dalvik/2.1.0 (Linux; U; Android 10; Redmi K20 Pro MIUI/V11.0.3.0.QFKINXM)"
2020-03-09T12:44:33.133234+00:00 heroku[router]: at=info method=POST path="/receiveWav/sav" host=audient.herokuapp.com request_id=b9271923-4c37-412c-bcd0-b8619ca96161 fwd="106.200.166.60" dyno=web.1 connect=1ms service=3462ms status=500 bytes=208 protocol=https
2020-03-09T12:44:33.233148+00:00 heroku[router]: at=info method=POST path="/receiveWav/sav" host=audient.herokuapp.com request_id=49d2f2bb-3226-4a69-922c-b1059f66aca5 fwd="106.200.166.60" dyno=web.1 connect=1ms service=3580ms status=500 bytes=208 protocol=https
I've tried this as well
val job = GlobalScope.launch{
val file = File(output)
try {
val httpclient = DefaultHttpClient()
val httppost = HttpPost("https://audient.herokuapp.com/receiveWav/sav")
val reqEntity = InputStreamEntity(
FileInputStream(file), -1)
reqEntity.setContentType("binary/octet-stream")
reqEntity.isChunked = false
httppost.entity = reqEntity
var response = httpclient.execute(httppost)
Log.d("HUSKY","${response}")
println(response)
} catch (e: Exception) {
println("${e}")
}
}
2020-03-09T14:12:55.180267+00:00 app[web.1]: /app/.heroku/python/lib/python3.6/site-packages/sklearn/base.py:318: UserWarning: Trying to unpickle estimator StandardScaler from version 0.21.3 when using version 0.22.2.post1. This might lead to breaking code or invalid results. Use at your own risk.
2020-03-09T14:12:55.180278+00:00 app[web.1]: UserWarning)
2020-03-09T14:12:55.181780+00:00 app[web.1]: [2020-03-09 14:12:55,181] ERROR in app: Exception on /receiveWav/sav [POST]
2020-03-09T14:12:55.181781+00:00 app[web.1]: Traceback (most recent call last):
2020-03-09T14:12:55.181781+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
2020-03-09T14:12:55.181782+00:00 app[web.1]: rv = self.dispatch_request()
2020-03-09T14:12:55.181783+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
2020-03-09T14:12:55.181783+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2020-03-09T14:12:55.181783+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 468, in wrapper
2020-03-09T14:12:55.181784+00:00 app[web.1]: resp = resource(*args, **kwargs)
2020-03-09T14:12:55.181784+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/views.py", line 89, in view
2020-03-09T14:12:55.181785+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2020-03-09T14:12:55.181785+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
2020-03-09T14:12:55.181786+00:00 app[web.1]: resp = meth(*args, **kwargs)
2020-03-09T14:12:55.181786+00:00 app[web.1]: File "/app/app.py", line 20, in post
2020-03-09T14:12:55.181787+00:00 app[web.1]: x , sr = librosa.load(audioFile,mono=True,duration=5)
2020-03-09T14:12:55.181787+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/librosa/core/audio.py", line 129, in load
2020-03-09T14:12:55.181788+00:00 app[web.1]: with sf.SoundFile(path) as sf_desc:
2020-03-09T14:12:55.181789+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 629, in __init__
2020-03-09T14:12:55.181789+00:00 app[web.1]: self._file = self._open(file, mode_int, closefd)
2020-03-09T14:12:55.181789+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 1182, in _open
2020-03-09T14:12:55.181790+00:00 app[web.1]: raise TypeError("Invalid file: {0!r}".format(self.name))
2020-03-09T14:12:55.188875+00:00 app[web.1]: TypeError: Invalid file: None
2020-03-09T14:12:55.196231+00:00 app[web.1]: 10.168.89.215 - - [09/Mar/2020:14:12:55 +0000] "POST /receiveWav/sav HTTP/1.1" 500 37 "-" "Apache-HttpClient/UNAVAILABLE (java 1.5)"
2020-03-09T14:12:55.524669+00:00 heroku[router]: sock=backend at=error code=H18 desc="Server Request Interrupted" method=POST path="/receiveWav/sav" host=audient.herokuapp.com request_id=c9e15867-f4b6-4ea4-b3c7-5107fb8493dd fwd="183.82.146.189" dyno=web.1 connect=0ms service=355ms status=503 bytes=208 protocol=https
2020-03-09T14:15:03.316967+00:00 app[web.1]: [2020-03-09 14:15:03,316] ERROR in app: Exception on /receiveWav/sav [POST]
2020-03-09T14:15:03.316977+00:00 app[web.1]: Traceback (most recent call last):
2020-03-09T14:15:03.316979+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
2020-03-09T14:15:03.316979+00:00 app[web.1]: rv = self.dispatch_request()
2020-03-09T14:15:03.316979+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
2020-03-09T14:15:03.316981+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2020-03-09T14:15:03.316981+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 468, in wrapper
2020-03-09T14:15:03.316981+00:00 app[web.1]: resp = resource(*args, **kwargs)
2020-03-09T14:15:03.316982+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/views.py", line 89, in view
2020-03-09T14:15:03.316983+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2020-03-09T14:15:03.316983+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
2020-03-09T14:15:03.316984+00:00 app[web.1]: resp = meth(*args, **kwargs)
2020-03-09T14:15:03.316984+00:00 app[web.1]: File "/app/app.py", line 20, in post
2020-03-09T14:15:03.316985+00:00 app[web.1]: x , sr = librosa.load(audioFile,mono=True,duration=5)
2020-03-09T14:15:03.316985+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/librosa/core/audio.py", line 129, in load
2020-03-09T14:15:03.316986+00:00 app[web.1]: with sf.SoundFile(path) as sf_desc:
2020-03-09T14:15:03.316986+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 629, in __init__
2020-03-09T14:15:03.316986+00:00 app[web.1]: self._file = self._open(file, mode_int, closefd)
2020-03-09T14:15:03.316987+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 1182, in _open
2020-03-09T14:15:03.316987+00:00 app[web.1]: raise TypeError("Invalid file: {0!r}".format(self.name))
2020-03-09T14:15:03.316996+00:00 app[web.1]: TypeError: Invalid file: None
2020-03-09T14:15:03.317808+00:00 app[web.1]: 10.63.251.11 - - [09/Mar/2020:14:15:03 +0000] "POST /receiveWav/sav HTTP/1.1" 500 37 "-" "Apache-HttpClient/UNAVAILABLE (java 1.5)"
2020-03-09T14:15:03.529280+00:00 heroku[router]: sock=backend at=error code=H18 desc="Server Request Interrupted" method=POST path="/receiveWav/sav" host=audient.herokuapp.com request_id=82498799-a9d8-498a-83ee-b6d9c8519558 fwd="183.82.146.189" dyno=web.1 connect=0ms service=214ms status=503 bytes=208 protocol=https
The fieldname wasn't mentioned in both the libraries (Loopj and Fuel HTTP) and hence Flask-restful didn't receive anything with the field name "file" while parsing the request.
The fixed code snippets are:
Fuel.upload("https://audient.herokuapp.com/receiveWav")
.add(FileDataPart(File(output),filename = "iam.wav", name="file"))
.also { println(it) }
.response { result ->
Log.d("HUSKY", "${result.get()}")
}
val client = AsyncHttpClient()
val params = RequestParams()
params.put("file", File(output))
client.post("https://audient.herokuapp.com/receiveWav", params,object : AsyncHttpResponseHandler() {
override fun onStart() {
// called before request is started
}
override fun onSuccess(
statusCode: Int,
headers: Array<Header?>?,
response: ByteArray?
) {
println("HTTP STATUS 200 OK")
}
override fun onFailure(
statusCode: Int,
headers: Array<Header?>?,
errorResponse: ByteArray?,
e: Throwable?
) {
// called when response HTTP status is "4XX" (eg. 401, 403, 404)
println("HTTP STATUS 400/500 NOT OK")
}
override fun onRetry(retryNo: Int) {
// called when request is retried
}
})