Use Flutter to send a http Post-Request (containing an image) to a Flask API

Question

I've trained a CNN on the CIFAR10 dataset (placeholder, will be replaced with a different model later) and integrated the model into a flask API. The API is hosted on Heroku, and I would now like to use Flutter / Dart to take pictures on my phone, send them to the Flask API, run my trained model on them and return the prediction.

Using python, I can easily make a post request to my API and return the predictions. Here is my simple python code for this:

import requests
import json

img = open('some-picture.jpg', 'rb')

files = {'image': img}
response = requests.post("url_to_api", files=files)

print(response.text)

I haven't been using Flutter / Dart for very long, and I gather that the process of making htpp requests is a little more complex than in python. Could someone give me some pointers or perhaps code that allows me to take a picture with my camera, upload it to my API, and store the response in a variable? Here's my (simplified) python code for the flask API:

from flask import Flask, request
import os
import numpy as np
from PIL import Image
from tensorflow import keras

app = Flask(__name__)
app.config["DEBUG"] = True

model = keras.models.load_model('cifar10_cnn.h5')
labels = ["Airplane", "Automobile", "Bird", "Cat", "Deer", "Dog", "Frog", "Horse", "Ship", "Truck"]


@app.route('/', methods=["POST"])
def predict():
    
    # stuff not relevant to question, left out for conciseness #
    
    file = request.files['image']

    image = Image.open(file).resize((32, 32))
    image = np.array(image)
    image = image / 255
    image = image.reshape(-1, 32, 32, 3)

    predictions = model.predict([image])
    index = np.argmax(predictions)
    results = {'Prediction:': labels[index]}

    return results

if __name__ == '__main__':
    app.run()

So far I know that Multipart files seem like the way to go, and that the Dio package might be worth looking into. If further tips or code could be provided I would be grateful.

Answer 1

You may already know how to choose an image from the gallery/camera (e.g. using image_picker library). You fill a class field like File image; with the result of that picker. This could be as easy as:

import 'dart:io';
import 'package:image_picker/image_picker.dart';
class _MyHomePageState extends State<MyHomePage> {
  File image;
  final picker = ImagePicker();

  pickImageFromGallery(ImageSource source) async {
    final image = await picker.getImage(source: source);

    setState(() {
      this.image = File(image.path);
    });
  }
}

(Change the ImageSource source to match your desire: camera or gallery)

Then you can upload that file to your api. Using http library:

import 'package:http/http.dart' as http;
import 'package:http_parser/http_parser.dart';

class _MyHomePageState extends State<MyHomePage> {
doUpload(){
    var request = http.MultipartRequest(
      'POST',
      Uri.parse("url_to_api"),
    );
    Map<String, String> headers = {"Content-type": "multipart/form-data"};
    request.files.add(
      http.MultipartFile(
        'image',
        image.readAsBytes().asStream(),
        image.lengthSync(),
        filename: "filename",
        contentType: MediaType('image', 'jpeg'),
      ),
    );
    request.headers.addAll(headers);
    print("request: " + request.toString());
    request.send().then((value) => print(value.statusCode));
}
}

For each one of these libraries, you have to add them as dependency inside pubspec.yaml in your flutter project:

cupertino_icons: ^0.1.3
http: ^0.12.2
image_picker: ^0.6.7