Flutter Dio receiving gibberish

Question

I am pulling data from eurostat but dio is receiving gibberish data even though the status code is successful, shown in this

Here is my code

class DatabaseHelperEurope extends DatabaseHelper {
  DatabaseHelperEurope(this.code);

  //initializes the database
  static Database? _database;
  Future<Database> get database async {
    return _database ??= await _initDatabase();
  }

  String code;
  String error = "";
  final String baseUrl = "https://ec.europa.eu/eurostat/api/dissemination";
  final dio = Dio();

  Future<Database> _initDatabase() async {
    var databasesPath = await getExternalStorageDirectory();
    var path = join(databasesPath!.path, "$code.csv");
    var exists = await databaseExists(path);

    if (!exists) {
      String url =
          "https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/PRC_HICP_MIDX/.I15..$code?format=SDMX-CSV";
      try {
        downloadFile(url, databasesPath!.path);
      } catch (e) {
        error = e.toString();
      }
    }

    return await openDatabase(path);
  }

  @override
  Future<IO.File?> downloadFile(String url, String path) async {
    final file = IO.File(join(path, "$code.csv"));

    final response = await dio.fetch(RequestOptions(
      baseUrl: url
    ));
    final raf = file.openSync(mode: IO.FileMode.write);
    raf.writeByte(response.data);
    debugPrint("data written");
    await raf.close();

    return file;
  }

  @override
  Future<List<DatabaseContent>> getDatabaseContent() async{
    Database db = await database;
    var databaseContents = await db.rawQuery(
        "SELECT * FROM europe_price_index_all WHERE geo like 'DE' and coicop like 'CP00' and unit like 'I15';");
    List<DatabaseContent> databaseContentList = databaseContents.isNotEmpty
        ? databaseContents.map((e) => DatabaseContent.fromMap(e)).toList()
        : [];
    return databaseContentList;
  }

I tried using fetch and get but I get the same result, I tried fetching the data from Python using requests and its output should be like this.

Answer 1

SDMX-CSV != CSV

You are requesting a SDMX-CSV but trying to treat it as a normal CSV document. Looking at the SDMX homepage, it becomes clear that this data format is optimized for the exchange of data.

In general: If I see this type of symbols I would always check:

1. Is the encoding right? (Setting, UTF-8 should solve most of these issues)
2. Is there some sort of compression going on? (See documentation which compression is used) However, this was unlikely.
3. Is the response in the form of arbitrary binary data? (See documentation of service on how to decode it properly)

For SDMX: It looks like much of the Standard is still not finished, so documentation is sparse. However I could find this document and this cheat sheet. In any case you are going to have to read a lot through the documentation in order to find out what's wrong.

I also noticed that you didn't define any headers, so I would suggest adding the following (it's also noted in the documentation):

Content-Type: application/vnd.sdmx.data+csv; version=1.0.0