Azure Custom Vision: Is it possible to store image metadata (NOT tags) in the Custom Vision environment?

Question

I have images with important file metadata (e.g. provenance and processing history) stored locally or in Azure blob storage.

I would like to import (POST) these to the Azure Custom Vision environment (via the API or GUI) (see e.g. https://southcentralus.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Training_3.0) for training while (i) retaining those image metadata and (ii) being able to retrieve them via (a) the Custom Vision API and (b) the Custom Vision GUI.

An example use case would be to purge images of a certain provenance from the Custom Vision store because of a GDPR-related customer request [Aside: I appreciate that Azure Cognitive Services can anyway use the data for improving their models etc.].

As far as I can tell the only way to reference an image POSTed to Custom Vision is via its UUID. Is there any other way to reference metadata stored with that image or:

1. Would that constitute a feature request?

2. Could the image metadata be stored inside the image (e.g. JPEG EXIF) (assuming it is possible to retrieve the image itself from the Custom Vision "environment", which it may not be)?

3. Otherwise, is the only solution to store the returned Custom Vision image UUID in a database elsewhere alongside the required metadata?

NB In the above, by metadata I do not mean tags/labels in the image model-side sense, but rather data-side file metadata.

[Note that Azure Cognitive Services is using stackoverflow for Q&A, so this question is I believe appropriate for stackoverflow.]

Thanks as ever!

Answer 1

I am not part of Microsoft, so it is only my opinion based on my usage of Custom Vision.

I understand your use-case, especially regarding GDPR as you mentioned, but currently adding metadata is not a feature, whether through the API or the GUI.

To answer your questions:

Would that constitute a feature request?

Definitely. You can create an item on UserVoice for this feature (but 1st check if there is not another related existing item): https://cognitive.uservoice.com/forums/598141-custom-vision-service

Could the image metadata be stored inside the image (e.g. JPEG EXIF) (assuming it is possible to retrieve the image itself from the Custom Vision "environment", which it may not be)?

You can get the images your previously posted using GetImagesByIds method from the API for example (or GetTaggedImages / GetUntaggedImages). Remember that the images you post are treated: thumbnail images, resized images are generated based on what you posted. These methods provides the links to the images.

I made a quick try by: - Uploading an image to CustomVision and adding a tag - Getting its id - Getting the image through the API

Good news: on the image downloaded with the "originalImageUri" link, I still have some EXIF available (just needed to rename the file as ".jpg" after download):

Otherwise, is the only solution to store the returned Custom Vision image UUID in a database elsewhere alongside the required metadata?

Right now it is clearly the best solution in my opinion