How to read predict() result in Tensorflowjs using a SavedModel

Question

Code using tfjs-node:

const model = await tf.node.loadSavedModel(modelPath);
const data = fs.readFileSync(imgPath);
const tfimage = tf.node.decodeImage(data, 3);
const expanded = tfimage.expandDims(0);
const result = model.predict(expanded);
console.log(result);
for (r of result) {
   console.log(r.dataSync());
}

Output:

(8) [Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor]
Float32Array(100) [48700, 48563, 48779, 48779, 49041, 48779, ...]
Float32Array(400) [0.10901492834091187, 0.18931034207344055, 0.9181075692176819, 0.8344497084617615, ...]
Float32Array(100) [61, 88, 65, 84, 67, 51, 62, 20, 59, 9, 18, ...]
Float32Array(9000) [0.009332209825515747, 0.003941178321838379, 0.0005068182945251465, 0.001926332712173462, 0.0020033419132232666, 0.000742495059967041, 0.022082984447479248, 0.0032682716846466064, 0.05071520805358887, 0.000018596649169921875, ...]
Float32Array(100) [0.6730095148086548, 0.1356855034828186, 0.12674063444137573, 0.12360832095146179, 0.10837388038635254, 0.10075071454048157, ...]
Float32Array(1) [100]
Float32Array(196416) [0.738592267036438, 0.4373246729373932, 0.738592267036438, 0.546840488910675, -0.010780575685203075, 0.00041256844997406006, 0.03478313609957695, 0.11279871314764023, -0.0504981130361557, -0.11237315833568573, 0.02907072752714157, 0.06638012826442719, 0.001794634386897087, 0.0009463857859373093, ...]
Float32Array(4419360) [0.0564018189907074, 0.016801774501800537, 0.025803595781326294, 0.011671125888824463, 0.014013528823852539, 0.008442580699920654, ...]

How do I read the predict() response for object detection? I was expecting a dictionary with num_detections, detection_boxes, detection_classes, etc. as described here.

I also tried using tf.execute(), but it throws me the following error: UnhandledPromiseRejectionWarning: Error: execute() of TFSavedModel is not supported yet.

I'm using efficientdet/d0 downloaded from here.

Answer 1

When you download the tensor using dataSync() it just keeps the value. If you wanted the object with a description of each of the results without the tensors you would just have to console.log(result). Then you expand the result from your log in the browsers console it should return something like this:

Tensor {
  "dataId": Object {},
  "dtype": "float32",
  "id": 160213,
  "isDisposedInternal": false,
  "kept": false,
  "rankType": "2",
  "scopeId": 365032,
  "shape": Array [
    1,
    3,
  ],
  "size": 3,
  "strides": Array [
    3,
  ],
}

The output of your console.log(result) contains 8 tensors within it which shows that it is correct. You are looping over each of the results and each of the outputs should follow this format :

['num_detections', 'detection_boxes', 'detection_classes', 'detection_scores', 'raw_detection_boxes', 'raw_detection_scores, 'detection_anchor_indices', 'detection_multiclass_scores']