Search code examples
javascriptnode.jsmachine-learningartificial-intelligenceonnx

How to load an onnx model using ONNX.js

I am trying to import an ONNX model using onnxjs, but I get the below error:

Uncaught (in promise) TypeError: cannot resolve operator 'Cast' with opsets: ai.onnx v11

Below shows a code snippet from my html file.

<html>
  <head> </head>
  <body>
    <!-- Load ONNX.js -->
    <script src="https://cdn.jsdelivr.net/npm/onnxjs/dist/onnx.min.js"></script>
    <!-- Code that consume ONNX.js -->
    <script>
        async function test(){
            const sess = new onnx.InferenceSession()
            await sess.loadModel('./onnx_model.onnx')
            }
        test()
    </script>
  </body>
</html>

How to solve this?

Solution

  • This will load Resnet 50 model

    const sess = new onnx.InferenceSession()
async function test(){
  console.time("loading model")
  await sess.loadModel('https://microsoft.github.io/onnxjs-demo/resnet50v2.onnx')
  console.timeEnd("loading model")

  console.log("model loaded");
}
document.querySelector('#load').addEventListener('click', test);
    <script src="https://cdn.jsdelivr.net/npm/onnxjs/dist/onnx.min.js"></script>

<button id="load">load model</button>

    The message suggest something to do with the Cast operator not being supported by opset 11, maybe you want to use Cast-9. Maybe you have to generate a new model.

    Edit

    Your model loads using onnxruntime python

    sess = onnxruntime.InferenceSession('../../Downloads/onnx_model.onnx');
{i.name: i.shape for i in sess.get_inputs()}
{o.name: o.shape for o in sess.get_outputs()}

    {'input_ids': ['batch', 'sequence'],
 'attention_mask': ['batch', 'sequence'],
 'token_type_ids': ['batch', 'sequence']}

{'output_0': ['batch', 2]}

    An Idea

    You will probably have to debug it your self, hopefully the only problem is the cast operator.

    You start looking here the operator support for onnxjs, and rewrite parts of the model where the operator appears.

    For instance, the Cast operator appears only once, you can locate it as follows

    import onnx
model = onnx.load('../../Downloads/onnx_model.onnx')
for node in model.graph.node:
    if 'cast' in node.op_type.lower():
        print(node.name, node.op_type)

    That will print

    Cast_2 Cast

    Using https://netron.app/ (Or the desktop version) you can see that it is netron-view

    So you should simply rewrite how your attention mask is processed in the model, a possible solution would be to let unsqueeze and cast operations outside the model.