How to use deeplearn.js in a React component

Question

I am currently working on creating a project with react and deeplearn.js, and have reached a roadblock when combining the two. In my react application I am importing this deeplearnjs library model which I am using to do classification. Unfortunately, when I try to call the predict() method I get the following error:

TypeError: _this.variables is undefined

For the following part of code:

SqueezeNet.prototype.predictWithActivation = function (input, activationName) {
   var _this = this;
   var _a = this.math.scope(function () {
      var activation;
      var preprocessedInput = _this.math.subtract(input.asType('float32'), _this.preprocessOffset);

When I use the generated Javascript in a normal HTML it works perfectly, so I am unsure why I am getting this error within react. I have a feeling it has to do with stricter React rules or Javascript versioning, but I am not sure.

Thanks!

UPDATE

The simplest way to reproduce this is the following:

1. Create a new React app with create-react-app
2. Run yarn add deeplearn and yarn add deeplearn-squeezenet

3. Modify App.js to the following:

   import React, { Component } from 'react';
   import {ENV, Array3D} from 'deeplearn';
   import {SqueezeNet} from 'deeplearn-squeezenet';
   
   class App extends Component {
     constructor() {
       super();
       var net = new SqueezeNet(ENV.math);
       net.load();
   
       var img = new Image(227, 227);
       img.src = 'boat.jpg';
       img.onload = function () {
         var pixels = Array3D.fromPixels(img)
         var res = net.predict(pixels);
       };
     }
     render() {
       return (
         <div></div>
       );
     }
   }
   
   export default App;
   

4. Download the following file into the public folder: https://raw.githubusercontent.com/PAIR-code/deeplearnjs/master/models/squeezenet/cat.jpg

5. Run yarn start

For reference I am using react 16.2.0

Answer 1

Your code is presumably failing because some of the method calls are asynchronous (.load() for example).

Here is how you would make your example work with React:

1. Create a new React app with create-react-app
2. Run yarn add deeplearn and yarn add deeplearn-squeezenet
3. Add cat.jpg to the public folder

4. Modify App.js as below

   import React, { Component } from 'react';
   import { ENV, Array3D } from 'deeplearn';
   import { SqueezeNet } from 'deeplearn-squeezenet';
   
   const math = ENV.math;
   const squeezeNet = new SqueezeNet(math);
   
   class App extends Component {
     constructor() {
       super();
       this.state = {
         statusText: 'Loading Squeezenet...'
       }
     }
   
     buildSuggestions(obj){
       return Object.keys(obj).map(
         key => `${obj[key].toFixed(5)}: ${key}`
       );
     }
   
     imageLoadHandler(e) {
       const img = e.target;
       squeezeNet.load()
         .then(() => {
           this.setState({ statusText: 'Predicting...' });
           const pixels = Array3D.fromPixels(img);
           const result = squeezeNet.predict(pixels);
           this.setState({ statusText: '' });
           squeezeNet.getTopKClasses(result, 5)
             .then((obj) => {
               this.setState({ statusText: this.buildSuggestions(obj) });
             });
         });
     }
   
     render() {
       const text = Array.isArray(this.state.statusText)?
         this.state.statusText :
         [this.state.statusText];
   
       return (
         <div>
           <img src="cat.jpg"
                alt="cat"
                onLoad={this.imageLoadHandler.bind(this)}
           />
           <div id="result">
             { text.map(el => <div key={el}>{el}</div>) }
           </div>
         </div>
       );
     }
   }
   
   export default App;
   

5. Then run yarn start