I'm trying to follow a tutorial on YT on how to write a basic neural network, but it's written in python. I'm trying to write everything using javascript and I haven't found a way to read the npz file into JS.
I have tried npyjs
and tfjs-npy-node
, but neither works. Using npy.js yields the error:
Only absolute URLs are supported
Using tfjs-npy-node yields any one of these errors, depending on how I pass it the file:
Expected provided filepath () to have file extension .npy
Error: assert failed
Error: Not a numpy file
Are there any functional libraries that work in Node and will read/parse the file without me needing to create an entire project dedicated to it? (looking at you, tensorflow)
Edit: Examples
npyjs: import npyjs from 'npyjs';
const n = new npyjs();
const dataPath = path.resolve('./data/mnist.npz')
console.log(dataPath)
let data = n.load(dataPath);
console.log('NPZ data:', data);
result:
D:\Projects\Programming\Personal\neural-networks-> tutorial\js\data\mnist.npz NPZ data: Promise { } D:\Projects\Programming\Personal\neural-networks-tutorial\js\node_modules\node-fetch\lib\index.js:1327 throw new TypeError('Only absolute URLs are supported'); ^
TypeError: Only absolute URLs are supported
I assume it's related to this issue, but following the "working" solution: import npyjs from 'npyjs';
const n = new npyjs();
const dataPath = path.resolve('./data/mnist.npz')
console.log(dataPath)
let buf = fs.readFileSync(dataPath);
let data = n.parse(buf.buffer)
console.log('NPZ data:', data);
gives:
SyntaxError: Unexpected token '', "!�&vt" is not valid JSON at JSON.parse () at npyjs.parse (file:///D:/Projects/Programming/Personal/neural-networks-tutorial/js/node_modules/npyjs/index.js:126:29)
tfjs-npy-node:
import {npz} from "tfjs-npy-node"
(same as before)
let data = await npz.load(dataPath);
gives:
Error: Could not load D:\Projects\Programming\Personal\neural-networks-tutorial\js\data\mnist.npz: No backend found in registry. at Object.load (D:\Projects\Programming\Personal\neural-networks-tutorial\js\node_modules\tfjs-npy-node\dist\src\npz.js:41:15)
Project Structure: root
|- data
-|- mnist.npz
|- program.js
Edit: Sometimes simply googling the answer instead oif a sub propblem is best: answer
The answer for tfjs-npy-node
is to install a different library @tensorflow/tfjs-backend-cpu
and import import "@tensorflow/tfjs-backend-cpu"
into the project. This allows it to read the file, but only shows teh tensors as far as I can tell.
Here is what worked for me(based on this):
import path from "path"
import npyjs from 'npyjs'
import JSZip from 'jszip'
import fs from "fs"
const dataPath = path.resolve('./data/mnist.npz')
let contents = fs.readFileSync("./data/mnist.npz")
const jsZip = new JSZip()
const npzFiles = await jsZip.loadAsync(contents)
const _npyjs_ = new npyjs()
let data = Object.entries(npzFiles.files)
for (const [npy_filename, npy_data] of Object.entries(npzFiles.files)) {
if (!npy_filename.endsWith('.npy')) {
console.error('error .npy')
}
const npy_array_buffer = await npzFiles.files[npy_filename].async("arraybuffer")
console.log({ npy_filename, npy_data, npy_array_buffer: npy_array_buffer })
data[npy_filename] = await _npyjs_.parse(npy_array_buffer)
}