I'm attempting to use the "file-type" NPM module (which I have working on the server) client side to validate mime type prior to a file upload to an S3 bucket.
The readme for the module includes an example of using it in the browser:
const FileType = require('file-type/browser');
const url = 'https://upload.wikimedia.org/wikipedia/en/a/a9/Example.jpg';
(async () => {
const response = await fetch(url);
const fileType = await FileType.fromStream(response.body);
console.log(fileType);
//=> {ext: 'jpg', mime: 'image/jpeg'}
})();
That obviously won't work directly in the browser due to the "require", and if I link to the NPM file directly, I get:
Uncaught ReferenceError: require is not defined
So I've tried using Webpack (with which I'm not well versed at all, but have followed the official Webpack tutorial, and a few other tutorials) to create a "main.js" file and then accessed that via a script tag.
For example, running this through Webpack:
import * as mimeFromBuffer from 'file-type/browser.js';
export function mimeTyping(mimeValidationBytes){
(async () => {
const fileType = await mimeFromBuffer(mimeValidationBytes);
console.log(fileType);
//=> {ext: 'jpg', mime: 'image/jpeg'}
})();
return;
}
Results in this when I call the mimeTyping client side:
Uncaught ReferenceError: mimeTyping is not defined
I've tried Vinay's answer from this question in the browser:
import fileType from 'file-type';
const blob = file.slice(0, fileType.minimumBytes);
const reader = new FileReader();
reader.onloadend = function(e) {
if (e.target.readyState !== FileReader.DONE) {
return;
}
const bytes = new Uint8Array(e.target.result);
const { ext, mime } = fileType.fromBuffer(bytes);
// ext is the desired extension and mime is the mimetype
};
reader.readAsArrayBuffer(blob);
Which gets me this:
Uncaught SyntaxError: Cannot use import statement outside a module
I've researched that error a bit, with a common solution being to add "type":"module" to package.json. This did not help (error was unchanged).
I found a similar question in the module's Github repository, which suggests:
Although this module is primary designed for Node.js, using it in the browser is possible as well. That is indeed where 'file-type/browser' intended for. It provides the right dependencies and right functions to the JavaScript module bundler. Some dependencies, which are typically present in a Node.js environment, but are missing in a browser environment, you may need to pass (polyfill) to your module bundler. Although in some cases it may a bit tricky to configure it, note that this is a pretty common task handled by a module bundler.
I'm struggling to understand what next steps that suggests.
Another comment at the bottom of that thread indicates the author had success with:
import { fromBuffer } from 'file-type/core';
...
const buffer = await uploadedFile.arrayBuffer();
const types = await fromBuffer(buffer);
I'm not sure how to implement that, and what other code I need (I'm guessing this gets passed to Webpack, and probably requires an export statement, and then an import on the client side?)
I've tried passing this into Webpack:
const fileType = require('file-type/browser');
module.exports = fileType;
But linking to the output file again gets:
Uncaught SyntaxError: Cannot use import statement outside a module
I think I understand conceptually what I need to do: pass the NPM module to Webpack, which in turn parses it and finds any dependencies, gets those dependencies, and creates a JavaScript file I can use client side. It seems I'm doing something in there wrong.
I've spent days trying to understand how to use NPM modules client side (still quite foggy in my mind) and trying all sorts of variants on the above code - would really appreciate some guidance (first time posting a question here - please go easy on me!).
Thank you!
Edit: I don't think this is a duplicate - I did review How to import JS library from node_modules, Meteor Npm-module client-side?, how to use node.js module system on the clientside, How to call a server-side NodeJS function using client-side JavaScript and Node Js application in client machine but no suggestion I tried in any of those seemed to help.
Finally got this working. In case anyone else is stuck on this, here's an explanation (apologies for the lack of brevity - probably this should be a blog post...).
To flesh out the use case a bit further, I'm using Uppy to allow users to upload files to an AWS S3 bucket. The way this works is that, when the user uploads a file, Uppy makes a call to my server where an AWS pre-signed URL is generated and passed back to the client. The client then uses that pre-signed URL to upload the file directly to the S3 bucket, bypassing the server, such that the file doesn't pass through the server at any point.
The problem I was attempting to solve was that files missing an extension ended up uploaded with the content / MIME type set as "application/octet", because it seems the browser, Uppy, and S3 all rely on the file extension to decide the file type (rather than parsing the so-called "magic bytes" of the file), and if the file extension is missing, AWS defaults to "application/octet". This causes issues when users attempt to open the file, as they are not handled correctly (i.e. a png file without an extension and with an "application/octet" content / MIME type opens a download dialog rather than being previewed, etc.). I also want to validate the MIME type / file type in cases even where the extension exists so that I can exclude certain types of files, and so the files get handled appropriately when they are later downloaded (where the MIME type will again be validated) if an incorrect file extension is used.
I use the "file-type" NPM module to determine the mimetype server side, and that's straight forward enough, but changing the file's content type / MIME type when generating the AWS pre-signed URL is not enough to fix the problem - it still gets uploaded as "application/octet". I wanted to use the same module client side so we get the exact same results on the client as on the server, and needed in any case to determine the MIME type and set it accordingly pre-upload but post-pre-signed URL. I had no idea how to do this (i.e. use "file-type" client side - the meat of my question).
I finally gave up on Webpack - nothing I tried worked. So I switched to Browserify, and the sample browser code at the "file-type" repository worked at once! So then I was left trying to figure out how to pass a function through Browserify to use in the client side code.
This proved impossible for me - I couldn't figure out how to pass the asynchronous IIFE through to my code. So instead, I moved my Uppy code into the code I pass to Browserify:
// Get the dependency, which we've added to node via "npm install file-type":
const FileType = require('file-type/browser');
// When the user adds a file for upload to Uppy...
uppy.on('file-added', (file) => {
// 1. Create a filereader:
const filereader = new FileReader();
filereader.onloadend = function(evt) {
// 4. Once the filereader has successfully finished reading the file...
if (evt.target.readyState === FileReader.DONE) {
// Get the unsigned 8 bit int8Array (ie Uint8Array) of the 600 bytes (this looks like '[119,80,78,71...]'):
const uint = new Uint8Array(evt.target.result);
// Determine the mime type using the "file-type" NPM package:
(async () => {
// Pass in our 600 bytes ("uint"):
const fileType = await FileType.fromBuffer(uint);
console.log(fileType); // outputs => {ext: 'jpg', mime: 'image/jpeg'}
// Do some validation here...
//
// Assign the results to the file for upload - we're done!:
file.extension = fileType.ext;
file.meta.type = fileType.mime;
file.type = fileType.mime;
})();
}
}
// 2. Grab the first 600 bytes of the file for mime type analysis server side - most mime
// types use the first few bytes, but some (looking at you, Microsoft...) start at the
// 513th byte; ISO CD files start at the 32,770th byte, so we'll ignore those rather than
// send that much data for each file - users of this system certainly aren't expected to
// upload ISO CD files! Also, some .zip files may start their identifying bytes at the
// 29,153nd byte - we ignore those too (mostly, .zip files start at the first, 31st, or 527th byte).
const blob = file.data.slice(0, 600);
// 3. Start reading those 600 bytes...continues above at 'filereader.onloadend':
filereader.readAsArrayBuffer(blob);
})
That all goes into a file I call "index.js" and then, having installed Browserify at the command line via "npm install -g browserify", I use this at the command line to create the file ("main.js") I link to in my client side code:
browserify index.js -o main.js