I'm trying to use a web worker to get some data and then pass them back to the main thread. I have tried with this code but it will not work as expected
onmessage = (e) => {
console.log(e);
if( e.data[0] === 'fetchData' ){
fetch('https://example.com/platform/api/v1/endpoint')
.then( (res) => res.json() )
.then( async (data) => {
const imagesData = await Promise.all(
data.map( async (item) => {
let res = await fetch(item.src);
let img = await res.blob();
let reader = new FileReader();
reader.readAsDataURL(img);
reader.onloadend = () => {
return {
title: item.title,
link: item.link,
src: reader.result
}
}
})
)
postMessage(imagesData);
})
}
}
The imagesData after a console.log
will contain an array of seven undefined elements, how I can fix this?
UPDATE
I've changed the code in vue front-end and into the worker and now I'm able to get the data but sometimes the worker will not work and I will not able to get the data or I get just two entries but the expected number is seven items for the front-end. Here is how I've modified the code, maybe I need to terminate the work before use another one? NB: I'm creating a tab override chrome extension
vue front-end code
<script>
const worker = new Worker('services.js');
export default {
name: 'App',
beforeCreate() {
worker.postMessage(['fetchData']);
},
created() {
this.init();
this.clock();
},
data() {
return {
mostVisited: [],
imagesData: [],
isLoading: true
}
},
methods: {
init() {
worker.onmessage = (e) => {
console.log(e);
this.imagesData = e.data;
this.isLoading = false;
}
browser.topSites.get().then( (sites) => this.mostVisited = sites );
} //end init
}
</script>
web worker code
onmessage = (e) => {
console.log(e);
if( e.data[0] === 'fetchData' ){
fetch('https://example.com/platform/api/v1/endpoint')
.then( (res) => res.json() )
.then( async (data) => {
let imagesData = [];
await Promise.all(
data.map( async (item) => {
let res = await fetch(item.src);
let img = await res.blob();
let reader = new FileReader();
reader.readAsDataURL(img);
reader.onloadend = () => {
imagesData.push({ title: item.title, link: item.link, src: reader.result });
}
})
)
postMessage(imagesData);
}); // end then(data)
}
}
You are not waiting for the asynchronous FileReader to have read your files before resolving the outer Promise, so the Promise.all
Promise resolves after let img = await res.blob();
but before onloadend
does.
Since you are in a Worker context, you can use the synchronous FileReaderSync API, which would give something like
.then( async (data) => {
let imagesData = [];
await Promise.all(
data.map( async (item) => {
let res = await fetch(item.src);
let img = await res.blob();
let reader = new FileReaderSync();
const result = reader.readAsDataURL(img);
imagesData.push({ title: item.title, link: item.link, src: result });
})
)
postMessage(imagesData);
});
But I'm 99% confident that you don't even need that data: URL, and that it will do more harm than anything.
Remember that the data: URL from a FileReader always encode to base64, which will produce a DOMString containing 134% of the original data, that you multiply per 2 since DOMStrings are stored in UTF-16.
Then, to pass that data to the main context, the browser will have to serialize the data using the structured clone algorithm, and for DOMStrings that means a simple copy in memory. Each image now takes in memory about 5 times its real size, and it's even before the main context starts parsing it again to binary data so it can build the pixels data...
Instead, simply pass the Blob you get as img
.
Blobs' inner data is passed by reference by the structured clone algorithm, so all you copy is the small js wrapper around.
Then when you need to display these images on the main context, use URL.createObejctURL(img)
which will return a blob: URL directly pointing to that same Blob's data, which is still stored only once in memory.
.then( async (data) => {
let imagesData = [];
await Promise.all(
data.map( async (item) => {
let res = await fetch(item.src);
let img = await res.blob();
const url = URL.createObjectURL(img);
imagesData.push({ title: item.title, link: item.link, src: url, file: img });
})
)
postMessage(imagesData);
});