I am working on a little project where I need to store business hours in my MongoDB database. I decided to use a bitmask for the entire week represented by 336bits which means that every bit represents 30 minutes of the day. (Using a bitmask has no special reason. I am willing to change the way I'm storing business hours, but I'm curious anyway.)
I thought it would be convenient if I'd use the $bitsAnySet operator to query based on the bitmask, because I could just figure out the position that needed to be set if a customer queries for open businesses a t 4pm for example.
Reading the docs about the operator I found out that you can provide a array of positions you want checked, which would be the perfect fit for my needs.
But here's the catch. I can't figure out how those positions work. The docs state, that position 0 will be on the least significant bit (the one most right) meaning a document with a number like 254 (0b1111 1110) stored as the business hours would not be found if querying $bitsAnySet: [0], because the far most right bit is set to 0 in this example.
Using the operator this way, testing it on my test data I always ran into problems, because the positioning didn't seem to match.
So I wrote myself a little script (in JavaScript running on NodeJS v12 connecting to MongoDB using Mongoose) that was simply trying all the possible positions:
Here my Script:
require ('mongoose').connect ('mongodb://localhost/test', {useCreateIndex: true, useNewUrlParser: true, useFindAndModify: false, useUnifiedTopology: true}).then (() => {
let buffer = Buffer.alloc (6, 0);
// 0xC0 => 0b1100 0000
buffer.writeUInt8 (0xC0, 0);
// 0x07 => 0b0000 0111
buffer.writeUInt8 (0x07, 5);
// buffer should be: 11000000 00000000 00000000 00000000 00000000 00000111
new SomeModel ({data: buffer}).save ().then (model => {
let promises = [];
// just trying out each position
for (let index = 0; index < 48; index ++)
promises.push (SomeModel.findOne ({data: {$bitsAnySet: [index]}}))
Promise.all (promises).then (resolved => {
resolved = resolved.map ((item, index) => {
if (item !== null)
return {index, item: item.data.toString ('hex')};
else return null;
});
// yes, I know it's not very performant to go over the entire array a second time,
// because I could have done it in the first round,
// but this was just some quick and dirty testing :D
resolved = resolved.filter (item => item !== null);
console.log (resolved);
})
}).catch (console.log);
}).catch (console.log);
This was my little testing script, and I expected an output like:
[
{index: 0, item: 'c00000000007'},
{index: 1, item: 'c00000000007'},
{index: 2, item: 'c00000000007'},
{index: 41, item: 'c00000000007'},
{index: 42, item: 'c00000000007'},
]
because when looking at the binary representation coming from the right it should first find the three ones at the positions 0, 1 and 2 and then it should find the last two ones at the very left end.
But I got this:
[
{ index: 6, item: 'c00000000007' },
{ index: 7, item: 'c00000000007' },
{ index: 40, item: 'c00000000007' },
{ index: 41, item: 'c00000000007' },
{ index: 42, item: 'c00000000007' }
]
This blew my mind, because it was totally unexpected for me. Suddenly it started looking like MongoDB is starting from the left. It found the two ones before the three ones, and the two ones are located at the very left of the binary data. Which also confuses me is that it doesn't seem to start at position 0.
If I'm not wrong my data starts with two ones and ends with three. So a $bitsAnySet: [0] should be true either way.
Does anyone of you know what I'm doing wrong here? I'm very thankful for your help!
Note: Correct me if I'm wrong, but I believe that Promise.all does not change the order of the array, meaning that it couldn't just be that one promise just resolved before the other changing its position in the 'promises' array.
Note: The model does not have any additional fields. It just has the default _id field and the data field.
Fortunately I was able to figure this one out myself. This has to do with endianness. In my mind I was thinking in the big endian way therefore assuming it to be stored as follows:
Hex: c0 00 00 00 00 07
Bin: 11000000 00000000 00000000 00000000 00000000 00000111
According to the BSON Specification BSON is
serialized in little-endian format.
therefore its actually stored like this:
Hex: 07 00 00 00 00 C0
Bin: 00000111 00000000 00000000 00000000 00000000 11000000
This explains why the indexes where mixed around.