FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

Question

I trained a HSMW Index using faiss on Windows. I wanted to deploy this to an Azure Function App, hence I just pip to install faiss-cpu==1.7.3 rather than the recommended install with conda.

The code that searches for the most similar vector is

distances, indices = index.search(vector, k)

works without issue on Windows, Python 3.10.4, faiss-cpu==1.7.3

However, in the Azure Function App and also on Ubuntu, it throws the following error.

  File "/home/vboxuser/Documents/VectorDB/src/match_vector.py", line 30, in get_matching_vectors
    distances, indices = index.search(vector, k)
  File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/__init__.py", line 322, in replacement_search
    self.search_c(n, swig_ptr(x), k, swig_ptr(D), swig_ptr(I))
  File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/swigfaiss.py", line 5436, in search
    return _swigfaiss.IndexHNSW_search(self, n, x, k, distances, labels)
TypeError: in method 'IndexHNSW_search', argument 3 of type 'float const *'

I would love to understand what causes this and whether there is a work around so that I can run it in the Azure Function App.

Thanks

Answer 1

According to the solution in this github issue, Refer here:- TypeError: in method 'IndexFlat_add', argument 3 of type 'float const *' · Issue #461 · facebookresearch/faiss · GitHub Make sure the generated vector types or array is of numpy.float32

Code 1:-

import numpy as np
d = 128
n = 10000
xb = np.random.rand(n, d).astype(np.float32)

vector = [0.5] * d
vector = np.array(vector).astype(np.float32)

distances = np.linalg.norm(xb - vector, axis=1)

k = 5
indices = np.argsort(distances)[:k]

print("Distances:", distances[indices])
print("Indices:", indices)

Output:-

Code 2:-

import numpy as np

d = 128

n = 10000

x = np.random.rand(n, d).astype(np.float32)

print (x)

Output:-