Find image from a database of images

I'm moving my first steps with OpenCV in Python.
What I'd wish to do is given an image, find its "original" one from a collection of reference images. Just to be clear, the query image is a simple photo of the whole image (card), so it's not the scenario "find an object inside a photo", but "just" a similarity test.
My final database will be pretty large (about 25 000 images), but I started doing some tests on a smaller scale (only 270 images).
Recognition works perfectly, however it's pretty slow: it takes 8 seconds to iterate over all 270 images. I was able to speed up the job by saving the descriptors to disk and load them, instead of calculating them; anyway it's still slow.

So I started to work on FLANN: I get some results, but my main problem is to find the matching images. I get a whole array of points, but I don't know how to fetch the right image.
This is my code:

scanned = 'tests/temp_bw.png'
surf = cv2.xfeatures2d.SURF_create(400)
surf.setUpright(True)

img1 = cv2.imread(scanned, 0)
kp1, des1 = surf.detectAndCompute(img1, None)

FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)

des_all = None
for filename in os.listdir('images'):
    img2 = cv2.imread('images/' + filename, 0)
    kp2, des2 = surf.detectAndCompute(img2, None)
    if des_all is None:
        des_all = des2
    else:
        des_all = np.concatenate((des_all, des2))

flann = cv2.flann.Index()
print "Training..."
flann.build(des_all, index_params)
print "Matching..."
indexes, matches = flann.knnSearch(des1, 10)
# and now???

Any suggestions on how I can reference back the most similar image?

EDIT This is the shape of des2

print des2.shape
(1731, 64)
(2144, 64)
(1811, 64)
(1649, 64)

These are the return values from flann.knnSearch:

pprint(matches[0])
array([ 0.0463759], dtype=float32)

pprint(indexes[0])
array([249106], dtype=int32)

Solution

While building your des_all, make a list that has the same length as the number of descriptors, where each element is the index of the image this descriptor is from.

Once you have your matched features, you can look this up in the list, which image the feature was matched in. Then you can do a simple majority vote on which image is correct, or use the inverse of the distances to weigh the votes by importance.

Edit: If you modify your for loop like this, you can build the list that has the number of which image has generated which features:

for index, filename in enumerate(os.listdir('images')):
    img2 = cv2.imread('images/' + filename, 0)
    kp2, des2 = surf.detectAndCompute(img2, None)
    indexList.extend([index]*des2.shape[0])

If you do the voting by majority voting, it can be done like this (each feature votes for the image it's matched to, and the image with the most votes wins)

votes = indexList[indexes]
from collections import Counter
votes = Counter()
likelyMatch = votes.most_common()[0][0]

If you want to weigh using the distance of the matches, you should remember that the distances in matches are squared, and you should take the square root. These distances can be used as inverse weights, to make each feature vote with higher or lower weight, depending on how good of a match it is.