DeepFace for extracting vector information of an image

Question

for doing face recognition with Deepface I am trying to extract vector information of an image to store in db. So that next time in order to match, I will extract the new image's vector information and look into the db . If the search has results then its a match. I used verify method of the DeepFace but its comparing between 2 images and returning with this:

from deepface import DeepFace
import os

detected_face = DeepFace.detectFace("sly.jpg")
print (detected_face)

this is the output for above:

result  = DeepFace.verify("sly1.jpg","sly2.jpg");

for this I get:

Using VGG-Face model backend and cosine distance.
{'verified': True, 'distance': 1.1920928955078125e-07, 'max_threshold_to_verify': 0.4, 'model': 'VGG-Face', 'similarity_metric': 'cosine'}

This is the comparison result, but I need information of only one image without comparison because I will have lots of records to search(for vector info) when a new face will be tested. Any help will be appreciated.

Answer 1

I am assuming it is this repo among others with same name and installed with setup.py or pip install deepface.

I tested this on google colab. For using locally use cv2.imshow(...) instead of cv2_imshow(...).

Downloading test images

!wget "http://*.jpg" -O "1.jpg"

!wget "https://*.jpg" -O "2.jpg"

Check image

import cv2
from google.colab.patches import cv2_imshow
im1 = cv2.imread("1.jpg")
#cv2.imshow("img", im1)
cv2_imshow(im1)

Face Detection

The output from DeepFace.detectFace returns normalized cropped face. For mtcnn I got image of shape (224, 224, 3). You can verify and view the image with,

from deepface import DeepFace
import cv2
from google.colab.patches import cv2_imshow

#backends = ['opencv', 'ssd', 'dlib', 'mtcnn']
backends = ['mtcnn']
for backend in backends:
  #face detection and alignment
  detected_face = DeepFace.detectFace("1.jpg", detector_backend = backend)
  
  print(detected_face)
  print(detected_face.shape)

  im = cv2.cvtColor(detected_face * 255, cv2.COLOR_BGR2RGB)
  #cv2.imshow("image", im)
  cv2_imshow(im)

Output

[[[0.12156863 0.05882353 0.02352941]
  [0.2901961  0.18039216 0.1254902 ]
  [0.3137255  0.20392157 0.14901961]
  ...
  [0.06666667 0.01176471 0.01176471]
  [0.05882353 0.01176471 0.00784314]
  [0.03921569 0.00784314 0.00392157]]

 [[0.26666668 0.2        0.16470589]
  [0.19215687 0.08235294 0.02745098]
  [0.33333334 0.22352941 0.16862746]
  ...
  [0.03921569 0.00392157 0.00392157]
  [0.04313726 0.00784314 0.00784314]
  [0.04313726 0.         0.00392157]]

 [[0.11764706 0.05098039 0.01568628]
  [0.21176471 0.10588235 0.05882353]
  [0.44313726 0.3372549  0.27058825]
  ...
  [0.02352941 0.00392157 0.        ]
  [0.02352941 0.00392157 0.        ]
  [0.02745098 0.         0.        ]]

 ...

 [[0.24313726 0.1882353  0.13725491]
  [0.24313726 0.18431373 0.13725491]
  [0.22745098 0.16470589 0.11372549]
  ...
  [0.654902   0.69803923 0.78431374]
  [0.62352943 0.67058825 0.7529412 ]
  [0.38431373 0.4117647  0.45882353]]

 [[0.23529412 0.18039216 0.12941177]
  [0.22352941 0.16862746 0.11764706]
  [0.22745098 0.16470589 0.11764706]
  ...
  [0.6392157  0.69803923 0.78039217]
  [0.6156863  0.6745098  0.75686276]
  [0.36862746 0.40392157 0.4627451 ]]

 [[0.21568628 0.16862746 0.10980392]
  [0.2        0.15294118 0.09803922]
  [0.20784314 0.14901961 0.10196079]
  ...
  [0.6313726  0.6901961  0.77254903]
  [0.6039216  0.6627451  0.74509805]
  [0.36078432 0.39607844 0.4509804 ]]]
(224, 224, 3)

Face Embedding

Since, you are looking for embedding vector you can get it with below. It is a modified version of verify function. I kept option for two images, distance calculation, verification, but you can modify it generate face embedding for a single face only. I did not remove any unused imports.

"""
Modified verify function for face embedding generation
backends = ['opencv', 'ssd', 'dlib', 'mtcnn']
"""

from keras.preprocessing import image
import warnings
warnings.filterwarnings("ignore")
import time
import os
from os import path
from pathlib import Path
import gdown
import numpy as np
import pandas as pd
from tqdm import tqdm
import json
import cv2
from keras import backend as K
import keras
import tensorflow as tf
import pickle

from deepface import DeepFace
from deepface.basemodels import VGGFace, OpenFace, Facenet, FbDeepFace, DeepID
from deepface.extendedmodels import Age, Gender, Race, Emotion
from deepface.commons import functions, realtime, distance as dst


def FaceEmbeddingAndDistance(img1_path, img2_path = '', model_name ='Facenet', distance_metric = 'cosine', model = None, enforce_detection = True, detector_backend = 'mtcnn'):

  #--------------------------------
  #ensemble learning disabled.
  
  if model == None:
    if model_name == 'VGG-Face':
      print("Using VGG-Face model backend and", distance_metric,"distance.")
      model = VGGFace.loadModel()

    elif model_name == 'OpenFace':
      print("Using OpenFace model backend", distance_metric,"distance.")
      model = OpenFace.loadModel()

    elif model_name == 'Facenet':
      print("Using Facenet model backend", distance_metric,"distance.")
      model = Facenet.loadModel()

    elif model_name == 'DeepFace':
      print("Using FB DeepFace model backend", distance_metric,"distance.")
      model = FbDeepFace.loadModel()
    
    elif model_name == 'DeepID':
      print("Using DeepID2 model backend", distance_metric,"distance.")
      model = DeepID.loadModel()
    
    elif model_name == 'Dlib':
      print("Using Dlib ResNet model backend", distance_metric,"distance.")
      from deepface.basemodels.DlibResNet import DlibResNet #this is not a must because it is very huge.
      model = DlibResNet()

    else:
      raise ValueError("Invalid model_name passed - ", model_name)
  else: #model != None
    print("Already built model is passed")

  #------------------------------
  #face recognition models have different size of inputs
  #my environment returns (None, 224, 224, 3) but some people mentioned that they got [(None, 224, 224, 3)]. I think this is because of version issue.
    
  if model_name == 'Dlib': #this is not a regular keras model
    input_shape = (150, 150, 3)
  
  else: #keras based models
    input_shape = model.layers[0].input_shape
    
    if type(input_shape) == list:
      input_shape = input_shape[0][1:3]
    else:
      input_shape = input_shape[1:3]
    
  input_shape_x = input_shape[0]
  input_shape_y = input_shape[1]

  #------------------------------

  #tuned thresholds for model and metric pair
  threshold = functions.findThreshold(model_name, distance_metric)

  #------------------------------
  

  #----------------------
  #crop and align faces

  img1 = functions.preprocess_face(img=img1_path, target_size=(input_shape_y, input_shape_x), enforce_detection = enforce_detection, detector_backend = detector_backend)
  img2 = functions.preprocess_face(img=img2_path, target_size=(input_shape_y, input_shape_x), enforce_detection = enforce_detection, detector_backend = detector_backend)

  #----------------------
  #find embeddings

  img1_representation = model.predict(img1)[0,:]
  img2_representation = model.predict(img2)[0,:]

  print("FACE 1 Embedding:")
  print(img1_representation)

  print("FACE 2 Embedding:")
  print(img2_representation)

  #----------------------
  #find distances between embeddings

  if distance_metric == 'cosine':
    distance = dst.findCosineDistance(img1_representation, img2_representation)
  elif distance_metric == 'euclidean':
    distance = dst.findEuclideanDistance(img1_representation, img2_representation)
  elif distance_metric == 'euclidean_l2':
    distance = dst.findEuclideanDistance(dst.l2_normalize(img1_representation), dst.l2_normalize(img2_representation))
  else:
    raise ValueError("Invalid distance_metric passed - ", distance_metric)

  print("DISTANCE")
  print(distance)

  #----------------------
  #decision

  if distance <= threshold:
    identified =  "true"
  else:
    identified =  "false"

  print("IDENTIFIED")
  print(identified)

Above function is called via,

FaceEmbeddingAndDistance("1.jpg", "2.jpg", model_name='Facenet', detector_backend = 'mtcnn')

Output

FACE 1 Embedding:
[-0.7229302  -1.766835   -1.5399052   0.59634393  1.203212   -1.693247
 -0.90845925  0.5264039   2.148173   -0.9786542  -0.00369854 -1.2710322
 -1.5515596  -0.4111185  -0.36896533 -0.30051672  0.35091963  0.5073533
 -1.7270111  -0.5230838   0.3376239  -1.0811361   1.5242224  -0.6137103
 -1.3100258   0.80050004 -0.7087368  -0.64483845  1.0830203   2.6056807
 -0.76527536 -0.83047277 -0.7335422  -0.01964059 -0.86749244  2.9645889
 -2.426583   -0.11157394 -2.3535717  -0.65058017  0.30864614 -0.77746457
 -0.6233895   0.44898677  2.5578005  -0.583796    0.8406945   1.1105415
 -1.652044   -0.6351479   0.07651432 -1.0454555  -1.8752071   0.50948805
 -1.6050931  -1.1769634  -0.02965304  1.5107706   0.83292925 -0.5382068
 -1.5981512  -0.6405941   0.5521577   0.22957848  0.506649    0.24680384
 -0.91464925 -0.18441322 -0.6801975  -1.0448433   0.52288735 -0.79405725
  0.5974493  -0.40668172 -0.00640235 -0.742475    0.1928863   0.31236258
 -0.37383577 -1.5883486  -1.5336255  -0.74254227 -0.8524561  -1.4625055
 -2.718953   -0.7180952  -1.2140683  -0.5232462   1.2576898  -1.1097553
  2.3971314   0.8855096  -0.16556528 -0.07307663 -1.8778017   0.8690948
 -0.39043528 -0.5494097  -2.2382076   0.7101087   0.15859437  0.2959841
  0.8605075  -0.2040207   0.77952844  0.04542177  0.92514265 -1.988945
  0.9418363   1.6509243  -0.20324889  0.2974357   0.37681833  1.095943
  1.6308782  -1.2553837  -0.10246387 -1.4697052  -0.5832107  -0.34192032
 -1.1347024   1.5154309  -0.00527111 -1.165709   -0.7296148  -0.20767921
  1.2530949  -0.9487353 ]
FACE 2 Embedding:
[ 0.9399996   1.3996615  -1.2931366   0.6869738  -0.03219241  0.96111965
  0.7378809  -0.24804354 -0.8128112   0.19901593  0.48911542 -0.91603553
 -1.1671298   0.88576627  0.25427592  1.1395477   0.45400882 -1.4845027
 -0.90582514 -1.1371222   0.47669724  1.2933927   1.4533392  -0.46943524
  0.10245587 -1.4916894  -2.3223586  -0.10979578  1.7803721   1.0051152
 -0.09164213 -0.64848715 -1.4191641   1.811776    0.73174113  0.2582223
 -0.26430857  1.7021953  -1.0571098  -1.1215096   0.3606074   1.5136883
 -0.30045512  0.26225814 -0.19101554  1.269355    1.0674374  -0.2550623
 -1.0582973   1.7474637  -1.7739134  -0.67914337 -0.1877765   1.1581128
 -2.281225    1.3955555  -1.2690883  -0.16299461  1.337664   -0.8831901
 -0.6862674   2.0526903  -0.6325836   1.333468   -0.10851342 -0.64831966
 -1.0277263   1.4572504  -0.29905424 -0.33187118 -0.54727656  1.1528811
  0.12454037 -1.5835186  -0.2271783   1.3911225   1.0170195   0.5741334
 -1.3088373  -0.5950714  -0.6856393  -0.910367   -2.0136826  -0.73777384
  0.319223   -2.1968741   0.9673934  -0.604423   -0.08049382 -1.948634
  1.88159     0.20169139  0.7295723  -1.0224706   1.2995481  -0.3402595
  1.1711328  -0.64862376  0.42063504 -0.01502114 -0.7048841   1.4360497
 -1.2988033   0.31773448  1.534014    0.98858756  1.3450235  -0.9417385
  0.26414695 -0.01988658  0.7418235  -0.04945141 -0.44838902  1.5288658
 -1.1905407   0.13961646 -0.17101136 -0.18599203 -1.9648114   0.66071814
 -0.07431012  1.5870664   1.5989372  -0.21751085  0.78908855 -1.5576671
  0.02266342  0.20999858]
DISTANCE
0.807837575674057
IDENTIFIED
false