I'm working on facial recognition software as a hobby to learn more about how to use OpenCV.
My software includes a function to detect and generate 20 dataSet images and saves it with an ID of the user I have assigned in the database, it then trains it and adds it to my ".trainedData.yml" file which I use to recognize the faces.
I have a SQL database where I store profiles of the people and use my webcam to capture videos.
When I run the current software, in every frame, I check the confidence level, if it's good enough I look up in the database to return the information of the profile found. Following that I call a function to insert into the database data so I can keep track of when and where the face was detected.
The issue: This insert query is being processed every frame creating a lot of inserts and sometimes even failing when multiples are being done at the same time.
Is there a way to make this more efficient without having to insert data every frame and perhaps do it every 5 or 10 seconds?
Please find my code below
Thank you in advance,
import cv2
from dbconnect import mySQL
import geocoder
faceDetect=cv2.CascadeClassifier('Classifiers\haarcascade_frontalface_alt.xml')
cam = cv2.VideoCapture(0)
rec = cv2.face.LBPHFaceRecognizer_create()
rec.read(r'trainner\trainningData.yml')
def getProfile(Id):
mySQL.execute("SELECT * FROM people where id ="+ str(Id))
cursor = mySQL.fetchall()
print(cursor)
profile = None
for row in cursor:
profile = row
#mySQL.close()
return profile
def insertProfile(Id):
try:
query = "insert into memberLocation values (null,"+ str(Id) +", now(), 'LOCATION DETECTED')"
print(query)
mySQL.execute(query)
print("Entry inserted successfuly")
mySQL.commit()
except:
print("Failed to insert user " + str(Id))
def draw_border(frame, pt1, pt2, color, thickness, r, d):
x1,y1 = pt1
x2,y2 = pt2
# Top left
cv2.line(frame, (x1 + r, y1), (x1 + r + d, y1), color, thickness)
cv2.line(frame, (x1, y1 + r), (x1, y1 + r + d), color, thickness)
cv2.ellipse(frame, (x1 + r, y1 + r), (r, r), 180, 0, 90, color, thickness)
# Top right
cv2.line(frame, (x2 - r, y1), (x2 - r - d, y1), color, thickness)
cv2.line(frame, (x2, y1 + r), (x2, y1 + r + d), color, thickness)
cv2.ellipse(frame, (x2 - r, y1 + r), (r, r), 270, 0, 90, color, thickness)
# Bottom left
cv2.line(frame, (x1 + r, y2), (x1 + r + d, y2), color, thickness)
cv2.line(frame, (x1, y2 - r), (x1, y2 - r - d), color, thickness)
cv2.ellipse(frame, (x1 + r, y2 - r), (r, r), 90, 0, 90, color, thickness)
# Bottom right
cv2.line(frame, (x2 - r, y2), (x2 - r - d, y2), color, thickness)
cv2.line(frame, (x2, y2 - r), (x2, y2 - r - d), color, thickness)
cv2.ellipse(frame, (x2 - r, y2 - r), (r, r), 0, 0, 90, color, thickness)
video_capture = cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_SIMPLEX
while True:
# try to get coordinates
# g = geocoder.ip('me') # coordinates are way off correct but pulling internet service location not current location
# print(g.latlng)
# Capture frame-by-frame
ret, frame = video_capture.read()
if ret==False:
continue
frame = cv2.flip(frame, 1) # Flip image
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceDetect.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags=cv2.CASCADE_SCALE_IMAGE
)
for (x, y, w, h) in faces:
draw_border(frame, (x, y), (x + w, y + h), (255, 0, 105),4, 15, 10)
#cv2.rectangle(frame, (x,y), (x+w, y+h), (0,0,255), 2)
roi_gray = gray[y:y+h, x:x+w]
roi_color = frame[y:y+h, x:x+w]
nbr_predicted, conf = rec.predict(gray[y:y+h, x:x+w])
if conf < 70:
profile=getProfile(nbr_predicted)
if profile != None:
cv2.putText(frame, "Confidence Level: "+str(round(conf,2))+ "%", (x, y+h+30), font, 0.4, (255, 0, 105), 1)
cv2.putText(frame, "Name: "+str(profile[1]), (x, y+h+50), font, 0.4, (255, 0, 105), 1)
cv2.putText(frame, "Age: " + str(profile[2]), (x, y + h + 70), font, 0.4, (255, 0, 105), 1)
cv2.putText(frame, "Gender: " + str(profile[3]), (x, y + h + 90), font, 0.4, (255, 0, 105), 1)
#call function to insert into database
insertProfile(nbr_predicted)
else:
cv2.putText(frame, "Confidence Level: "+str(round(conf,2))+ "%", (x, y+h+30), font, 0.4, (0, 0, 255), 1)
cv2.putText(frame, "Name: Unknown", (x, y + h + 50), font, 0.4, (0, 0, 255), 1)
cv2.putText(frame, "Age: Unknown", (x, y + h + 70), font, 0.4, (0, 0, 255), 1)
cv2.putText(frame, "Gender: Unknown", (x, y + h + 90), font, 0.4, (0, 0, 255), 1)
# Display the resulting frame
cv2.imshow('frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
video_capture.release()
cv2.destroyAllWindows()
You can make a list where to collect the nbr_predicted instances, add a counter for each succesful recognition and a parameter (each_N_passes); then do batch-inserts at each N recognitions:
nbr_predicted_list = []
passes = 0
batchStep = 10
while True:
...
if conf < 70:
#Instead of this:
#call function to insert into database
#insertProfile(nbr_predicted)
#DO:
passes++
nbr_predicted_list.append(nbr_predicted)
if passes % batchPasses == 0:
for pr in nbr_predicted_list:
insertProfile(pr)
nbr_predicted_list.clear()
Also, try to increase the delay time: if cv2.waitKey(1) & 0xFF == ord('q'):, unless maximum framerate is critical.
Change it to say:
cv2.waitKey(delay)
Where:
delay = 1000/frameRate # ms
(The actual framerate would be lower due to the recognition time etc.)
...
There could be reduction of the number of transactions also if you can check repetitive profiles and do not insert them twice in a batch (if that's desirable), also if you cache the profiles once read with an Id in getProfile (if they are constant during a run).
If they are cached, in the next call take the Id from the dictionary and return the profile without an SQL transaction.
If speed is needed, try also to remove the excessive print calls (or make them with the shortest possible output and less new lines/terminal scroll), logging may be faster.
A more sophisticated solution may use a thread or multiprocessing. Also one way to prevent race conditions for the inserts may be to add short cv.waitKey or sleep for 1 millisecond or whatever between each call to the DB; it's better in a thread though.
Note that if you delay the SQL access too much and collect too many candidate inserts (if say using max 30 fps, delay 33, and 10 seconds, if each frame has a recognition: 300 items), and if you push them in the main thread at once, that may be blocking and it may "throttle" the video framerate during these transfers.
...
Another general idea could be the function for insertion to the DB to receive not just one, but many collected data items and then one call to that function to insert many items at once: possibly the SQL command itself and the database scheme can be such that one INSERT to add many rows, if that may solve a bottleneck with too many simple single transactions.
For example, many data items could be added at once in an intermediate table as varchar rows with items encoded in CSV format etc. as strings. Then some more sophisticated SQL command or just another script in another proces or thread can watch that table, sample the new rows (SELECT * where time ... after some lastly sampled current time or some id etc.), parse them and then insert the parsed items into another "final" table with the default "uncompressed" structure, while adding sleep etc. if needed.