HAAR face detection doesnt work as expected

I'm currently working on face detection using OpenCV. I finished training by using my positive and negative images with a cascade trainer GUI. However when I run my code to test for face detection, only the first image works(only when .detectMultiScale is set to gray,1.01,7) and it doesn't work on the other images.

import cv2
import numpy as np

face_cascade = cv2.CascadeClassifier('classifier/cascade.xml')

img = cv2.imread('p/pic2.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

faces = face_cascade.detectMultiScale(gray,1.01,7)
for (x,y,w,h) in faces:
    img = cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)

cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()

All of my positive images are in gray scale and their size is (600,600). My negative image sizes are bigger but not in gray scale. There are 80 positive images and 160 negative. I'm sure if this is what's effecting the results.

Solution

First of all your both negative and positive images should be in grayscale.
Secondly, you didnt talk about how you labeled your positive images to get positive vectors.
Thirdly, your number of images(pos and neg) is not enough and difficult to get good results.(900 neg and 1800 pos is suggested as minimum)
Lastly,I assume you are using opencv_traincascade so in which stage of xml did you finish training.

My suggestion is following this tutorial by following the documentation. I used this tutorial and trained many objects before so it is good to follow