Detect Skin and Capture Image

Question

Update 1

I have an idea what inRange function does. But I don't want to apply mask and show the new image with skin color. What I want to do is to know if the image contains skin color and cover larger area.

What I want to do

I want to capture a picture whenever finger is detected inside a boundary. Its dimensions are known.

Struggling points

1. Manipulate image data in native code.
2. Detecting skin in live camera, so whenever that particular area is focused and skin is detected, snap should be taken

What I have done

I am using JNI Layer to perform the operation. I am able to get Mat from image data using this tutorial, but don't know how to manipulate poutPixels. The format is NV21 and I am not sure how to do operations on it.

I need to crop image and then detect if there's skin present in the image. I have successfully cropped the image to the desired dimension, but has no clue to move forward to detect skin. I want this method to return true or false.

Here is the code:

jbyte * pNV21FrameData = env->GetByteArrayElements(NV21FrameData, 0);
jint * poutPixels = env->GetIntArrayElements(outPixels, 0);

Mat mNV(height, width, CV_8UC3, (unsigned char*)pNV21FrameData);
Mat finalImage(height, width, CV_8UC3, (unsigned char*) poutPixels);

jfloat wScale = (float) width/screenWidth;
jfloat hScale = (float) height/screenHeight;

float temp = rectX * wScale;
int x = (int) temp;
temp = rectY * hScale;
int y = (int) temp;

int cW = (int) (width * wScale);
int cH =  (int) (height * hScale);

cH = cH/2;


Rect regionToCrop(x, y, cW, cH);
mNV = mNV(regionToCrop);
finalImage = finalImage(regionToCrop);

//detect skin and return true or false

I have read about inRange function, but I don't know how to check whether there's skin or not.

Questions

1. Am I on the right path to proceed further?
2. The image format I am getting is NV21. Is it a 8UC1 or it can be 8UC3 too?
3. How to proceed from here to start detecting skin?

Any help is appreciated.

Answer 1

I have solved my problem by extracting skin color range and making all pixels equal to zero. Below are the steps.

1. Convert the image to HSV

First convert image to HSV.

Mat mHsv = new Mat(rows, cols, CvType.CV_8UC3);
Imgproc.cvtColor(mRgba, mHsv, Imgproc.COLOR_RGB2HSV);

2. Get range of skin color

Skin color range may vary, but this one is working fine for me.

Mat output = new Mat();
Core.inRange(mHsv, new Scalar(0, 0.18*255, 0), new Scalar(25, 0.68*255, 255), output);

3. Extract this Skin Range channel

Now extract this channel while making skin pixels equal to zero

Mat mExtracted = new Mat();
Core.extractChannel(output, mExtracted, 0);

Now you have mExtracted matrix, in which skin colored pixels are 0 and rests are 255 (or skin color, I am not sure).

4. Get count of zeros

Since 0 now is actually skin color area, what you can do is to define a threshold which suits your need. According to my need, I want skin to cover more than half of the area, so I made my logic accordingly.

int n = Core.countNonZero(mExtracted);
int check = (mExtracted.rows() * mExtracted.cols())/2;

if(n >= check && isFocused) {
    //Take picture
}