Socket programming with usbcam using python 3.4 opencv 3.2

Question

I am doing a project that needs to live stream the video at the client side using USB webcam at the server side.

I am using (Opencv 3.2.0 + python 3.4.3).

The below given code works fine, I want to know how this code actually works and how it is transferring the frames to the client.

There is a delay in this code, and how to overcome that delay. I tried to set resolution but it gives error at the client side like ([Error]: total size of new array must be unchanged).when i set it to default resolution (i.e.,640*480) it works fine.

Help me 
    1. to set the resolution. -answered
    2. why dtype uint8 is used? -answered
    3. what is role fileDescriptor? -answered

updated questions:

1. In server side - I am getting a problem like ( ctrl+c needs to be pressed two times to stop the server - why is that happening ) - new
2. server is alternatively running successfully (i.e., for the first time it is not connecting, and second time it is connecting ,third time is getting stopped, fourth time running good -why is it happening ? - new
3. How to reduce delay ? - new

Lets see who will answer all these questions !!!

server:

import cv2
import time
import json
import socket
import base64
import numpy as np
from threading import Thread

SERVER_IP = "x.x.x.x"
SERVER_PORT = xxxx
MAX_NUM_CONNECTIONS = 20
DEVICE_NUMBER = 0

class ConnectionPool(Thread):

    def __init__(self, ip_, port_, conn_, device_):
        Thread.__init__(self)
        self.ip = ip_
        self.port = port_
        self.conn = conn_
        self.device = device_
        print("[+] New server socket thread started for " + self.ip + ":" +str(self.port))

    def run(self):
        try:
            while True:
                ret, frame = self.device.read()
                a = b'\r\n'
                data = frame.tostring()
                da = base64.b64encode(data)
                self.conn.sendall(da + a)

        except Exception as e:
            print("Connection lost with " + self.ip + ":" + str(self.port) +"\r\n[Error] " + str(e.message))
        self.conn.close()

if __name__ == '__main__':
    cap = cv2.VideoCapture(DEVICE_NUMBER)
    print("Waiting connections...")
    connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    connection.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    connection.bind((SERVER_IP, SERVER_PORT))
    connection.listen(MAX_NUM_CONNECTIONS)
    while True:
        (conn, (ip, port)) = connection.accept()
        thread = ConnectionPool(ip, port, conn, cap)
        thread.start()
    connection.close()
    cap.release()

client:

import cv2
import socket
import base64
import numpy as np

IP_SERVER = "x.x.x.x"
PORT_SERVER = xxxx
TIMEOUT_SOCKET = 10
SIZE_PACKAGE = 4096


IMAGE_HEIGHT = 480
IMAGE_WIDTH = 640
COLOR_PIXEL = 3  # RGB


if __name__ == '__main__':
    connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    connection.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    connection.settimeout(TIMEOUT_SOCKET)
    connection.connect((IP_SERVER, PORT_SERVER))

    while True:
        try:
            fileDescriptor = connection.makefile(mode='rb')
            result = fileDescriptor.readline()
            fileDescriptor.close()
            result = base64.b64decode(result)
            frame = np.fromstring(result, dtype=np.uint8)
            frame_matrix = np.array(frame)
            frame_matrix = np.reshape(frame_matrix, (IMAGE_HEIGHT, IMAGE_WIDTH,COLOR_PIXEL))
            cv2.imshow('Window title', frame_matrix)

            if cv2.waitKey(1) & 0xFF == ord('q'):
                break

        except Exception as e:
            print("[Error] " + str(e))

    connection.close()

Answer 1

1. to set the resolution.

You have to change it in both places, server side and client side. Most probably the error you have comes from the reshape function. This function will take the data from the image and reshape it, this means that the data will be untouched and only the size will changed... In this case it is used to change a 1D data array to a 2D matrix 3 channel matrix (however the data order is unchanged). How to change it, you have to change it first in the server part.

Where it says

ret, frame = self.device.read()

After it you can add the resize function of OpenCV:

frame = cv2.resize(frame ,(width/2, height/2), interpolation = cv2.INTER_LINEAR )

This will be half the size. Now in the client side you can reduce to:

IMAGE_HEIGHT = 480 / 2
IMAGE_WIDTH = 640 / 2

2. why dtype uint8 is used?

This comes from the image representation used by default by OpenCV (BGR), this means that each pixel color it is represented by 3 channels, Blue Green and Red. Each of this channels goes from 0-255 in value, which means that it can be represented with 8 bits, and that is why dtype uint8. Later on the function reshape will create pixels from 3 continuous values (COLOR_PIXEL value).

In this part the code reads a line of text, which is the image data and puts it in an 1D array. This will tell that each 8bit of that line is a value and should be seen as an integer.

3. what is role fileDescriptor?

This is one way to read data from a socket. See the documentation. This way you can use readline, and get a chunk of data limited by a character of new line. In you case '\r\n'. See the lines:

a = b'\r\n'
data = frame.tostring()
da = base64.b64encode(data)
self.conn.sendall(da + a)

a is the return character, which is sent after the whole image data represented as a string and encoded with base 64.