Create a Live Streaming Video Chat App without voice using cv2 module of Python

Create Socket:

try    
    skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    print("Socket successfully created")
except socket.error as err:
    print("Socket creation failed with error {}".format(err)):

Server-Side Code (server.py):

import cv2
import socket
import pickle
import struct
# Server socket setup
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host_name = socket.gethostname()
host_ip = socket.gethostbyname(host_name)
print('HOST IP:', host_ip)
port = 9999
socket_address = (host_ip, port)
server_socket.bind(socket_address)
server_socket.listen(5)
print("Waiting for a connection...")
client_socket, client_addr = server_socket.accept()
print("CONNECTED!")
# Open the camera
cap = cv2.VideoCapture(0)
while True:
    # Read a frame from the camera
    ret, frame = cap.read()
    # Serialize the frame using pickle
    data = pickle.dumps(frame)
    # Pack the serialized data and send it to the client
    message_size = struct.pack("L", len(data))
    client_socket.sendall(message_size + data)
    # Display the frame (optional)
    cv2.imshow('Server', frame)
    cv2.waitKey(1)
# Release the camera and close the connection
cap.release()
cv2.destroyAllWindows()
client_socket.close()
server_socket.close()

1. We import the necessary libraries: cv2 for OpenCV, socket for networking, pickle for serialization, and struct for handling packed data.
2. Set up the server socket using socket. socket() to create a new TCP socket.
3. Get the host IP using socket. gethostname() and bind the server socket to the IP and a specified port number (here, it's 9999).
4. Start listening for incoming connections using server_socket.listen().
5. When a client connects, we accept the connection using server_socket.accept(), which returns the client socket and address.
6. Initialize the camera using cv2.VideoCapture(0) (assuming the first camera device is used).
7. In a loop, we capture frames from the camera using cap. read().
8. Serialize the frame into bytes using pickle.dumps(frame), converting it to a format suitable for transmission.
9. Use struct.pack("L", len(data)) to pack the length of the serialized data into a fixed-size byte format (4 bytes in this case).
10. Send the packed message size and the serialized frame data to the client using client_socket.sendall(message_size + data).
11. Optionally, display the frames on the server side using cv2.imshow() and cv2.waitKey(1) (which waits for 1ms to handle GUI events).
12. The loop continues indefinitely until the connection is closed or interrupted.

Client-Side Code (client.py):

import cv2
import socket
import pickle
import struct
# Client socket setup
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host_ip = '192.168.118.67' # Replace with the server IP address
port = 9999
client_socket.connect((host_ip, port))
data = b""
payload_size = struct.calcsize("L")
while True:
    while len(data) < payload_size:
        packet = client_socket.recv(4 * 1024)
        if not packet:
            break
        data += packet
    if not data:
        break
    packed_msg_size = data[:payload_size]
    data = data[payload_size:]
    msg_size = struct.unpack("L", packed_msg_size)[0]
    while len(data) < msg_size:
        data += client_socket.recv(4 * 1024)
    frame_data = data[:msg_size]
    data = data[msg_size:]
    # Deserialize the frame using pickle
    frame = pickle.loads(frame_data)
    # Display the received frame
    cv2.imshow('Client', frame)
    cv2.waitKey(1)
# Close the connection and destroy any open windows
cv2.destroyAllWindows()
client_socket.close()

1. Similar to the server side, we import the required libraries: cv2, socket, pickle, and struct.
2. Set up the client socket using socket. socket() to create a new TCP socket.
3. Connect the client socket to the server IP and port number specified in the host_ip and port variables.
4. We initialize variable data and payload_size to handle the received data.
5. In a loop, we receive data from the server in chunks using client_socket.rev (4 * 1024) and reconstruct the complete message until the entire frame is received.
6. We use struct. Unpack("L", packed_msg_size)[0] to unpack the packed message size and obtain the actual message size.
7. We then extract the frame data from the received data using slicing (data[:msg_size]).
8. We deserialize the received frame using a pickle. loads(frame_data) to convert it back to a NumPy array.
9. We display the received frame on the client side using cv2.imshow() and cv2.waitKey(1) to handle GUI events.
10. The loop continues until there is no more data to receive.

Conclusion:

The code above demonstrates a simple video streaming application where the server captures video from the camera and sends it to the client using OpenCV, socket, pickle, and struct. The client receives and displays the video frames. However, this is just a basic example without handling advanced features like signalling, multiple clients, and audio support, which are essential for a complete video chat application. For a more comprehensive solution, consider using specialized real-time communication libraries like WebRTC or Socket.IO.

Thank you.