Image Recognition of PCB Components for Manufacturing Processes

The first step of this project involved performing object masking to isolate key features of the motherboard to ensure that they can be recognized, strengthening the model’s recognition capability. To achieve this process, the object was read into Python using OpenCV and manipulated to extract the motherboard image from the rest of the image. 

Edge detection is a common method used to determine the important lines that highlight the image. The specific methods of edge detection include corner and contour detection. This detection method focused on the brightness of pixels as a scale of detecting the points of interest. Pixels with the same brightness are analyzed together to determine parts of an image. 

One of the edge detection methods is corner detection, focused on determining the corners of the object in the image and extracting the image by inferring line coincidence. OpenCV provides two methods of corner detection: Canny and Harris. Additionally, contour detection is applied to extract the lines of an image such that the outline of these objects is enclosed to isolate key features. The regions with high intensity are highlighted and filtered for the largest contours, thus extracting the motherboard image.

To enhance the extracted image, the hierarchy must be adjusted to include additional child contours, ensuring that previously ignored components are accounted for. Additionally, applying further preprocessing techniques, such as Gaussian blur, can help reduce image noise, thereby improving accuracy. This eliminates false contours, such as shadows from the I/O ports that were included in the extracted image.

To train the model, the directory structure was required to be configured. This involved organizing the image dataset and setting up the program to initiate training. A Google Drive folder was created to store the training, testing, and evaluation data, along with the data.yaml file. The code environment was set up using Google Colab. However, due to Colab's inactivity timeout, the Ultralytics package had to be reinstalled for each run. The Google Drive was then mounted using the google.colab package. The YOLOv8 nano model, named yolov8n, was imported from the Ultralytics package. Finally, the model was trained with the following hyperparameters: 100 epochs, an image size of 1100px, and a batch size of 4. Due to the power requirements of this project, the training of this model used Google Colab eGPU's, totaling to 10 hours of runtime to successfully train the model.

This project was intended to develop an understanding of object masking and training a computer vision model to recognize and ensure the correct placement of PCB components. Object masking was performed on an image of a motherboard using the OpenCV package with Spyder IDE to program the required operations. The motherboard was successfully extracted from the background using contour detection, a form of edge detection. Furthermore, a computer vision model, YOLOv8 nano, was trained using the Google Colab platform. The model was provided with folders representing training and validation folders containing image data of components for the model to train with. The model was developed using the following hyperparameters: 100 epochs, 1100px image size, and a batch size of 4. An evaluation folder was provided to measure the performance of the model in recognizing the installed components on a Raspberry Pi, Arduino Uno, and Arduino MEGA2560. 

Although the model recognized majority of the components, it experienced some failure in recognizing components and labeling components according to their expected label. With further data augmentation, the model's prediction ability can be drastically improved.  However due to runtime limitations, the model was not retrained.