An enhancement algorithm for head characteristics of caged chickens detection based on cyclic consistent migration neural network (2024)

PMCID: PMC11017354

Zhenwei Yu,^⁎† Liqing Wan,^*† Khurram Yousaf,^‡ Hai Lin,^§ Ji Zhang,^*† Hongchao Jiao,^§ Geqi Yan,^§ Zhanhua Song,^* and Fuyang Tian^⁎†,1

Author information Article notes Copyright and License information PMC Disclaimer

Experimental Platform

The video data acquisition equipment used in this experiment is the poultry house inspection robot. The specific structure of the poultry house inspection robot is shown in Figure 1.

The infrared thermal imager (3), Azure Kinect DK (4), and Environmental detector (5) are fixed on the Telescopic rod (2), which is controlled by PLC, driven by servomotor, and fixed on the Inspection car (1). The Azure Kinect DK is a developer kit and PC external with a 1-megapixel advanced depth camera, 360° microphone array, 12-megapixel full HD camera, and orientation sensor for advanced computer vision and voice model development. It is less than 13 cm long and only 4 cm thick. Table 1 shows the detailed parameters of the device.

The inspection car (1) is based on SIMATIC S7-200 SMART PLC as the core and is equipped with a 16-wire Lidar, brushless DC motor, and encoder. The inspection car (1) is controlled by PLC and circulates in the chicken house under the action of LIDAR and navigation radar (9). The infrared thermal imager (3) is used to collect the thermal infrared images of chickens. Azure Kinect DK4 is used to capture RGB images of chickens. Telescopic rod (2) is used to adjust the height position of infrared thermal imager (3), Azure Kinect DK (4), environmental detector (5); The platform swing mechanism (6) and precision electric rotating platform (7) adjust the shooting angles of the infrared thermal imager (3), Azure Kinect DK (4), and environmental detector (5); The environmental detector (5) collects data on temperature, humidity, light intensity, and carbon dioxide concentration. The collected thermal infrared images, RGB images, temperature, humidity, light intensity, carbon dioxide concentration, and other data information are uploaded to server (8). Server (8) employs a data enhancement algorithm to screen the images and transmit them to the workstation.

Image Acquisition

In this study, image data were collected from the cage chicken breeding center affiliated with Shandong Agricultural University. The specific steps for image acquisition were as follows: Firstly, the AzureKinect sensor SDK on the server was configured, where the SDK included AzureKinect Viewer, AzureKinect Recorder, and AzureKinect Firmware Tool. Secondly, the navigation system was employed to establish the inspection route for the car to ensure that enough samples were collected. Then the server was connected with a USB-C cable and the lens angle was adjusted using the Azure Kinect viewer to ensure that the video content captured was complete. The frame count was set to 30 and the video resolution to 1920 × 1080p, each recording lasting 30 s, after the completion of the setting input instructions to start recording. The recordings featured light brown shell laying hens aged 18 to 25 wk within the cage door closed and within the cage door open. In order to obtain sufficient data set and satisfy sample diversity, multiple videos were recorded. The videos captured in two different states of the same cage were recorded in 1 group, and 4 groups were recorded in each cage. A total of 24 videos were recorded in six cages, of which 12 were with the coop open and 12 were with the coop closed. Finally, 21,600 images were extracted frame by frame. The dataset comprises 10,800 images with the cage door closed and an additional 10,800 images with the cage door open.

Generative Adversarial Networks

Generative Adversarial Networks (GAN) is a novel class of deep generative models that has recently gained significant attention (Saxena and Cao, 2021). Generative Adversarial Networks are machine-learning models composed of two neural networks: a Generator and a Discriminator. The fundamental concept behind GANs is to teach the network to produce lifelike data samples by placing generators and discriminators in opposition.

The generator aims to acquire the skill to generate novel samples resembling real data instances, while the discriminator's objective is to differentiate between genuine samples and generated ones. The generator takes random noise as input and produces synthetic samples. The discriminator categorizes the samples to determine whether they are real or generated. The generator and discriminator engage in iterative adversarial training, ultimately achieving the outcome wherein samples generated by the generator become indistinguishable from real ones.

The training process of GANs can be regarded as a zero-sum game process. Generators and discriminators engage in competitive training, iteratively adjusting parameters to enhance their respective capabilities. The generator's aim is to mislead the discriminator to the extent that it struggles to accurately ascertain sample authenticity. Conversely, the discriminator strives to meticulously differentiate between real and generated samples.

One of the primary strengths of GANs lies in their capacity to generate data that closely mirrors the distribution of the training data. This characteristic renders them exceptionally valuable in diverse applications, including but not limited to image generation, text generation, data augmentation, and anomaly detection. In the realm of image generation, GANs have found utility in crafting authentic images, including those of human faces, that find application in computer graphics and virtual reality domains.

Cyclic Consistent Migration Neural Network

The Cyclic Consistent Migration Neural Network (CCMNN) represents an unsupervised GANs network designed for image style conversion. By capturing the mapping relationships between different domains, it is possible to convert an image from one domain to another without the paired training data. Pix2Pix predates its emergence and aims at achieving image style conversion; however, it bears significant limitations. The primary challenge stems from the requirement that the two styles of images used for training need to be paired. In reality, acquiring a sufficient number of paired images exhibiting disparate styles yet identical content proves to be a daunting task. Consequently, this paper employs the CCMNN network model to eliminate the cage door from images featuring closed cage doors.

At the heart of CCMNN lies the concept of training through an adversarial network featuring two generators and 2 discriminators. One generator converts images from the source domain into the target domain, while the other generator transforms images from the target domain into the source domain. Two discriminators are employed to differentiate between the generated images and real images. The training process of CCMNN unfolds in 2 stages. During the initial stage, emphasis is placed on computing the adversarial loss. Through computation of the adversarial loss function, the network aims to diminish the disparity between the generated and real images. Additionally, it strives to amplify the discriminator's capacity to accurately distinguish between the two, thereby encouraging the generator to produce increasingly lifelike images.

In the second stage, focus shifts to the computation of cyclic consistency loss. Images from domain A are transformed into domain B images via the generatorA2B, and subsequently, these transformed images are reconverted to domain A images through the generatorB2A. Cyclic consistency loss is determined by evaluating the pixel-level disparity between the image reverted to domain A and the initial image within domain A. Likewise, a corresponding cyclic consistency loss is derived by conducting analogous operations on images from domain B. The cumulative cyclic consistency loss is then obtained by adding the two individual cyclic consistency losses. During the training process, constant optimization of both adversarial loss and cyclic consistency loss ensues, contributing to the enhancement of the CCMNN's capacity for image conversion and preservation of consistency. The network structure of CCMNN is shown in Figure 2. Figure 3 shows the generator in the CCMNN structure and Figure 4 shows the Discriminator in the CCMNN structure.

YOLOv8

Built upon the accomplishments of its predecessors, YOLOv8 stands as a cutting-edge model that introduces fresh features and enhancements to bolster performance and flexibility (Sharma et al., 2023). Notable innovations comprise a novel backbone network, an advanced Anchor-Free detection header, and a fresh loss function capable of operating across a range of hardware platforms, spanning CPUs to GPUs.

The Backbone and Neck segments of the backbone network supersede YOLOv8’s C3 structure with the C2f architecture, incorporating varying channel counts for diverse scale models. With the C2f structure featuring additional residual connections, facilitates a more copious gradient flow. Thus, the substitution of YOLOv8’s C3 structure with the C2f architecture equips YOLOv8 to harness a richer reservoir of gradient flow data, all while maintaining its lightweight nature.

The Head section brings forth two significant enhancements compared to YOLOv8. To start with, the architecture has transitioned from YOLOv8’s coupled Head to the contemporary mainstream Decoupled head structure (Decoupled Head), isolating the classification and detection components. Secondly, there has been a shift from Anchor-Based to an Anchor-Free paradigm. While prioritizing enhanced detection efficacy, the network structure has been streamlined.

When computing Loss, an adopted strategy is the task-aligned assigner positive sample matching, along with the incorporation of distribution focal loss (DFL). Positive samples are chosen by the Task-Aligned Assigner based on weighted scores derived from classification and regression scores, thereby training the network to prioritize dynamic attention and superior positive samples. Distribution focal loss aims to optimize the probability of the two positions nearest to the label using cross-entropy, expediting the network's ability to rapidly concentrate on the distribution of proximate regions aligned with the target location.

In the training part, the training method of YOLOX is adopted, and the Mosaic enhancement operation is turned off in the last 10 epochs, which can effectively improve the accuracy. The network structure of YOLOv8 is shown in Figure 5 below.

MSE=1mn∑i=0m−1∑j=0n−1∥I(i,j)−K(i,j)∥2

(1)

PSNR=10·log10(MAXI2MSE)

(2)

SSIM(X,Y)=l(X,Y)⋅c(X,Y)⋅s(X,Y)

(3)

c(X,Y)=2σXσY+C2σX2+σY2+C2

(4)

s(X,Y)=2σXY+C3σXσY+C3

(5)

l(X,Y)=2μXμY+C1μX2+μY2+C1

(6)

C1=(k1L)2

C2=(k2L)2

C3=C2/2

k1=0.01k2=0.03

A=TP+TNP+N

(7)

P=TPTP+FP

(8)

R=TPTP+FN

(9)

AP=1101∑R∈{0,0.01,...1}pinterp(R)

(10)

pinterp(R)=maxR′≥Rp(R′)

(11)

mAP=1k∑i=1kAPi

(12)

Table 2

Classification for different categories.

Table 3

CCMNN parameters.

Table 4

Configuration of training parameters for each group.

Table 5

YOLOv8 parameter settings.

Table 6

Training results of YOLOv8 models in each group.

Table 7

Test results of group UCI models on test sets.

Table 8

Test results of group POI models on test sets.

Table 9

Test results of group UOI models on test sets.

Analysis of Experimental Results

It can be seen from the table that compared with group UOI, the precision of the four network models YOLOv8s, YOLOv8n, YOLOv8m and YOLOv8x trained by group UCI decreased by 14.4, 14.1, 13.7, and 14.2%, and the recall rates decreased by 4, 3.4, 5.4, and 5.9%. Clearly, the precision and recall of each model in-group UCI on the validation set are markedly lower than those in-group UOI. Combined with the detection images generated by the models, it becomes evident that the primary reason is the incomplete capture of the shape and features of the comb and eyes of caged chickens by the target detection algorithm due to the presence of the cage door. The key feature points and texture information are challenging to extract and analyze using the detection algorithm. Consequently, the detection boxes of the models in-group UCI are unable to precisely align with the positions of the comb and the eyes of the chickens, leading to instances of missed and false detections. This discrepancy contributes to a lower detection performance in-group UCI compared to group UOI.

The precision and recall of each model in-group POI on the validation set are likewise inferior to those in group UOI. In comparison to group UOI, the overall precision of the four network models YOLOv8s, YOLOv8n, YOLOv8m, and YOLOv8x trained by group POI decreased by 4.2, 3, 3, and 5.6%, and the recall rates decreased by 2.1, 2.4, 2.7, and 3.7%. This indicates that while the CCMNN network eliminates the occlusion caused by the cage door on the comb and eyes of the chickens, and manages to recover the occluded area's contents to a certain degree, there may persist issues including information loss, suboptimal image quality, challenges in fully restoring intricate occlusions, and inadequacies in the post-occlusion removal processing by the CCMNN network after the occlusion is removed. These factors affect the performance of each model in-group POI, resulting in lower precision and recall rates compared to those in group UOI.

While the precision and recall rates of groups A and B are lower than those of group UOI, the precision and recall rates of each model in-group POI are notably superior to those in group UCI. In comparison to group UCI, the overall precision of the 4 network models YOLOv8s, YOLOv8n, YOLOv8m, and YOLOv8x achieved by group POI increased by 10.2, 11.1, 10.7, and 10.4%, and the recall rates increased by 1.9, 1, 2.7, and 2.2%. This is primarily due to the removal of the cage door in the image through CCMNN network processing, which restores the information of the obstructed portion covering the comb and chicken eyes, enhancing the completeness of these features. The elimination of the cage door liberates the target detection algorithm from occlusion interference, enabling it to focus more on the comb and chicken eyes. This improved visibility of key characteristics empowers the algorithm to capitalize on the restored information, resulting in more precise feature capture, and accurate comb and chicken eye identification and localization. The removal of the cage door makes the target detection algorithm no longer subject to the interference of occlusion, and focuses more on the comb and the chicken eyes, making full use of the recovered information to better capture important features, and more accurately distinguish and locate the comb and the chicken eyes.

The precision of each model in groups A, B, and C on the test set composed of images with the cage door opened after CCMNN network processing has been significantly improved compared to the precision on the test set composed of images without CCMNN network processing. The precision of group UCI has been improved by 4, 7.8, 5.6, and 1.1%. The precision of group POI has been improved by 25.6, 20.9, 25.2, and 22.5%. The precision of group UOI has been improved by 11.1, 13.1, 18.1, and 12.6%. Upon comparing the aforementioned 3 groups, it becomes evident that the CCMNN network processing has notably enhanced the detection performance of each model.

In summary, a cage-gate removal algorithm based on deep learning proposed in this paper utilizes the Cyclic Consistent Migration Neural Network (CCMNN), a network model based on consistent transfer neural networks, automatically eliminates the presence of cage doors that may affect the image's detection performance, while recovering occluded information and enhancing the overall integrity of the caged chicken's comb and eyes. As a result, the object detection algorithm is no longer hampered by occlusion, allowing it to effectively utilize the restored information for improved target identification, enhanced feature localization, and more accurate detection of the caged chicken's comb and eyes.

Compared With Existing Interference Removal Studies

In order to remove interference in images, and improve the detection effect, the researchers designed a variety of different methods. Sun et al. (2022) proposed an image enhancement approach for fish detection in complex underwater environments. This method can significantly improve detection accuracy. Ye et al. (2022) designed an end-to-end automatic pest detection framework based on a multi-scale attention-UNet (MA-UNet) model and monophasic images. Compared with the traditional model, the proposed model achieves a much better recall rate of 57.38% in detecting pest infested forest areas, while the recall rates of the Support Vector Machine (SVM), UNet, attention-UNet, and MedT models are 14.38, 49.33, 48.02, and 33.64%, respectively. The CCMNN network proposed in this paper as a cage door removal algorithm based on deep learning, achieves automatic elimination and restoration of some key information in the image, more complete outline images of caged chickens can be obtained using this algorithm, and the precision in detecting comb and eyes of caged chickens can be enhanced.

In previous studies, to mitigate the interference caused by the presence of cage doors and acquire comprehensive characteristic data of caged chickens, most researchers used the overhead view angle for detection and identification. However, the front-view camera can capture more details of caged chickens, which is more advantageous for the monitoring and identification of caged chickens. Therefore, in order to obtain better monitoring and identification effects and promote the development of monitoring technology, further research investment is essential in the monitoring and identification of frontal perspectives of caged chickens in future studies. The CCMNN network, proposed in this paper as a cage door removal algorithm based on deep learning, provides a reference for subsequent poultry researchers to deploy detection equipment, thereby ensuring that poultry detection systems can capture comprehensive front-facing image data, leading to enhanced accuracy in positioning and detection.

Discussion of Model Generalization Capability

The generalization capability of a model refers to the capability of the model to effectively adapt to unfamiliar data instances. An adequately generalizable model is characterized by its ability to effectively generalize its learned patterns and relationships to novel, previously unobserved data instances, rather than merely performing well on the training dataset. The generalization capability of a model is a pivotal metric for assessing the efficacy of a deep learning model. Hence, the examination of the generalization capability of the proposed CCMNN model is deliberated in this manuscript to gauge its viability for practical deployment. To achieve this aim, multiple validation experiments were conducted on various chicken breeds and different chicken cages. The specific test results are shown in the Figure 8.

In group A of Figure 8, the proposed CCMNN model effectively achieves precise image transformation while preserving intricate details of the original image by eliminating the cage door. Upon substituting HY-LINEVARIETYBROWN in group A with Gallus gallus domesticus in group B, the model demonstrates the capability to produce an image of the cage door opened while maintaining the intricate details of the original image without altering the cage structure. Comparing groups B and C, it is evident that the performance of the CCMNN model remains consistent regardless of the shift from horizontal cooperation to vertical cooperation within the same chicken species. Through a comparative analysis between group A and group D, it can be inferred that the conversion efficacy of the CCMNN model remains unaffected even in scenarios where there are alterations in the chicken type and chicken cage type. However, after comparing the performance of the model across different chicken breeds and various chicken coops, we conclude that the model exhibits outstanding generalization capability and broad applicability, making it effectively extendable to diverse chicken breeds and coop environments.

Limitation Analysis

In the detection of caged chickens, not only the cage door will block the characteristics of chickens, but also each chicken will block each other, because of which target detection algorithm cannot accurately identify the comb and eyes. In this study, only the influence of cage door was considered, and the mutual occlusion of chickens was not considered.

Therefore, in the subsequent study, a data enhancement algorithm was considered to expand the data set ensuring coverage of various chicken states. Improve the CCMNN algorithm, introduce more complex generator and discriminator structure, and add advanced neural network structure such as attention mechanism to better capture the subtle features in the image. Meanwhile, efforts were drawn to use the filling algorithm to estimate the content of the occluded part in order to recover it better. In addition to images, other modal information such as sound and video was also considered to be introduced. For the caged chicken scene, multi-modal information fusion can provide a richer feature representation for the model, which is conducive to the accuracy and fidelity of image conversion.

DISCLOSURES

The authors declare no conflict of interest.