A Computer Vision System for Missing Tablets Detection
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Journal of Physics: Conference Series
PAPER • OPEN ACCESS
A Computer Vision System for Missing Tablets Detection
To cite this article: Xuhan Zhao 2021 J. Phys.: Conf. Ser. 1827 012029
View the article online for updates and enhancements.
This content was downloaded from IP address 46.4.80.155 on 22/09/2021 at 23:25ICETIS 2021 IOP Publishing
Journal of Physics: Conference Series 1827 (2021) 012029 doi:10.1088/1742-6596/1827/1/012029
A Computer Vision System for Missing Tablets Detection
Xuhan Zhao
Paul G. Allen School of Computer Science and Engineering, University of
Washington, Seattle, 98195, USA
Author’s E-mail: zhaox43@cs.washington.edu
Abstract: In this paper, we established a computer vision system for missing tablets detection
in pharmaceutical industry during the manufacturing process. The images of blister pack with
tablets in RGB color format were acquired by using a CCD industrial color camera on the
industrial assembly line in real time. In order to reduce the effect of environmental brightness,
the obtained RGB format images were transformed into HSI color format. According to the
threshold of the hue and saturation, the image segmentation was carried out. Then, the tablets
could be identified and determined. The Canny edge detection and Hough circle detection was
used to mark the locations of tablets. Based on the locations and number of circles, the location
of the missing tablets was obtained. This computer vision system could boost efficiency,
reduce the cost of labor, and guarantee the safety of drug consumers.
1. Introduction
As the quality of living is developing at a rapid pace, the pharmaceutical industry in China is in line
with the international standards. Therefore, the Chinese government and pharmaceutical industry
follow stricter and stricter rules on pharmaceutical packaging. Internationally, drug manufacturing and
quality control follow the rule of Good Manufacturing Practices for Drugs (GMP). The GMP standard
is a methodical and systematic guidance, which has become the fundamental criteria for
pharmaceutical industry and quality control. The GMP achieves quality expectation through full-scale
management and rigorous monitor. At the same time, the GMP prevents contamination, mixture, and
error of drug during production. To protect the security and rights of drug consumers, the
pharmaceutical companies have already taken positive action to decrease or eliminate pills defect.
Nowadays, most pharmaceutical companies employ specialists to check pill losses and damages.
However, checking manually comes with obvious flaw. Working for a long time exhausts visual
system, which decreases the accuracy. In addition, checking manually is intense with low efficiency.
Currently, computer vision has significant applications in many fields such as manufacturing industry,
object recognition and location, medical diagnostics, automatic recognition, sorting. For example, Z.
Chaczko et al [1] designed a low-cost solution of tablet inspection system based on machine vision by
using a dedicated sequence of operation to perform dispensing, scanning and sorting using mini
factory setup. R. Xiao and his co-workers [2] proposed a fully automatic algorithm named adaptive
geometrical vessel tracking for coronary artery identification in X-ray angiograms. Z. Liang and his
colleagues [3] proposed a new image tracking approach for high similarity drug tablets based on light
intensity reflective energy and ANN recognition. Replacing labor with computer vision techniques to
recognize contamination, mixture, and error in pharmaceutical packaging will boost the efficiency of
assembly line and improve detection accuracy and automation level.
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution
of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Published under licence by IOP Publishing Ltd 1ICETIS 2021 IOP Publishing
Journal of Physics: Conference Series 1827 (2021) 012029 doi:10.1088/1742-6596/1827/1/012029
In this study, we designed a computer vision system for missing tablets detection in pharmaceutical
industry during the manufacturing process. The images of blister pack with tablets in RGB color
format were acquired by using a CCD industrial color camera on the industrial assembly line in real
time. In order to reduce the effect of environmental brightness, the obtained RGB format images were
transformed into HSI color format. According to the threshold of the hue and saturation, the image
segmentation was carried out. Then, the tablets could be identified and determined. The Canny edge
detection and Hough circle detection was used to mark the locations of tablets. Based on the locations
and number of circles, the location of the missing tablets was obtained. This machine vision system
could be used for high-efficient drug quality control.
2. Experimental Setup
Figure 1. The location identification process of each tablet.
Fig. 1 displays the procedure of the tablets identification process. First, the images of blister pack with
tablets in RGB color format were acquired by using a CCD industrial color camera (2048×1536 pixels,
12 fps frame rate) on the industrial assembly line in real time. However, the calculated RGB values of
images were usually different, which was greatly affected by the light intensity of the shooting
location on actual industrial assembly line. In order to reduce the effect of environmental brightness,
the obtained RGB format images were transformed into HSI color format. The RGB values of the
obtained images were transformed into the HSI color format by the following equation [4, 5]:
B G
H (1)
360 B G
cos 1 (
R G R B ) (2)
R G R B G B
2
2
min R, G, B
S 1 (3)
I
2ICETIS 2021 IOP Publishing
Journal of Physics: Conference Series 1827 (2021) 012029 doi:10.1088/1742-6596/1827/1/012029
I
R G B (4)
3
where R is the value of red, G is the value of green, B is the value of blue, H is the value of hue, S is
the value of saturation, I is the value of Intensity. According to the calculated hue and saturation
values, the characteristic of tablets could be determined. The corresponding threshold of the hue and
saturation for tablets was set, according to which the image segmentation from the hue and saturation
diagram was carried out. Then, the tablets could be identified and determined. After that, the Gaussian
filter was applied to the image for the upcoming image decomposition and edge detection. Finally, the
Canny edge detection and Hough circle detection was used to mark the locations of tablets. Based on
the locations and number of circles, the location of the missing tablets was obtained. Finally, the
machine could replenish the tablets at certain locations according to the displayed results.
3. Results and discussion
Fig.2 represents the images of different blister packs obtained by the CCD industrial camera,
respectively. As shown in Fig. 2(a), the full filled blister packs have 18 tablets, which were perfectly
fitting into their corresponding positions. As for Fig. 2(b) and 2(c), one or two tables were missing,
which was mainly due to the malfunctioning of tablet dispatching machine. According to the images,
the hue and saturation values of the tablets could be obtained before image processing on these figures.
Figure 2. The images of different blister packs obtained by the CCD industrial camera.
Fig. 3 shows the image processing result of full filled blister pack based on computer vision
technology. First, the image segmentation was carried out according the threshold of hue and
saturation, as exhibited in Fig. 3(b). By image segmentation, the figure was divided into different
regions, based on which the location of each tablet could be identified. Then, the Canny edge detection
was applied to detect the edge of each tablet based on the result from image decomposition. And then,
the Hough circle detection was carried out, as distributed in Fig. 3(c). From Fig. 3(c), it is observed
that all the tablets were correctly marked. At the same time, the center coordinates of these circles
were output for the sake of checking if each cavity was filled with tablet.
3ICETIS 2021 IOP Publishing
Journal of Physics: Conference Series 1827 (2021) 012029 doi:10.1088/1742-6596/1827/1/012029
Figure 3. The image processing result of full filled blister pack.
Fig. 4 presents the image processing result of blister pack with one tablet missing. From the
obtained image, the tablet on the second row and third column was missing. Following the same
process, there was not red solid circle at the corresponding position after image decomposition, as
shown in Fig. 4(b). Then, there was not any circle mark at the 2nd row and 3rd column in the result
after edge detection and Hough circle detection. As a result, compared with the coordinates of centers
obtained from the full filled case, the location of missing tablet was displayed, which provided the
information for the following pill refill.
Figure 4. The image processing result of blister pack with one tablet missing.
To further test the accuracy of the system, the number of missing tablets was increased to two. Fig.
5 exhibits the image processing result of blister pack with two tablets missing. Following the same
process, two missing tablets were accurately detected, the location of which was output. Therefore, it
is demonstrated that the designed computer vision system has good accuracy.
4ICETIS 2021 IOP Publishing
Journal of Physics: Conference Series 1827 (2021) 012029 doi:10.1088/1742-6596/1827/1/012029
Figure 5. The image processing result of blister pack with two tablets missing.
4. Conclusion
In conclusion, a computer vision system was designed for pharmaceutical industry to reduce
contamination, mixture and error of drug during the manufacturing process. A CCD industrial color
camera was used to obtain the images of blister pack with tablets in RGB color format. Then, the
obtained RGB format images were transformed into HSI color space to reduce the effect of
environmental brightness. The hue and saturation values of the tablets was calculated, according to
which the threshold was set for the following image segmentation. To mark the locations of tablets,
the Canny edge detection and Hough circle detection were carried out, based on which the location of
the missing tablets could be obtained. This machine vision system could be used for high-efficient
drug quality control.
Acknowledgements
This study was supported by the Scientific Research and Training Program.
References
[1] Chaczko Z., Kale A. (2011) Automated tablet quality assurance and identification for hospital
pharmacies, Int. J. Electron. Telecommun., 57(2), 153–158.
[2] Xiao R., Yang J., Goyal M., Liu Y., Wang Y. (2013) Automatic vasculature identification in
coronary angiograms by adaptive geometrical tracking, Comput. Math. Method M., 2013,
796342.
[3] Liang Z., Zhou L., Liu X., Wang, X. (2014) Image tracking for the high similarity drug tablets
based on light intensity reflective energy and artificial neural network, Comput. Math.
Method M., 2014, 304685.
[4] Muhammad K., Ahmad J., Farman H., Zubair M. (2015) A novel image steganographic approach
for hiding text in color images using HSI color model, Multimedia.
[5] Chen S., Feng R., Zhang Y., Zhang, C. (2019) Aerial image matching method based on HSI hash
learning. Pattern Recogn. Lett., 117, 131-139.
5You can also read
