Fruit recognition system using color filters and histograms

Nowadays, many children and adults do not know the type or name of fruits, especially if the fruit is a rare one. In this paper, a system was developed that can recognize fruit names in real time using a camera as a visual sensor. The camera captured the image and processed using image processing. This paper proposed a method using HSL color filters, RGB histograms and shapes of fruit objects to detect and recognize fruits. The proposed method was divided into two processes, namely the training and testing processes. The training process was carried out to obtain a database of each fruit. The first process of training was object detection using an HSL color filter. The calculation of the RGB histogram was conducted on the HSL color filtered object. After that, the object's roundness was measured. Meanwhile, the testing process was done by looking for the similarity of the histogram data of the test object with the reference object by using the histogram distance equation. The similarity of the object was determined by the distance value of the histogram of the tested fruit with the referenced fruit. Similar objects would have histogram distances less than the threshold values. Tests were implemented in several types of fruit. The test results showed the system could recognize fruit names accurately. E-ISSN 2548-7779 ILKOM Jurnal Ilmiah Vol. 13, No. 2, August 2021, pp. 140-147 141 Sugandi & Mahdaliza (Fruit recognition system using color filters and histograms) presented in section 4. Conclusions and some suggestions for future improvement of the system were presented in section 5. Method In this article, an algorithm was developed to perform automatic and real time recognition of fruit names. The fruit image was detected using an HSL color filter. The detected fruit was calculated based on its RGB histogram and shape (roundness). The matching process was done by comparing the test image with the reference image. The complete steps of the algorithm procedure used in this article were described in Figure 1. A. Hue, Saturation and Luminance (HSL) Color Filters The HSL color filter was a color filter used to distinguish one part of an object's color from another. HSL color filters are widely used to distinguish objects, especially if the background conditions are changing due to the influence of light. Compared to the original Red, Green and Blue (RGB) colors, HSL colors are easier to use to distinguish one object from another [13] – [16]. The conversion of the original RGB color to HSL was formulated in equation (1) r = R 255 ; g = G 255 ; b = B 255 d = max( r, g, b) − min( r, g, b) L = max(R, G, B) +min(R, G, B) 2 (1)


Introduction
Fruits play an important role for our daily nutrition. Knowing the name of the fruit and knowing the nutrition contained in a fruit is knowledge that we should have. However, nowadays many children and adults do not know the type or name of fruits, especially if the fruit is rare. Thus, it would be very useful if there was an application that could help them recognize the names of these fruits.
To overcome the problems, several methods have been conducted by several researchers. The fruit recognition process has been carried out conventionally based on the image, one of which used the Discrete Cosine Transform method [1]. In another study, the recognition process was carried out for the classification of types of fruit maturity [2]. Meanwhile [3] used the shape of color, texture in identifying mango fruit through the Learning Vector Quantization algorithm. Zawbaa et al performed automatic fruit recognition using color and shape [4]. A classification was done using the K-Nearest Neighborhood (K-NN) and Support Vector Machine (SVM) algorithms. The use of artificial intelligence for fruit recognition was developed to recognize several types of fruit [5] - [12]. Hou et al [5], Sakib et al [6], Hussain et al [7] and Firdaus et al [8] used Convolution Neural Network (CNN) in recognizing fruit names. They managed to get above accuracy between 95% -99%. The use of deep learning has been used to further improve recognition performance [9] - [11], while the real application of fruit recognition has been implemented in a robot to harvest the recognized fruit [12]. Some of the studies above can answer the existing problems although they still leave some errors that must be corrected, one of which is the need for a system that can recognize fruit in real time as done by Kang et al [12].
To overcome the problems above and answer the existing challenges, in this study a fruit recognition system was proposed which was developed automatically and real time using a camera as a visual sensor. The method developed is based on HSL color filters, RGB histograms and fruit object shapes. The HSL filter could detect fruit objects based on Hue, Saturation and Luminance colors. The Red, Green, Blue (RGB) histogram was calculated from the results of detecting fruit objects by the HSL color filter. In addition, to determine the shape of the fruit object, the roundness factor of the fruit object was also calculated. The training process was performed on the three parameters for each training object. The recognition process was run by comparing the three parameters of the test object with the calculation results from the reference object.
To facilitate discussion, this article was structured as follows. In part 2, the theoretical basis related to the proposed method was explained. Section 3 described the method developed in this article. The test results and discussion were

Research Article
Open Access (CC-BY-SA)

Abstract
Nowadays, many children and adults do not know the type or name of fruits, especially if the fruit is a rare one. In this paper, a system was developed that can recognize fruit names in real time using a camera as a visual sensor. The camera captured the image and processed using image processing. This paper proposed a method using HSL color filters, RGB histograms and shapes of fruit objects to detect and recognize fruits. The proposed method was divided into two processes, namely the training and testing processes. The training process was carried out to obtain a database of each fruit. The first process of training was object detection using an HSL color filter. The calculation of the RGB histogram was conducted on the HSL color filtered object. After that, the object's roundness was measured. Meanwhile, the testing process was done by looking for the similarity of the histogram data of the test object with the reference object by using the histogram distance equation. The similarity of the object was determined by the distance value of the histogram of the tested fruit with the referenced fruit. Similar objects would have histogram distances less than the threshold values. Tests were implemented in several types of fruit. The test results showed the system could recognize fruit names accurately.
presented in section 4. Conclusions and some suggestions for future improvement of the system were presented in section 5.

Method
In this article, an algorithm was developed to perform automatic and real time recognition of fruit names. The fruit image was detected using an HSL color filter. The detected fruit was calculated based on its RGB histogram and shape (roundness). The matching process was done by comparing the test image with the reference image. The complete steps of the algorithm procedure used in this article were described in Figure 1.

A. Hue, Saturation and Luminance (HSL) Color Filters
The HSL color filter was a color filter used to distinguish one part of an object's color from another. HSL color filters are widely used to distinguish objects, especially if the background conditions are changing due to the influence of light. Compared to the original Red, Green and Blue (RGB) colors, HSL colors are easier to use to distinguish one object from another [13] - [16].

B. RGB Histogram
The histogram was calculated from the detected object image. Each object image was calculated using histogram based on the frequency of occurrence of intensity in an image. The recognition process was obtained by comparing the histogram of the test object with the histogram in the data base (reference). Figure 2 shows the image display along with its histogram.

C. Training Process
The training process is depicted in Figure 1 (Training Algorithm). The training process began with taking an image by the camera. The image captured by the camera had an RGB color model which was the original color of each object. To detect fruit objects from the image, an HSL color filter was used. By adjusting the value of each HSL filter with the color pattern of the detected fruit, only fruit objects that appeared in the image were detected. An example of fruit image detection results is shown in Figure 3 using the HSL value: The HSL filter interval changed depending on the fruit detected, the ambient light and the background of the object.

a. Original image
b. Image with HSL filter  The next process was to calculate the roundness of the object. The roundness was calculated from the ratio of the length of the vertical and horizontal diameters (2). (2) The calculation process was shown in Figure 5. The roundness that was close to 1 indicated that the object was round. When it was smaller or larger than 1 then the object tended to be oval or elongated.
Sugandi & Mahdaliza (Fruit recognition system using color filters and histograms)

D. Testing Process
The testing process was conducted in stages similar to the training process. In this process, the similarity of one object with another object was calculated through the RGB histogram and roundness parameters. Histogram similarity was calculated using the histogram distance equation as equation (3).
Where d is the distance between histograms. pk and qk are histograms of the test object and the reference to the kth bin. The value of d was compared with a threshold value (threshold) to ensure that a test object was similar to the reference object. This threshold value was determined experimentally.

Results and Discussion
The experiments were conducted with several types of fruit including citrus fruits, pears, dragon fruit, red apples, green apples, bananas and persimmons. There were two experimental stages, namely the training stage to store the fruit object data base and the testing stage to recognize fruit names. The first stage was the training process. In this process several pieces were detected using an HSL color filter. The histogram and roundness of the fruits were calculated and then stored as a data base for the fruits. The training data for each type of fruit was taken from 4 fruit seeds such as 4 oranges, 4 dragon fruit seeds and so on which were then the average of the histogram and roundness data was measured. The second stage was the testing stage. Similar to the first stage, all fruits to be recognized were detected by the HSL color filter. Then, the histogram and roundness were calculated and stored as test data. The testing process was run by comparing the test data for each fruit with the existing data base.
The experimental results are shown in Figures 6 to Figure 12. The HSL color filter successfully detected the fruit as indicated by the red squares in each image. The RGB histogram succeeded in recognizing the fruit name by comparing the histogram value of the fruit being tested with the histogram stored in the system. As can be seen in each image, using training data of 4 fruit seeds for each type of fruit, the developed system succeeded in recognizing the names of fruits without errors.
The developed system recognized fruit in real time. The surrounding environment was one of the factors of success or error in recognizing fruit names. One of the obstacles in the process of introducing this fruit was the changing lighting conditions. One of these conditions could be overcome by setting the same lighting for each test.      Table 1 shows the results of measuring the roundness of each type of fruit. The measurement results show that the roundness of the fruit was close to 1, indicating that the fruit was round. Meanwhile, for roundness which was far from the value of 1, it shows that it was not round. In this example, it was bananas with a roundness value of 0.50.

Conclusion
This research succeeded in developing a fruit name recognition system using HSL color filters, RGB histograms and roundness in real time using a camera as a visual sensor. The developed system succeeded in recognizing each fruit accurately. One of the obstacles in the process of introducing this fruit was the changing lighting conditions. One of these conditions could be overcome by setting the same lighting for each test. One of the challenges of real-time detection was changing environmental conditions, especially lighting. By using only one object characteristic, i.e., color, environmental changes could greatly affect the recognition results. The addition of some object features such as textures and objects could reduce recognition errors. In addition, the addition of training objects could also be used to increase the accuracy of fruit recognition. Some of these shortcomings should be solved for future system development. .