Contactless Elevator Button Control System Based on Weighted K-NN Algorithm for AI Edge Computing Environment

Sang-Yub Lee¹, In-Pyo Cho¹ and Chung-Pyo Hong^2,*

¹Korea Electronic Technology Institute, Republic of Korea

²Division of Computer Engineering, Hoseo University, Republic of Korea

E-mail: syublee@keti.re.kr; inpyo.cho@keti.re.kr; cphong@hoseo.edu

*Corresponding Author

Received 17 April 2021; Accepted 04 November 2021; Publication 07 January 2022

Abstract

In recent years, attempts have been made to create a door-opening or elevator button that operates based on gestures when entering and exiting a building. This can consider the convenience of an individual carrying luggage, and in some cases, has the advantage of preventing the spread of disease between people through contact. In this study, we propose a method for operating elevator buttons without contact. Elevators cannot utilize high-performance processors owing to production costs. Therefore, this paper introduces a prototype of a low-performance processor-based system that can be used in elevators, and then introduces a weighted K-nearest neighbors (K-NN) based user gesture learning and number matching method for application in an optimal non-contact button control method that can be used in such an environment. As a result, through the proposed method, a performance gain of 7.5% in comparison to a conventional K-NN method and a performance improvement of 9.7% compared to a radial basis function were achieved in a relatively low-performance processor-based system.

Keywords: Machine learning, magnetic sensor, weighted K-NN, K-NN, RBF, contactless, elevator control, low-power, edge computing.

1 Introduction

Gesture-based machine control methods have been extensively studied. For example, in the past, air conditioning, lighting, and audio control in cars were operated using physical buttons. However, in recent years, a method of control through human gestures has appeared as an alternative to rotary or physical buttons. For example, moving the hand toward the left can decrease the audio volume, and moving the hand toward the right can increase the volume. In addition, in recent years, there have been attempts to develop a door-opening or elevator button that operates based on gestures when entering and exiting a building. This approach can consider the convenience of an individual carrying luggage, and in some cases, has the advantage of preventing the spread of disease between people through physical contact. In this study, we propose a method for operating elevator buttons without contact. Elevators cannot utilize high-performance processors owing to the significant production costs involved. Therefore, in this paper, a prototype of a low-performance processor-based system that can be used in elevators is introduced, along with a weighted K-NN-based user gesture learning and number matching method for use as an optimal non-contact button control method. As a result, through the proposed method, a performance gain of 7.5% compared to the conventional K-nearest neighbors (K-NN) method and a performance improvement of 9.7% compared to the radial basis function (RBF) method in a relatively low-performance processor-based system were achieved. The remainder of this paper is organized as follows. Section 2 introduces previous related studies. The proposed scheme is then presented in Section 3. Section 4 provides the experimental results, and finally, Section 5 provides some concluding remarks.

2 Related Work

In this section, we describe previous methods for providing contactless human–machine interfaces based on machine-learning techniques, and we also describe low-power processors for AI-embedded platforms to implement a contactless elevator button controller.

2.1 Contactless Human–Machine Interface

Various techniques are used in elevator button control systems. Elevator button recognition technology using a neural network is an approach that determines which button the user has pressed using a camera and applying a method for detecting the region of interest from internal images. Although the use of a camera is advantageous in terms of accuracy and implementation, a pressing motion cannot be recognized when the button is blocked from the camera by numerous users present in the elevator [1]. Although a method for solving this problem has been suggested [2], if an image cannot be secured owing to the location of the camera and obstacles blocking its view, recognition cannot be achieved.

In general, a method that recognizes user motion by extending human–computer interaction (HCI) based gesture recognition technology has been developed.

HCI is the most contactable infotainment factor in the wearable device market, and the HCI technique [3] is a significant part of our daily lives [4]. Many products have adopted human gesture recognition techniques as natural human interfaces for more easily commanding machines or computers [5, 6]. The most intuitive and simple technique is a hand-gesture technique. Many gesture recognition algorithms have been developed and studied over the past few years. The proposed applications of gesture recognition include intelligent wheelchairs [7], interactive presentation systems [8], automatic user state recognition for television control systems [9], and robot-assisted living [10]. There are two main types of hand gesture recognition: vision- [11, 12] and inertial-sensor-based methods [5, 6, 13–15]. A vision-based method is used for 3D recognition in large systems, such as the image processing technology. An inertial sensor based approach is usually applied to estimate the real-time gait cycle and recognize hand gestures [16].

Hand gesture recognition using inertial sensor methods has also been studied. Some researchers have applied machine learning (ML) to gesture recognition using small sensors [17–20]. The authors of [19] proposed a method for acceleration data sequence based dynamic hand gesture recognition and employed a long short-term memory recurrent neural network.

However, in this study, we do not use a camera in an elevator, but apply a magnetic field sensor to determine whether the user operates according to the change in capacitance, and utilize a non-contact method to determine whether the button is active. In particular, a technique to improve the recognition accuracy by applying ML on a low-spec processor for application to an actual system is proposed.

2.2 Low-Power Processor Modules for Embedded AI Systems

Existing inertial sensors have been used to introduce a low-spec gesture recognition system using neuromorphic semiconductors [21]. Similarly, the conventional method of gesture recognition involves applying the axis sensor value of the inertial sensor to the neuromorphic semiconductor. In this study, however, the neuromorphic semiconductor uses the area classification weighting algorithm to recognize the magnetic field sensor value. A new method is proposed to improve the recognition accuracy in detecting which button has been pressed.

In addition, in the case of developing a system capable of sensitively accommodating tactical sensing using a magnetic field sensor [22, 23], it is difficult to estimate the user behaviors because the distribution of sensor values is not uniform and spreads widely owing to the characteristics of the magnetic field sensor. Therefore, there is a limitation in recognizing the user motions using only magnetic field sensor data, and it is necessary to increase the recognition accuracy by applying a data pre-processing technology and an ML algorithm for the preprocessed data.

As described above, conventional technologies require high-speed data processing as sensing data increase in number and the recognition accuracy decreases by determining the recognition through the threshold setting according to the defined algorithm. In this paper, we proposed a method to improve the recognition accuracy on a low-spec processor through ML, rather than simply setting a threshold using a neuromorphic processor.

The implementation of neuromorphic processors can achieve a remarkable reduction in computation time and power consumption [25], and realizing on-chip neuromorphic computing instead of CPUs and GPUs also helps apply the AI system not only to high-performance servers but also to edge devices of the Internet of Things (IoT). For applying the AI edge as an embedded system, it is essential for the computation module to be realized with restricted resources and operated under low power [26]. In this study, we used a low-power processor, called Intellino [24], with less resource usage for embedded AI systems. Intellino operates at low power and fully excludes multiplications owing to the optimized AI algorithms.

3 Proposed Elevator Button Control Scheme

In this section, the hardware configuration for the contactless elevator button control system, which consists of a main control unit (MCU) and a neural processing unit (NPU) to support the machine learning framework. The proposed weighted K-NN-based button-recognition scheme is then described.

3.1 Hardware Configuration for Machine Learning Framework

In this study, we implemented a system that detects changes in magnetic field sensors and recognizes buttons activated by users. As shown in the figure, the system module consists of a near field sensor (NFS) that recognizes the near field of a magnetic field and a field analyzing IC (FAIC) that transmits the sensor value to the main control unit (MCU) and analyzes the protocol of the transmitted value. In particular, to learn and recognize the machine learning algorithm proposed in this paper, a neural processing unit (NPU) is used, and the NPU applied at this time uses the Intellino-based parallel processing function. Figure 1 shows a conceptual diagram of the proposed hardware configuration. In addition, the principle of button recognition based on a magnetic sensor is described in Figure 2.

Figure 1 Conceptual diagram of proposed hardware configuration.

Figure 2 Principle of button recognition based on magnetic sensor.

The data sensing range of a given magnetic near-field sensor (NFS) is also shown in Table 1.

Table 1 Sensing data range of magnetic sensor

	X axis	Y axis	Z axis
Data Range	0 $\sim$ 65,535	30,000 $\sim$ 40,000	8,000 $\sim$ 12,000

3.2 Button Recognition Based on Machine Learning Method

There are two principal goals in designing the proposed scheme: The first is to provide a weighted value based on the frequency of each button usage. The second is to obtain the best-trained weighted K-NN model based on the Z-axis value. To achieve these two goals, we define a Z-axis-based weighted K-NN and an appropriate algorithm. In this section, we present the weighted K-NN model and its related scheme for training the given model.

3.2.1 Space weighted K-NN algorithm

A k-NN is a machine learning algorithm for classification or regression based on distance computation. The k-NN classifies a test dataset by comparing the distance between the test and training datasets. For supervised learning, the training dataset should be learned before classification, and each training dataset has its own category and location information (vector) for the distance calculation. Because the test dataset also has its own vector, the algorithm performs the classification by calculating the distance between each training dataset vector and test dataset vector. After taking the test dataset, the algorithm has the number of distance results, which is equal to the number of trained datasets. The algorithm takes k variables and classifies the test dataset as the major category among the nearest distance of the k-trained dataset. When the value of k is 1, the test dataset is classified as the nearest trained dataset. When the value of k is 3, the test dataset finds the three nearest trained datasets and is classified as the major category among the sorted datasets. The k-NN can prevent an under-fitting or over-fitting through the k variable and achieve a classification using only the distance calculation.

Similarly, the space-weighted k-NN algorithm proposed in this paper may be set according to the depth value, called the Z parameter. Another criterion is assigned to the feature owing its capability to divide the button recognition area into a re-mapped set of classes. In the conventional k-NN algorithm, the distance is used as a basis to weigh the contribution of each k-number of neighbors in the class assignment process and define the confidence of having data belonging to the classes.

The proposed space-weighted k-NN algorithm reduces the range of adjacent neighbors by observing the user distribution of button usage and re-establishing a button recognition area that fits the frequency of use. To find a button selected by the user within a reduced range, a Z value is assigned to a button with a high frequency of use, and the recognition rate is improved such that a button with a high frequency is selected even within the same area.

3.2.2 Button recognition process

In this study, information on X, Y, and Z (depth) can be obtained according to the change in the capacitance checked at the button to be pressed by the user, as shown in the Figure 2, and the range of the collected data, as shown in the Table 1. The collected data were compared with the number of buttons according to the preset X and Y values to check whether the corresponding button has been activated.

Because the X, Y, and Z values collected from the side of magnetic sensor have a large amount of noise, a preprocessing is conducted to remove the noise component through a Gaussian filter that can reflect the shape of the user’s finger as much as possible. After noise removal through this approach, the data indicate the button number in the corresponding area through the use of the k-NN algorithm. Unlike existing research methods, in this study, after observing a certain distribution of collected data, the number of users with a high-frequency of use are subdivided for application in the k-NN algorithm. In the subdivided region, the weight of the Z value (depth) is increased according to the usage frequency, and thus the distance measurement for recognition in the k-NN algorithm is easily applied. This method was applied to each button area and defined as a space-weighted k-NN, which is introduced in Section 3.

The traditional k-NN classification algorithm has certain limitations, including complex calculations and a performance dependent solely on the training set. To overcome these limitations, an improved version of a k-NN is proposed herein, which uses a genetic algorithm combined with a weighted k-NN to improve the classification performance, and the experiment results show that our proposed algorithm outperforms the KNN with greater accuracy [27].

Figure 3 Button recognition process based on weighted K-NN algorithm.

4 Evaluation

In this section, we present the experimental results for a comparison of the proposed scheme with other approaches. The experiment was conducted in two parts. The first is to evaluate the performance of the hand gesture algorithm, and the second is to evaluate the button recognition accuracy. For the hand gesture algorithm, we adopted convolutional neural network (CNN), radial basis function (RBF), and conventional K-NN algorithms as comparatives. For a comparison of the accuracy, an RBF, a KNN, and the proposed scheme are compared with each other.

4.1 Prototype Implementation

For the experiment, we implemented a prototype of the proposed hardware configuration. With the implemented system, we conducted the experiment and obtained some data to evaluate the efficiency of our novel design for the elevator button control system. The implemented prototype is shown in Figure 4.

Figure 4 Prototype of proposed system.

4.2 Gesture Recognition Algorithm

Through this experiment, we checked the performance of the gesture recognition algorithm without considering the hardware performance constraints. The method using a CNN was confirmed to achieve the highest performance. However, CNNs require significant computing power. Basically, relatively simple machine learning algorithms, such as an RBF or K-NN, can be applied through hardware that can be designed specifically for elevator button systems. Based on this, the RBF, conventional K-NN, and proposed scheme are compared. As a result, it was confirmed that the proposed scheme improved the performance by 9.7% compared to the RBF, which had a limitation in clearly classifying the boundary, and 7.5% compared to the conventional K-NN. Table 2 shows the experiment results in accord with different gesture recognition algorithms.

Table 2 Sensing data range of magnetic sensor

Algorithm	Max Accuracy (%)	Min MSE	Data Type	No. of Testing Samples
CNN	98.7	0.12	Image	110
RBF	83.8	2.16	Sensor data	3,000
KNN	86.0	1.83	Sensor data	3,000
Proposed	93.5	0.32	Sensor data	3,000
Algorithm

Figure 5 Button recognition accuracy: (a) RBF, (b) KNN, and (c) proposed algorithm.

4.3 Button Recognition Accuracy

We checked the elevator button recognition performance of the proposed method. In this experiment, we compared our proposed scheme with the RBF and conventional K-NN methods. All 3,000 testing samples were trained using each method, and the results were derived in the form of the mean squared error, which represents the average squared difference between the estimated and actual values. In this case, the estimated value is the inferred button number, and the actual value is the real button number. However, the proposed scheme shows that when the number of epochs is approximately 10, the MSE already converges to 10 to the zeroth power or less. By contrast, the other methods do not converge to this power even after exceeding 20 epochs. Figure 5 shows the mean squared error of the experiment.

5 Conclusion

In recent years, gesture-based machine control methods have been extensively studied. Consequently, methods of control through human gesture appear instead of a rotary or physical button method. For example, moving the hand to the left decreases the audio volume, and moving the hand to the right increases the volume. This trend is applied to various manipulations required when entering and exiting a building. For example, a door-opening button or an elevator operation button may be considered. In some cases, this contactless approach has the advantage of preventing the spread of disease between people through physical contact. In this study, we propose a method that can operate elevator buttons without such contact. Elevators cannot utilize high-performance processors owing to their significant production costs. Therefore, in this paper, a prototype low-performance processor-based system that can be used in elevators is introduced, along with a weighted K-NN-based user gesture learning and number matching method as an optimal non-contact button control method. The results of the proposed method show a performance gain of 7.5% compared to the conventional K-NN method and a performance improvement of 9.7% compared to the RBF approach.

Consequently, it is possible to construct an efficient noncontact elevator operation system using the proposed scheme.

Acknowledgement

This study was conducted as part of the Industrial Core Technology Development Project of the Ministry of Commerce, Industry, and Energy, and the Korea Industrial Technology Evaluation and Management Service (20006566: Development of embedded artificial intelligence module and system based on neuromorphic).

References

[1] Zijian Dong, Delong Zhu and Max Q. H Meng, “An Autonomous Elevator Button Recognition System Based on Convolutional Neural Networks,” in Proc. IEEE Int. Conference on Robotics and Biometrics, pp. 2533–2538, Dec. 2017.

[2] Heon-Hui Kim, Dae-Jin Kim and Kwang-Hyun Park, “Robust Elevator Button Recognition in the Presence of Partial Occlusion and Clutter by Specular Reflections,” IEEE Trans. Ind. Electron., vol. 59, no. 3, pp. 1597–1611, Mar. 2012.

[3] E. Sato, T. Yamaguchi, and F. Harashima, “Natural interface using pointing behavior for human–robot gestural interaction,” IEEE Trans. Ind. Electron., vol. 54, no. 2, pp. 1105–1112, Apr. 2007.

[4] S. Zhou et al., “2D human gesture tracking and recognition by the fusion of MEMS inertial and vision sensors,” IEEE Sensors Journal, vol. 14, no. 4, pp. 1160–1170, Apr. 2014.

[5] R. Xu, S. Zhou, and W. J. Li, “MEMS accelerometer based nonspecific user hand gesture recognition,” IEEE Sensors Journal, vol. 12, no. 5, pp. 1166–1173, May 2012.

[6] R. Xie, X. Sun, X, Xia, and J. Cao, “Matching-Based Extensible Hand Gesture Recognition,” IEEE Sensors J., vol. 15, no. 6, pp. 3475–3483, June 2015

[7] T. Lu, “A motion control method of intelligent wheelchair based on hand gesture recognition,” in Proc. 8th IEEE Conf. Ind. Electron. Appl. (ICIEA), pp. 957–962. Jun. 2013,

[8] B. Zeng, G. Wang, and X. Lin, “A hand gesture based interactive presentation system utilizing heterogeneous cameras,” Tsinghua Sci. Technol., vol. 17, no. 3, pp. 329–336, Jun. 2012.

[9] S. Lian, W. Hu, and K. Wang, “Automatic user state recognition for hand gesture based low-cost television control system,” IEEE Trans. Consum. Electron., vol. 60, no. 1, pp. 107–115, Feb. 2014.

[10] C. Zhu and W. Sheng, “Wearable sensor-based hand gesture and daily activity recognition for robot-assisted living,” IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 41, no. 3, pp. 569–573, May 2011.

[11] S. Berman and H. Stern, “Sensors for gesture recognition systems,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 3, pp. 277–290, May 2012.

[12] J. Alon, V. Athitsos, Q. Yuan, and S. Sclaroff, “A unified framework for gesture recognition and spatiotemporal gesture segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 9, pp. 1685–1699, Sep. 2009.

[13] J. K. Oh et al., “Inertial sensor based recognition of 3-D character gestures with an ensemble classifiers,” in Proc. 9th Int. Workshop Frontiers Handwriting Recognit. (IWFHR), pp. 112–117, Oct. 2004.

[14] S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character recognition using MEMS motion sensing technology,” in Proc. IEEE/ASME Int. Conf. Adv. Intell. Mechatron., pp. 1418–1423, Jul. 2008.

[15] A. Akl, C. Feng, and S. Valaee, “A novel accelerometer-based gesture recognition system,” IEEE Trans. Signal Process., vol. 59, no. 12, pp. 6197–6205, Dec. 2011.

[16] C. C. Yang, Y. L. Hsu, K. S. Shih, and J. M. Lu, “Real-Time Gait Cycle Parameter Recognition Using a Wearable Accelerometry System,” Journal of Sensors, pp. 7314-7326, Nov. 2011.

[17] T. Schlomer, B. Poppinga, N. Henze and S. Boll, “Gesture recognition with a Wii controller,” in Proc. 2th Tangible and Embedded Interaction (TEI’08), 2008, pp. 11–14.

[18] G. Costante, L. Porzi, O. Lanz, P. Valigi and E. Ricci, “Personalizing a smartwatch-based gesture interface with transfer learning,” in Proc. Signal Processing Conference (EUSIPCO), 2014, pp. 2530–2534.

[19] S. Shin and W. Sung, “Dynamic Hand Gesture Recognition for Wearable Devices with Low Complexity Recurrent Neural Networks,” in Proc. IEEE International symposium on circuits and systems (ISCAS 2016), 2016, pp. 2274–2277.

[20] G. Devineau, F. Moutarde, W. Xi and J. Yang, “Deep Learning for Hand Gesture Recognition on Skeletal Data,” in Proc. IEEE International Conferce on Automatic Face & Gesture Recognition 2018, pp. 106–113.

[21] Genuino 101. https://www.arduino.cc/en/Main/ArduinoBoard101

[22] Haiming Huang, Junhao Lin, Linyuan Wu, Bin Fang, Zhenkun Wen, and Fuchun Sunj, “Machine Learning-Based Multi-Modal Information Perception for Soft Robotic Hands,” Tsinghua Science and Technology, pp. 255–269, April, 2020.

[23] Gregory de Boer, Nicholas Raske, Hongbo Wang, Junwai Kow, Ali Alazmani, Mazdak Ghajari, Peter Culmer and Robert Hewson, “Design Optimisation of a Magnetic Field Based Soft Tactile Sensor,” IEEE Sensors, no. 9, vol. 2, pp. 1–20, 2017.

[24] Young Hyun Yoon, Dong Hyun Hwang, Jun Hyeok Yang and Seung Eun Lee, “Intellino: Processor for Embedded Artificial Intelligence,” electronics, no. 9, vol. 1169, pp. 1–12, 2020.

[25] Kang, H.J., “Short floating-point representation for convolutional neural network inference,” IEICE Electron. Express, vol. 16, pp. 1–11, 2019.

[26] Kota A., Kodai U., Yuka O., Kazutoshi H., Ryota U., Takumi K., Masayuki I., Tetsuya A., and Shinya T.Y., “Hardware/Algorithm Co-Design for Accurate Quantized Neural Networks,” IEICE Trans. Inf. Syst. E102.D, pp. 2341–2353, 2019.

[27] Gou, Jianping Du, Lan Zhang, Yuhong and Xiong, Taisong, “A New Distance-weighted k -nearest Neighbor Classifier,”. J. Inf. Comput. Sci., 9, 2011.

Biographies

Sang-Yub Lee received BS and MS degrees in electronic and electrical engineering from the Yonsei University, South Korea, in 2005. He received a PhD degree in computer and information science from Korea University, South Korea, in 2019. He is a principal researcher at the Korea Electronics Technology Institute. He was with Samsung Electro-Mechanics as a senior engineer at the Central R&D center. His research is focused on energy IT and digital twin platforms.

In-Pyo Cho received BS and MS degrees in computer science from Yonsei University, Seoul, South Korea, in 2012. He is a senior researcher at the Korea Electronics Technology Institute. He was with LG Electronics as a senior engineer at the Central R&D Center. His research focuses on energy IT and digital twin platforms.

Chung-Pyo Hong received BS and MS degrees in computer science from Yonsei University, Seoul, Korea, in 2004 and 2006, respectively. In 2012, he received a PhD in computer science from Yonsei University, Seoul, Korea. He is currently an associate professor of Computer Engineering at Hoseo University, Asan, Korea. His research interests include machine learning, explainable AI, and data science.