The Implementation of Ocular Health Service System Using Android Platform

Woongsik Kim

Dept. Medical IT Engineering, Konyang University, #158, Daejeon, South Korea
E-mail: wskim@konyang.ac.kr

Received 24 February 2022; Accepted 19 April 2022; Publication 10 August 2022

Abstract

As the life expectancy of human increases, having a long and healthy life, Well-Aging, Wellness, and Anti-Aging become more important. There is a paradigm shift from diagnosis and treatment in the healthcare field to prognosis and prevention in daily life. The human part with the most capillary blood vessels is the inside of human eyes or the fundus oculi. These capillary blood vessels show characteristic changes prior to chronic diseases such as diabetes or hypertension. In this study, a system is being developed to regularly collect data from the user, convert them into a database, and analyze to inform and warn any characteristic changes to users as they occur, such that users can proactively take care of their own eyes.

Keywords: Ocular health, u-healthcare, measurement system, android Platform, service system.

1 Introduction

In developing countries, medical facilities are poor, so there are few opportunities to receive professional treatment. Therefore, 90% of blindness occurs in developing countries. 80% of these blindness can be prevented or treated if they are treated [1]. This study is to predict circulation disorders by photographing fundus novels with smartphones for telemedicine and observing changes in blood vessels. First, a camera adaptor, including a reflective refraction module, has been developed.

Figure 1 System configuration.

Figure 1 shows the overall system configuration. It can be installed on a smartphone to align the light source axis of the smartphone and camera at a certain distance (about 3 cm or less) for more precise and larger images. Second, an automatic focusing app to achieve a fundus oculi images have been developed and tested. All components have been developed for Android Studio. Usage of Android camera includes the Intent (using the camera provided by Android) and Surface view (customizing with previews). The easier method has been selected after several tests. The app performs the fundus oculi photographing, the transfer of images to the server, and request and reception of measurement data. Third, fundus oculi chronic disease determination algorithm is to be developed to determine such diseases as glaucoma, diabetic retinopathy or macular degeneration using the fundus oculi images and the latest AI algorithm deep learning [13]. In addition, a system for classifying ratings according to severity using Fast R-CNN and Random Forest techniques is being studied [14] and a method for classifying them as ensemble techniques using fundus images [15].

2 Methods

2.1 Proposed Process

The main process of the ocular health service system is to prepare modules, take photos of the app and save the server of the inside picture taken, prepare opinions of the doctor about the photos taken on the web page, and confirm the opinions of the physician created by the app as shown below Figure 2.

Figure 2 Service process.

When an application starts filming, the camera app inside the smartphone is launched, and the lens part of the camera module is closely attached to the eye before filming is carried out. After shooting, pictures are saved in the internal memory of the smartphone and uploaded to the server through the save button.

Figure 3 Ocular shoot.

The doctor logs in to the server and checks the list of photos taken by the patient on the patient care menu. Select one picture from the Picture list to see the data that predicted lesions through Deep Learning. The doctor only needs to look at the photos of abnormalities through Deep Learning and write a doctor’s comment on the photos.

2.2 Development of Shooting Module

A 3D printer was used to develop a camera module prototype to create our camera module. The 3D printer is suited for research on hardware that is used for small products. The module can be divided into the main part and lens part, and the lens part has a switch to turn on and off the pre-light for filming. The camera’s performance is capable of supporting rear OIS and has 12 million pixels. It also uses dual-pixel phase difference detection AF, with a sensor size of 1/2.55 inches, aperture value of F/1.7, and sensor ratio of 4:3.

Figure 4 (1) Camera module assembly (left-main module, right-lens module). (2) Camera module assembly view. (3) Camera module fitted.

2.3 Development of App Program

Users (patients) can use the app on a smart phone to easily take Ocular pictures. Users can log in with their own ID and password. Photography uses a built-in camera on a smart phone. The pictures taken can be found inside the app and in the photo library. When checking a picture in an app, the status of the eye care taken through deep learning can be determined and stored. These results do not represent accurate diagnosis as data to help with comment and care. The eye health service app is used by the user (patient). Users capture and store their eyes by sending them to their mobile phones and servers. The saved photo can be checked by the administrator and commented on the health of the eyes, which can be checked by the user on the app. Membership registration is only available to general patients. The administrator registers the members directly with the database for medical license verification and medical verification. Through a database linked to the Web, doctors check the results of the retina of patients with a high risk that is read through the AI algorithm and inform the patient of the risk.

Figure 5 App launch screen.

2.4 Development of Web Program

I have developed a Web program that allows you to connect to your PC in an internet environment. The Web is used by the user (patient) or administrator (pharmacist), and the expert identifies the user’s eye image assigned to him or her and makes comments about that image. Comments are written mainly on health conditions that appear in images. Users determine whether or not to receive care by checking images taken and transmitted by the app and referring to expert comments written about the images. It is also implemented to enable communication with other users. Membership registration is only available to general patients. The administrator registers the members directly with the database for medical license verification and medical verification. Doctors can check the results of the retina of patients with a high risk, through a database linked to the Web. The results have been analysis by the AI algorithm. Although the initial implementation of the retinopathy reading algorithm through deep learning was carried out directly, the performance of the currently prepared equipment and equipment takes a long time to learn, and the project is difficult to progress due to the need to continue. Therefore, in this study, we will perform learning using the Exception-v3 learning model distributed by Google first, to obtain a learning model, apply it in the project, and develop and apply the model directly in the future.

Figure 6 System configuration diagram.

We set our system configuration diagram as shown in Figure 6. Eye health service, which implements Python-produced learning models with JAVA Spring.Predict.py, Data Control for forecasting and learning at Python to import them into web servers.py, implement Fundus Learn.py and save as a Python application that can run through Application.py. Then, the read Python Exe is implemented in JAVA to convert the image read into the Image Loader into a file that can be read by the Python application into TSV maker.

2.5 Deep Learning Algorithm

The Deep Learning algorithm is used by retaining the Exception v3 in Google. For the data set used for learning, use the data set of the Kinetic Retinopathy Detection Component of the Kaggle. There are total of 35,124 images on the left and right sides, 9,314 images of diabetic retinopathy and 25,810 images on the right. Of these, 90% are used for learning, and 10% for testing is used for random images. When learning with Exception v3, the Tensorflow library allows you to obtain a text file containing the learned model file, protocol buffer(PB) and label. When implementing an Android app, deep learning is possible using these files and the Tensorflow Interface library, using models already learned in the app. Anaconda was installed to use Python in the Windows environment, and Jupyter notebook has been used to implement. A total of 35,126 ophthalmological-Retinopathy-Detection Competitions in Kaggle were used, and the study was conducted with 25,810 normal images and 9,314. Figure 7 is representation of ocular sample.

Figure 7 Ocular sample.

Based on the CNN(Convolution Neural Network) use the softmax and sigmoid functions of the Tensorflow library and learn by adjusting the image to size 8x6 to improve learning speed. In this study, the Deep Learning Model Exception v3 used by Google is used. Figure 8 shows the structure of the model. Use a small unit of Conv (Convolution Layer). Deep learning works better when the network is deep, and the layer is wider. However, it is difficult to learn because problems such as Overfeiting and Vanishing occur when actual implementation occurs. To solve this problem, the overall network age was reduced while detailed matrix computations were processed to be as dense as possible.

Figure 8 Structure of Inception v3.

The first part is the standard Convolution Layer, and then the basic Inception Layer is used. The basic Exception Layer used here is the Exception Module defined from v1 to the Exception Module, which can be raised to a smaller extent and used as a combination of Inception-1 $\sim$ 3 modules in Figure 9.

Figure 9 Basic Inception-v3 structure.

Perform re-training on the ocular image data set in Exception v3. When training is completed, the protocol buffer file and the Labels text file are displayed in the output, including the configuration of the model, such as setting the light or bias value based on the learning result. Use the Tensorflow Interface Library to implant it into the Android App.

Figure 10 Configuration diagram for applying Exception v3.

Figure 11 Test result.

Figure 12 Left – Normal Ocular Picture, right – Ocular picture of abnormal judgment.

3 Results and Discussion

Based on the current findings, there was a huge difference between the diabetic core and the Normal core (0.2 or higher) in the case of the diabetic image, and the normal score in the normal image was higher due to the small difference (0.2 or less). Based on this, it will be possible to categorize and read Diabetic and Normal.

When the lesion is predicted through Deep Learning, the module determine whether it is normal or not. It bordered green line on the image. If there is no issue on the provided picture. Otherwise, it shows a red border on the image. The physician can comment on regards to the result using a commenting button.

4 Conclusion

In this study, an ophthalmic health service system capable of telemedicine was implemented to help treatments in countries or regions with poor medical facilities such as developing countries. The camera module made using a 3D printer and a smartphone were fused to obtain retinal data. Apps and web programs were developed and implemented so that anyone could easily access them. In addition, rapid response is possible by reading retinal data through AI algorithms. The establishment of a new system is expected to enable smooth treatment in countries or regions where treatment is difficult. Modern medicine has been shifting from disease treatments to preventions. Through this study and development, we hope to be able to respond in a timely manner to changes in future medical trends. The basic concept of this study is a new system that can measure eye health anytime and anywhere in real-time. Besides, the prevention of eye diseases can be implemented through the fusion of mobile devices and smartphones. A smartphone is used to notify any emergency such that it can be treated in time before it deteriorates further and to contribute to a healthy life.

References

[1] World Health Organization. 2003. Up to 45 million blind people globally – and growing. https://www.who.int/news/item/09-10-2003-up-to-45-million-blind-people-globally---and-growing

[2] Caroline Schmidt-Lucke, Reduced number of circulating endothelial progenitor cells predicts future cardiovascular events: proof of concept for the clinical importance of endogenous vascular repair (2005).

[3] Tailoi Chan-Ling, Desmin ensheathment ratio as an indicator of vessel stability: evidence in normal development and in retinopathy of prematurity (2004).

[4] J. J. Kim, Role of FOXO1A in the regulation of insulin-like growth factor-binding protein-1 in human endometrial cells: interaction with progesterone receptor (2005).

[5] Mirela Anghelina, Monocytes/macrophages cooperate with progenitor cells during neovascularization and tissue repair: conversion of cell columns into fibrovascular bundles (2006).

[6] Lori Jerome, Recombinant human insulin-like growth factor binding protein 3 inhibits growth of human epidermal growth factor receptor-2-overexpressing breast tumors and potentiates herceptin activity in vivo (2006).

[7] Budenz Donald L., Blindness and Visual Impairment in an Urban West African Population: The Tema Eye Survey (2012).

[8] Elisa Spano, Low-Power Wearable ECG Monitoring System for Multiple-Patient Remote Monitoring (2016).

[9] Kligfield Paul, Recommendations for the Standardization and Interpretation of the Electrocardiogram (2007).

[10] Nitish V. Thakor, Ground-Free ECG Recording with Two Electrodes (1980).

[11] Joshi Sanjay, Low-power low-noise analog signal conditioning chip with on-chip drivers for healthcare applications (2012).

[12] Jap Budi Thomas, Comparing combinations of EEG activity in train drivers during monotonous driving (2010).

[13] R. Raman, Fundus photograph-based deep learning algorithms in detecting diabetic retinopathy (2019).

[14] Jung, Y.H., CNN based Diabetic Retinopathy Feature Extraction and Grade Classification (2019).

[15] Cho, H.S., Deep learning model for glaucoma diagnosis and its stages classification based on fundus images (2019).

Biography

Woongsik Kim was team Leader of Korea Information and Communications, Development; Director of CECROP Electronic Communication Business Division; Senior Researcher of Korea Software Promotion Agency and Professor, Department of Medical Artificial Intelligence, Konyang University.