An Automation Script Generation Technique for the Smart Home
DOI:
https://doi.org/10.13052/jwe1540-9589.2222Keywords:
Home automation, automation script generation, first-order logic, natural language processingAbstract
A home automation system means monitoring and controlling various kinds of devices in the home remotely using the Internet of things (IoT). Technologies such as natural language processing techniques, user-friendly visual programming, and machine intelligence programming are already available for home automation. For such systems, the increase in the number of devices often makes users focused on the system’s ability to perform complex or composing tasks. However, some existing natural language processing systems can only perform simple tasks and cannot meet users’ needs. Thus, it is difficult for users to develop the home automation systems they need using visual programming systems because of the large amount of programming knowledge required. Meanwhile, automatic programming without user action can only write a few lines of code and implement little functionality. There are relatively few tools available for generating home automation scripting languages. To address this problem, we propose a practical method for generating executable home automation scripts using Chinese texts. Our method includes the following steps: it extracts critical information from the command sentences in Chinese; it uses first-order logic to check the validity of the extracted information; based on the validation, the correct sentences are mapped into the intermediate language scripts, which can interface with different home platforms. We conducted experiments on Home Assistant, converted intermediate scripts to Home Assistant, and collected 600 scenario descriptions. The experimental results show that the method can automatically generate executable scripts for the Home Assistant platform, and the correct rate was 93.66%.
Downloads
References
A. Cyril Jose and R. Malekian, “Smart Home Automation Security: A Literature Review,” Smart Comput. Rev., no. Rtdm, pp. 6–10, 2015, doi: 10.6029/smartcr.2015.04.004.
C. Paul, A. Ganesh, C. Sunitha, “An overview of IoT based smart homes,” Proc. 2nd Int. Conf. Inven. Syst. Control. ICISC 2018, no. Icisc, pp. 43–46, 2018, doi: 10.1109/ICISC.2018.8398858.
M. Shahjalal, M. K. Hasan, M. M. Islam, M. M. Alam, M. F. Ahmed, Y. M. Jang, “An Overview of AI-Enabled Remote Smart- Home Monitoring System Using LoRa,” 2020 Int. Conf. Artif. Intell. Inf. Commun. ICAIIC 2020, 2020, pp. 510–513, doi: 10.1109/ICAIIC48513.2020.9065199.
A. Alhammadi, A. Alzaabi, B. Almarzooqi, S. Alneyadi, Z. Alhashmi, and M. Shatnawi, “Survey of IoT-Based Smart Home Approaches,” 2019 Adv. Sci. Eng. Technol. Int. Conf. ASET 2019, 2019, pp. 1–6, doi: 10.1109/ICASET.2019.8714572.
J. A. Fadhil, K. Region, O. A. Omar, K. Region, Q. I. Sarhan, K. Region, “A Survey on the Applications of Smart Home Systems,” 2020 International Conference on Computer Science and Software Engineering (CSASE), Duhok, Iraq, 2020, pp. 168–173, doi: 10.1109/CSASE48920.2020.9142103.
M. Asadullah and A. Raza, “An overview of home automation systems,” 2016 2nd Int. Conf. Robot. Artif. Intell. ICRAI 2016, 2016, pp. 27–31, doi: 10.1109/ICRAI.2016.7791223.
K. Agarwal, A. Agarwal, G. Misra, “Review and performance analysis on wireless smart home and home automation using IoT,” Proc. 3rd Int. Conf. I-SMAC IoT Soc. Mobile, Anal. Cloud, I-SMAC 2019, 2019, pp. 629–633, doi: 10.1109/I-SMAC47947.2019.9032629.
J. Jaihar, N. Lingayat, P. S. Vijaybhai, G. Venkatesh, K. P. Upla, “Smart home automation using machine learning algorithms,” 2020 Int. Conf. Emerg. Technol. INCET, 2020, pp. 1–4, 2020, doi: 10.1109/INCET49848.2020.9154007.
M. Gamba, A. Gonella, C. E. Palazzi, “Design issues and solutions in a modern home automation system,” 2015 Int. Conf. Comput. Netw. Commun. ICNC 2015, 2015, pp. 1111–1115, doi: 10.1109/ICCNC.2015.7069505.
C. J. Baby, F. A. Khan, J. N. Swathi, “Home automation using IoT and a chatbot using natural language processing,” 2017 Innov. Power Adv. Comput. Technol. i-PACT, 2017, pp. 1–6, 2017, doi: 10.1109/IPACT.2017.8245185.
Y. Inayama and H. Hosobe, “Toward an efficient user interface for block-based visual programming,” Proc. IEEE Symp. Vis. Lang. Human-Centric Comput. VL/HCC, 2018, pp. 293–294, doi: 10.1109/VLHCC.2018.8506530.
T. Zeng, Y. Liu, X. Ma, X. Bao, J. Qiu, L. Zhan, “Auto-programming for numerical data based on remnant-standard-deviation-guided gene expression programming,” 2009 Fifth International Conference on Natural Computation, Tianjian, China, 2009, pp. 124–128, doi: 10.1109/ICNC.2009.617.
S. R. Swamy, K. S. Nandini Prasad, P. Tripathi, “Smart home lighting system,” Proc. 2020 Int. Conf. Smart Innov. Des. Environ. Manag. Plan. Comput. ICSIDEMPC, 2020, pp. 75–81, doi: 10.1109/ICSIDEMPC49020.2020.9299585.
C. Xie, H. Qi, L. Ma, J. Zhao, “DeepVisual: A visual programming tool for deep learning systems,” IEEE Int. Conf. Progr. Compr., May 2019, pp. 130–134, doi: 10.1109/ICPC.2019.00028.
H. Naito, T. Yokogawa, N. Igawa, S. Amasaki, H. Aman, K. Arimoto, “A node-style visual programming environment for the nuXmv model checker,” 2020 IEEE 9th Glob. Conf. Consum. Electron. GCCE, 2020, pp. 71–75, doi: 10.1109/GCCE50665.2020.9291945.
H. Kamada and K. Nishikawa, “The visual interactive programing learning system using image processing,” 2016 Third Int. Conf. Comput. Meas. Control Sens. Netw., 2016, pp. 158–161, doi: 10.1109/CMCSN.2016.21.
R. Anbunathan and A. Basu, “Automation framework for test script generation for android mobile,” Adv. Intell. Syst. Comput., vol. 731, pp. 571–584, 2019, doi: 10.1007/978-981-10-8848-3_55.
H. Tanno and X. Zhang, “Test script generation based on design documents for web application testing,” Proc. Int. Comput. Softw. Appl. Conf., vol. 3, pp. 672–673, 2015, doi: 10.1109/COMPSAC.2015.74.
S. Goel and R. Sharma, Economic Analysis of Solar Water Pumping System for Irrigation, in: Sharma, R., Mishra, M., Nayak, J., Naik, B., Pelusi, D. (eds) Green Technology for Smart City and Society. Lecture Notes in Networks and Systems, vol. 151, Singapore: Springer, https://doi.org/10.1007/978-981-15-8218-9_13.
S. Oyucu, “Integration of cloud-based speech recognition system to the Internet of Things based smart home automation,” HORA 2021 – 3rd Int. Congr. Human-Computer Interact. Optim. Robot. Appl. Proc., 2021, pp. 30–32, doi: 10.1109/HORA52670.2021.9461360.
A. Meliones and D. Giannakis, “Visual programming of an interactive smart home application using LabVIEW,” IEEE Int. Conf. Ind. Informatics, 2013, pp. 655–660, doi: 10.1109/INDIN.2013.6622961.
Z. Li, Y. Xiao, S. Liang, S. Wang, “Design of Smart home management system based on MQTT and FBP,” Proc. 2018 Chinese Autom. Congr. CAC 2018, 2019, pp. 3086–3091, doi: 10.1109/CAC.2018.8623113.
M. Campbell, “Automated coding:” Computer, Vol. 53, No. 2, pp. 2020–2022, 2020, doi: 10.1109/MC.2019.2957958.
J. Yi, S. Fu, S. Cui, C. Zhao, “A deep contractive auto-encoding network for machinery fault diagnosis,” Isc. 2018 – 18th Int. Symp. Commun. Inf. Technol., 2018, pp. 85–89, doi: 10.1109/ISCIT.2018.8587983.
X. Yang, H. Zhang, J. Cai, “Auto-encoding and distilling scene graphs for image captioning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 8828, no. c, pp. 1–14, 2020, doi: 10.1109/TPAMI.2020.3042192.
H. Li, Y.-P. Wang, T.-J. Mu, “Nerva: Automated application synthesis for humanoid robot from user natural language description,” Commun. Inf. Syst., vol. 17, no. 1, pp. 45–64, 2017, doi: 10.4310/cis.2017.v17.n1.a3.
V. Le, S. Gulwani, Z. Su, “SmartSynth,” Mobisys ’13: Proceeding of the 11th Annual International Conference On Mobile Systems, Applications, And Services, 2013, p. 193, doi: 10.1145/2462456.2464443.
S. R. Joseph, H. Hloman, K. Letsholo, K. Sedimo, “Natural language processing: A review,” Int. J. Res. Eng. Appl. Sci., vol. 6, no. 3, pp. 1–8, 2016, available at: http://www.euroasiapub.org.
S. J. Segalowitz and H. Chevalier, “Event-related potential (ERP) research in neurolinguistics: Part I. Techniques and applications to lexical access,” Handbook of Neurolinguistics, Academic Press, 1998. https://doi.org/10.1016/B978-012666055-5/50009-5.
D. Stringer, “Lexical semantics: Relativity and transfer,” Appl. Linguist. Teach. Cult. Linguist. Divers. Learn., pp. 180–203, 2019, doi: 10.4018/978-1-5225-8467-4.ch007.
S. K. Joseph, “Natural language processing tutorial,” Tutorials Point Pvt. Ltd., pp. 1–13, 2019, available at: https://store.tutorialspoint.com.
Y. Y. Hsu and H. Y. Kao, “Curatable named-entity recognition using semantic relations,” IEEE/ACM Trans. Comput. Biol. Bioinforma., vol. 12, no. 4, pp. 785–792, 2015, doi: 10.1109/TCBB.2014.2366770.
A. Fern, “Lecture Notes: First-Order Logic: Syntax and Semantics Syntax of FO Logic,” pp. 1–9, 2010, available at: https://web.engr.oregonstate.edu/~afern/classes/cs532/notes/fo-ss.pdf
J. Laird, “A compositional cost model for the λ
-calculus,” Proc. Symp. Log. Comput. Sci., vol. 2021-June, 2021, doi: 10.1109/LICS52264.2021.9470567.
M. Biernacka, D. Biernacki, S. Lenglet, P. Polesiuk, D. Pous, A. Schmitt, “Fully abstract encodings of λ
-calculus in HOcore through abstract machines,” Proc. Symp. Log. Comput. Sci., pp. 1–12, 2017, doi: 10.1109/LICS.2017.8005118.
T. Lampert, “Minimizing disjunctive normal forms of pure first-order logic,” Log. J. IGPL, vol. 25, no. 3, pp. 325–347, 2017, doi: 10.1093/jigpal/jzx003.
A. Sernadas, “Fibring modal first-order logics: Completeness preservation,” Log. J. IGPL, vol. 10, no. 4, pp. 413–451, 2002, doi: 10.1093/jigpal/10.4.413.
C. Calvès and M. Fernández, “Matching and alpha-equivalence check for nominal terms,” J. Comput. Syst. Sci., vol. 76, no. 5, pp. 283–301, 2010, doi: 10.1016/j.jcss.2009.10.003.
S. Guerrini, “Linear β
-reduction,” Electron. Proc. Theor. Comput. Sci. EPTCS, vol. 238, pp. 44–53, 2017, doi: 10.4204/EPTCS.238.5.
P. H. Azevedo De Amorim, D. Kozen, R. Mardare, P. Panangaden, M. Roberts, “Universal semantics for the stochastic λ
-calculus,” Proc. Symp. Log. Comput. Sci., vol. 2021, June 2021, doi: 10.1109/LICS52264.2021.9470747.
S. Bird, E. Klein, E. Loper, “NLTK tutorial: Introduction to natural language processing,” English, vol. 66, p. 22, 2005.
V. Sinha, F. Doucet, C. Siska, R. Gupta, S. Liao, A. Ghosh, “YAML: A tool for hardware design visualization and capture,” Proc. Int. Symp. Syst. Synth., vol. January 2000, pp. 9–14, 2000, doi: 10.1109/ISSS.2000.874023.
