An Automation Script Generation Technique for the Smart Home


  • Jiayi Kuang School of Software, Yunnan University, Kunming, Yunnan, China
  • Gang Xue School of Software, Yunnan University, Kunming, Yunnan, China
  • Zeming Yan School of Software, Yunnan University, Kunming, Yunnan, China
  • Jing Liu School of Software, Yunnan University, Kunming, Yunnan, China



Home automation, automation script generation, first-order logic, natural language processing


A home automation system means monitoring and controlling various kinds of devices in the home remotely using the Internet of things (IoT). Technologies such as natural language processing techniques, user-friendly visual programming, and machine intelligence programming are already available for home automation. For such systems, the increase in the number of devices often makes users focused on the system’s ability to perform complex or composing tasks. However, some existing natural language processing systems can only perform simple tasks and cannot meet users’ needs. Thus, it is difficult for users to develop the home automation systems they need using visual programming systems because of the large amount of programming knowledge required. Meanwhile, automatic programming without user action can only write a few lines of code and implement little functionality. There are relatively few tools available for generating home automation scripting languages. To address this problem, we propose a practical method for generating executable home automation scripts using Chinese texts. Our method includes the following steps: it extracts critical information from the command sentences in Chinese; it uses first-order logic to check the validity of the extracted information; based on the validation, the correct sentences are mapped into the intermediate language scripts, which can interface with different home platforms. We conducted experiments on Home Assistant, converted intermediate scripts to Home Assistant, and collected 600 scenario descriptions. The experimental results show that the method can automatically generate executable scripts for the Home Assistant platform, and the correct rate was 93.66%.


Download data is not yet available.

Author Biographies

Jiayi Kuang, School of Software, Yunnan University, Kunming, Yunnan, China

Jiayi Kuang is a master student at the School of Software, Yunnan University, China. She received the B.Eng. degree from Zaozhuang University, China, in 2019. Her research interests include web service and automation technology research.

Gang Xue, School of Software, Yunnan University, Kunming, Yunnan, China

Gang Xue received the B.Eng. degree from Wuhan Technical University of Surveying and Mapping in 2000. He received the M.Eng. and Ph.D. degrees from Yunnan University in 2006 and 2009, respectively. From September 2019 to July 2020, he worked at Zhejiang University as a visiting scholar. He is currently an associate professor at the School of Software, Yunnan University, China. His research interests include service computing, edge computing, and embedded systems.

Zeming Yan, School of Software, Yunnan University, Kunming, Yunnan, China

Zeming Yan is a master student at the School of Software, Yunnan University, China. He received the B.Eng. degree from Jishou University, China, in 2019. His research interests include service computing and edge computing.

Jing Liu, School of Software, Yunnan University, Kunming, Yunnan, China

Jing Liu received the Ph.D. degree in computer application technology from the University of Electronic Science and Technology of China in 2003. From September 2003 to July 2005, he was with No. 30 Institute of China Electronics Technology Group Corporation as a postdoctoral fellow. From September 2005 to December 2012, he had been an assistant professor at Sun Yat-Sen University. Since January 2013, he has been an associate professor at Yunnan University. His current research interests include applied cryptography and network security.


A. Cyril Jose and R. Malekian, “Smart Home Automation Security: A Literature Review,” Smart Comput. Rev., no. Rtdm, pp. 6–10, 2015, doi: 10.6029/smartcr.2015.04.004.

C. Paul, A. Ganesh, C. Sunitha, “An overview of IoT based smart homes,” Proc. 2nd Int. Conf. Inven. Syst. Control. ICISC 2018, no. Icisc, pp. 43–46, 2018, doi: 10.1109/ICISC.2018.8398858.

M. Shahjalal, M. K. Hasan, M. M. Islam, M. M. Alam, M. F. Ahmed, Y. M. Jang, “An Overview of AI-Enabled Remote Smart- Home Monitoring System Using LoRa,” 2020 Int. Conf. Artif. Intell. Inf. Commun. ICAIIC 2020, 2020, pp. 510–513, doi: 10.1109/ICAIIC48513.2020.9065199.

A. Alhammadi, A. Alzaabi, B. Almarzooqi, S. Alneyadi, Z. Alhashmi, and M. Shatnawi, “Survey of IoT-Based Smart Home Approaches,” 2019 Adv. Sci. Eng. Technol. Int. Conf. ASET 2019, 2019, pp. 1–6, doi: 10.1109/ICASET.2019.8714572.

J. A. Fadhil, K. Region, O. A. Omar, K. Region, Q. I. Sarhan, K. Region, “A Survey on the Applications of Smart Home Systems,” 2020 International Conference on Computer Science and Software Engineering (CSASE), Duhok, Iraq, 2020, pp. 168–173, doi: 10.1109/CSASE48920.2020.9142103.

M. Asadullah and A. Raza, “An overview of home automation systems,” 2016 2nd Int. Conf. Robot. Artif. Intell. ICRAI 2016, 2016, pp. 27–31, doi: 10.1109/ICRAI.2016.7791223.

K. Agarwal, A. Agarwal, G. Misra, “Review and performance analysis on wireless smart home and home automation using IoT,” Proc. 3rd Int. Conf. I-SMAC IoT Soc. Mobile, Anal. Cloud, I-SMAC 2019, 2019, pp. 629–633, doi: 10.1109/I-SMAC47947.2019.9032629.

J. Jaihar, N. Lingayat, P. S. Vijaybhai, G. Venkatesh, K. P. Upla, “Smart home automation using machine learning algorithms,” 2020 Int. Conf. Emerg. Technol. INCET, 2020, pp. 1–4, 2020, doi: 10.1109/INCET49848.2020.9154007.

M. Gamba, A. Gonella, C. E. Palazzi, “Design issues and solutions in a modern home automation system,” 2015 Int. Conf. Comput. Netw. Commun. ICNC 2015, 2015, pp. 1111–1115, doi: 10.1109/ICCNC.2015.7069505.

C. J. Baby, F. A. Khan, J. N. Swathi, “Home automation using IoT and a chatbot using natural language processing,” 2017 Innov. Power Adv. Comput. Technol. i-PACT, 2017, pp. 1–6, 2017, doi: 10.1109/IPACT.2017.8245185.

Y. Inayama and H. Hosobe, “Toward an efficient user interface for block-based visual programming,” Proc. IEEE Symp. Vis. Lang. Human-Centric Comput. VL/HCC, 2018, pp. 293–294, doi: 10.1109/VLHCC.2018.8506530.

T. Zeng, Y. Liu, X. Ma, X. Bao, J. Qiu, L. Zhan, “Auto-programming for numerical data based on remnant-standard-deviation-guided gene expression programming,” 2009 Fifth International Conference on Natural Computation, Tianjian, China, 2009, pp. 124–128, doi: 10.1109/ICNC.2009.617.

S. R. Swamy, K. S. Nandini Prasad, P. Tripathi, “Smart home lighting system,” Proc. 2020 Int. Conf. Smart Innov. Des. Environ. Manag. Plan. Comput. ICSIDEMPC, 2020, pp. 75–81, doi: 10.1109/ICSIDEMPC49020.2020.9299585.

C. Xie, H. Qi, L. Ma, J. Zhao, “DeepVisual: A visual programming tool for deep learning systems,” IEEE Int. Conf. Progr. Compr., May 2019, pp. 130–134, doi: 10.1109/ICPC.2019.00028.

H. Naito, T. Yokogawa, N. Igawa, S. Amasaki, H. Aman, K. Arimoto, “A node-style visual programming environment for the nuXmv model checker,” 2020 IEEE 9th Glob. Conf. Consum. Electron. GCCE, 2020, pp. 71–75, doi: 10.1109/GCCE50665.2020.9291945.

H. Kamada and K. Nishikawa, “The visual interactive programing learning system using image processing,” 2016 Third Int. Conf. Comput. Meas. Control Sens. Netw., 2016, pp. 158–161, doi: 10.1109/CMCSN.2016.21.

R. Anbunathan and A. Basu, “Automation framework for test script generation for android mobile,” Adv. Intell. Syst. Comput., vol. 731, pp. 571–584, 2019, doi: 10.1007/978-981-10-8848-3_55.

H. Tanno and X. Zhang, “Test script generation based on design documents for web application testing,” Proc. Int. Comput. Softw. Appl. Conf., vol. 3, pp. 672–673, 2015, doi: 10.1109/COMPSAC.2015.74.

S. Goel and R. Sharma, Economic Analysis of Solar Water Pumping System for Irrigation, in: Sharma, R., Mishra, M., Nayak, J., Naik, B., Pelusi, D. (eds) Green Technology for Smart City and Society. Lecture Notes in Networks and Systems, vol. 151, Singapore: Springer,

S. Oyucu, “Integration of cloud-based speech recognition system to the Internet of Things based smart home automation,” HORA 2021 – 3rd Int. Congr. Human-Computer Interact. Optim. Robot. Appl. Proc., 2021, pp. 30–32, doi: 10.1109/HORA52670.2021.9461360.

A. Meliones and D. Giannakis, “Visual programming of an interactive smart home application using LabVIEW,” IEEE Int. Conf. Ind. Informatics, 2013, pp. 655–660, doi: 10.1109/INDIN.2013.6622961.

Z. Li, Y. Xiao, S. Liang, S. Wang, “Design of Smart home management system based on MQTT and FBP,” Proc. 2018 Chinese Autom. Congr. CAC 2018, 2019, pp. 3086–3091, doi: 10.1109/CAC.2018.8623113.

M. Campbell, “Automated coding:” Computer, Vol. 53, No. 2, pp. 2020–2022, 2020, doi: 10.1109/MC.2019.2957958.

J. Yi, S. Fu, S. Cui, C. Zhao, “A deep contractive auto-encoding network for machinery fault diagnosis,” Isc. 2018 – 18th Int. Symp. Commun. Inf. Technol., 2018, pp. 85–89, doi: 10.1109/ISCIT.2018.8587983.

X. Yang, H. Zhang, J. Cai, “Auto-encoding and distilling scene graphs for image captioning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 8828, no. c, pp. 1–14, 2020, doi: 10.1109/TPAMI.2020.3042192.

H. Li, Y.-P. Wang, T.-J. Mu, “Nerva: Automated application synthesis for humanoid robot from user natural language description,” Commun. Inf. Syst., vol. 17, no. 1, pp. 45–64, 2017, doi: 10.4310/cis.2017.v17.n1.a3.

V. Le, S. Gulwani, Z. Su, “SmartSynth,” Mobisys ’13: Proceeding of the 11th Annual International Conference On Mobile Systems, Applications, And Services, 2013, p. 193, doi: 10.1145/2462456.2464443.

S. R. Joseph, H. Hloman, K. Letsholo, K. Sedimo, “Natural language processing: A review,” Int. J. Res. Eng. Appl. Sci., vol. 6, no. 3, pp. 1–8, 2016, available at:

S. J. Segalowitz and H. Chevalier, “Event-related potential (ERP) research in neurolinguistics: Part I. Techniques and applications to lexical access,” Handbook of Neurolinguistics, Academic Press, 1998.

D. Stringer, “Lexical semantics: Relativity and transfer,” Appl. Linguist. Teach. Cult. Linguist. Divers. Learn., pp. 180–203, 2019, doi: 10.4018/978-1-5225-8467-4.ch007.

S. K. Joseph, “Natural language processing tutorial,” Tutorials Point Pvt. Ltd., pp. 1–13, 2019, available at:

Y. Y. Hsu and H. Y. Kao, “Curatable named-entity recognition using semantic relations,” IEEE/ACM Trans. Comput. Biol. Bioinforma., vol. 12, no. 4, pp. 785–792, 2015, doi: 10.1109/TCBB.2014.2366770.

A. Fern, “Lecture Notes: First-Order Logic: Syntax and Semantics Syntax of FO Logic,” pp. 1–9, 2010, available at:

J. Laird, “A compositional cost model for the λ

-calculus,” Proc. Symp. Log. Comput. Sci., vol. 2021-June, 2021, doi: 10.1109/LICS52264.2021.9470567.

M. Biernacka, D. Biernacki, S. Lenglet, P. Polesiuk, D. Pous, A. Schmitt, “Fully abstract encodings of λ

-calculus in HOcore through abstract machines,” Proc. Symp. Log. Comput. Sci., pp. 1–12, 2017, doi: 10.1109/LICS.2017.8005118.

T. Lampert, “Minimizing disjunctive normal forms of pure first-order logic,” Log. J. IGPL, vol. 25, no. 3, pp. 325–347, 2017, doi: 10.1093/jigpal/jzx003.

A. Sernadas, “Fibring modal first-order logics: Completeness preservation,” Log. J. IGPL, vol. 10, no. 4, pp. 413–451, 2002, doi: 10.1093/jigpal/10.4.413.

C. Calvès and M. Fernández, “Matching and alpha-equivalence check for nominal terms,” J. Comput. Syst. Sci., vol. 76, no. 5, pp. 283–301, 2010, doi: 10.1016/j.jcss.2009.10.003.

S. Guerrini, “Linear β

-reduction,” Electron. Proc. Theor. Comput. Sci. EPTCS, vol. 238, pp. 44–53, 2017, doi: 10.4204/EPTCS.238.5.

P. H. Azevedo De Amorim, D. Kozen, R. Mardare, P. Panangaden, M. Roberts, “Universal semantics for the stochastic λ

-calculus,” Proc. Symp. Log. Comput. Sci., vol. 2021, June 2021, doi: 10.1109/LICS52264.2021.9470747.

S. Bird, E. Klein, E. Loper, “NLTK tutorial: Introduction to natural language processing,” English, vol. 66, p. 22, 2005.

V. Sinha, F. Doucet, C. Siska, R. Gupta, S. Liao, A. Ghosh, “YAML: A tool for hardware design visualization and capture,” Proc. Int. Symp. Syst. Synth., vol. January 2000, pp. 9–14, 2000, doi: 10.1109/ISSS.2000.874023.




How to Cite

Kuang, J. ., Xue, G. ., Yan, Z. ., & Liu, J. . (2023). An Automation Script Generation Technique for the Smart Home. Journal of Web Engineering, 22(02), 221–254.