Fruit Picking Robot Arm Training Solution Based on Reinforcement Learning in Digital Twin

Authors

  • Xinyuan Tian Macau Institute of Systems Engineering, Macau University of Science and Technology, Taipa, 999078, Macau, China
  • Bingqin Pan Sichuan Digital Transportation Tech Co. Ltd., Chengdu, 610095, Sichuan, China
  • Liping Bai Macau Institute of Systems Engineering, Macau University of Science and Technology, Taipa, 999078, Macau, China
  • Guangbin Wang School of Mechanical and Electrical Engineering, Lingnan Normal University, Zhanjiang, 524000, Guangdong, China
  • Deyun Mo 1) Macau Institute of Systems Engineering, Macau University of Science and Technology, Taipa, 999078, Macau, China, 2)School of Mechanical and Electrical Engineering, Lingnan Normal University, Zhanjiang, 524000, Guangdong, China

DOI:

https://doi.org/10.13052/jicts2245-800X.1133

Keywords:

Robot arm, digital twin, Reinforcement Learning, unity, ML-agent

Abstract

In the era of Industry 4.0, digital agriculture is developing very rapidly and has achieved considerable results. Nowadays, digital agriculture-based research is more focused on the use of robotic fruit picking technology, and the main research direction of such topics is algorithms for computer vision. However, when computer vision algorithms successfully locate the target object, it is still necessary to use robotic arm movement to reach the object at the physical level, but such path planning has received minimal attention. Based on this research deficiency, we propose to use Unity software as a digital twin platform to plan the robotic arm path and use ML-Agent plug-in as a reinforcement learning means to train the robotic arm path, to improve the accuracy of the robotic arm to reach the fruit, and happily the effect of this method is much improved than the traditional method.

Downloads

Download data is not yet available.

Author Biographies

Xinyuan Tian, Macau Institute of Systems Engineering, Macau University of Science and Technology, Taipa, 999078, Macau, China

Xinyuan Tian received his B.S. degree in Information Security in 2020 from the School of cyber science and engineering of Wuhan University, Hubei, China. He received his M.S. degree in Information Security in 2022 from the University of Glasgow. He is currently a Ph.D. student at the Faculty of Innovation Engineering of Macau University of Science and Technology in Macau, China. His research interests are related to Artificial Intelligence, Pattern Recognition, Intelligent Science and Systems and information security.

Bingqin Pan, Sichuan Digital Transportation Tech Co. Ltd., Chengdu, 610095, Sichuan, China

Bingqin Pan, graduated from the National University of Singapore with a Master’s Degree majoring in Industry 4.0, and graduated from Wuhan University with a Bachelor’s Degree majoring in Electrical Engineering and Automation. Now he working in the Institute of Future Transportation Engineering in Sichuan Digital Transportation Technology Co., Ltd, working as an engineer in Technology and Innovation Department. His research interests include Big data, digital twin and intelligent transportation.

Liping Bai, Macau Institute of Systems Engineering, Macau University of Science and Technology, Taipa, 999078, Macau, China

Liping Bai is a researcher at Macau Institute of Systems Engineering in the Macau University of Science and Technology, China and she is currently an Associate Professor. Her research interests are related to Industry Engineering, Operations Research, Production Planning and Control, Information system, E-Commerce. She has published research papers at national and international journals, conference proceedings as well as chapters of books.

Guangbin Wang, School of Mechanical and Electrical Engineering, Lingnan Normal University, Zhanjiang, 524000, Guangdong, China

Guangbin Wang is a professor of Lingnan Normal University, a member of the Academic Committee of Lingnan Normal University and a leader in robotics. His research direction mainly focuses on high-end equipment such as wind turbines and aviation engines, conducting research on complex electromechanical system dynamics, modern sensing and detection, equipment health monitoring and fault diagnosis, and mechanical optimization design and remanufacturing for equipment maintenance.

Deyun Mo, 1) Macau Institute of Systems Engineering, Macau University of Science and Technology, Taipa, 999078, Macau, China, 2)School of Mechanical and Electrical Engineering, Lingnan Normal University, Zhanjiang, 524000, Guangdong, China

Deyun Mo received his M.S. degree in Mechanical Engineering in 2022 from the Guangdong University of Technology. He is currently a Ph.D. student at the Faculty of Innovation Engineering of Macau University of Science and Technology in Macau, China. He is a senior experimentalist in School of Mechanical and Electrical Engineering at Lingnan Normal University. His research interests include high speed machining equipment, machine learning, and artificial intelligence.

References

Rose, D. C., and Chilvers, J. (2018). Agriculture 4.0: Broadening responsible innovation in an era of smart farming. Frontiers in Sustainable Food Systems, 2, 87.

Smith, M. J. (2018). Getting value from artificial intelligence in agriculture. Animal Production Science, 60(1), 46–54.

Rotz, S., Duncan, E., Small, M., Botschner, J., Dara, R., Mosby, I., … and Fraser, E. D. (2019). The politics of digital agricultural technologies: a preliminary review. Sociologia Ruralis, 59(2), 203–229.

Inshakova, A. O., Frolova, E. E., Rusakova, E. P., and Kovalev, S. I. (2020). The model of distribution of human and machine labor at intellectual production in industry 4.0. Journal of Intellectual Capital, 21(4), 601–622.

Daudelin, J., Jing, G., Tosun, T., Yim, M., Kress-Gazit, H., and Campbell, M. (2018). An integrated system for perception-driven autonomy with modular robots. Science Robotics, 3(23), eaat4983.

Zhang, Y., Li, M., Qiao, J., and Liu, G. (2008). A segmentation algorithm for apple fruit recognition using artificial neural network. Aktualni zadaci mehanizacije poljoprivrede. Zbornik radova, 35. međunarodnog simpozija iz podruèja mehanizacije poljoprivrede, Opatija, Croatia, 11-15 veljaèe 2008., 359–367.

Wang, Z. (2018, September). Robot obstacle avoidance and navigation control algorithm research based on multi-sensor information fusion. In 2018 11th International Conference on Intelligent Computation Technology and Automation (ICICTA) (pp. 351–354). IEEE.

Au, W., Chung, H., and Chen, C. (2016). Path planning and assembly mode-changes of 6-DOF Stewart-Gough-type parallel manipulators. Mechanism and Machine Theory, 106, 30–49.

Gómez-Bravo, F., Carbone, G., and Fortes, J. C. (2012). Collision free trajectory planning for hybrid manipulators. Mechatronics, 22(6), 836–851.

Yang, H., Li, L., and Gao, Z. (2017). Obstacle avoidance path planning of hybrid harvesting manipulator based on joint configuration space. Transactions of the Chinese Society of Agricultural Engineering, 33(4), 55–62.

Wang, C., Tang, Y., Zou, X., SiTu, W., and Feng, W. (2017). A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot. Optik, 131, 626–631.

Cao, X., Zou, X., Jia, C., Chen, M., and Zeng, Z. (2019). RRT-based path planning for an intelligent litchi-picking manipulator. Computers and electronics in agriculture, 156, 105–118.

Garg, G., Kuts, V., and Anbarjafari, G. (2021). Digital twin for fanuc robots: Industrial robot programming and simulation using virtual reality. Sustainability, 13(18), 10336.

Xue, C., Qiao, Y., and Murray, N. (2020, August). Enabling human-robot-interaction for remote robotic operation via augmented reality. In 2020 IEEE 21st International Symposium on “A World of Wireless, Mobile and Multimedia Networks” (WoWMoM) (pp. 194–196). IEEE.

Li, Y., Zhang, Q., Xu, H., Lim, E., and Sun, J. (2022). Virtual monitoring system for a robotic manufacturing station in intelligent manufacturing based on Unity 3D and ROS. Materials Today: Proceedings.

Juliani, A., Berges, V. P., Teng, E., Cohen, A., Harper, J., Elion, C., … and Lange, D. (2018). Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627.

Lin, M., Shan, L., and Zhang, Y. (2020, September). Research on robot arm control based on Unity3D machine learning. In Journal of Physics: Conference Series (Vol. 1633, No. 1, p. 012007). IOP Publishing.

Liu, X., Jiang, D., Tao, B., Jiang, G., Sun, Y., Kong, J., … and Chen, B. (2021). Genetic algorithm-based trajectory optimization for digital twin robots. Frontiers in Bioengineering and Biotechnology, 9.

Juliani, A., Berges, V. P., Teng, E., Cohen, A., Harper, J., Elion, C., … and Lange, D. (2018). Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627.

Akimov, S. S. (2022, May). Solution of the inverse kinematic problem for the KUKA KR AGILUS robotic arm. In Computer Applications for Management and Sustainable Development of Production and Industry (CMSD2021) (Vol. 12251, pp. 115–119). SPIE.

Hung, P. T., Truong, M. D. D., and Hung, P. D. (2022). Tuning Proximal Policy Optimization Algorithm in Maze Solving with ML-Agents. In International Conference on Advances in Computing and Data Sciences (pp. 248–262). Springer, Cham.

Urmanov, M., and Alimanova, M. (2020). Training a single Machine Learning Agent using Reinforcement Learning and Imitation Learning methods in Unity environment. Suleyman Demirel University Bulletin: Natural and Technical Sciences, 52(1).

Recht, B. (2019). A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems, 2, 253–279.

Terry, J., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R., … and Ravi, P. (2021). Pettingzoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34, 15032–15043.

Ding, T., Zeng, Z., Bai, J., Qin, B., Yang, Y., and Shahidehpour, M. (2020). Optimal electric vehicle charging strategy with Markov decision process and reinforcement learning technique. IEEE Transactions on Industry Applications, 56(5), 5811–5823.

Puterman ML. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons; 2014 Aug 28.

Downloads

Published

2023-09-11

How to Cite

Tian, X. ., Pan, B. ., Bai, L. ., Wang, G. ., & Mo, D. . (2023). Fruit Picking Robot Arm Training Solution Based on Reinforcement Learning in Digital Twin. Journal of ICT Standardization, 11(03), 261–282. https://doi.org/10.13052/jicts2245-800X.1133

Issue

2023: Vol 11 Iss 3

Section

Intelligent System Concepts, architecture, standards, tools and applications