A Novel Proximal Policy Optimization Approach for Filter Design

Authors

  • Dongdong Fan Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China Shenzhen 518110, China
  • Shuai Ding 1) Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China Shenzhen 518110, China 2) Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China
  • Haotian Zhang Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China
  • Weihao Zhang School of Materials and Energy, University of Electronic Science and Technology of China Chengdu 610054, P.R. China
  • Qingsong Jia Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China
  • Xu Han Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China
  • Hao Tang Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China
  • Zhaojun Zhu Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China
  • Yuliang Zhou School of Aeronautics and Astronautics, University of Electronic Science and Technology of China Chengdu 610054, China

DOI:

https://doi.org/10.13052/2024.ACES.J.390502

Keywords:

bandpass filters (BPF), coupling matrix synthesis, Proximal Policy Optimization (PPO)

Abstract

This paper proposes a proximal policy optimization (PPO) algorithm for coupling matrix synthesis of microwave filters. With the improvement of filter design requirement, the limitations of traditional methods such as limited applicability are becoming more and more obvious. In order to improve the filter synthesis efficiency, this paper constructs a reinforcement learning algorithm based on Actor-Critic network architecture, and designs a unique filter coupling matrix synthesis reward function and action function, which can solve combinatorial optimization problems stably.

Downloads

Download data is not yet available.

Author Biographies

Dongdong Fan, Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China Shenzhen 518110, China

Dongdong Fan was born in Shanxi, China,in 1996. He received the B.E. degree in Optoelectronic information science and engineering from the Nanyang Institute of Technology of China, in 2019, where he is currently pursuing the M.E. degree in electronic information engineering with the School of Physics. His current research interests include radio-frequency circuit and filter.

Shuai Ding, 1) Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China Shenzhen 518110, China 2) Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China

Shuai Ding received the Ph.D. degree in radio physics from the University of Electronic Science and Technology of China (UESTC), Chengdu, in 2013. From 2013 to 2014, he was a Postdoctoral Associate with the École Polytechnique de Montréal, Montréal, QC, Canada. In 2015, he joined UESTC, where he is currently an Associate Professor. He has authored or coauthored over 80 publications in refereed journals and international conferences/symposia. His current research interests include time-reversed electromagnetics and its applications to communication and energy transmission, phased array, analog signal processing, and microwave circuits. He has served as a TPC Member for various conferences and a reviewer for several peer-reviewed periodicals and international conferences/symposia.

Haotian Zhang, Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China

Haotian Zhang was born in Henan, China,in 1998. He is received the M.E. degree in physics with the School of Physics from the University of Electronic Science and Technology of China, in 2023. His current research interests include machine learning, antenna arrays, and filter.

Weihao Zhang, School of Materials and Energy, University of Electronic Science and Technology of China Chengdu 610054, P.R. China

Weihao Zhang was born in Handan, China,in 1995. He received theB.E. degree in fundamental science (mathematics and physics) from the University of Electronic Science and Technology of China, in 2018, where he is currently pursuing the Ph.D.degree in Electronic Information Materials and Components with University of Electronic Science and Technology of China, ChengDu, China. His current research interests include integrated magnetic devices and fabrication technologies, filter, filtering antenna, metasurface, antenna array.

Qingsong Jia, Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China

Qingsong Jia was born in Sichuan, China,in 1997. He received the B.E. degree in electronic information science and technology from the University of Electronic Science and Technology of China, in 2019, where he is currently pursuing the Ph.D.degree in electromagnetic field and microwave technology with the School of Physics. His current research interests include metasurface, antenna arrays, and the application of radio OAM vortex wave.

Xu Han, Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China

Xu Han was born in Sichuan, China, in 1995. He received the B.E. degree in electronic information science and technology from the University of Electronic Science and Technology of China, in 2018, where he is currently pursuing the Ph.D.degree in electromagnetic field and microwave technology with the School of Physics. His current research interests include metasurface, antenna arrays, and phase array.

Hao Tang, Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China

Hao Tang was born in Hebei, China, in 1998. He received the B.E. degree in Internet of Things Engineering from the Chengdu University of Technology of China,in 2020, where he is currently pursuing the Ph.D. degree in physics with the School of Physics. His current research interests include metasurface and antenna arrays.

Zhaojun Zhu, Institute of Applied Physics, University of Electronic Science and Technology of China Chengdu 610054, China

Zhaojun Zhu was born in Sichuan, China, in 1978. He received the B.S. degree and the Ph.D.degree in physical electronics from the University of Electronic Science and Technology of China(UESTC), Chengdu, in 2002 and 2007, respectively. Since 2012, he has been an Associate Professor with UESTC. His research interests include the design of microwave and millimeter-wave circuits.

Yuliang Zhou, School of Aeronautics and Astronautics, University of Electronic Science and Technology of China Chengdu 610054, China

Yuliang Zhou received the B.S. degree in applied physics from the University of Electronic Science and Technology of China, Chengdu, China, in 2012, where he is currently pursuing the Ph.D. degree in communication and information systems. From 2017 to 2018, he was with the Microwave Laboratory, University of Pavia, Pavia, Italy. His current research interests include substrate integrated circuits, leaky-wave antennas, and systems for wireless communication.

References

D. Liang, X. Zhang, B. Yang, and D. Young, “Overview of base station requirements for RF and microwave filters,” in 2021 IEEE MTT-S International Microwave Filter Workshop (IMFW), pp. 46-49, 2021.

R. J. Cameron, C. M. Kudsia, and R. R. Mansour, Microwave Filters for Communication Systems: Fundamentals, Design, and Applications. Hoboken, NJ: John Wiley & Sons, 2018.

F. Feng, C. Zhang, J. Ma, and Q.-J. Zhang, “Parametric modeling of EM behavior of microwave components using combined neural networks and pole-residue-based transfer functions,” IEEE Transactions on Microwave Theory and Techniques, vol. 64, no. 1, pp. 60-77, 2016.

M. Ohira, A. Yamashita, Z. Ma, and X. Wang, “Automated microstrip bandpass filter design using feedforward and inverse models of neural network,” in 2018 Asia-Pacific Microwave Conference (APMC), pp. 1292-1294, 2018.

M. Ohira, K. Takano, and Z. Ma, “A novel deep-Q-network-based fine-tuning approach for planar bandpass filter design,” IEEE Microwave and Wireless Components Letters, vol. 31, no. 6, pp. 638-641, 2021.

M. Ohira, A. Yamashita, Z. Ma, and X. Wang, “A novel eigenmode-based neural network for fully automated microstrip bandpass filter design,” in 2017 IEEE MTT-S International Microwave Symposium (IMS), pp. 1628-1631, 2017.

B. Liu, H. Yang, and M. J. Lancaster, “Global optimization of microwave filters based on a surrogate model-assisted evolutionary algorithm,” IEEE Transactions on Microwave Theory and Techniques, vol. 65, no. 6, pp. 1976-1985, 2017.

J. L. Chávez-Hurtado and J. E. Rayas-Sánchez, ‘‘Polynomial-based surrogate modeling of RF and microwave circuits in frequency domain exploiting the multinomial theorem,” IEEE Transactions on Microwave Theory and Techniques, vol. 64, no. 12, pp. 4371-4381, 2016.

L. Bi, W. Cao, W. Hu, and M. Wu, “Intelligent tuning of microwave cavity filters using granular multi-swarm particle swarm optimization,” IEEE Transactions on Industrial Electronics, vol. 68, no. 12, pp. 12901-12911, 2021.

S. Koziel, J. Meng, J. W. Bandler, M. H. Bakr, and Q. S. Cheng, “Accelerated microwave design optimization with tuning space mapping,” IEEE Transactions on Microwave Theory and Techniques, vol. 57, no. 2, pp. 383-394, 2009.

Q. S. Cheng, J. W. Bandler, and S. Koziel, “Space mapping design framework exploiting tuning elements,” IEEE Transactions on Microwave Theory and Techniques, vol. 58, no. 1, pp. 136-144, 2010.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.

M. A. Nielsen, Neural Networks and Deep Learning, vol. 25. San Francisco, CA: Determination Press, 2015.

S. Hochreiter, “The vanishing gradient problem during learning recurrent neural nets and problem solutions,” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, no. 02, pp. 107-116, 1998.

Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, p. 436, 2015.

X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” Journal of Machine Learning Research, vol. 15, pp. 315-323, 2011.

G. E. Dahl, T. N. Sainath, and G. E. Hinton, “Improving deep neural networks for LVCSR using rectified linear units and dropout,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8609-8613, 2013.

Downloads

Published

2024-05-31

How to Cite

[1]
D. Fan, “A Novel Proximal Policy Optimization Approach for Filter Design”, ACES Journal, vol. 39, no. 05, pp. 390–395, May 2024.

Issue

2024: Vol 39 Iss 5

Section

General Submission

Categories