Development of an Equation-Free Surrogate Model using Deep Learning Algorithm for Heat Transfer Simulation
DOI:
https://doi.org/10.13052/ejcm2642-2085.3341Keywords:
Neural network, deep learning, weakly supervised learning, steady state heat equation, boundary conditionAbstract
The significant computational costs and time associated with accurate simulation of physical phenomena make the simulation of nonlinear systems based on differential equations impractical for real-time prediction. Deep learning with its high potential in understanding nonlinear and unknown phenomena can be a suitable alternative to equation-based modeling. However, the success of deep learning highly relies on the availability of large-scale labeled data. To solve this problem, weakly supervised learning helps us. This algorithm can train models using only a limited amount of labeled data. In this work, a new definition of the loss function was presented, which can greatly reduce our need to prepare labels for network training through weak supervision. We used this approach for 2D heat transfer modeling. The present work consists of two steps: (1) Extracting the equilibrium temperature pattern directly from only 400 thermal data and encoding it in a convolutional kernel that forms the loss function; and (2) unsupervised training of the model using this loss function instead of the labels without observing any thermal data. The effectiveness of the proposed model in terms of accuracy, the number of labeled data used, and the time required for training the network was evaluated and compared with three supervised models trained on large data sets. Despite using less data, our model achieved higher accuracy compared to a supervised model trained from direct observation of 5000 labeled thermal data (0.68% vs. 1.5% error), which has a longer training time than our model (20 vs. 12 hours); and the cGAN-based model despite using more than 10 times more labeled thermal data (0.68% vs. 1% error).
Downloads
References
Mattheij, R., Sjoerd, W., “Partial differential equations: modeling, analysis, computation”, Society for Industrial and Applied Mathematics, (2005).
Sadeghi, E, S., Cronin, L.,Sadeghi, S., Pakzad, S., “Input estimation of nonlinear systems using probabilistic neural network”, Journal of Mechanical Systems and Signal Processing, Vol. 166, No. 9, (2021).
Moghanlo, S., Alavinejad, M., Oskoei, V., Najafi, H., “Using artificial neural networks to model the impacts of climate change on dust phenomenon in the Zanjan region”, Journal of Urban Climate, Vol. 35, (2021).
Graf, R.; Kolerski, T.; Zhu, S., “Predicting Ice Phenomena in a River Using the Artificial Neural Network and Extreme Gradient Boosting”, Resources, Vol. 11, No. 2, (2022).
Pichi, F., Ballarin, F., Rozza, G., “An artificial neural network approach to bifurcating phenomena in computational fluid dynamics”, Journal of Computers & Fluids, Vol. 254, (2023).
Ma, J., Wang, J., Han, Y., Dong, S., “Towards data-driven modeling for complex contact phenomena via self-optimized artificial neural network methodology”, Journal of Mechanism and Machine Theory, Vol. 182, (2023).
Bongard, J., Lipson, H., “Automated reverse engineering of nonlinear dynamical systems”, National Acad Sciences, USA 104, (2007), 9943–9948.
Schmidt, M., Lipson, H., “Distilling free-form natural laws from experimental data”, Journal of Science, (2009), 81–85.
Lusch, B., Kutz, J.N., Brunton, S.L., “Deep learning for universal linear embeddings of nonlinear dynamics”, Journal of Nature Communications, (2018).
Ruthotto, L., Osher, S., Li, W., Nurbekyan, L., “A machine learning framework for solving high-dimensional mean field game and mean field control problems”, Journal of PNAS, Vol. 117, No. 17, (2020), 9183–9193.
Ding, Y., Hua, L., Li, S., “Research on computer vision enhancement in intelligent robot based on machine learning and deep learning”, Journal of Neural Computing and Applications, Vol. 34, (2022), 2623–2635.
Lauriola, I., Lavelli, A., Aiolli, F., “An introduction to deep learning in natural language processing: Models, techniques, and tools”, Journal of Neurocomputing, Vol. 470, (2022), 443–456.
Johnson, M., Schuster, M., Le, Q., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., Hughes, M., & Dean, J., “Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”, Transactions of the Association for Computational Linguistics, Vol. 5, (2017), 339–351.
Gumbs, A., Grasso, V., Bourdel, N., Croner, R., Spolverato, G., Frigerio, I., Illanes, A., Abu Hilal, A., Park, A., Elyan, E., “The Advances in Computer Vision That Are Enabling More Autonomous Actions in Surgery: A Systematic Review of the Literature”, Vol. 22, No. 13, (2022).
Thepade, S. D., Dindorkar, M.R., Chaudhari, P.R., Bang, S.V., “Enhanced Face Presentation Attack Prevention Employing Feature Fusion of Pre-trained Deep Convolutional Neural Network Model and Thepade’s Sorted Block Truncation Coding”, International Journal of Engineering (IJE), Transactions A: Basics, Vol. 36, No. 04, (2023), 807–816.
El-Amir, H., Hamdy, M., “Deep Learning Pipeline: Building a Deep Learning Model with TensorFlow”, Apress, (2019).
Venigandla, K., Tatikonda, V., “Improving Diagnostic Imaging Analysis with RPA and Deep Learning Technologies ”, Journal of Power System Technology, Vol. 45, No. 4, (2021).
Liu, X., Gao, K., Liu, B., Pan, C., “Advances in Deep Learning-Based Medical Image Analysis”, Journal of Health Data Science, (2021).
Ramcharan, A., Baranowski, K., McCloskey, P., Ahmed, B., “Deep Learning for Image-Based Cassava Disease Detection”, Journal of Frontiers in Plant Science, Vol. 8, (2017).
Issa, D., Demirci, M.F., Yazici,A., “Speech emotion recognition with deep convolutional neural networks”, Journal of Biomedical Signal Processing and Control, Volume 59, (2020).
Saleem, N., Khattak, M., Qazi, A., “Supervised Speech Enhancement Based on Deep Neural Network”, Journal of Intelligent & Fuzzy Systems, Vol. 37, No. 4, (2019), 5187–5201.
Wang, D., Chen, J., “Supervised Speech Separation Based on Deep Learning: An Overview”, in IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 26, No. 10, (2018), 1702–1726.
Chan, Z., Lau, C., Thang, K., “Visual Speech Recognition of Lips Images Using Convolutional Neural Network in VGG-M Model”, Journal of Information Hiding and Multimedia Signal Processing, Vol. 11, No. 3, (2020).
Song, Z., “English speech recognition based on deep learning with multiple features”, Journal of Computing, Vol. 102, (2020), 663–682.
Araújo, A., Pereira, A., Benevenuto, F., “A comparative study of machine translation for multilingual sentence-level sentiment analysis”, Journal of Information Sciences, Vol. 512, (2020).
Andrabi, A., Wahid, A., “Machine Translation System Using Deep Learning for English to Urdu”, Journal of Computational Intelligence and Neuroscience, (2022).
Petousis, P., Han, S., Aberle, D., Bui, A., “Prediction of lung cancer incidence on the low-dose computed tomography arm of the National Lung Screening Trial: A dynamic Bayesian network”, Journal of Artificial Intelligence in Medicine, Vol. 72, (2016), 42–55.
Aberle, D., Adams, A., Berg, C., Black, W., “Reduced lung-cancer mortality with low-dose computed tomographic screening”, Journal of National Lung Screening Trial Research Team, (2011).
D’Souza, G., Siddalingaswamy, P.C., Pandya, M.A., “AlterNet-K: a small and compact model for the detection of glaucoma”, Journal of Biomedical Engineering Letter, Vol. 14, (2024), 23–33.
Zhou, Z., “A brief introduction to weakly supervised learning”, Journal of National Science Review, Vol. 5, No. 1, (2018), 44–53.
Wang, R., Chen, B., Meng, D., Wang, L., “Weakly Supervised Lesion Detection from Fundus Images”, in IEEE Transactions on Medical Imaging, Vol. 38, No. 6, (2019), 1501–1512.
Nguyen-Duc, T., Yoo, I., Thomas, L., Kuan, A., “Weakly supervised learning in deformable EM image registration using slice interpolation”, In IEEE 16th International Symposium on Biomedical Imaging, (2019), 670–673.
Costa, P., Galdran, A., Smailagic, A., Campilho, A., “A weakly-supervised framework for interpretable diabetic retinopathy detection on retinal images”, IEEE Access 6, (2018), 18747–18758.
Ganji, D., Sabzehmeidani, Y., Sedighiamiri, A., “Nonlinear System in Heat Transfer Mathematical Modeling and Analytical Methods”, (2018), 35–36.
Daileda, R. C., “The two-dimensional heat equation”, Trinity University, San Antonio, Texas, (2012).
Lu, C., Liu, Q., Sun, Q., Hsieh, C., Zhang, S., Shi, L., Lee, C., “Deep Learning for Optoelectronic Properties of Organic Semiconductors”, Journal of Physical Chemistry, Vol. 124, No. 13, (2020), 7048–7060.
Ryczko, K., Strubbe, D., Tamblyn, I., “Deep Learning and Density Functional Theory”, (2019).
Bikmukhametov, T., Jäschke, J., “Combining machine learning and process engineering physics towards enhanced accuracy and explainability of data-driven models”, Elsevier, Vol. 138, (2020).
Frank, M., Drikakis, D., Charissis, V., “Machine-Learning Methods for Computational Science and Engineering”, Computation, (2020).
Kochkov, D., Smith, J. A., Alieva, A., Hoyer, S., “Machine learning–accelerated computational fluid dynamics”, Journal of PNAS, University of California, Vol. 118, No. 21, (2021).
Mario, l., Stathi, F., Anil A. B., Chris D. C., “Current and emerging deep-learning methods for the simulation of fluid dynam”, Journal of Proc. R. Soc. A, Vol. 479, No. 2275, (2023).
Li, J., Zhang, M., Martins, J., Shu, C., “Efficient Aerodynamic Shape Optimization with Deep-Learning-Based Geometric Filtering”, AIAA Journal, Vol. 58, No. 10, (2020).
Yan, X., Zhu, J., Kuang, M., Wang, X., “Aerodynamic shape optimization using a novel optimizer based on machine learning techniques”, Journal of Aerospace Science and Technology, Vol. 86, (2019).
J.Russell, S., Norvig, P., “Artificial Intelligence: A Modern Approach, Third Edition”, Prentice Hall, (2010).
Talo, M., “Automated classification of histopathology images using transfer learning”, Journal of Artificial Intelligence in Medicine, Vol. 101, (2019).
Yousefi, S., Nie, Y., “Transfer Learning from Nucleus Detection to Classification in Histopathology Images”, IEEE 16th International Symposium on Biomedical Imaging, Venice, Italy, (2019).
Ker, J., Wang, L., Rao, J., Lim, T., “Deep Learning Applications in Medical Image Analysis”, in IEEE Access, Vol. 6, (2018), 9375–9389.
Aljuaid, A., Anwar, M., “Survey of Supervised Learning for Medical Image Processing”, Journal of Computer Science, (2022).
Ronneberger, O., Fischer, P., Brox, T., “U-Net: Convolutional Networks for Biomedical Image Segmentation”, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer International Publishing, (2015), 234–241.
Jiang, J., Hu, Y. C., Tyagi, N., Zhang, P., Rimner, A., Mageras, G. S., Deasy, J. O., Veeraraghavan, H., “Tumor-aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation”, Medical image computing and computer-assisted intervention, (2018), 777–785.
Hinton, G. E., Salakhutdinov, R. R., “Reducing the dimensionality of data with neural networks”, Journal of Science, New York, N.Y., Vol. 313, No. 5786, (2006), 504.
Yin, XX., Sun, L., Fu, Y., Lu, R., Zhang, Y., “U-Net-Based Medical Image Segmentation”, Journal of Healthc Eng, (2022).
Siddique, N., Paheding, S., Elkin, C., Devabhaktuni, V., “U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications”, in IEEE Access, Vol. 9, (2021).
Chicco, D., Warrens, M. J., Jurman, G., “ The coefficient of determination R-squaredis more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation”, Journal of Computer Science, (2021).
Raissi, M., Yazdani, A., Karniadakis, G., “Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data”, Journal of Science, (2018).
Kingma, D. P., Ba, j., “Adam: A Method for Stochastic Optimization”, computer science, conference paper at the 3rd International Conference for Learning Representations, San Diego, (2014).
Barati, A., Gomes, J., Pande, V., “Deep Learning the Physics of Transport Phenomena”, (2017).
Edalatifar, M., Tavakoli, M. B., Ghalambaz, M., Setoudeh, F., “Using deep learning to learn physics of conduction heat transfer”, Journal of Thermal Analysis and Calorimetry, (2020).
O’Shea, K., Nash, R., “An Introduction to Convolutional Neural Network”, Journal of Neural and Evolutionary Computing, (2015).
Li, z., Yang, w., Peng, S., Liu, S., “A Survey of Convolutional Neural Networks: Analysis”, IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, No. 12, (2022).
B. L. Lecun, Y., Bengio, Y., et al., “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, Vol. 86, No. 11, (1998), 2278–2324.
Nair, v., Hinton, G. E., “Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair”, (2010).
Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E., “Squeeze-and Excitation Networks.”, Journal of Computer Science, (2019).
Howard, A., Chu, G., Chen, L., Chen, B., Tan, M., Wang, W., Zhu, Y., “Searching for MobileNetV3,” (2019).
Krizhevsky, A., Sutskever, I., Hinton, G., “ImageNet Classification with Deep Convolutional Neural Networks,” Journal of Advances in Neural Information Processing Systems, vol. 25, no. 2, (2012).
Cruz-Mota, J., Bogdanova, I., Paquier, B., Bierlaire, M., “Scale Invariant Feature Transform on the Sphere: Theory and Applications”, Journal of Computer Vision, (2012), 217–241.
Ahonen, T., Hadid, A., Pietikäinen, M., “Face description with local binary patterns: application to face recognition”, Journal of IEEE Trans Pattern Anal Mach Intell, (2006).
Hawkins, D. M., “The problem of overfitting”, Journal of Chemical Information and computer sciences, Vol. 44, No. 1, (2004).
Gholamalinezhad, G., Khosravi, H., “Pooling Methods in Deep Neural Networks, a Review”, Journal of Computer Science, (2020).
Hameed, A., Karlik, B., Shukri, M., “Back-propagation Algorithm with Variable Adaptive Momentum”, Journal of LATEX Templates, (2016).
