Real Time Traffic Prediction Based On Social Media Text Data Using Deep Learning
DOI:
https://doi.org/10.13052/jmm1550-4646.18211Keywords:
Traffic prediction, Spark, Big data architecture ,Real time traffic analysisAbstract
Due to urbanization Traffic management is one of the major issues in contemporary civic management, considering this circumstance traffic analysis is turning into the need of the present world. Text data generated by Twitter, Facebook and other social media platforms can be used for traffic management. Big data helps in traffic prediction and traffic analysis of advancing metropolitan zones. Constant traffic investigation requires preparing of information streams that are produced persistently to increase fast experiences. To measures stream information at a fast rate advancements on high figuring limit is required. Social media text data can be processed by using batch processing and stream processing with big data architecture through Spark and Hadoop framework. In this paper big data architecture is proposed for real time traffic text data analysis. In architecture Spark and Kafka are used in combination. Kafka helps in pipelines text data used in conjunction with spark stream processing engine. Big data architecture using Spark, Kafka with ability for processing and preparing huge measure of information, have settled the serious issue of handling and putting away constantly streaming data. The traffic information from Twitter API is streamed. In The proposed model pointed toward ensemble neural network model to reduce the variance in results for better prediction foreseeing traffic stream text data by incorporating Spark and Kafka that will be of an extraordinary incentive to the public authority for traffic management and analysis.
Downloads
References
L. Zhu, F. R. Yu, Y. Wang, B. Ning and T. Tang, “Big Data Analytics in Intelligent Transportation Systems: A Survey,” in IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 1, pp. 383–398, Jan. 2019, doi: 10.1109/TITS.2018.2815678.
Amini, S. et al. “Big data analytics architecture for real-time traffic control.” 2017 5th IEEE International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS) (2017): 710–715.
Hua-pu Lu, Zhi-yuan Sun, Wen-cong Qu, “Big Data-Driven Based Real-Time Traffic Flow State Identification and Prediction”, Discrete Dynamics in Nature and Society, vol. 2015, Article ID 284906, 11 pages, 2015.
X. Wang, C. Wang, J. Zhang, M. Zhou and C. Jiang, “Improved Rule Installation for Real-Time Query Service in Software-Defined Internet of Vehicles,” in IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 2, pp. 225–235, Feb. 2017
Merlin Mathew (merlinthemathew@gmail.com), Aishwarya Balakrishnan (aishu9298@gmail.com), Bikky Kumar Goit (navabryt@gmail.com), Mounica. B (premkumarmounica@gmail.com), “Dynamic Toll Charge using openCV and Spark”, International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 5 Issue 3, pp. 33–37, May–June 2019.
L.U. Laboshin, A.A. Lukashin, and V.S. Zaborovsky. 2017. The Big Data Approach to Collecting and Analyzing Traffic Data in Large Scale Networks. Procedia Comput. Sci. 103, C (March 2017), 536–542.
Prathilothamai M., Lakshmi, A. M. Sree, and Viswanthan, D., “Cost effective road traffic prediction model using Apache spark”, Indian Journal of Science and Technology, vol. 9, 2016.
G. Kothai, E. Poovammal, Gaurav Dhiman, Kadiyala Ramana, Ashutosh Sharma, Mohammed A. AlZain, Gurjot Singh Gaba, Mehedi Masud, “A New Hybrid Deep Learning Algorithm for Prediction of Wide Traffic Congestion in Smart Cities”, Wireless Communications and Mobile Computing, vol. 2021, Article ID 5583874, 13 pages, 2021.
W. Liyong and P. Vateekul, “Improve Traffic Prediction Using Accident Embedding on Ensemble Deep Neural Networks,” 2019 11th International Conference on Knowledge and Smart Technology (KST), 2019, pp. 11–16.
G Guerreiro, P Figueiras, R Silva, R Costa, R J Goncalves, “An architecture for big data processing on intelligent transportation systems. An application scenario on highway traffic flows”, 2016 IEEE 8th International Conference on Intelligent Systems (IS), 2016.
P Figueiras, G Guerreiro, R Costa, Z Herga, A Rosa and R J Goncalves, “Real-Time Monitoring of Road Traffic using Data Stream Mining”, IEEE International Conference on Engineering, Technology and Innovation, 2018.
A. I. Maarala, M. Rautiainen, M. Salmi, S. Pirttikangas and J. Riekki, “Low latency analytics for streaming traffic data with Apache Spark,” 2015 IEEE International Conference on Big Data (Big Data), Santa Clara, CA, 2015, pp. 2855–2858.
B. Zhou, J. Li, X. Wang, Y. Gu, L. Xu, Y. Hu, L. Zhu, “Online Internet traffic monitoring system using spark streaming,” in Big Data Mining and Analytics, vol. 1, no. 1, pp. 47–56, March 2018.
Biem, E. Bouillet, H. Feng, A. Ranganathan, A. Riabov and O. Verscheure, “Real-Time Traffic Information Management using Stream Computing,” IEEE Data Engineering Bulletin, vol. 33, no. 2, pp. 64–68, 2010
M. M. Rathore, H Son, A. Ahmad, A. Paul, G Jeon, “Real-Time Big Data Stream Processing Using GPU with Spark Over Hadoop Ecosystem”, International Journal of Parallel Programming, vol. 46, Issue 3, pp. 630–646, 2018.
B. Yadranjiaghdam, S. Yasrobi and N. Tabrizi, “Developing a Real-Time Data Analytics Framework for Twitter Streaming Data”, IEEE International Congress on Big Data (BigData Congress), 2017, pp. 329–336.
A. I. Maarala, M. Rautiainen, M. Salmi, S. Pirttikangas and J. Riekki, “Low latency analytics for streaming traffic data with Apache Spark,” 2015 IEEE International Conference on Big Data (Big Data), Santa Clara, CA, 2015, pp. 2855–2858.
Yu, L., Wang, S., and Lai, K.K. “Credit risk assessment with a multistage neural network ensemble learning approach”, Expert Systems with Applications, 2008, 34, pp. 1434–1444.
X. Qiu, L. Zhang, Y. Ren, P. N. Suganthan and G. Amaratunga, “Ensemble deep learning for regression and time series forecasting,” 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL), 2014, pp. 1–6.
Yu, L., Lai, K.K., and Wang, S. “Multistage RBF neural network ensemble learning for exchange rates forecasting”. Neurocomputing, 2008, 71, pp. 3295–3302.
W. Liyong and P. Vateekul, “Improve Traffic Prediction Using Accident Embedding on Ensemble Deep Neural Networks,” 2019 11th International Conference on Knowledge and Smart Technology (KST), 2019, pp. 11–16.
Lai, K. K., Yu, L., Wang, S. Y., and Zhou, L. G. “Credit risk analysis using a reliability-based neural network ensemble model”, Lecture Notes in Computer Science, 2006, 4132, pp. 682–690.
Weissenbacher D, Sarker A, Klein A, O’Connor K, Magge A, Gonzalez-Hernandez G. “Deep neural networks ensemble for detecting medication mentions in tweets”, Journal of the American Medical Informatics Association, 2019, Vol. 26, Issue 12, pp. 1618–1626.
Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., and Muller, P. “Deep Neural Network Ensembles for Time Series Classification”, 2019 International Joint Conference on Neural Networks (IJCNN), 2019, pp. 1–6.
Marushko, E.E., Alexandr Doudkin. “Ensembles of neural networks for forecasting of time series of spacecraft telemetry”, Optical Memory and Neural Networks, 26(1), 2017, pp. 47–54.
L.N. Araujo J.T. Belotti, Taalves, Tadano, H. Siqueira. “Ensemble method based on Artificial Neural Networks to estimate air pollution health risks”, Environmental Modelling and Software, 123, 2019.
Liu, Y., Yao, X. “Ensemble learning via negative correlation”, Neural networks, 12(10), 1999, pp. 1399–1404.
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J. and Wozniak, M. “Ensemble Learning for Data Stream Analysis: a Survey”, Information Fusion 37, 2017, pp. 132–156.
H. Zhang, M. Liptrott, N. Bessis and J. Cheng, “Real-Time Traffic Analysis using Deep Learning Techniques and UAV based Video,” 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2019, pp. 1–5, doi: 10.1109/AVSS.2019.8909879.
Shekhar, H., Setty, S., and Mudenagudi, U. (2016). Vehicular traffic analysis from social media data. 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).
Chicco D., Warrens M.J., Jurman G. 2021. The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Comput. Sci.
Barata, M. and Bernardino, J. Cassandra’s Performance and Scalability Evaluation. DOI: 10.5220/0005980101270134 In Proceedings of the 5th International Conference on Data Management Technologies and Applications (DATA 2016), pages 127–134.
Gupta, A., Tyagi, S., Panwar, N., Sachdeva, S. and Saxena, U. “NoSQL databases: Critical analysis and comparison,” 2017 International Conference on Computing and Communication Technologies for Smart Nation (IC3TSN), 2017, pp. 293–299.
https://www.sciencedirect.com/topics/computer-science/apache-spark
