HIDDEN WEBPAGES DETECTION USING DISTRIBUTED LEARNING AUTOMATA
Keywords:
hidden Web, learning automata, distributed learning automata, DLAAbstract
Webpages directly connected to each other on the Web can be reached easily by following hyperlinks. Those webpages that are not linked by hyperlinks comprises hidden Web and it is challenging to find them. Furthermore, most of webpages in hidden Web are generated dynamically. This paper proposes first time an algorithm to find webpages in hidden Web using distributed learning automata. Learning automata use its self-learning characteristic of taking action based on the action probabilities using pairs. These actions may lead the current webpage to hidden webpages that are generated dynamically. At each stage of the proposed algorithm, we determine the edge that should be chosen to reach webpage of interest. The proposed algorithm is validated on four different websites from dmoz.org. Precision-recall curve and coverage plot in the results section shows the effectiveness of the proposed algorithm.
Downloads
References
He B., Patel M., Zhang Z., and Chang K. C., Accessing the Deep Web : A Survey, 2000.
Bergman M. K., White Paper: The Deep Web: Surfacing Hidden Value, J. Electron. Publ., 7(1), Aug. 2001.
Oommen B. J. and de St. Croix E. V , Graph partitioning using learning automata, IEEE Trans. Comput., 45(2), Feb. 1996, 195–208.
Srikantakumar P. R. and Narendra K. S., A Learning Model for Routing in Telephone Networks, SIAM J. Control Optim., 20 (1), 1982, 34–57.
Oommen B. J., T Roberts. D., Continous learning automata Solutions to the capacity assignment problem, IEEE Transactions on Computers, 49(6), 2000, 608-620.
Meybodi M. R. and Beigy H., New learning automata based algoritbms for adaptation of backpropagation algorithm pararmeters, Int. J. Neural Syst., 12(3), 2002, 45–67.
Meybodi M. R. and Beigy H., A note on learning automata-based schemes for adaptation of BP parameters, Neurocomputing, vol. 48, 2002, 957–974.
Narendra K. S. and Thathachar M. A. L., Learning Automata - A Survey, IEEE Trans. Syst. Man. Cybern., vol. SMC-4 (4), Jul. 1974, 323–334.
Mousavian A., Rezvanian A., and Meybodi M. R., Cellular learning automata based algorithm for solving minimum vertex cover problem, in 2014 22nd Iranian Conference on Electrical Engineering (ICEE), 2014, 996–1000.
Beigy H. and Meybodi M. R., utilizing distributed learning automata to solve stochastic shortest path problems, Int. J. Uncertainty, Fuzziness Knowledge-Based Syst., 14(5), Oct. 2006, 591–615
Broder A. et al., Graph structure in the Web, Comput. Networks, 33(1), Jun. 2000, 309–320.
Khomami M. M. D, Bagherpour N., Sajedi H., and Meybodi M. R., A new distributed learning automata based algorithm for maximum independent set problem, in 2016 Artificial Intelligence and Robotics (IRANOPEN), 2016, 12–17.
