A Novel Negative Sampling Based on Frequency of Relational Association Entities for Knowledge Graph Embedding

Authors

  • Wanhua Cao College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China and Wuhan Digital Engineering Research Institute, Wuhan, 430205, China
  • Yi Zhang College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China and Wuhan Digital Engineering Research Institute, Wuhan, 430205, China
  • Juntao Liu Wuhan Digital Engineering Research Institute, Wuhan, 430205, China
  • Ziyun Rao Wuhan Digital Engineering Research Institute, Wuhan, 430205, China

DOI:

https://doi.org/10.13052/jwe1540-9589.2068

Keywords:

Knowledge Embedding, negative Sampling, frequency of relational association entities, knowledge Graph.

Abstract

Knowledge graph embedding improves the performance of relation extraction and knowledge reasoning by encoding entities and relationships in low-dimensional semantic space. During training, negative samples are usually constructed by replacing the head/tail entity. And the different replacing relationships lead to different accuracy of the prediction results. This paper develops a negative triplets construction framework according to the frequency of relational association entities. The proposed construction framework can fully consider the quantitative of relations and entities in the dataset to assign the proportion of relation and entity replacement and the frequency of the entities associated with each relationship to set reasonable proportions for different relations. To verify the validity of the proposed construction framework, it is integrated into the state-of-the-art knowledge graph embedding models, such as TransE, TransH, DistMult, ComplEx, and Analogy. And both the evaluation criteria of relation prediction and entity prediction are used to evaluate the performance of link prediction more comprehensively. The experimental results on two commonly used datasets, WN18 and FB15K, show that the proposed method improves entity link and triplet classification accuracy, especially the accuracy of relational link prediction.

Downloads

Download data is not yet available.

Author Biographies

Wanhua Cao, College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China and Wuhan Digital Engineering Research Institute, Wuhan, 430205, China

Wanhua Cao is a PhD supervisor at the College of Computer Science and Technology, Harbin Engineering University (HEU). He received his Bachelor’s degree in Huazhong University of Science & Technology (HUST) and Master’s degree in China Ship Research and Development Academy. He is a deputy director, researcher at Wuhan Digital Engineering Institute. His main research interest is decision support.

Yi Zhang, College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China and Wuhan Digital Engineering Research Institute, Wuhan, 430205, China

Yi Zhang is a Ph.D. student at the College of Computer Science and Technology, Harbin Engineering University (HEU) from Autumn 2015. He received his Bachelor’s degree in School of Computer Science & Technology from Huazhong University of Science & Technology (HUST) in 2007, Master’s degree in School of software engineering from University of Science and Technology of China (USTC) in 2011. He is now a senior engineer in Wuhan Digital Engineering Research Institute. He is currently completing a doctorate in Computer Science at the Harbin Engineering University. His research interest covers knowledge computing, knowledge graph, and database technology.

Juntao Liu, Wuhan Digital Engineering Research Institute, Wuhan, 430205, China

Juntao Liu received the BS and MS degrees both in Computer Science from Ordnance Engineering College, Shijiazhuang, China, in 2002 and 2005, respectively, and PhD degree in Communication and Information Systems from Huazhong University of Science & Technology (HUST), Wuhan, China, in 2014. He is now a senior engineer in Wuhan Digital Engineering Research Institute. His research interests include data mining, machine learning and computer vision.

Ziyun Rao, Wuhan Digital Engineering Research Institute, Wuhan, 430205, China

Ziyun Rao is a Master student at Wuhan Digital Engineering Institute. She received her Bachelor’s degree from Huazhong University of Science & Technology (HUST). Her research interest covers knowledge graph and recommendation system.

References

Y. Dai, S. Wang, N. Xiong, W. Guo, ‘A Survey on Knowledge Graph Embedding: Approaches, Applications and Benchmarks’, Electronics, 2020.

J. Liu, ‘Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors’, Information Processing & Management, 2021, 58(3):102522.

B. Shao, X. Li, G. Bian, ‘A Survey of Research Hotspots and Frontier Trends of Recommendation Systems from the Perspective of Knowledge Graph’, Expert Systems with Applications, 2020(165):113764.

X. Li, H. Zang, X. Yu, et al. ‘On improving knowledge graph facilitated simple question answering system’, Neural Computing and Applications, 2021(2).

K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, ‘Freebase: A collaboratively created graph database for structuring human knowledge’, in Proc. of ACM SIGMOD Int. Conf. on Manage. Data, 2008, pp. 1247–1250.

F.M. Suchanek, G. Kasneci, and G. Weikum, ‘YAGO: A core of semantic knowledge’, in Proc. 16th Int. Conf. on World Wide Web, 2007, pp. 697–706.

B. Yang, WT. Yih, X. He, J. Gao, and L. Deng, ‘Embedding entities and relations for learning and inference in knowledge bases’, in Proc. Int. Conf. Learn. Represent., 2015.

T. Trouillon, J. Welbl, S. Riedel, E. Gaussier, and G. Bouchard, ‘Complex embeddings for simple link prediction’, in Proc. 33rd Int. Conf. Mach. Learn., 2016, pp. 2071–2080.

H. Liu, Y. Wu, and Y. Yang, ‘Analogical inference for multirelational embeddings,’ in Proc. 34th Int. Conf. Mach. Learn., 2017, pp. 2168–2178.

A. Bordes, X. Glorot, J. Weston, and Y. Bengio, ‘A semantic matching energy function for learning with multi-relational data’, Mach. Learn., vol. 94, no. 2, pp. 233–259, 2014.

S. Guo, Q. Wang, L. Wang, B. Wang, and L. Guo, ‘Semantically smooth knowledge graph embedding’, in Proc. 53rd Annu. Meeting Assoc. Comput. Linguistics 7th Int. Joint Conf. Natural Language Process., 2015, pp. 84–94.

A. Bordes, N. Usunier, A. García-Durán, J. Weston, and O. Yakhnenko, ‘Translating embeddings for modeling multi-relational data’, in Adv. Neural Inf. Process. Syst., 2013, pp. 2787–2795.

Z. Wang, J. Zhang, J. Feng, and Z. Chen, ‘Knowledge graph embedding by translating on hyperplanes’, in Proc. 28th AAAI Conf. Artif. Intell., 2014, pp. 1112–1119.

Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu, “Learning entity and relation embeddings for knowledge graph completion,” in Proc. 29th AAAI Conf. Artif. Intell., 2015, pp. 2181–2187.

G. Ji, S. He, L. Xu, K. Liu, and J. Zhao, ‘Knowledge graph embedding via dynamic mapping matrix’, in Proc. 53rd Annu. Meeting Assoc. Comput. Linguistics 7th Int. Joint Conf. Natural Language Process., 2015, pp. 687–696.

H. Xiao, M. Huang, Y. Hao, and X. Zhu, ‘TransA: An adaptive approach for knowledge graph embedding’, in arXiv:1509.05490, 2015.

S. He, K. Liu, G. Ji, and J. Zhao, ‘Learning to represent knowledge graphs with Gaussian embedding’, in Proc. 24th ACM Int. Conf. Inf. Knowl. Manage., 2015, pp. 623–632.

G. Ji, K. Liu, S. He, and J. Zhao, ‘Knowledge graph completion with adaptive sparse transfer matrix’, in Proc. 30th AAAI Conf. Artif. Intell., 2016, pp. 985–991.

H. Xiao, M. Huang, Y. Hao, et al. ‘TransG : A Generative Mixture Model for Knowledge Graph Embedding’, Computer Science, 2015.

A. Bordes, X. Glorot, J. Weston, ‘Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing’, International Conference on Artificial Intelligence & Statistics, 2012.

Downloads

Published

2021-10-18

Issue

Section

Articles