A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems

Authors

  • Frank van Harmelen Department of Computer Science, Vrije Universiteit Amsterdam, Netherlands
  • Annette ten Teije Department of Computer Science, Vrije Universiteit Amsterdam, Netherlands

Keywords:

Hybrid systems, neurosymbolic systems, knowledge representation, machine learning, design patters

Abstract

We propose a set of compositional design patterns to describe a large variety of systems that combine statistical techniques from machine learning with symbolic techniques from knowledge representation. As in other areas of computer science (knowledge engineering, software engineering, ontology engineering, process mining and others), such design patterns help to systematize the literature, clarify which combinations of techniques serve which purposes, and encourage re-use of software components. We have validated our set of compositional design patterns against a large body of recent literature.

Downloads

Download data is not yet available.

Author Biographies

Frank van Harmelen, Department of Computer Science, Vrije Universiteit Amsterdam, Netherlands

Frank van Harmelen is professor of Knowledge Representation and Reasoning at the VU University Amsterdam, and adjunct professor at Wuhan University and Wuhan University of Science and Technology. He played a leading role in the development of the Semantic Web, which aims to make data on the web semantically interpretable by machines through formal representations. He was a contributor to the Web Ontology Language OWL, now a standard in worldwide commercial use, and the basis for an entire research community. He is one of the architects of the semantic storage engine Sesame (now RDF4J). This work received the 10-year impact award of the Semantic Web community. He co-authored the Semantic Web Primer, the first text book on the semantic web (now translated into 5 languages), and he co-edited the standard reference work in his field (The Handbook of Knowledge Representation). He is a member of the Royal Netherlands Academy of Arts and Sciences, the Royal Holland Academy of Sciences, and Academia Europea, and fellow of the European Association for Artificial Intelligence.

Annette ten Teije, Department of Computer Science, Vrije Universiteit Amsterdam, Netherlands

Annette ten Teije is an associate professor at the Vrije Universiteit Amsterdam. Her interests are in knowledge modelling, representation and reasoning in particular in the medical domain. She earned a PhD (1997) from the University of Amsterdam (SWI) for her thesis entitled “Automated configuration of problem solving methods in diagnosis”. She was involved in a number of EU-funded projects IBROW project under the FET-O programme, Protocure-II project, concerned with formal modelling and verification of medical guidelines and protocols, WS-DIAMOND FET-Open project concerned with self-healing web-services, FP7-ICT EURECA project concerned with enabling information re-use by linking clinical research and clinical care. She was program co-chair for the 18th International Conference on Knowledge Engineering and Knowledge Management (2012), and was the general chair of the 16th conference on Artificial Intelligence in Medicine.

References

Ellen Agerbo and Aino Cornils. “How to Preserve the Benefits of Design Patterns”. In: Proceedings of the 1998 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications (OOPSLA ’98), Vancouver, British Columbia, Canada, October 18–22, 1998. Ed. by Bjørn N. Freeman-Benson and Craig Chambers. ACM, 1998, pp. 134–143. doi: 10.1145/286936.286952. URL: https://doi.org/10.1145/286936.286952.

Muhammad Nabeel Asim et al. “A survey of ontology learning techniques and applications”. In: Database 2018 (2018), bay101.

Stephen H. Bach et al. “Hinge-Loss Markov Random Fields and Probabilistic Soft Logic”. In: Journal of Machine Learning Research 18 (2017), 109:1–109:67. URL: http://jmlr.org/papers/v18/15-631.html.

Stephan Baier, Yunpu Ma, and Volker Tresp. “Improving Visual Relationship Detection Using Semantic Modeling of Scene Descriptions”. In: The Semantic Web – ISWC 2017 – 16th International Semantic Web Conference, Vienna, Austria, October 21–25, 2017, Proceedings, Part I. Ed. by Claudia d’Amato et al. Vol. 10587. Lecture Notes in Computer Science. Springer, 2017, pp. 53–68. ISBN: 978-3-319-68287-7. doi: 10.1007/978-3-319-68288-4_4. URL: https://doi.org/10.1007/978-3-319-68288-4%5C 4.

A. Barron, J. Rissanen, and Bin Yu. “The minimum description length principle in coding and modeling”. In: IEEE Transactions on Information Theory 44.6 (Oct. 1998), pp. 2743–2760. ISSN: 0018-9448. doi: 10.1109/18.720554.

Peter W. Battaglia et al. “Relational inductive biases, deep learning, and graph networks”. In: CoRR abs/1806.01261 (2018). arXiv: 1806. 01261. URL: http://arxiv.org/abs/1806.01261.

R. P. Jagadeesh Chandra Bose and Wil M. P. van der Aalst. “Abstractions in Process Mining: A Taxonomy of Patterns”. In: Business Process Management, 7th International Conference, BPM 2009, Ulm, Germany, September 8–10, 2009. Proceedings. Ed. by Umeshwar Dayal et al. Vol. 5701. Lecture Notes in Computer Science. Springer, 2009, pp. 159–175. doi: 10.1007/978-3-642-03848-8 12. URL: https://doi.org/10.1007/978-3-642-03848-8%5C 12.

R. P. Jagadeesh Chandra Bose and Wil M. P. van der Aalst. “Abstractions in Process Mining: A Taxonomy of Patterns”. In: Business Process Management, 7th International Conference, BPM 2009, Ulm, Germany, September 8–10, 2009. Proceedings. Ed. by Umeshwar Dayal et al. Vol. 5701. Lecture Notes in Computer Science. Springer, 2009, pp. 159–175. doi: 10.1007/978-3-642-03848-8 12. URL: https://doi.org/10.1007/978-3-642-03848-8%5C 12.

Léon Bottou. “From Machine Learning to Machine Reasoning”. In: CoRR abs/1102.1808 (2011). arXiv: 1102.1808. URL: http://arxiv.org/abs/1102.1808.

Zied Bouraoui, Shoaib Jameel, and Steven Schockaert. “Inductive Reasoning about Ontologies Using Conceptual Spaces”. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4–9, 2017, San Francisco, California, USA. Ed. by Satinder P. Singh and Shaul Markovitch. AAAI Press, 2017, pp. 4364–4370. URL: http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14916.

Jiaoyan Chen et al. “Knowledge-Based Transfer Learning Explanation”. In: Principles of Knowledge Representation and Reasoning: Proceedings of the Sixteenth International Conference, KR 2018, Tempe, Arizona, 30 October – 2 November 2018. Ed. by Michael Thielscher, Francesca Toni, and Frank Wolter. AAAI Press, 2018, pp. 349–358. URL: https://aaai.org/ocs/index.php/KR/KR18/paper/view/18054.

Hui-Qing Chong, Ah-Hwee Tan, and Gee Wah Ng. “Integrated cognitive architectures: a survey”. In: Artif. Intell. Rev. 28.2 (2007), pp. 103–130. doi: 10.1007/s10462-009-9094-9. URL: https://doi.org/10.1007/s10462-009-9094-9.

Rudi Cilibrasi and Paul M. B. Vitányi. “Clustering by compression”. In: IEEE Trans. Information Theory 51.4 (2005), pp. 1523–1545. doi: 10.1109/TIT.2005.844059. URL: https://doi.org/10.1109/TIT.2005.844059.

Stefania Costantini. “Meta-reasoning: A Survey”. In: Computational Logic: Logic Programming and Beyond, Essays in Honour of Robert A. Kowalski, Part II. Ed. by Antonis C. Kakas and Fariba Sadri. Vol. 2408. Lecture Notes in Computer Science. Springer, 2002, pp. 253–288. doi: 10.1007/3-540-45632-5 11. URL: https://doi.org/10.1007/3-540-45632-5%5C_11.

Adnan Darwiche. “Human-level Intelligence or Animal-like Abil-ities?” In: Commun. ACM 61.10 (Sept. 2018), pp. 56–67. ISSN: 0001-0782. doi: 10.1145/3271625. URL: http://doi.acm.org.vu-nl.idm.oclc.org/10.1145/3271625.

Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. “Lifted Rule Injection for Relation Embeddings”. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1–4, 2016. Ed. by Jian Su, Xavier Carreras, and Kevin Duh. The Association for Computational Linguistics, 2016, pp. 1389–1399. URL: http://aclweb.org/anthology/D/D16/D16-1146.pdf.

Ivan Donadello, Luciano Serafini, and Artur S. d’Avila Garcez. “Logic Tensor Networks for Semantic Image Interpretation”. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017. Ed. by Carles Sierra. ijcai.org, 2017, pp. 1596–1602. ISBN: 978-0-9992411-0-3. doi: 10.24963/ijcai.2017/221. URL: https://doi.org/10.24963/ijcai.2017/221.

Monireh Ebrahimi et al. “Reasoning over RDF Knowledge Bases using Deep Learning”. In: CoRR abs/1811.04132 (2018). arXiv: 1811.04132. URL: http://arxiv.org/abs/1811.04132.

Erich Gamma. “Design Patterns – Past, Present and Future”. In: The Future of Software Engineering. Ed. by Sebastian Nanz. Springer, 2010, p. 72. doi: 10.1007/978-3-642-15187-3 4. URL: https://doi.org/10.1007/978-3-642-15187-3%5C 4.

Erich Gamma et al. Design Patterns: Elements of Reusable Object-Oriented Software. 1st ed. Addison-Wesley Professional, 1994. ISBN: 0201633612.

Aldo Gangemi and Valentina Presutti. “Ontology Design Patterns”. In: Handbook on Ontologies. Ed. by Steffen Staab and Rudi Studer. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 221–243. doi: 10.1007/978-3-540-92673-3 10. URL: https://doi.org/10.1007/978-3-540-92673-3 10.

Marta Garnelo, Kai Arulkumaran, and Murray Shanahan. “Towards Deep Symbolic Reinforcement Learning”. In: CoRR abs/1609.05518 (2016). arXiv: 1609.05518. URL: http://arxiv.org/abs/1609.05518.

Hector Geffner. “Model-free, Model-based, and General Intelligence”. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13–19, 2018, Stockholm, Sweden. Ed. by Jérôme Lang. ijcai.org, 2018, pp. 10–17. ISBN: 978-0-9992411-2-7. doi: 10.24963/ijcai.2018/2. URL: https://doi.org/10.24963/ijcai.2018/2.

Lise Getoor. “Probabilistic Soft Logic: A Scalable Approach for Markov Random Fields over Continuous-Valued Variables – (Abstract of Keynote Talk)”. In: Theory, Practice, and Applications of Rules on the Web – 7th International Symposium, RuleML 2013, Seattle, WA, USA, July 11–13, 2013. Proceedings. Ed. by Leora Morgenstern et al. Vol. 8035. Lecture Notes in Computer Science. Springer, 2013, p. 1. ISBN: 978-3-642-39616-8. doi: 10.1007/978-3-642-39617-5 1. URL: https://doi.org/10.1007/978-3-642-39617-5%5C 1.

Lise Getoor. “Statistical Relational Learning: Unifying AI and DB Perspectives on Structured Probabilistic Models”. In: Proceedings of the 36th ACM SIGMOD-SIGACT-SIGAI Sympo-sium on Principles of Database Systems, PODS 2017, Chicago, IL, USA, May 14–19, 2017. Ed. by Emanuel Sallinger, Jan Van den Bussche, and Floris Geerts. ACM, 2017, p. 183. ISBN: 978-1-4503-4198-1. doi: 10.1145/3034786. 3056450. URL: https://doi.org/10.1145/3034786.3056450.

Frank van Harmelen, Vladimir Lifschitz, and Bruce W. Porter, eds. Handbook of Knowledge Representation. Vol. 3. Foundations of Artificial Intelligence. Elsevier, 2008. ISBN: 978-0-444-52211-5. URL: http://www.sciencedirect.com/science/bookseries/15746526/3.

Patrick Hohenecker and Thomas Lukasiewicz. “Deep Learning for Ontology Reasoning”. In: CoRR abs/1705.10342 (2017). arXiv: 1705. 10342. URL: http://arxiv.org/abs/1705.10342.

Rodrigo Toro Icarte et al. “Teaching Multiple Tasks to an RLAgent using LTL”. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2018, Stockholm, Sweden, July 10–15, 2018. Ed. by Elisabeth André et al. International Foundation for Autonomous Agents and Multiagent Systems Richland, SC, USA / ACM, 2018, pp. 452–461. URL: http://dl.acm.org/citation.cfm?id=3237452.

Katsumi Inoue, Hayato Ohwada, and Akihiro Yamamoto. “Special issue on inductive logic programming”. In: Machine Learning 106 (2017), pp. 1863–1865.

Daphne Koller and Nir Friedman. Probabilistic Graphical Models – Principles and Techniques. MIT Press, 2009. ISBN: 978-0-262-01319-2. URL: http://mitpress.mit.edu/catalog/item/default.asp? ttype=2%5C&tid=11886.

Stasinos Konstantopoulos and Angelos Charalambidis. “Formulating description logic learning as an Inductive Logic Programming task”. In: FUZZ-IEEE 2010, IEEE International Conference on Fuzzy Systems, Barcelona, Spain, 18–23 July, 2010, Proceedings. IEEE, 2010, pp. 1–7. doi: 10.1109/FUZZY.2010.5584417. URL: https://doi.org/10.1109/FUZZY.2010.5584417.

Reinier Kop et al. “Predictive modeling of colorectal cancer using a dedicated pre-processing pipeline on routine electronic medical records”. In: Comp. in Bio. and Med. 76 (2016), pp. 30–38.

Brenden M. Lake et al. “Building Machines That Learn and Think Like People”. In: The Behavioral and brain sciences 40 (2017), e253.

Gary Marcus. “Deep Learning: A Critical Appraisal”. In: CoRR abs/1801.00631 (2018). arXiv: 1801.00631. URL: http://arxiv.org/ abs/1801.00631.

Maximilian Nickel et al. “A Review of Relational Machine Learning for Knowledge Graphs”. In: Proceedings of the IEEE 104.1 (2016), pp. 11–33. doi: 10.1109/JPROC.2015.2483592. URL: https://doi.org/10.1109/JPROC.2015.2483592.

Ankur Padia, David Martin, and Peter F. Patel-Schneider. “Automating Class/Instance Representational Choices in Knowledge Bases”. In: Knowledge Engineering and Knowledge Management – 21st International Conference, EKAW 2018, Nancy, France, November 12–16, 2018, Proceedings. Ed. by Catherine Faron-Zucker et al. Vol. 11313. Lecture Notes in Computer Science. Springer, 2018, pp. 273–288. doi: 10.1007/978-3-030-03667-6 18. URL: https://doi.org/10.1007/978-3-030-03667-6%5C 18.

Heiko Paulheim. “Knowledge graph refinement: A survey of approaches and evaluation methods”. In: Semantic Web 8.3 (2017), pp. 489–508. doi: 10.3233/SW-160218. URL: https://doi.org/10.3233/SW-160218.

Judea Pearl. “Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution”. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, Marina Del Rey, CA, USA, February 5–9, 2018. Ed. by Yi Chang et al. ACM, 2018, p. 3. doi: 10.1145/3159652.3176182. URL: https://doi.org/10.1145/3159652.3176182.

Don Perlis et al. “The Internal Reasoning of Robots”. In: Proceedings of the Thirteenth International Symposium on Commonsense Reasoning, COMMONSENSE 2017, London, UK, November 6–8, 2017. Ed. by Andrew S. Gordon, Rob Miller, and György Turán. Vol. 2052. CEUR Workshop Proceedings. CEUR-WS.org, 2017. URL:http://ceur-ws.org/Vol-2052/paper16.pdf.

Jay Pujara et al. “Using Semantics and Statistics to Turn Data into Knowledge”. In: AI Magazine 36.1 (2015), pp. 65–74. URL: http:// www.aaai.org/ojs/index.php/aimagazine/article/view/2568.

Matthew Richardson and Pedro M. Domingos. “Markov logic networks”. In: Machine Learning 62.1-2 (2006), pp. 107–136. doi: 10.1007/s10994-006-5833-1. URL: https://doi.org/10.1007/s10994-006-5833-1.

Fabrizio Riguzzi, Elena Bellodi, and Riccardo Zese. “A History of Probabilistic Inductive Logic Programming”. In: Front. Robotics and AI 2014 (2014).

Tim Rocktäschel and Sebastian Riedel. “End-to-end Differentiable Proving”. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA. Ed. by Isabelle Guyon et al. 2017, pp. 3791–3803. URL: http://papers.nips.cc/paper/6969-end-to-end-differentiable-proving.

Md. Kamruzzaman Sarker et al. “Explaining Trained Neural Networks with Semantic Web Technologies: First Steps”. In: Proceedings of the Twelfth International Workshop on Neural-Symbolic Learning and Reasoning, NeSy 2017, London, UK, July 17–18, 2017. Ed. by Tarek R. Besold, Artur S. d’Avila Garcez, and Isaac Noble. Vol. 2003. CEUR Workshop Proceedings. CEUR-WS.org, 2017. URL: http://ceur-ws.org/Vol-2003/NeSy17%5C paper4.pdf.

Guus Schreiber et al. “CommonKADS: A Comprehensive Methodology for KBS Development”. In: IEEE Expert 9.6 (1994), pp. 28–37. doi: 10.1109/64.363263. URL: https://doi.org/10.1109/64.363263.

Guus Schreiber et al. Knowledge Engineering and Management The CommonKADS Approach. MIT Press, 1999.

Steffen Staab and Rudi Studer, eds. Handbook on Ontologies. International Handbooks on Information Systems. Springer, 2009. ISBN: 978-3-540-70999-2. doi: 10.1007/978-3-540-92673-3. URL: https://doi.org/10.1007/978-3-540-92673-3.

Peter Struss. “Model-based Problem Solving”. In: Handbook of Knowledge Representation. Ed. by Frank van Harmelen, Vladimir Lifschitz, and Bruce W. Porter. Vol. 3. Foundations of Artificial Intelligence. Elsevier, 2008, pp. 395–465. doi: 10.1016/S1574-6526(07)03010-6. URL: https://doi.org/10.1016/S1574-6526(07) 03010-6.

Ilaria Tiddi, Mathieu d’Aquin, and Enrico Motta. “Data Patterns Explained with Linked Data”. In: Machine Learning and Knowledge Discovery in Databases – European Conference, ECML PKDD 2015, Porto, Portugal, September 7– 11, 2015, Proceedings, Part III. Ed. by Albert Bifet et al. Vol. 9286. Lecture Notes in Computer Science. Springer, 2015, pp. 271–275. doi: 10.1007/978-3-319-23461-8 28. URL: https://doi.org/10.1007/978-3-319-23461-8%5C 28.

Q. Wang et al. “Knowledge Graph Embedding: A Survey of Approaches and Applications”. In: IEEE Transactions on Knowledge and Data Engineering 29.12 (Dec. 2017), pp. 2724–2743. ISSN: 1041-4347. doi: 10. 1109/TKDE.2017.2754499.

Daniel S. Weld and Gagan Bansal. “Intelligible Artificial Intelligence”. In: CoRR abs/1803.04263 (2018). arXiv: 1803.04263. URL: http://arxiv.org/abs/1803.04263.

Wilson Wong, Wei Liu, and Mohammed Bennamoun. “Ontology learning from text: A look back and into the future”. In: ACM Comput. Surv. 44.4 (2012), 20:1–20:36. doi: 10.1145/2333 112.2333115. URL: https://doi.org/10.1145/2333112.2333115.

Jingyi Xu et al. “A Semantic Loss Function for Deep Learning with Symbolic Knowledge”. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10–15, 2018. Ed. by Jennifer G. Dy and Andreas Krause. Vol. 80. JMLR Workshop and Conference Proceedings. JMLR.org, 2018, pp. 5498–5507. URL: http://proceedings.mlr.press/v80/xu18h.html.

Jihong Yan et al. “A retrospective of knowledge graphs”. In: Frontiers of Computer Science 12.1 (Feb. 2018), pp. 55–74. ISSN: 2095-2236. doi: 10.1007/s11704-016-5228-9. URL: https://doi.org/10.1007/s11704-016-5228-9.

Fan Yang, Zhilin Yang, and William W. Cohen. “Differentiable Learning of Logical Rules for Knowledge Base Reasoning”. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA. Ed. by Isabelle Guyon et al. 2017, pp. 2316–2325. URL: http://papers.nips.cc/paper/6826-differentiable-learning-of-logical-rules-for-knowledge-base-reasoning.

Downloads

Published

2019-01-01

How to Cite

Frank van Harmelen, & Annette ten Teije. (2019). A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems. Journal of Web Engineering, 18(1-3). Retrieved from https://journals.riverpublishers.com/index.php/JWE/article/view/3175

Issue

2019: Vol 18 Iss 1-3

Section

Articles