• PATRICK PHILIPP Institute AIFB, Karlsruhe Institute of Technology Karlsruhe, Germany
  • MARIA MALESHKOVA Institute AIFB, Karlsruhe Institute of Technology Karlsruhe, Germany
  • ACHIM RETTINGER Institute AIFB, Karlsruhe Institute of Technology Karlsruhe, Germany
  • DARKO KATIC HIS, Karlsruhe Institute of Technology Karlsruhe, Germany


Sequential Decision Making, Linked APIs, Meta Learning, Medical Assistance, Entity Linking


Current developments in the medical domain, not unlike many other sectors, are marked by the growing digitalization of data, including patient records, study results, clinical guidelines or imagery. This trend creates the opportunity for the development of innovative decision support systems to assist physicians in making a diagnosis or preparing a treatment plan. Similar conditions hold for the Web, where massive amounts of raw text are to be processed and interpreted automatically, e.g. to eventually add new information to a knowledge base. To this end, complex tasks need to be solved, requiring one or more interpretation algorithms (e.g. image- or natural language processors) to be chosen and executed based on heterogeneous data. We, therefore, propose the first approach to a semantic framework for sequential decision making and develop the foundations of a Linked agent who executes interpretation algorithms available as Linked APIs [43] on a data-driven, declarative basis [45] by integrating structured knowledge formalized with the Resource Description Framework (RDF), and having access to meta components for planning and learning from experience. We evaluate our framework based on automatically processing brain images, the ad-hoc combination of surgical phase recognition algorithms and experiential learning to optimally pipeline entity linking approaches.


Download data is not yet available.


Albrecht, M., Donnelly, P., Bui, P., Thain, D.: Makeflow: A portable abstraction for data intensive

computing on clusters, clouds, and grids. In: Proceedings of the 1st ACM SIGMOD Workshop on

Scalable Workflow Execution Engines and Technologies. pp. 1:1–1:13. SWEET ’12, ACM (2012)

Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.G.: Dbpedia: A nucleus for a

web of open data. In: The Semantic Web, 6th International Semantic Web Conference, 2nd Asian

Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11-15, 2007. pp.

–735 (2007)

Basave, A.E.C., Rizzo, G., Varga, A., Rowe, M., Stankovic, M., Dadzie, A.: Making sense of

microposts (#microposts2014) named entity extraction & linking challenge. In: Proceedings of

the 4th Workshop on Making Sense of Microposts co-located with the 23rd International World

Wide Web Conference (WWW 2014), Seoul, Korea, April 7th, 2014. pp. 54–60 (2014)

Benjamins, V.R., Pierret-Golbreich, C.: Assumptions of problem-solving methods. In: European

Knowledge Acquisition Workshop, EKAW. pp. 1–16 (1996)

Beygelzimer, A., Riabov, A., Sow, D., Turaga, D.S., Udrea, O.: Big data exploration via automated

orchestration of analytic workflows. In: ICAC. pp. 153–158. San Jose, CA (2013)

Blankenberg, D., Kuster, G.V., Coraor, N., Ananda, G., Lazarus, R., Mangan, M., Nekrutenko,

A., Taylor, J.: Galaxy: a web-based genome analysis tool for experimentalists. Current protocols

in molecular biology pp. 19–10 (2010)

Christensen, E., Curbera, F., Meredith, G., Weerawarana, S., et al.: Web services description

language (wsdl) 1.1 (2001)

Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P., Mayani, R., Chen,

W., da Silva, R.F., Livny, M., Wenger, R.K.: Pegasus, a workflow management system for science

automation. Future Generation Comp. Syst. 46, 17–35 (2015)

Doshi, P., Goodwin, R., Akkiraju, R., Verma, K.: Dynamic workflow composition using markov

decision processes. In: ICWS. pp. 576–582 (2004)

Fahringer, T., Prodan, R., Duan, R., Hofer, J., Nadeem, F., Nerieri, F., Podlipnig, S., Qin, J.,

Siddiqui, M., Truong, H.L., Villazon, A., Wieczorek, M.: ASKALON: A Development and Grid

Computing Environment for Scientific Workflows, pp. 450–471. Springer London (2007)

Fensel, D., Motta, E., van Harmelen, F., Benjamins, V.R., Crubézy, M., Decker, S., Gaspari, M.,

Groenboom, R., Grosso, W.E., Musen, M.A., Plaza, E., Schreiber, G., Studer, R., Wielinga, B.J.:

The unified problem-solving method development language UPML. Knowl. Inf. Syst. 5(1), 83–131


Feurer, M., Klein, A., Eggensperger, K., Springenberg, J.T., Blum, M., Hutter, F.: Efficient and

robust automated machine learning. In: NIPS. pp. 2962–2970 (2015)

Fikes, R., Nilsson, N.J.: STRIPS: A new approach to the application of theorem proving to problem

solving. Artif. Intell. 2(3/4), 189–208 (1971)

Filguiera, R., Klampanos, I., Krause, A., David, M., Moreno, A., Atkinson, M.: dispel4py: A

python framework for data-intensive scientific computing. In: Proceedings of the 2014 International

Workshop on Data Intensive Scalable Computing Systems. pp. 9–16. DISCS ’14, IEEE Press (2014)

Finkel, J.R., Grenager, T., Manning, C.D.: Incorporating non-local information into information

extraction systems by gibbs sampling. In: ACL 2005, 43rd Annual Meeting of the Association

for Computational Linguistics, Proceedings of the Conference, 25-30 June 2005, University of

Michigan, USA (2005)

Garijo, D., Gil, Y.: A new approach for publishing workflows: abstractions, standards, and linked

data. In: WORKS. pp. 47–56 (2011)

Gemmeke, P., Maleshkova, M., Philipp, P., Götz, M., Weber, C., Kämpgen, B., Zelzer, S., Maier-

Hein, K., Rettinger, A.: Using linked data and web apis for automating the pre-processing of

medical images. COLD (ISWC) (2014)

Gil, Y., Gonzalez-Calero, P.A., Kim, J., Moody, J., Ratnakar, V.: A semantic framework for automatic

generation of computational workflows using distributed data and component catalogues.

Journal of Experimental & Theoretical Artificial Intelligence 23(4), 389–467 (2011)

Hendler, J.A., Tate, A., Drummond, M.: AI planning: Systems and techniques. AI Magazine 11(2),

–77 (1990)

Hoffart, J., Yosef, M.A., Bordino, I., Fürstenau, H., Pinkal, M., Spaniol, M., Taneva, B., Thater,

S., Weikum, G.: Robust disambiguation of named entities in text. In: Proceedings of the 2011

Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July

, John McIntyre Conference Centre, Edinburgh, UK, A meeting of SIGDAT, a Special Interest

Group of the ACL. pp. 782–792 (2011)

Jain, A., Ong, S.P., Chen, W., Medasani, B., Qu, X., Kocher, M., Brafman, M., Petretto, G.,

Rignanese, G., Hautier, G., Gunter, D.K., Persson, K.A.: Fireworks: a dynamic workflow system

designed for high-throughput applications. Concurrency and Computation: Practice and Experience

(17), 5037–5059 (2015)

Katic, D., Wekerle, A.L., Gärtner, F., Kenngott, H.G., Müller-Stich, B.P., Dillmann, R., Speidel,

S.: Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative contextaware

assistance. In: Proc. of Information Processing in Computer Assisted Interventions (IPCAI).

pp. 158–167 (2014)

Klusch, M., Gerber, A., Schmidt, M.: Semantic web service composition planning with owls-xplan.

In: Int. AAAI Fall Symposium on Agents and the Semantic Web. pp. 55–62 (2005)

Klusch, M.: Semantic web service coordination. CASCOM: Intelligent service coordination in the

semantic web pp. 59–104 (2008)

Kopecky, J., Gomadam, K., Vitvar, T.: hrests: An html microformat for describing restful web services.

In: Web Intelligence and Intelligent Agent Technology, 2008. WI-IAT’08. IEEE/WIC/ACM

International Conference on. vol. 1, pp. 619–625. IEEE (2008)

Kopecký, J., Vitvar, T., Bournez, C., Farrell, J.: SAWSDL: semantic annotations for WSDL and

XML schema. IEEE Internet Computing 11(6), 60–67 (2007)

Martin, D.L., Burstein, M.H., McDermott, D.V., McIlraith, S.A., Paolucci, M., Sycara, K.P.,

McGuinness, D.L., Sirin, E., Srinivasan, N.: Bringing semantics to web services with OWL-S.

World Wide Web 10(3), 243–277 (2007)

Mendes, P.N., Jakob, M., García-Silva, A., Bizer, C.: Dbpedia spotlight: shedding light on the

web of documents. In: Proceedings the 7th International Conference on Semantic Systems, ISEMANTICS

, Graz, Austria, September 7-9, 2011. pp. 1–8 (2011)

Nguyen, P., Hilario, M., Kalousis, A.: Using meta-mining to support data mining workflow planning

and optimization. J. Artif. Intell. Res. (JAIR) 51, 605–644 (2014)

Oinn, T., Greenwood, M., Addis, M., Alpdemir, M.N., Ferris, J., Glover, K., Goble, C., Goderis,

A., Hull, D., Marvin, D., Li, P., Lord, P., Pocock, M.R., Senger, M., Stevens, R., Wipat, A., Wroe,

C.: Taverna: Lessons in creating a workflow environment for the life sciences: Research articles.

Concurr. Comput. : Pract. Exper. 18(10), 1067–1100 (Aug 2006)

van Otterlo, M.: The logic of adaptive behavior: knowledge representation and algorithms for

adaptive sequential decision making under uncertainty in first-order and relational domains, vol.

Ios Press (2009)

Philipp, P., Katic, D., Maleshkova, M., Rettinger, A., Speidel, S., Wekerle, A.L., Kämpgen, B.,

Kenngott, H., Studer, R., Dillmann, R., Müller, B.: Towards cognitive pipelines of medical assistance

algorithms. In: Proc. Computer Assisted Radiology and Surgery (CARS) (2015)

Philipp, P., Maleshkova, M., Götz, M., Weber, C., Kämpgen, B., Zelzer, S., Maier-Hein, K., Rettinger,

A.: Automatisierte verarbeitung von bildverarbeitungsalgorithmen mit semantischen technologien.

In: Bildverarbeitung für die Medizin (BVM). pp. 263–268 (2015)

Puppe, F.: Systematic introduction to expert systems - knowledge representations and problemsolving

methods. Springer (1993)

Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John

Wiley & Sons, Inc., New York, NY, USA, 1st edn. (1994)

Rice, J.R.: The algorithm selection problem. Advances in Computers 15, 65–118 (1976)

Roman, D., Keller, U., Lausen, H., de Bruijn, J., Lara, R., Stollberg, M., Polleres, A., Feier, C.,

Bussler, C., Fensel, D.: Web service modeling ontology. Applied Ontology 1(1), 77–106 (2005)

Ruiz, P., Poibeau, T.: Combining open source annotators for entity linking through weighted

voting. In: Proceedings of* SEM 2015. Fourth Joint Conference on Lexical and Computational

Semantics (2015)

Russell, S.J., Norvig, P.: Artificial Intelligence - A Modern Approach (3. internat. ed.). Pearson

Education (2010)

Sirin, E., Hendler, J.A., Parsia, B.: Semi-automatic composition of web services using semantic

descriptions. In: WSMAI. pp. 17–24 (2003)

Sirin, E., Parsia, B., Wu, D., Hendler, J.A., Nau, D.S.: HTN planning for web service composition

using SHOP2. J. Web Sem. 1(4), 377–396 (2004)

Speck, R., Ngomo, A.N.: Ensemble learning for named entity recognition. In: The Semantic Web -

ISWC 2014 - 13th International Semantic Web Conference, Riva del Garda, Italy, October 19-23,

Proceedings, Part I. pp. 519–534 (2014)

Speiser, S., Harth, A.: Integrating linked data and services with linked data services. In: The

Semantic Web: Research and Applications, pp. 170–184. Springer (2011)

Stadtmüller, S., Norton, B.: Scalable discovery of linked apis. IJMSO 8(2), 95–105 (2013)

Stadtmüller, S., Speiser, S., Harth, A., Studer, R.: Data-fu: A language and an interpreter for

interaction with read/write linked data. In: WWW. pp. 1225–1236 (2013)

Strehl, A.L., Littman, M.L.: Online linear regression and its application to model-based reinforcement

learning. In: Advances in Neural Information Processing Systems 20, Proceedings of the

Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British

Columbia, Canada, December 3-6, 2007. pp. 1417–1424 (2007)

Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings

of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada,

May 8-12, 2007. pp. 697–706 (2007)

Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-weka: combined selection and

hyperparameter optimization of classification algorithms. In: SIGKDD. pp. 847–855 (2013)

Usbeck, R., Ngomo, A.N., Röder, M., Gerber, D., Coelho, S.A., Auer, S., Both, A.: AGDISTIS -

graph-based disambiguation of named entities using linked data. In: The Semantic Web - ISWC

- 13th International Semantic Web Conference, Riva del Garda, Italy, October 19-23, 2014.

Proceedings, Part I. pp. 457–471 (2014)

Usbeck, R., Röder, M., Ngomo, A.N., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti,

M., Cherix, D., Eickmann, B., Ferragina, P., Lemke, C., Moro, A., Navigli, R., Piccinno, F., Rizzo,

G., Sack, H., Speck, R., Troncy, R., Waitelonis, J., Wesemann, L.: GERBIL: general entity annotator

benchmarking framework. In: Proceedings of the 24th International Conference on World

Wide Web, WWW 2015, Florence, Italy, May 18-22, 2015. pp. 1133–1143 (2015)

Verborgh, R., Harth, A., Maleshkova, M., Stadtmüller, S., Steiner, T., Taheriyan, M., Van de

Walle, R.: Survey of Semantic Description of REST APIs, pp. 69–89. Springer New York (2014)

Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artificial Intelligence

Review 18(2), 77–95 (2002)

Vitvar, T., Kopecký, J., Viskova, J., Fensel, D.:Wsmo-lite annotations for web services. In: ESWC.

pp. 674–689 (2008)

Wood, I., Vandervalk, B., McCarthy, L., Wilkinson, M.: Owl-dl domain-models as abstract workflows.

In: Leveraging Applications of Formal Methods, Verification and Validation. Applications

and Case Studies, Lecture Notes in Computer Science, vol. 7610, pp. 56–66. Springer Berlin Heidelberg