A Landscape Analysis of Standardisation in the Field of Artificial Intelligence

Wolfgang Ziegler

Fraunhofer Institute for Algorithms and Scientific Computing SCAI, Sankt Augustin, Germany

E-mail: wolfgang.ziegler@scai.fraunhofer.de

Received 27 August 2019; Accepted 04 January 2020; Publication 23 April 2020

Abstract

While projects, developments and applications addressing and using artificial intelligence (AI) are rather multifaceted and their number is constantly increasing, the standardisation activities in the field of artificial intelligence are limited, their number is significantly lower and does not increase at the same pace.

The European funded project StandICT.eu aims at supporting European experts’ presence in and contributions to international standardisation activities in ICT. The focus of the project is on the 5 priority domains identified by the European Commission (Cloud Computing, IoT, Big Data, Cyber Security, 5G) and on Artificial Intelligence while being open for other relevant topics defined in the annual European Rolling Plan for ICT Standardisation. The project has two main outcomes: (i) increased contribution of European experts in international standardisation through support by providing grants for planned contributions of successful applications of experts, and (ii) an online observatory of published standards and ongoing standardisation activities in the areas mentioned before. This observatory (called Standards Watch) is accessible through the project’s web site and open for contributions and comments from registered users. As part of the effort for the Standards Watch the projects has prepared a comprehensive analysis of the international standardisation landscape in the AI field, that comprises a description of the ICT standards and ongoing work at international level in the field of AI across the standardisation organisations already active in the field.

In this article we will present results of our work where we have analysed the work of the 5 international and European Standards Development Organisations (SDOs) IEEE,¹ ISO/IEC,² ITU-T,³ ETSI⁴ and CEN-CENELEC⁵ developing standards in the field of AI. The first 4 bodies have already been active several years in AI standardisation at the time of writing this article. CEN-CENELEC has launched a focus group in 2019 which aims at producing a roadmap for AI standardisation. For these 4 SDOs information on their active groups, details of their work and the respective state/outcome is provided in the main part of this article. Followed by the same exercise for the two identified Standards Settings Organisations (SSOs): W3C⁶ and IRTF,⁷ their active groups and the respective state/outcome.

With this information an initial analysis of the AI standardisation landscape as Q3 2019 is performed with the main outcomes that (i) the number of working groups chaired by Europeans is significant and (ii) that there is room for European experts contributing to ongoing and future standardisation work.

The article is concluded by considerations on future priorities.

Keywords: Artificial intelligence, standardisation.

1 Introduction

Over the last years we can observe that developments and applications addressing and using artificial intelligence (AI) are rather multifaceted and their number is constantly increasing. However, standardisation activities in the field of artificial intelligence are limited, their number is significantly lower and does not increase at the same speed as developments and applications. Several organisations of the European Commission (EC) active in the field of AI identified the need for a deeper analysis with a focus on European participation. The European funded project StandICT.eu [18] was mandated to carry out this analysis.

The StandICT.eu project aims at supporting European experts’ presence in and contributions to international standardisation activities in ICT. The focus of the project is on the 5 priority domains identified by the European Commission (Cloud Computing, IoT, Big Data, Cyber Security, 5G) [19] and on Artificial Intelligence while being open for other relevant topics defined in the annual European Rolling Plan for ICT Standardisation [1].

The project has two main outcomes: (i) increased contribution of European experts in international standardisation through support by providing grants to experts for planned contributions of successful applicants responding to StandICT.eu’s open calls, and (ii) an online observatory of published standards and ongoing standardisation activities in the areas mentioned before. The Standards Watch (the collaborative web space of the observatory) is accessible through the project’s web site and open for contributions and comments from registered users.

The comprehensive analysis of the international standardisation landscape in the AI field has been performed as part of the effort for the Standards Watch. The analysis comprises a description of the ICT standards and ongoing work at international level in the field of AI and a first analysis of the landscape.

For the purposes of the present document, the following definitions will apply:

Standards Setting Organization (SSO): Any entity whose primary activities are developing, coordinating, promulgating, revising, amending, reissuing, interpreting, or otherwise maintaining specifications and standards that address the interests of a wide base of users outside the standards development organization. The SSOs investigated in this report are: W3C and IRTF.

Standards Development Organization (SDO): A Standards Setting Organization that has a formal recognition by international treaties, regulation, etc. The SDOs examined in this report are: IEEE, ETSI, ISO/IEC, ITU-T, CEN-CENELEC. Note: The SDOs are a subset of the SSOs.

Standards: A Standard is an output from an SDO.

Specifications: A Specification is an output from an SSO that may become a standard when ratified by an SDO.

The rest of the article is structured as follows. In Section 2 we will present results of our work regarding the 5 Standards Development Organisations (SDOs): IEEE, ISO/IEC, ITU-T, ETSI and CEN-CENELEC. Followed by the same exercise for the two identified Standards Settings Organisations (SSOs): W3C and IRTF, their active groups and the respective state/outcome. In Section 3 we provide an initial analysis of the AI standardisation landscape as of early 2019 based on this information presenting the main outcomes. Section 4 wrap up our finding and conclude with considerations and recommendations on future priorities.

2 Standardisation Activities

2.1 Standards Development Organisations

StandICT.eu has created a comprehensive overview on activities of Standards Setting Organisations (SSO) and Standards Developing Organisations (SDO) with activities related to AI. As of Spring 2019 five SDOs and 2 SSOs had active groups working on standardisation or related activities.

A brief description of the respective active groups of the five SDOs (already active before 2019 or starting in 2019) and details on their standardisation activities can be found in sections 2.1.1 IEEE, 2.1.2 ISO/IEC, 2.1.4 ETSI and 2.1.3 ITU-T and 2.1.5 CEN-CENELEC.

Figure 1 depicts the number of their respective working groups, study groups, focus groups, community groups, committees and initiatives with activities related to AI standardisation. Working Groups typically aim at producing standards or other normative documents. Study and Focus Groups are usually dedicated to explore an area and to eventually use the results to create a Working Group. ISO Committees host Working Groups and Study Groups working in the same field. IEEE Initiatives host different activities dedicated to explore an area and to eventually use the results to create Working Groups. In summary, with ETSI and CEN-CENELEC there are 2 European Standards Organisations (ESOs) with ongoing or preparatory activities in the field of AI standardisation out of 5 SDOs in total.

2.1.1 IEEE

In 2019 the following 14 working groups have been identified in IEEE each of which is focussing on a distinct standard. All 14 standards are under development. Moreover, IEEE has launched the initiative Symbiotic Autonomous Systems (SAS) which aims at taking the lead in developing the new field of Symbiotic Systems Science, fostering interdisciplinary technology deployments that take into account Ethical, Legal, and Societal considerations, and promoting humancentric economic growth. More working groups may be started from SAS in future.

Figure 1 Number of active groups with standardisation or related activities per SDO/SSO.

2.1.1.1 EMELC-WG

The Engineering Methodologies for Ethical LifeCycle Concerns Working Group produces the P7000 standard Model Process for Addressing Ethical Concerns During System Design.

The standard establishes a process model by which engineers and technologists can address ethical consideration throughout the various stages of system initiation, analysis and design. Expected process requirements include management and engineering view of new IT product development, computer ethics and IT system design, value-sensitive design, and, stakeholder involvement in ethical IT system design. Engineers, technologists and other project stakeholders need a methodology for identifying, analysing and reconciling ethical concerns of end users at the beginning of systems and software life cycles.

The purpose of this standard is to enable the pragmatic application of this type of Value-Based System Design methodology which demonstrates that conceptual analysis of values and an extensive feasibility analysis can help to refine ethical system requirements in systems and software life cycles.

This standard will provide engineers and technologists with an imple-mentable process aligning innovation management processes, IT system design approaches and software engineering methods to minimise ethical risk for their organisations, stakeholders and end users.

2.1.1.2 ASV WG_P7001

The Autonomous Systems Validation Working Group_P7001 produces the P7001 standard Transparency of Autonomous Systems.

This standard describes measurable, testable levels of transparency, so that autonomous systems can be objectively assessed and levels of compliance determined.

A key concern over autonomous systems (AS) is that their operation must be transparent to a wide range of stakeholders, for different reasons. (i) For users, transparency is important because it builds trust in the system, by providing a simple way for the user to understand what the system is doing and why. If we take a care robot as an example, transparency means the user can quickly understand what the robot might do in different circumstances, or if the robot should do anything unexpected, the user should be able to ask the robot ‘why did you just do that?’. (ii) For validation and certification of an AS transparency is important because it exposes the system’s processes for scrutiny. (iii) If accidents occur, the AS will need to be transparent to an accident investigator; the internal process that led to the accident need to be traceable. Following an accident (iv) lawyers or other expert witnesses, who may be required to give evidence, require transparency to inform their evidence. And (v) for disruptive technologies, such as driverless cars, a certain level of transparency to wider society is needed in order to build public confidence in the technology. For designers, the standard will provide a guide for self-assessing transparency during development and suggest mechanisms for improving transparency (for instance the need for secure storage of sensor and internal state data, comparable to a flight data recorder or black box).

2.1.1.3 PDP

The Personal Data Privacy Working Group produces the P7002 standard Data Privacy Process.

The purpose of this standard is to have one overall methodological approach that specifies practices to manage privacy issues within the systems/software engineering life cycle processes.

This standard defines requirements for a systems/software engineering process for privacy–oriented considerations regarding products, services, and systems utilising employee, customer or other external user’s personal data. It extends across the life cycle from policy through development, quality assurance, and value realisation. It includes a use case and data model (including metadata). It applies to organisations and projects that are developing and deploying products, systems, processes, and applications that involve personal information. By providing specific procedures, diagrams, and checklists, users of this standard will be able to perform a conformity assessment on their specific privacy practices. Privacy impact assessments (PIAs) are described as a tool for both identifying where privacy controls and measures are needed and for confirming they are in place.

2.1.1.4 ALGB-WG

The Algorithmic Bias Working Group produces the P7003 standard Algorithmic Bias Considerations.

This standard is designed to provide individuals or organisations creating algorithms, largely in regards to autonomous or intelligent systems, certification-oriented methodologies to provide clearly articulated accountability and clarity around how algorithms are targeting, assessing and influencing the users and stakeholders of said algorithm. Certification under this standard will allow algorithm creators to communicate to users, and regulatory authorities, that up-to-date best practices were used in the design, testing and evaluation of the algorithm to avoid unjustified differential impact on users.

This standard describes specific methodologies to help users certify how they worked to address and eliminate issues of negative bias in the creation of their algorithms, where “negative bias” infers the usage of overly subjective or uniformed data sets or information known to be inconsistent with legislation concerning certain protected characteristics (such as race, gender, sexuality, etc.); or with instances of bias against groups not necessarily protected explicitly by legislation, but otherwise diminishing stakeholder or user well-being and for which there are good reasons to be considered inappropriate. Possible elements include (but are not limited to): benchmarking procedures and criteria for the selection of validation data sets for bias quality control; guidelines on establishing and communicating the application boundaries for which the algorithm has been designed and validated to guard against unintended consequences arising from out-of-bound application of algorithms; suggestions for user expectation management to mitigate bias due to incorrect interpretation of systems outputs by users (e.g. correlation vs. causation).

2.1.1.5 WG-CSDG

The Working Group for Child and Student Data Governance produces the P7004 Standard on Child and Student Data Governance.

The standard defines specific methodologies to help users certify how they approach accessing, collecting, storing, utilising, sharing, and destroying child and student data. The standard provides specific metrics and conformance criteria regarding these types of uses from trusted global partners and how vendors and educational institutions can meet them.

This standard is designed to provide organisations handling child and student data governance-oriented processes and certifications guaranteeing the transparency and accountability of their actions as it relates to the safety and wellbeing of children, their parents, the educational institutions where they are enrolled, and the community and societies where they spend their time, both on and offline. It is also designed to help parents and educators, with an understanding that most individuals may not be tech-savvy enough to understand underlying issues of data usage, but still must be properly informed about the safety of their children’s data and provided with tools and services that provide proper opportunities for content based, preinformed choice regarding their family’s data.

2.1.1.6 EDG-WG

The Employer Data Governance working group produces the P7005 Standard for Transparent Employer Data Governance.

The standard defines specific methodologies to help employers to certify how they approach accessing, collecting, storing, utilising, sharing, and destroying employee data. The standard provides specific metrics and conformance criteria regarding these types of uses from trusted global partners and how vendors and employers can meet them.

This standard is designed to provide organisations with a set of clear guidelines and certifications guaranteeing they are storing, protecting, and utilising employee data in an ethical and transparent way. It is also designed to help employers with an understanding that most individuals may not be tech-savvy enough to understand underlying issues of data usage, but still must be properly informed about the safety of their employee data to be provided with tools and services that provide proper opportunities for content based, pre informed choice regarding how they share their information in the workplace. Modelled after the EU GDPR legislation, this Standard will be designed to be a form of “GDPR for Employees” guaranteeing that workers facing widespread automation issues potentially displacing their jobs will have control and influence over the personal information that directly represents a core asset of their identity and lives whether derived from work-flow monitoring or personal data storage.

2.1.1.7 WG-PDAI

The Personal Data AI Agent Working Group produces the P7006 Standard for Personal Data Artificial Intelligence (AI) Agent.

This standard describes the technical elements required to create and grant access to a personalised Artificial Intelligence (AI) that will comprise inputs, learning, ethics, rules and values controlled by individuals.

With the advent and rise of AI there is a risk that machine-to-machine decisions will be made with black-box inputs determined without input transparency to humans. In order to enable ethics-based AI, individuals will require the means to influence and determine the values, rules and inputs that guide the development of personalised algorithms and Artificial Intelligence. They will need an agent that can negotiate their individual rights and agency in a system of shared social norms, ethics and human rights that also foresee and helps the individual mitigate ethical implications of data processing. This approach will enable individuals to safely organise and share their personal information at a machine-readable level and enable a personalised AI to act as a proxy for machine-to-machine decisions. A key goal for the creation of this standard is to educate government and commercial actors why it is in their best interests to create the mechanisms for individuals to train Personal AI Agents to move beyond asymmetry and harmonise personal data usage for the future.

2.1.1.8 EDARR-wg

The WG for the Adoption of: Robots and robotic devices: Guide to the ethical design and application for robots and robotic systems produces the P7007 Ontological Standard for Ethically Driven Robotics and Automation Systems.

The standard establishes a set of ontologies with different abstraction levels that contain concepts, definitions and axioms which are necessary to establish ethically driven methodologies for the design of Robots and Automation Systems.

The standard establishes a set of definitions and their relationships that will enable the development of Robotics and Automation Systems in accordance with worldwide Ethics and Moral theories, with a particular emphasis on aligning the ethics and engineering communities to understand how to pragmatically design and implement these systems in unison. These definitions allow for a precise communication among global experts of different domains that includes Robotics, Automation and Ethics.

The use of ontologies for representing knowledge in any domain has several benefits that include (a) formal definition of concepts of a particular domain in a language-independent representation, i.e., they are not dependent of a specific programming language, however, they are formally described to be implemented in a target language; (b) tools for analysing concepts and their relationships in searching of inconsistency, incompleteness and redundancy; (c) language for being used in the communication process among Robots from different manufacturers; etc.

2.1.1.9 Ethical Nudging

The Working Group for Ethically Driven Nudging for Robotic, Intelligent and Autonomous Systems produces the P7008 Standard for Ethically Driven Nudging for Robotic, Intelligent and Au tonomous Systems.

“Nudges” as exhibited by robotic, intelligent or autonomous systems are defined as overt or hidden suggestions or manipulations designed to influence the behaviour or emotions of a user. This standard establishes a delineation of typical nudges (currently in use or that could be created). It contains concepts, functions and benefits necessary to establish and ensure ethically driven methodologies for the design of the robotic, intelligent and autonomous systems that incorporate them.

The Standard for Ethically Driven Nudging for Robotic, Intelligent and Autonomous Systems establishes a set of definition of functions and their relationships with benefits depending on cultural aspects of users (well-being, health, . . .) that enables the development of Robotics, Intelligent and Autonomous Systems in accordance with worldwide Ethics and Moral theories, with a particular emphasis on aligning the ethics and engineering communities to understand how to pragmatically design and implement these systems in unison. This standard along with definitions allows for precise communication among global experts of different domains that includes Robotics, Artificial Intelligence, Autonomous Systems, and Ethics.

2.1.1.10 Fail-Safe Design

The working group Standard for Fail-Safe Design of Autonomous and Semi-Autonomous Systems produces the P7009 Standard for Fail-Safe Design of Autonomous and Semi-Autonomous Systems.

Autonomous and semi-autonomous systems which remain operational after an intended or unintended malfunction can disadvantage and harm users, society, and the environment. There is a need for definition of effective failsafe mechanisms to help mitigate risks related to system malfunction and provide developers, installers and operators with clear technical criteria to terminate unsuccessful or compromised operations in a safe and consistent manner.

This standard establishes a practical, technical baseline of specific methodologies and tools for the development, implementation, and use of effective fail-safe mechanisms in autonomous and semi-autonomous systems. The standard includes (but is not limited to): clear procedures for measuring, testing, and certifying a system’s ability to fail safely on a scale from weak to strong, and instructions for improvement in the case of unsatisfactory performance. The standard serves as the basis for developers, as well as users and regulators, to design fail-safe mechanisms in a robust, transparent, and accountable manner.

2.1.1.11 Wellbeing for Ethical AI

The working group Wellbeing Metrics Standard for Ethical Artificial Intelligence and Autonomous Systems produces the P7010 Wellbeing Metrics Standard for Ethical Artificial Intelligence and Autonomous Systems.

This standard establishes wellbeing metrics relating to human factors directly affected by intelligent and autonomous systems and establishes a baseline for the types of objective and subjective data these systems should analyse and include (in their programming and functioning) to proactively increase human wellbeing.

The Wellbeing Metrics Standard for Ethical Artificial Intelligence and Autonomous Systems enables programmers, engineers, and technologists to better consider how the products and services they create can increase human wellbeing based on a wider spectrum of measures than growth and productivity alone. Today, affective systems utilising emotion recognising sensors are quantified primarily by their economic value in the market place beyond their efficacy within certain fields (psychology, etc.). While it is often understood that ethical considerations for intelligent and autonomous systems might hinder innovation by the introduction of unwanted regulation, without metrics that value mental and emotional health at both an individual and societal level, innovation is impossible to quantify. The introduction and use of these metrics for programmers and technologists means that beyond economic increase human wellbeing can be measured and better improved.

2.1.1.12 NST_WG

The News Site Trustworthiness Working Group produces the P7011 Standard for the Process of Identifying and Rating the Trustworthiness of News Sources.

The purpose of the standard is to address the negative impacts of the unchecked proliferation of fake news by providing an open system of easy-to-understand ratings. In so doing, it shall assist in the restoration of trust in some purveyors, appropriately discredit other purveyors, provide a disincentive for the publication of fake news, and promote a path of improvement for purveyors wishing to do so. The standards shall target a representative sample set of news stories in order to provide a meaningful and accurate rating scorecard.

This standard provides semi-autonomous processes using standards to create and maintain news purveyor ratings for purposes of public awareness. It standardises processes to identify and rate the factual accuracy of news stories in order to produce a rating of online news purveyors and the online portion of multimedia news purveyors. This process will be used to produce truthfulness scorecards through multi-faceted and multi-sourced approaches. The standard defines an algorithm using open source software and a score card rating system as methodology for rating trustworthiness as a core tenant in an effort to establish trust and acceptance.

2.1.1.13 MRPT_WG

The Working Group on Machine Readable Privacy Terms (SSIT/SC/ MachReadPrivacy) produces the P7012 Standard for Machine Readable Personal Privacy Terms.

The standard identifies/addresses the manner in which personal privacy terms are proffered and how they can be read and agreed to by machines.

The purpose of the standard is to provide individuals with means to proffer their own terms respecting personal privacy, in ways that can be read, acknowledged and agreed to by machines operated by others in the networked world. In a more formal sense, the purpose of the standard is to enable individuals to operate as first parties in agreements with others–mostly companies–operating as second parties.

2.1.1.14 AFAT_WG

The Benchmarking Accuracy, Increasing Transparency, and Governing Use of Automated Facial Analysis Technology Working Group produces the P7013 Inclusion and Application Standards for Automated Facial Analysis Technology.

The standard provides phenotypic and demographic definitions that technologists and auditors can use to assess the diversity of face data used for training and benchmarking algorithmic performance, establishes accuracy reporting and data diversity protocols/rubrics for automated facial analysis, and outlines a rating system to determine contexts in which automated facial analysis technology should not be used.

Research continues to show that artificial intelligence which is used for automated facial analysis is susceptible to bias that can exacerbate human prejudice and systematically disadvantage individuals based on gender, ethnicity, age, and other factors.

The purpose of the standard is to provide inclusion guidelines for developing and benchmarking automated facial analysis technology to mitigate demographic and phenotypic bias and discrimination.

The reporting rubrics/protocols established in this standard serve to increase transparency of this automated technology so that developers and decision makers can compare available options to choose the most appropriate technology based on target populations and intended use cases. Given the sensitivity of the biometric data provided from a human face, the standard also delineates appropriate and inappropriate uses of automated facial analysis based on accuracy and values established by a global community.

2.1.1.15 SAS

The Symbiotic Autonomous Systems Initiative fosters studies and applications focused on the convergence of human augmentation with the increasing intelligence and awareness of artefacts, leading towards a symbiosis of humans and machines. This will have significant implications for human society as a whole, affecting culture and the economy and prompting new questions about our place on Earth.

2.1.2 ISO/IEC

The following paragraphs show the two sub committees (SC42 and SC27) with 9 working groups active in 2019 with the goal to develop 13 AI standards in ISO/IEC (SC42) and 1 Big Date security standard (SC27).

2.1.2.1 JTC1 SC42

The Artificial Intelligence Committee is managing standardisation in the area of Artificial Intelligence, serves as the focus and proponent for JTC1’s standardisation program on Artificial Intelligence, and provides guidance to JTC1, IEC, and ISO committees developing Artificial Intelligence applications.

2.1.2.2 JTC1 SC42 AG1 and AG2

AG1 is the Advisory Group 1 on AI Management Systems Standard with terms of reference to produce internal reports towards JTC1. AG2 is the Advisory Group 2 on AI Systems Engineering responsible to carry out a number of studies on, e.g. gap analysis regarding existing engineering practices and AI practices, maintenance of AI systems, Machine Learning life cycle, etc. The outcome will be used for WG4.

2.1.2.3 JTC1 SC42 AHG1, AGH2, and AGH3

The Dissemination and outreach Working Group, Liaison with SC38 Working Group, and the Intelligent systems engineering Working Group are not producing standards and are not presented here.

2.1.2.4 JTC1 SC42 JWG1

The Governance implications of AI Working Group produces the standard NP 38507 – Information technology – Governance of IT – Governance implications of the use of artificial intelligence by organisations.

JWG1 is a joint working group with JTC1 SC40 IT Service Management and IT Governance. A working draft (WD) study has been initiated for NP 38507.

The following sections briefly describe the other 5 active working groups of SC42.

2.1.2.5 JTC1 SC42 WG1

The Foundational standards Working Group produces two standards WD 22989 – Artificial intelligence – Concepts and terminology and WD 23053 Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML).

With such a diverse set of stakeholders for AI, it is essential to have foundational standards that provide for a framework and common set of vocabulary. Not only does this enable stakeholders of different backgrounds and perspectives to talk the same language, it also sets the stage of how the different stakeholders and technology providers/users interact with one another. Progressing these two foundational standards is a priority of SC42.

2.1.2.6 JTC1 SC42 WG2

The Big data working group will be based on the program of work on Big Data (JTC 1/WG 9) which has been disbanded and the program transferred to JTC 1/SC 42. The group has published three standards and is working on three standards.

The published standards are ISO/IEC 20546:2019 – Information technology – Big data – Overview and vocabulary, ISO/IEC TR 20547-2:2018 – Information technology – Big data reference architecture – Part 2: Use cases and derived requirements, and ISO/IEC TR 20547-5:2018 – Information technology – Big data reference architecture – Part 5: Standards roadmap.

The three standards under development are ISO/IEC TR 20547-1: Information technology – Big data reference architecture – Part 1: Framework and application process, ISO/IEC DIS 20547-3: Information technology Big data reference architecture – Part 3: Reference architecture and ISO/IEC AWI 24668: Information technology – Artificial intelligence – Process management framework for Big data analytics.

JTC 1’s program on Big Data, initiated a few years ago, has started two foundational multi–part projects around overview and vocabulary as well as a Big Data Reference Architecture (BDRA). These projects have received tremendous interest from the industry. As we look to the arc of future work, the roadmap for big data aligns well with that of SC42. Moreover, from a data science perspective, expert participation, use cases and applications, future anticipated work on analytics, and the role of systems integration (working with other ISO, IEC and JTC 1 committees on application areas), the program of work of big data and the initial program of work identified for SC42 line up well together. From an industry practice point of view, it is hard to imagine applications where one technology is present without the other. As a consequence, JTC 1’s program on Big Data is now included in SC42 as WG2.

2.1.2.7 JTC1 SC42 WG3

The Trustworthiness Working Group is producing the four standards ISO/IEC NP TR 24027: Information technology – Artificial Intelligence (AI) – Bias in AI systems and AI aided decision making, ISO/IEC NP TR 24028: Information technology – Artificial Intelligence (AI) – Overview of trustworthiness in Artificial Intelligence, ISO/IEC NP TR 24029-1: Artificial Intelligence (AI) – Assessment of the robustness of neural networks – Part 1: Overview, and ISO/IEC AWI 23894: Information Technology – Artificial Intelligence – Risk Management.

WG3 – Trustworthiness is based on the outcome of the SC42 Study Group 2. The working group takes on the three approved trustworthiness projects:

TR on Bias in AI systems and AI aided decision making
TR on Overview of trustworthiness in Artificial Intelligence
TR on Assessment of the robustness of neural networks – Part 1: Overview

Additionally the working group works on risk management in AI.

2.1.2.8 JTC1 SC42 WG4

The Use cases and applications Working Group produces the standard ISO/IEC NP TR 24030 – Information technology – Artificial Intelligence (AI) – Use cases.

WG4 – Use cases and applications will be based on the outcome of the SC42 Study Group 3 which aimed at:

Identifying different AI application domains (e.g., social networks and embedded systems) and the different context of their use (e.g., fintech, health care, smart home, and autonomous cars).
Collecting representative use cases.
Describing applications and use cases using the terminology and concepts defined in ISO/IEC AWI 22989 and ISO/IEC AWI 23053 and extend the terms as necessary.
Developing new work item proposals as appropriate and recommend placement.

2.1.2.9 JTC1 SC42 WG 5

The working group Computational approaches and computational characteristics of AI systems produces the standard ISO/IEC AWI TR 24372: Information technology – Artificial intelligence (AI) – Overview of computational approaches for AI systems. WG5 is based on the outcome of the former SC42 Study Group 1.

2.1.2.10 JTC1 SC27

The Information security, cybersecurity and privacy protection committee is working in its WP4 on security aspects of Big Data. SC27 engages in active liaison and collaboration with appropriate bodies to ensure the proper development and application of SC27 standards and technical reports in relevant areas.

2.1.2.11 JTC1 SC27 WG4

The Security controls and services working group produces the standard ISO/IEC AWI 20547-4 – Information technology – Big data reference architecture – Part 4: Security and privacy.

Originally this work was done in JTC 1/WG 9 which has been disbanded in 2017. With the establishment of SC42, JTC 1 transferred the JTC 1/WG 9 program of work to SC42 Artificial Intelligence.

Part 4 has been delegated from WG9 to SC27 WG4 and will return to SC42 once completed. Besides SC27 WG4 also SC27 WG5 – Identity Management and Privacy Technologies is contributing to 20547-4.

2.1.3 ITU-T

ITU-T has set-up 2 Focus Groups to do supportive research for the application of AI in the health sector and for using machine learning for future networks. None of the groups aims at developing a standard but rather will produce different reports and specifications.

2.1.3.1 FG-AI4H

The Artificial Intelligence for Health Focus Group engages researchers, engineers, practitioners, entrepreneurs and policy makers, to enable leveraging such solutions in practice.

FG-AI4H has defined a number of task to accomplish including, e.g., the following:

Developing a list of standards bodies, forums, consortia, regulators, core research organizations, engineering teams, health professionals, entrepreneurs, digital health policy makers and other entities dealing with aspects of AI and to establish liaisons and relationships with some of the listed organizations;
Organising thematic workshops and forums on AI for health, which will bring together all stakeholders, and promote the FG activities and encourage both ITU members and non-ITU members to join its work;
Gathering information on initiatives pertaining to AI for health and to identify existing standards, AI methods, best practises and challenges for the adoption;
Drafting technical reports and specifications for assessment frameworks for AI for health, including for example data formats, interfaces, architecture, and protocols. Note, it is not intended to specify the AI for health algorithms themselves as an ITU Recommendation;
Writing a report(s) of the FG activities including a recommendation how to proceed with AI for health standardization after the FG finished its work.

The list of tasks is rather generic. The status of the activities of the focus group is work in progress with no tangible output yet in Q3 2019.

2.1.3.2 FG-5GML

The Machine Learning for Future Networks including 5G Focus Group would play a role in providing a platform to study and advance the various ML approaches for future networks including 5G.

Specific Tasks and Deliverables are, e.g., the following:

Providing terminology and taxonomy for ML in the context of future networks, as well as a guideline on the approaches, tools, applications and platforms related to this topic;
Gathering information on initiatives pertaining to ML for future networks and to identify existing standards, ML methods, best practises and challenges for the adoption of ML in future networks;
Organising thematic workshops and forums on ML for future networks, which will bring together all stakeholders, and promote the FG activities and encourage both ITU members and non-ITU members to join its work;
Analysing possible requirements on ML applied to future networks;
Drafting technical reports and specifications for ML for future networks, including interfaces, network architectures, protocols, algorithms and data formats;
Sending the final deliverables to ITU-T Study Group 13 at least four calendar weeks before the parent group’s next meeting in accordance with Recommendation ITU-T A.7.

The list of tasks is rather generic. The status of the activities of the focus group is work in progress with no tangible output yet in Q3 2019.

Moreover, given the activities in IEEE and ISO/IEC the most pressing issues requiring standardisation are already addressed by these. Thus, to avoid duplicate, overlapping or competing standards the suggested non-standardsrelated work of the two focus groups seems more appropriate.

2.1.4 ETSI

ETSI has set-up 3 working groups. Two of them are focusing on the use of AI technologies for network management. No standards development is foreseen. In contrast, the third group (SAI-ISG) is dedicated to prestandardisation work a focus on later developments of technical specifications that mitigate against threats arising from the deployment of AI, and threats to AI.

2.1.4.1 SAI ISG

The Secure AI Industry Specification Group will develop an ETSI consensual view of the technical knowledge required to develop technical specifications that mitigate against threats arising from the deployment of AI, and threats to AI, from both other AIs, and from conventional sources.

As a pre-standardisation activity the ISG SAI is intended to frame the security concerns arising from AI and to build the foundation of a longer-term response to the threats from AI in sponsoring the future development of normative technical specifications.

The underlying rationale for ISG SAI is that autonomous mechanical/computing entities may make decisions that act against the relying parties either by design or as a result of malicious intent. The conventional cycle of risk analysis and countermeasure deployment represented by the Identify-Protect-Detect-Respond cycle needs to be re-assessed when an autonomous machine is involved. The purpose of the ISG SAI is to develop the technical knowledge that acts as a baseline in ensuring that AI is secure. The stakeholders impacted by the activity of the ISG includes all the member groups represented in ETSI and some of the wider societal environment that AI will be deployed in. This includes end users, manufacturers, operators and governments, and the activity of the ISG will include gathering concerns of each stakeholder group to ensure that ETSI and the output of the ISG SAI address those concerns.

2.1.4.2 ZSM ISG

The Zero touch network and Service Management Industry Specification Group is conceived as a next-generation management system that leverages the principles of Network Functions Virtualisation (NFV) and Software Defined Networking (SDN). It will be designed for the new, cloud-based network infrastructures and functions, and based on cloud-native principles to address zero touch (fully automated) management and operation.

The group strives for enabling agile, efficient and qualitative management and automation of future services.

2.1.4.3 ENI ISG

The Experiential Networked Intelligence Group is defining a Cognitive Network Management architecture, using Artificial Intelligence (AI) techniques and context-aware policies to adjust offered services based on changes in user needs, environmental conditions and business goals.

The use of Artificial Intelligence techniques in the network management system should solve some of the problems of future network deployment and operation.

2.1.5 CEN-CENELEC

CEN-CENELEC has implemented a focus Group on Artificial Intelligence. The kick-off meeting of the group was in April 2019. Together with other stakeholders the group aims at developing a roadmap for AI standardization by 2020.

2.2 Standards Settings Organisations

The two SSOs with already active groups in 2018 or groups starting in 2019 are W3C and IRTF (the research sister organisation of the IETF). A brief description of the respective active groups and details can be found in Sections 2.2.1 (W3C) and 2.2.2 (IRTF).

2.2.1 W3C

While W3C has a number of published standards and such under development that are not explicitly developed for AI but are well suited for usage in the AI field (e.g. data description or formatting standards) a new activity has been launched directly addressing AI. The group studies AI knowledge representation. No standards development is planned.

2.2.1.1 AI KR

Artificial Intelligence Knowledge Representation Community Group The overall goal/mission of this community group is to explore the requirements, best practices and implementation options for the conceptualisation and specification of domain knowledge in AI.

Particular emphasis will be placed on the identification and the representation of AI facets and various aspects (technology, legislation, ethics, etc.) with the purpose to facilitate knowledge exchange and reuse.

Therefore, the proposed outcomes could be instrumental to research and advancement of science and inquiry, as well as to increase the level of public awareness in general to enable learning and participation. Proposed outcomes:

A comprehensive list of open access resources in both AI and KR (useful to teaching and research)
A set of metadata derived from these resources
A concept map of the domain
A natural language vocabulary to represent various aspects of AI
One or more encoding/implementations/ machine language version of the vocabulary, such as ChatBot Natural Language Understanding and Natural Language Generation
Methods for KR management, especially Natural Language Learning/ Semantic Memory

2.2.2 IRTF

The three research groups of IRTF work on studies on the possible role of AI in future networks. No standards development is planned.

2.2.2.1 NMLRG

The Network Machine Learning Research Group provided a forum for researchers to explore the potential of machine learning technologies for networks. In particular, the NMLRG will work on potential approaches that apply machine learning technologies in network control, network management, and supplying network data for upper-layer applications.

The research group has been abandoned.

2.2.2.2 NMRG

The Network Management Research Group provides a forum for researchers to explore new technologies for the management of the Internet. In particular, the NMRG will work on solutions for problems that are not yet considered well understood enough for engineering work within the IETF.

The initial focus of the NMRG will be on higher-layer management services that interface with the current Internet management framework. This includes communication services between management systems, which may belong to different management domains, as well as customer-oriented management services. The NMRG is expected to identify and document requirements, to survey possible approaches, to provide specifications for proposed solutions, and to prove concepts with prototype implementations that can be tested in large-scale real-world environments.

The IETF Operations and Management Area Directors are members of the NMRG mailing list and invited to NMRG meetings in order to ensure free flow of information in both directions, and to avoid duplication of work with the various IETF working groups.

The group will report its progress through a publicly accessible web site and presentations at IETF meetings. Specifications developed by the NMRG will be submitted for publication as Experimental or Informational RFCs.

2.2.2.3 COINRG

The Computation in the Network Research Group will explore existing research and foster investigation of Compute In the Network and resultant impacts to the data plane. The goal is to investigate how to harness and to benefit from this emerging disruption to the Internet architecture to improve network and application performance as well as user experience. COIN will encourage scrutiny of research solutions that comprehend the re-imagining of the network to be a place where routing, compute, and storage blend.

COIN will address both controlled environments such as DCN and the ongoing shift from data center (DC) toward edge computing and will debate whether this shift can be viewed as a cloud continuum. COIN specifically will focus on the evolution necessary for networking to move beyond packet interception as the basis of network computation. While existing DCs employ rudimentary languages for programming switch, richer programmability is required to support emerging workloads, such as edge network analytics, machine learning and deep learn. Such applications not only need access to more general-purpose languages, but also need to operate in conjunction with local and remote caches, dynamic control points, and data stewardship.

3 Landscape Analysis

In this section a first analysis of the AI standardisation landscape is presented. Based on the information collected in Section 2 several charts depict the results of evaluations of, e.g., the European impact estimated through the distribution of countries of the working group chairs, the thematic distribution of AI standardisation across SDOs and SSO.

3.1 Ongoing Standardisation Activities

Figure 3 presents the different activities as described in Section 3.1 in a matrix of themes (challenges, pre-standardisation, standards) and SDOs/SSOs. As can be seen, there is little overlap in activities of different SDOs/SSOs with respect to the themes.

Only for AI usage four SDOs (IEEE, ITU-T, ETSI, IOS/IEC) and one SSO (IRTF) have parallel activities. However, these activities are either in different fields or are of different nature, e.g. standards development (ISO/IEC) versus research (IRTF), AI for network management (ETSI) versus the symbiotic autonomous systems initiative (IEEE).

Two SDOs have non-overlapping activities related to AI security: ISO/IEC JTC1 SC27 is preparing a security standard for Big Data (AWI 20547-4) while the new ETSI SAI ISG group will focus on a Threat Ontology and a document providing the Securing AI Problem Statement.

The last theme with two SDOs (IEEE and ISO/IEC) having parallel activities is AI foundational standards. Again, there is no risk of competing developments. While IEEE is focussing on a standard for Fail-Safe Design of Autonomous and Semi-Autonomous Systems the ISO/IEC JTC1 SC42 WG 1 is focusing on defining AI concepts and terminology (WD 22989) and a framework for AI systems using machine learning (WD 23053).

3.2 European Contribution to the Above and Upside Potential

The world map presented in Figure 2 shows the distribution of the group chairs of the working and study groups across the region North America and Canada, the Asia-Pacific region and Europe.

Figure 4 provides a detailed view on the countries of the 15 Europeans chairing working groups or study groups of the SDOs and SSOs with activities in AI standardisation. As can be seen, as of early 2019 the Eastern part of Europe and the South-Eastern part do not send out chairs to these groups. This corresponds to the fact that applications for StandICT.eu grants for supporting European contributions to ICT standardisation from these countries are quite limited.

Figure 2 Countries of the group chairs.

Figure 3 AI related activities: themes/challenges/areas vs. SDOs and SSOs.

Figure 4 Distribution of European chairs.

Figure 5 presents an overview on the different kind of groups addressing AI themes in the SDOs and SSOs. Despite the fact that there are four times more working groups than study or research groups a closer look on the goals of the different working groups (depicted in Figure 6 shows that as of end of early 2019 about half of the activities are not focusing on defining a standard but rather focus on studying the respective fields and aim at delivering technical reports (which might well be the base for a new standardisation activity later).

Figure 5 Relation of number of working groups to study groups per SDO and SSO.

Figure 6 Relation of standardisation to pre-standardisation activities.

Figure 7 provides an overview on themes and the number of AI activities. The themes Ethics, Trustworthiness and Transparency of Autonomous Systems all relate to activities aiming at ensuring that humans fully remain in control of AI technologies and include the highest number of activities. This coincides well with the current goals of the EU as presented for example in their Communication and its Annex [3, 6, 9] or with the findings of the first report of the High Level Expert Group on AI [11].

Figure 7 Themes and number of activities.

The second highest number of activities can be found in the theme usage of AI. The reason for this is that most of the SDOs and SSOs observed in this report define different (in general non-overlapping) use-cases depending on the focus of their AI standardisation activities.

The third highest number can be observed in the Big Data theme. This is due to the fact that the ISO/IEC JTC1 WG9 (Big Data) program was merged with JTC1 SC42 (Artificial Intelligence) in 2018 because of the increased relevance of (big) data in several AI approaches. The previous Big Data activities are now part of the Artificial Intelligence standardisation activities of ISO/IEC JTC1 SC42, though the integration with the other themes of SC42 still needs to be done.

Similar to AI usage, the theme AI foundational standards comprises activities with different foci.

As shown in Figure 8 early 2019 the number of Europeans chairing groups in the different SDOs and SSOs with activities related to AI outperforms those of the individual other countries and equals the number of these countries together. This indicates that there is a strong European guidance in the development of the respective standards. However, since the nationality of the active members is only available to the members of the individual groups the active participation of Europeans beyond the chair cannot be determined.

Figure 8 Country distribution of the group chairs.

The StandICT.eu approach to support European presence and contribution for this groups and especially for those not chaired by a European is therefore considered useful to increase the European impact.

3.3 Maturity of the Standards

Q3 2019 only three AI-related standards have been published yet. As can be seen in Figure 9 most groups are declaring their work being in progress. The three standards being published stem from the work of JTC1 WG9 and are standards for Big Data which now are part of the Artificial Intelligence activities after WG9 has been merged with JTC1 SC42.

The large number of standards activities being work in progress on the other hand clearly indicates the opportunity for Europeans to engage in and contribute to these groups and their respective standards.

Figure 9 AI standards maturity.

4 Future Priorities and Conclusions

The European Commission has launched several activities resulting in recommendations and plans to support the European engagement in world-wide ICT standardisation. A particular focus will be on Europe’s approach for AI and how to increase the European impact in the worldwide efforts to establish AI as broadly available and used technology.

Future priorities are developed and will be guided by these activities. Some more relevant being:

The Multi Stakeholder Platform on ICT Standardisation [17] and the annual Rolling Plan on ICT Standardisation [1]
The High Level Expert Group on Artificial Intelligence [11]
The AI Alliance [2]
The Coordinated Plan on Artificial Intelligence [9] and its Annex [6]
The Artificial Intelligence: A European Perspective [4]
The AI Watch realised and maintained by EC’s JRCs Ispra and Sevilla [5]

The activities are beneficial in several dimensions: The Rolling Plan on ICT Standardisation provides a comprehensive view on the ICT standardisation landscape that is considered relevant for building the Digital Single Market. The work of the High Level Expert Group on Artificial Intelligence puts emphasises on human-centric artificial intelligence by focusing on trustworthy AI as their north star, since human beings will only be able to confidently and fully reap the benefits of AI if they can trust the technology. The report Artificial Intelligence: A European Perspective as a comprehensive European view of Artificial Intelligence under different perspectives, e.g., legal, ethical, economical, cyber security. The Coordinated Plan on Artificial Intelligence and its Annex do not particularly address standardisation but it provides an analysis of the current situation regarding AI in Europe and defines actions to be taken in order to sustain the European competitiveness in the field and to advance the European industry. The AI Watch monitors the development, uptake and impact of Artificial Intelligence for Europe and aims at providing a comprehensive and detailed analysis of the field.

With this article StandICT.eu provides an overview on the state of the standardisation in the area of Artificial Intelligence as of early 2019.

The report presents the SDOs and SSOs with active groups working on AI. The work of the groups is presented in detail and forms the major part of the article.

The information provided clearly indicates that there are a number of significant activities though little outcome with respect to standardisation. Based on this information the report provides an initial analysis of the AI standardisation landscape. The analysis suggests that there is a significant European share in group chairing but there is still opportunity for European experts to contribute to the work in progress.

Acknowledgements

Research and effort to write this article have been co-funded within the project StandICT.eu by the European Union’s Horizon 2020 (H2020) research and innovation programme under Grant Agreement no 7804391.

¹Institute of Electrical and Electronics Engineers [13].

²International Organization for Standardization [15]/International Electrotechnical Com mission [12].

³International Telecommunication Union-Telecommunication Standardization Sector [16].

⁴European Telecommunications Standards Institute [10].

⁵European Committee for Standardization [7]-European Committee for Electrotechnical Standardization [8].

⁶World Wide Web Consortium [20].

⁷Internet Research Task Force [14].

References

[1] GROW/F3 – Rolling Plan on ICT standardisation 2018. Website. Online at https://joinup.ec.europa.eu/collection/rolling-plan-ict-standardisation/rolling-plan-2018-revision, visited 25 August 2019.

[2] European AI Alliance. Website. Online at https://ec.europa.eu/digital-single-market/en/european-ai-alliance, visited 25 August 2019.

[3] Communication of the EC on Artificial Intelligence. Website. Online at http://ec.europa.eu/newsroom/dae/document.cfm?doc_id=51625, visited 25 August 2019.

[4] Artificial Intelligence: A European Perspective. Website. Online at https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/artificial-intelligence-european-perspective, visited 25 August 2019.

[5] AI Watch – Monitor the development, uptake and impact of Artificial Intelligence for Europe. Website. Online at https://ec.europa.eu/knowledge4policy/ai-watch_en, visited 25 August 2019.

[6] Annex: Coordinated Plan:. Website. Online at https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=56017, visited 25 August 2019.

[7] European Committee for Standardization. Website. Online at https://www.cen.eu/, visited 20 January 2020.

[8] European Committee for Electrotechnical Standardization. Website. Online at https://www.cenelec.eu/, visited 20 January 2020.

[9] Coordinated Plan on Artificial Intelligence (COM(2018) 795 final). Website. Online at https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=56018, visited 25 August 2019.

[10] European Telecommunications Standards Institute. Website. Online at https://www.etsi.org/, visited 20 January 2020.

[11] High-Level Expert Group on Artificial Intelligence. Website. Online at https://ec.europa.eu/digital-single-market/en/high-level-expert-group-artificial-intelligence, visited 25 August 2019.

[12] International Electrotechnical Commission. Website. Online at https://www.iec.ch/, visited 20 January 2020.

[13] Institute of Electrical and Electronics Engineers. Website. Online at https://www.ieee.org/, visited 20 January 2020.

[14] Internet Research Task Force. Website. Online at https://irtf.org/, visited 20 January 2020.

[15] International Organization for Standardization. Website. Online at https://www.iso.org/, visited 20 January 2020.

[16] International Telecommunication Union – Telecommunication Standardization Sector. Website. Online at https://www.itu.int/en/ITU-T/, visited 20 January 2020.

[17] Multi Stakeholder Platform on ICT Standardisation. Website. Online at http://ec.europa.eu/growth/industry/policy/ict-standardisationen#platform_ict_standardisation, visited 25 August 2019.

[18] StandICT.eu European Project Web Portal. Website. Online at https://www.standict.eu/, visited 25 August 2019.

[19] Communication: ICT Standardisation Priorities for the Digital Single Market. Website. Online at http://ec.europa.eu/newsroom/dae/document.cfm?doc_id=15265, visited 25 August 2019.

[20] World Wide Web Consortium. Website. Online at https://www.w3.org/, visited 20 January 2020.

Biography

images

Wolfgang Ziegler holds a doctoral degree in informatics and a diploma in informatics and has over 30 years of experience in different positions and fields of applied research. Since 1987 he was a scientist in the German national research centre Gesellschaft für Mathematik und Datenverarbeitung (GMD), which was merged with the Fraunhofer Gesellschaft in 2002. For more than 17 years he is working in the Fraunhofer Institute SCAI. Since 2003 he is head of the Grid and Cloud Middleware Research Group of the Bioinformatics department of SCAI and has a long record in distributed computing research and development with a focus on Cloud Computing over the last years, research on Service Level Agreements and software licencing technologies for distributed environments. He has a long record in standardisation activities, e.g. in the Open Grid Forum since its beginning. He is a co-chair of the Open Grid Forum working group responsible for the WS Agreement and WS-Agreement Negotiation specification. Moreover, he is area director for applications of the Open Grid Forum and since mid of 2015 Vice President of Community of the Open Grid Forum. He was a member of the working group on SLA management of the Telemanagement Forum, contributed in the ETSI Cloud Standardisation Coordination phase 1 task force and was a member of the ETSI experts for the Cloud Standardisation Coordination phase 2 project funded by the European Commission. He is a member of the German national standards body DIN and active in several working groups of ISO JTC1. Since 2018 he is active in the European project StandICT.eu.