Author: Trusted Autonomous SystemsPublished: 28/06/2022Category: Body of KnowledgeLast updated: 08/08/2022

Key concepts: Assurance

Assurance is the quality of being certain about the truth of some claim, and is usually achieved through test and evaluation in compliance with specific standards or requirements.¹ For example, a vessel being surveyed by an accredited marine surveyor is a form of assurance. Another way to view assurance is the minimisation of uncertainty about a claim, for example compliance with specific technical standards, being true (or not true). In the context of the operation of autonomous systems in the air, maritime or land domains, we intrinsically have some notion of what an acceptably safe outcome for operations is (i.e. I should be able to get on a plane or ship and be very certain that I will be able to get off at my destination without harm).

Safety Assurance of these autonomous vessels, aircraft, vehicles and associated operations, personnel, organisations and external equipment are all the processes, checks, requirements etc. that are undertaken to confirm that the specific regulatory requirements are met, and therefore minimise uncertainty related to an operation being able to achieve safe outcomes.

Assurance provides a compelling case for the trustworthiness of a component, subsystem, system, algorithm, or entire operation. For example, part of the assurance process for an autonomous vessel would include a series of simulation-based testing, followed by physical testing (i.e. sea trials), together with the same, or similar, series of inspections from an accredited marine surveyor that a traditional vessel is subject to.²

Assurance can be viewed as a process or methodology for gaining trust and validation. For this process to be trusted, it must be fair, equitable, and transparent.

The assurance of RAS-AI is a complex problem, increased by conditions of uncertainty. Autonomous robots differ from most machines because of the computational components that lead to their intelligence and control. They also differ from most existing software systems because of their integration with physical machines. RAS-AI rely on observations perceived by sensors to decide how they should behave and control themselves, which is then translated to actuators, enabling robots to change behaviour, or state, and change in response to their physical environments.

A reliable and robust autonomous robot must be able to plan and make strategic decisions about behaviour and control, despite enduring uncertainty plaguing the physical world. At the very least, these systems must be able to handle the following three types of uncertainty: Nondeterministic effects of actions, partial observability due to errors and limitations in a system’s sensors and perceptions, and lack of information about the environment and its dynamics. The enduring nature and variety of uncertainty affects all components of an autonomous robot, prompting the need to assure a system at a range of levels, including algorithms, software, and hardware.

Assurance requirements pose multi-faceted issues:

1. Novel technical approaches for those assurances are required. The ability for users to easily update algorithms and software, which alters the performance of a system, implies that traditional machine assurance performed prior to deployment or sale, will no longer viable. Moreover, the high frequency of updates implies that tradition al certification that requires substantial time will no longer be practical. To alleviate these difficulties, automation of assurance will likely be needed; something like ‘ASsurance-as-a-Service’ (ASaaS), where APIs constantly ping RAS-AI to ensure abidance with various rules, frameworks, and behavioural expectations. There are exceptions to this, such as in contested or communications denied environments, or in underground or undersea mining; and these systems need their own risk assessments and limitations imposed. Indeed, self-monitors are already operating within some systems.
2. The assurance process will require stakeholders to possess sufficient technological and computational skills. Therefore, to ensure safe operation of future robotics systems, Australia needs to educate and prepare its technology developers, certifiers, and general population for more sophisticated assurance processes.
3. What would be the suitable regulatory environment for autonomous systems? The regulation content in the Knowledge Hub deals with the existing regulatory environment for various domains and how this is evolving—or needs to evolve—with the introduction of more autonomous systems.”³

Assuring elements of autonomous systems

Algorithms Algorithms are processes, heuristics, or sets of rules that are intended to achieve some form of outcome. These are not necessarily computational methods. One example of an algorithm is how an autonomous robot processes sensor data and makes decisions about its behaviour and control, so as to operate reliably and robustly to achieve pre-specified tasks. Algorithmic assurance is about reducing uncertainty of the implemented algorithm with respect to the desired function that needs to be undertaken. Assurance includes understanding the intended operational environmental, the way the algorithm models the world, and the way in which the function meets the expected requirements.

Software or Complex Electronic Hardware Currently, algorithmic assurance is generally managed within software development processes. They are not treated separately but should be. It is important to differentiate between algorithms and their software/complex electronic hardware implementations. Poor implementation of powerful algorithms will result in poor performance of an autonomous robot. Therefore, developers need to follow good software engineering practices. Software creation for safety related control systems of industrial robots is well described in standards such as IEC61508 and the machine specific version IEC62061. In aviation, the standard used for software design assurance onboard manned aircraft is RTCA DO-178C (and associated supplements). Furthermore, software testing should include whether the implementation of an algorithm satisfies the properties of the algorithms, both in terms of correctness and computational complexity. Examples of dangerous faults due to poor programming practices are abundant, including the unintended acceleration issue in Toyota vehicles, causing fatal crashes and a recall of multiple Toyota models

Effectors/Hardware The interface between algorithms and the external world are facilitated by effectors/ hardware. New types of hardware, such as soft and compliant robots, pose new difficulties in assurance. Such hardware is designed to increase safety. However, the dynamics of these robots depend on the interaction between the robot and their environments, which in turn raises questions on how to properly assure such machines.

Integrated Robotic System This assurance should ensure the system remains reliable in the event that one or more of its components are erroneous, including on seemingly simple minuscule issues, such as when the battery is low. System safety processes, such as SAE Aviation Recommended Practice 4761,9 are being updated to focus on the complex interactions between systems, rather than at the subsystem level.

---

1. Ortega, P. A., V Maini, and DeepMind Safety Team. “Building Safe Artificial Intelligence: Specification, Robustness, and Assurance.” DeepMind Safety Research (blog), September 27, 2018. https://deepmindsafetyresearch.medium.com/building-safe-artificial-intelligence-52f5f75058f1. ↩
2. Horne, R. “Autonomous and Remotely Operated Vessels 2021 to 2040, MIAL Future Leaders White Paper. Predictions for the Australian Maritime Industry 2040.” Maritime Industry Australia Limited, 2021. ↩
3. Devitt, S. K., R. Horne, Z. Assaad, E. Broad, H. Kurniawati, B. Cardier, A. Scott, et al. “Trust and Safety.” ArXiv:2104.06512 [Cs], April 13, 2021. http://arxiv.org/abs/2104.06512. ↩