Author: Trusted Autonomous SystemsPublished: 28/06/2022Category: Body of KnowledgeLast updated: 08/08/2022

Key concepts: Trust

Trust is “…the firm belief in the reliability, trust or ability of someone or something” and “is central for the acceptance and adoption of any technology, and it has both an intrinsic and an instrumental value… for autonomous vessels to be deployed, a wide societal trust in those vessels is needed, otherwise their deployment will be compromised.”¹

As robotics, autonomous systems, and AI become more widespread, existing safety and assurance processes will be challenged across industries. Ensuring that these systems are designed responsibly and robustly will be key to safeguarding trust.²

Trust exists in autonomous systems when their actions are coherent and justified under the circumstances, or where information from the autonomous system can be taken in good faith. Trust is contextual and is a function of the user’s experience.

Traditional technology is generally deemed safe if it complies with the regulatory framework set down and enforced by the Government or applicable regulator. For example, the Australian public would be entitled to assume a commercial vessel is safe if it has met the survey, operational, and competency-based requirements imposed by the Australian Maritime Safety Authority.

Trust is also supported by history of successful use – i.e., if a technology has been in use for one hundred years, there is a good understanding of the risks of failure and the results. Familiarity – if a technology has widespread usage and is visible, e.g. represented in news and popular media – and trust by association – e.g. an entity or person that is trusted uses or promotes use of a new technology – can also play a central role.

Autonomous technology must be deemed safe to be accepted by prospective users, Government, the applicable regulator, and the public. This would be a simpler proposition if the Government or applicable regulator had crafted well-tailored assurance and accreditation requirements, but that is rarely the case. The speed of technology development in the autonomous systems industry means that, early on, the central experts are the developers of the technology itself, and then the industry which is using it. In these circumstances industry-led projects to develop codes of practice or other regulatory content are vital, as they provide: a baseline level of requirement to ensure safety among early-adopters, a starting point for developing an appropriate regulatory approach, and a level of confidence for members of the public.

An example of an industry-led collaboration is the Maritime UK Autonomous Systems Regulatory Working Group (MASRWG), which published a Code of Practice for Maritime Autonomous Ships in November 2017 and have provided an updated version each 12 months. This Code provides a baseline set of requirements covering key issues such as design and manufacture, operation, skills and training, ship systems, cybersecurity, pilotage, and data recording. While the Code is not a formal part of the UK maritime regulatory framework, it is used by industry as the accepted standard for autonomous vessels and reflects current best practice.

A challenge for the Government or applicable regulator is to properly balance risk with regulatory overlay. If there is insufficient regulatory overlay to meet the risk of the operation, it will jeopardise trust in the technology and its regulation, but if the regulatory overlay is too heavy, it can stifle innovation and jeopardise trust in the intentions of the regulator. For this reason, adopting an appropriate regulatory approach is critical to ensuring innovation is supported, the benefits of autonomous systems are realised and trust is maintained between all stakeholders involved.”³

Risk & Safety: In Australia, there is a lack of established assurance frameworks to integrate autonomous systems into traditional systems, which means neither industry, testing facilities, nor regulators, have clear or consistent expectations or understanding of what assurance activities are required to demonstrate compliance with requirements to indicate safe operations. This is inefficient and compromises trust, which makes it a priority to address.

---

1. Glomsrud, J. A., A. Ødegårdstuen, A. L. St Clair, and Ø Smogeli. “Trustworthy versus Explainable AI in Autonomous Vessels.” In Proceedings of the International Seminar on Safety and Security of Autonomous Vessels (ISSAV) and European STAMP Workshop and Conference (ESWC) 2019, 37–47, 2019. https://doi.org/10.2478/9788395669606-004. ↩
2. Devitt, S. K., R. Horne, Z. Assaad, E. Broad, H. Kurniawati, B. Cardier, A. Scott, et al. “Trust and Safety.” ArXiv:2104.06512 [Cs], April 13, 2021. http://arxiv.org/abs/2104.06512. ↩
3. Horne, R., T. Putland, and M. Brady. “Regulating Trusted Autonomous Systems in Australia.” University of York: Assuring Autonomy International Programme, 2021. ↩