Author: Trusted Autonomous SystemsPublished: 28/06/2022Category: Body of KnowledgeLast updated: 08/08/2022

Key concepts: Accreditation

Accreditation is a broad term that refers to the permissioning mechanism that an autonomous system and its operator is required to hold to lawfully operate. It can include certification, accreditation, licences, permits, approvals, and determinations.

Currently, most autonomous systems across the air, land and maritime domains are subject to accreditation frameworks within existing  regulatory frameworks. For example, in the maritime domain, a vessel operator must hold a certificate of operation or an exemption from that requirement; a vessel must be the subject of a certificate of survey or an exemption from that requirement; and the vessel master and crew must hold the required certificates of competency or an exemption from that requirement.¹

In the context of safety regulation, ‘accreditation’ arrangements such as certification can be understood as ‘gatekeeping’ mechanisms by which a person or company cannot operate an artefact (be it an aircraft, vessel, car, truck or train) unless and until they have satisfied the criteria for issue of the relevant document. These criteria are generally tied to demonstrated compliance with regulations, standards, guidelines or codes of practice, and relate to a range of issues, including design, construction, survey, inspection, equipment, operation and crewing. Depending on the scheme, the regulated party may be required to obtain a report or assessment from a third party stating that the regulated party meets the required standards for issue of accreditation.²

Accreditation frameworks have, to date, been focussed on persons, companies (corporate persons), or artefacts. These frameworks needs to be adapted to encompass autonomous systems, as there are no specific standards or codes of practice that relate to autonomy. While new standards and codes are under development, it is likely that in the short term, they will be applied in combination with existing frameworks.

Accreditation frameworks often work alongside a suite of broad-based primary or general safety duties on designated duty holders,³ including a requirement that certain parties implement, maintain and/or comply with a safety management system. Safety management systems require varying levels of approval, verification, and oversight, depending on the regulatory framework. While certification and other permission-focussed schemes impose specific duties on the regulated party, general safety duties impose a broad obligation to ensure safety ‘so far as reasonably practicable’ in relation to persons doing things in relation to a particular thing or operation.

Risk-based accreditation frameworks

Through the increasing proliferation of ‘cooperative’ transport safety schemes involving federal, state and territory government cooperation, there has been increasing acceptance that in a rapidly changing environment, accreditation frameworks must be premised on a risk-based regulatory approach.⁴ This recognises the diversity of modern transport applications and that a ‘one size fits all’ approach, or creating prescriptive rules for each different application, is impractical and undesirable. Rather than creating multiple different schemes, accreditation frameworks should be ‘tiered’ to encompass different levels of risk.⁵ Under this approach, all regulated parties must meet the same objectives (for example, to be operated safely) but the best way to achieve those objectives varies between parties. This is not about setting different standards for different regulated parties but letting different parties show common objectives are met to the same standard in a manner suited to their operations. From a government perspective, risk-based approaches guide the effort level they should dedicate to particular classes of regulated parties. For example, this may involving scaling or tiering certification or permissioning requirements based on the risk profile of the operator, system or artefact. Under this approach:

• lower risk accreditation measures: regulated parties self-assess their compliance without independent oversight. An operator may be permitted to operate ‘as of right’ – that is, without making any application – so long as they ensure compliance with certain required outcomes
• medium risk accreditation measures: the certification or permissioning function is given or delegated to a third party instead of the government or regulator so that they are able to issue certification or permissions on the government’s behalf. The third party is usually subject to some kind of oversight process by government or the regulator
• high risk: the government or regulator retains the certification function, and conducts positive verification measures to establish compliance with the criteria for issue of that certification.⁶

Key flexibility mechanisms

Modern accreditation frameworks generally have a number of flexibility mechanisms. These can include:

• tiered legislation and regulation: properly tiered legislation set requirements in primary legislation – for example, the requirement for an operator to obtain a certificate of operation in the maritime domain. The legislation states the kinds of operators that require a certificate, and the criteria for the issue of a certificate may be prescribed in delegated legislation. Delegated legislation can be more easily amended and can in turn point to standards and codes of practice that are developed by regulators, standards authorities and industry bodies. This ensures that accreditation frameworks can maintain pace with technological developments.⁷ By contrast, legislation that is not risk-based can ‘hardwire’ the requirement for all operators to obtain certification, and limits or precludes any distinction in the requirements that different kinds of operators need to meet, and
• exemptions: some accreditation frameworks allow for the issue of exemptions from some or all the requirements of given legislation. This can assist when a regulated party can meet some, but not all, the criteria for accreditation. Alternatively, an exemption may be used to exempt a regulated party entirely from an accreditation requirement, on the basis that the party complies with certain conditions. Exemptions are only issued when, taken with conditions, doing so would not “jeopardise safety”, or “pose a significant safety risk”.⁸  Typically, exemptions are intended to provide short term relief from a particular requirement⁹
• equivalent means of compliance: in a scheme with performance-based standards that requires regulated parties to meet required outcomes, regulated parties may be able to choose between complying with a ‘deemed to satisfy solution’ or an equivalent means of compliance.¹⁰ Some schemes require an application for an equivalent means of compliance before they can be implemented, while others are allowed as of right so long as they meet specific criteria. Equivalent means of compliance must generally be as effective as the relevant deemed to satisfy solution in meeting the required outcome. They are usually of greatest utility in relation to physical structure (such as a vessel, aircraft, or heavy vehicle), and
• conditions: most accreditation frameworks allow for the regulator or certifier to impose conditions on a certificate or permissioning mechanism where it is considered necessary to ensure safe operations. There are usually legislative or regulatory criteria specifying or limiting matters that conditions can deal with.

Accreditation framework design and implementation

Risk-based accreditation frameworks require careful consideration of key factors at the legislative design stage. Key factors that impact whether an accreditation framework is risk-based:

1. whether the scheme designer – generally government – structures the legislation in close consultation with industry and includes inbuilt flexibility around the application of accreditation requirements. Previously, transport regulation often treated all regulated parties as ‘high risk’, regardless of their specific profile. However, in a risk-based scheme, government must not only support its proposed accreditation approach with risk-based evidence, but it must seek industry feedback on whether its approach is appropriate and risk analysis is correct.¹¹
2. whether participation in an accreditation scheme is voluntary or mandatory.¹² Where participation is voluntary, regulated parties choose either to follow prescriptive rules or to participate in an accreditation scheme, as long as they meet the scheme’s requirements. Where participation is mandatory, the scheme works as a permissioning mechanism or a licence to undertake an activity. Regulated parties have to give assurance they have the capacity and systems to manage the risk if they want to operate. Most transport accreditation schemes have mandatory components; however, some may make it voluntary but provide incentives – for example, more flexibility – if they do obtain accreditation. In a risk-based scheme, it may be appropriate that only ‘high-risk’ systems are subject to mandatory accreditation frameworks,¹³ and
3. the regulatory approach of the regulator to the administration of the accreditation framework, including their exercise of delegated legislative powers, and internal business enablers and processes that enable them to ensure that they are maximising the utility of flexibility mechanisms. Not only must accreditation frameworks empower regulators to administer accreditation requirements in a risk-based manner, but regulators must ensure that they have the capabilities to do so. The flexibility that may be created by allowing an operator to apply to a regulator for an exemption from a particular requirement may be reduced or removed when:
   • the operator does not understand the regulatory framework and does not have prior operational data to illustrate safe operations, and the regulator does not understand the platform or the technology that will enable safe operation. For example, an operator may identify the use of an AI-based system to mitigate an operational risk but does not explain how it works or provide evidence of its efficacy.

---

1. There are a range of efforts underway across the maritime, land and air domains to adapt and prepare respective regulatory frameworks and arrangements to appropriately provide for autonomous systems. These frameworks are at varying levels of development and sophistication, and are largely being led by governments and regulators, both in Australia and internationally, as well as through partnerships and engagement with other key stakeholders. These include the development of new legislation, standards, codes of practice, and guidance documents.  In the shorter term, the focus appears to be on the adaption of existing frameworks; whereas going forward, there will be a greater focus on accreditation of ‘systems’ and supporting technologies and infrastructure (including AI, machine learning and cyber-security). ↩
2. NTC, “Assurance Models.” National Transport Commission, 2019. https://www.ntc.gov.au/sites/default/files/assets/files/NTC%20Issues%20Paper%20-%20Assurance%20models.pdf. ↩
3. For example, for the maritime domain, see Part 3 of Schedule 1 of the Marine Safety (Domestic Commercial Vessel) National Law Act 2012 (National Law), which imposes general safety duties on the master and owner of the vessel, among others; for the land domain, see Part 1A.1 of the Heavy Vehicle National Law, which imposes a primary safety duty on parties in the chain of responsibility. Broad duties apply in relation to workplaces under Part 2 of the model Work Health and Safety Act 2011, which is the basis of state and territory WHS legislation. Among others, these duties apply to persons conducting a business or undertaking, manufacturers, designers, and workers. ↩
4. Productivity Commission, “National Transport Regulatory Reform: Productivity Commission Inquiry Report.” Canberra, Australia: Commonwealth of Australia, 2020. 94. https://www.pc.gov.au/inquiries/completed/transport/report/transport.pdf. ↩
5. NTC, “Assurance Models.” ↩
6. NTC, “Assurance Models.” ↩
7. NTC, “A Risk-Based Approach to Regulating Heavy Vehicles.” National Transport Commission, 2019. https://www.ntc.gov.au/sites/default/files/assets/files/NTC%20Issues%20Paper%20-%20A%20risk-based%20approach%20to%20regulatiing%20heavy%20vehicles.pdf. ↩
8. See, for example, in the maritime domain section 143 of the National Law; and in the land domain, see section 70 of the HVNL. ↩
9. In the land domain, see for example: National notices | NHVR.  In the maritime domain, see for example: General and specific exemptions (amsa.gov.au). In the air domain, see for example: Non-legislative instruments | Civil Aviation Safety Authority (casa.gov.au) ↩
10. In the maritime domain, see for example: Performance-Based Standards (PBS) | NHVR. In the maritime domain, see for example: Equivalent means of compliance (amsa.gov.au). In the air domain, see for example: Exclusions and AMOCs to Airworthiness Directives | Civil Aviation Safety Authority (casa.gov.au) ↩
11. In the maritime domain, there are a number of exemptions from key accreditation requirements in the National Law, including the requirement for certain kinds of vessels to have a certificate of survey. This indicates that there was not adequate risk-focussed consultation on the certification requirements in the National Law when it was being designed. ↩
12. NTC, “Assurance Models.” ↩
13. For example, the European Commission is currently considering accreditation and assurance arrangements for AI, with the intended focus on ‘high risk’ AI. It has conducted extensive consultation with industry on what will constitute ‘high risk’ AI. See European Commission (2020) White Paper: On Artificial Intelligence – A European approach to excellent and trust Available from commission-white-paper-artificial-intelligence-feb2020_en.pdf (europa.eu) ↩