Automated decision-making transparency—what APP entities need to know about the APP 1 amendments

The information to be disclosed in the privacy policy is:

1. the kinds of personal information used in the operation of such computer programs; and
2. the kinds of such decisions made solely by the operation of such computer programs; and
3. the kinds of such decisions for which a thing, that is substantially and directly related to making the decision, is done by the operation of such computer programs.

For the purposes of subclauses 1.7 and 1.8:

1. making a decision includes refusing or failing to make a decision; and
2. doing a thing includes refusing or failing to do a thing; and
3. a decision may affect the rights or interests of an individual, whether the rights or interests of the individual are adversely or beneficially affected; and
4. the following are examples of the kinds of decisions that may affect the rights or interests of an individual:
   1. a decision made under a provision of an Act or a legislative instrument to grant, or to refuse to grant, a benefit to the individual;
   2. a decision that affects the individual’s rights under a contract, agreement or arrangement;
   3. a decision that affects the individual’s access to a significant service or support.

The definition of ADM will capture a broader range of decisions than in other jurisdictions such as the EU GDPR, which targets decisions 'based solely on automated processing'. This means that APP entities also subject to the EU GDPR and deploying ADM systems globally will need to consider if they must be disclosed in their Privacy Policy, even if they do not meet the requirements for disclosure under GDPR.

The OAIC's commentary on the 'substantially and directly related' limb indicates that human oversight does not, by itself, prevent ADM disclosure. A computer program or formula that recommends a decision or guides a human decision-maker (including by categorising a person based on personal information) can still be 'substantially and directly related' to making a decision where the recommendation, guidance or categorisation is a key factor in the human's decision-making and has a direct connection with making the decision.⁴

This implies a broad range of decisions are intended to be captured. There is uncertainty as to the boundaries of this but, based on the current drafting, it is likely that mere automated execution of decisions made by humans would be captured. This would capture a large volume of modern processes that would need to be identified and disclosed in privacy policies.

The OAIC is inviting views on what factors are relevant to whether a computer program is 'substantially and directly related' to the human decision-maker's decision.

Whether a decision could reasonably be expected to significantly affect an individual's rights or interests may depend on the circumstances. Eg a decision may have greater impact on a child or vulnerable person than on individuals generally.

The Issues Paper states that '[t]he effects must be more than trivial, and must have the potential to significantly influence the circumstances of the individual concerned'.⁵

It refers to European Union (EU) guidelines on automated decision-making that also emphasise the distinct impact on minority groups and vulnerable adults. Examples from those guidelines are:

• a person known or likely to be in financial difficulties who is regularly targeted with adverts for high interest loans, and may sign up for these offers and potentially incur further debt;
• ADM resulting in differential pricing based on personal data could have a significant effect if eg prohibitively high prices effectively bar someone from certain goods or services; and
• if a credit card company reduced a customer’s card limit based on criteria such as an analysis of other customers living in the same area or shopping at the same stores, this could deprive someone of (or provide with) opportunities based on others' actions.⁶

The specific factors to consider when determining a 'significant impact' on rights or interests are open for consultation, including the extent to which vulnerability needs to be taken into account.

The Issues Paper clarifies that 'the extent of disclosure to ensure compliance with APP 1.7–1.9 will require a balance between providing enough meaningful information for individuals to understand the use of ADM and avoiding excessive detail that obscures the purpose of transparency and clear communication'.⁷

In considering best practice, the OAIC refers to various publications and international guidance, and summarises that entities must provide meaningful and accessible information to consumers. Disclosures should be:

• clearly articulated in plain language and easy to understand;
• structured to enable consumers to request further information, where required;
• appropriately tailored, being sufficiently specific to be meaningful, while avoiding
  overwhelming levels of detail;
• organised so that similar information is grouped in a logical manner; and
• framed in a way that allows the information and the decision to be challenged or contested.⁸

Critically, the level of granularity of disclosure required remains unclear. For example, it does not provide guidance as to whether organisations will need to distinguish in their policies between decisions solely made by a computer, as opposed to where a computer does a thing substantially and directly related to making a decision. The level of detail of required disclosure will have a significant impact on operationalisation and the compliance burden.

While the Issues Paper and the Explanatory Memorandum¹⁰ provide anchor examples,¹¹ the precise boundary of what is 'substantially and directly related' to making a decision in complex, real-world workflows remains unclear.

The OAIC is seeking views on relevant factors, including the degree of reliance on ADM output; the ability and likelihood of human override; the nature of the output (advisory versus determinative); the transparency and explainability of outputs; and the integration of ADM into the decision-making workflow.¹² How these factors interact and their relative weight is unresolved.

It is also unclear whether privacy policies will need to distinguish between decisions made solely by computer programs, and those substantially and directly related to a human making a decision.

The OAIC engages with the meaning of 'decision' across Australian legal frameworks, including administrative law and corporations law, as well as the EU GDPR. In those contexts, the definitions are broad, and include but are not limited to:

• making, suspending, revoking or refusing to make an order or determination or to issue a licence, authority or other instrument;
• giving, suspending, revoking or refusing to give a certificate, direction, an approval, consent or permission;
• imposing a condition or restriction;
• making a declaration, demand or requirement;
• retaining or refusing to deliver up an article;
• doing or refusing to do any other act or thing;
• something practically final or operative and determinative of an issue of fact;
• a measure evaluating personal aspects relating to a person.¹³

Specifically, the Issues Paper raises targeted advertising in a recruitment scenario, and whether a candidate not receiving a job advertisement (that then limits their employment options) is a 'decision'.¹⁴ Given it is not a determinative or definitive decision, or one that goes to the actual ability to apply for or receive suh a job, only its visibility, it would have broad implications where the scope of decision be deemed to cover such examples. Consultation responses on such edge cases to ensure that the boundaries of this obligation are not overly broad will be critical.  

While the Issues Paper provides useful examples of what it means to 'arrange for' ADM versus merely to 'operate' ADM, the boundary between these concepts in complex supply chains is uncertain—particularly where an entity uses embedded ADM features in commercial software without specifically procuring or directing those features.

The OAIC has acknowledged that further scenarios may require guidance and has invited submissions on this point.¹⁵

The factors that increase the likelihood that a decision could affect an individual's rights or interests remains subject to consultation. The OAIC asks what classes of persons should be considered vulnerable for these purposes, and why.

Guidance on this will be critical for entities that deal with potentially vulnerable cohorts.

The appropriate level of granularity to be included in privacy policy disclosures—eg whether entities must describe specific algorithms or only categories of ADM activity—is not yet clarified.

The OAIC will need to consider when setting guidance that a broad and uncertain scope could lead entities to over disclose by including dense, detailed information in privacy policies. Paradoxically, the more information included to hedge against non-compliance, the less comprehensible the policy becomes to the individuals it is designed to protect—undermining APP 1's transparency goal.

The ACCC has previously warned against information overload, as part of its digital platforms enquiry.