Using AI tools in your business creates privacy risks when personal information is entered into, disclosed through or generated by an AI system without the business first understanding how that information is handled. For Australian organisations covered by the Privacy Act 1988 (Cth), existing privacy obligations continue to apply when AI tools handle personal information, whether that is a staff member drafting an email with a chatbot or a recruitment tool screening job applications.
Businesses considering how AI adoption affects their obligations under the Australian Privacy Principles (APPs) can get tailored advice through Warlows Legal’s Technology, Privacy and Internet Law team, particularly where a new AI product is being introduced into existing systems or workflows.
How Personal Information Moves Through an AI System
AI privacy risk does not stop at what a staff member types into a prompt. A document upload, connected inbox or meeting transcription tool can all draw personal information into an AI system without someone deliberately entering each piece of information.
AI systems can also generate new personal information through inference, such as predicting a person’s characteristics from patterns in existing data. The Office of the Australian Information Commissioner (OAIC) confirms that where AI is used to generate or infer personal information, this is treated as a collection under APP 3. This means the collection must be reasonably necessary for the organisation’s functions or activities and carried out by lawful and fair means.
This makes data minimisation a practical starting point. Before entering material into an AI tool, businesses should ask whether personal information is actually needed for the task and whether identifying details can be removed first.
What to Check Before Selecting an AI Provider
Choosing an AI product involves more than comparing price and functionality. A business needs enough information about how the product works to assess the privacy risks involved in its intended use.
Due diligence should cover several practical questions. Has the product been tested for the specific use case your business has in mind? Does the provider retain inputs or outputs for further model training? Will the provider or another third party have access to what your business enters or generates? What security and access controls exist, and can higher-risk features be switched off? Where is the data processed, and could its use involve a cross-border disclosure of personal information?
The OAIC’s guidance on privacy and commercially available AI products recommends a privacy by design approach. Depending on the proposed use and level of privacy risk, a Privacy Impact Assessment can also help identify potential privacy impacts and appropriate safeguards before an AI product is adopted.
Know Where Your Information Goes
The scope of personal information an AI tool can reach often expands once it is connected to other business systems. An assistant linked to email, a customer relationship management platform or a document management system may have far broader access than the information employees type into individual prompts.
It is worth assessing an AI product within its actual operating environment rather than in isolation. Each new integration or enabled feature can change what personal information the tool can access and where that information may go. This makes it important to reassess the privacy risks as the technology becomes more embedded in the business.
Reviewing Privacy Policies and Collection Notices
Where AI changes how a business collects, uses or discloses personal information, existing privacy policies and collection notices should be reviewed to make sure they still accurately reflect what is happening.
APP 1 requires APP entities to maintain a clearly expressed and up-to-date privacy policy, while APP 5 requires reasonable steps to notify individuals of specified matters relating to the collection of their personal information.
The OAIC recommends that businesses provide clear and transparent information about relevant AI use in their privacy policies and notices. Public-facing AI tools, such as customer chatbots, should also be clearly identified as AI rather than presented as though a person is responding.
Setting Rules for Staff Use of AI Tools
Even where a business has approved certain AI products, staff may turn to free or consumer AI tools for everyday drafting, summarising or research. This can create privacy risks because personal information may leave the organisation without the safeguards applied to approved systems.
As a matter of best practice, the OAIC recommends that organisations do not enter personal information, particularly sensitive information, into publicly available generative AI tools because of the significant and complex privacy risks involved.
An internal AI use policy can name approved tools, set out what information should not be entered and give staff a clear approval process for new tools. Practical training can reinforce those requirements using examples relevant to everyday work.
Handling Sensitive Information in AI Systems
Sensitive information, including health information, biometric information and information about racial or ethnic origin, political opinions or criminal records, receives greater protection under the Privacy Act.
Where an AI system collects, generates or infers sensitive information, consent will generally be required under APP 3 unless an exception applies. The OAIC also makes clear that consent cannot simply be inferred because an individual has been given notice of the proposed collection.
Businesses should therefore set clear boundaries around which categories of information can be entered into approved AI systems and when further assessment or approval is required.
Human Oversight and Ongoing Monitoring
AI-generated information should not be treated as accurate simply because it reads convincingly. APP 10 requires APP entities to take reasonable steps to ensure the quality of personal information they collect, use and disclose. The OAIC also recommends having a human user responsible for verifying the accuracy of personal information generated or inferred by an AI system.
AI tools also change over time. Providers may update models, terms and default settings, while businesses may expand how a tool is used. Periodic reviews and human checks on relevant outputs can help ensure privacy safeguards continue to reflect how the technology is actually being used.
For wider privacy considerations, businesses can also review what the Privacy Act amendments mean for Australian start-ups.
Get Legal Advice About AI and Privacy
If your business is adopting AI or reviewing how existing tools handle personal information, Warlows Legal’s Technology, Privacy and Internet Law team can help you understand your privacy obligations and manage the legal risks.
Contact Warlows Legal to discuss your business’s use of AI.
The information in this article is general in nature and does not constitute legal advice about privacy obligations or the use of AI tools. Legal advice should be obtained for your individual circumstances.




