Could Microsoft’s AI Copilot Expose Your Confidential Emails?

Microsoft has recently confirmed an error in its AI work assistant, Microsoft 365 Copilot Chat, which mistakenly accessed and summarised confidential emails from users. This incident highlights the risks associated with rapidly evolving generative AI tools in workplace settings. Microsoft has since issued a fix to prevent such occurrences and reassured users that their data protection policies remain intact.

Last updated: 15 October 2023 (BST)

What’s happening now

Microsoft has acknowledged an issue with its AI-powered assistant, Microsoft 365 Copilot Chat, which inadvertently revealed confidential email content to some enterprise users. The bug allowed the tool to pull information from emails stored in users' drafts and sent items, including those marked with confidentiality labels. This error raises concerns about data security, especially as the technology is integrated into widely used applications like Outlook and Teams.

Key takeaways

Microsoft 365 Copilot Chat mistakenly accessed confidential emails, pulling data from drafts and sent folders.
The company has rolled out a fix and insists that access controls and data protection remained intact.
Experts warn that the rapid pace of AI development can lead to such errors becoming more common.

Timeline: how we got here

Here's a brief timeline of the events leading up to the current situation involving Microsoft 365 Copilot Chat:

January 2023: Microsoft reportedly becomes aware of the bug affecting Copilot Chat.
October 2023: Microsoft confirms the issue publicly following reports from tech news outlets.
October 2023: A configuration update is deployed worldwide for enterprise customers to rectify the issue.

What’s new vs what’s known

New today/this week

Microsoft has publicly addressed the error that led to confidential emails being summarised by Copilot Chat. They have implemented a configuration update intended to prevent this from happening again. The company states that no unauthorised access to information occurred, but the incident has raised significant concerns about data privacy and security in AI applications.

What was already established

Prior to this incident, organisations using Microsoft 365 Copilot Chat were assured of strict data protection measures. However, the recent blunder has revealed potential vulnerabilities in how AI tools interact with sensitive information, prompting a reevaluation of these safeguards.

Impact for the UK

Consumers and households

For regular users of Microsoft products, this incident may cause concern about the safety of their data. Users must be cautious about the information they share within AI-assisted tools, particularly those that may inadvertently expose sensitive content.

Businesses and jobs

The blunder raises questions for businesses relying on Microsoft 365 Copilot Chat. Companies may need to assess their use of generative AI tools and review data protection policies to mitigate risks associated with AI features that process sensitive information.

Policy and regulation

In light of this incident, there may be increased scrutiny on data privacy regulations, particularly concerning AI tools in the workplace. Businesses might face pressure to implement stricter controls and governance frameworks as they adopt these technologies.

Numbers that matter

1: The number of major incidents reported regarding the privacy breach in Microsoft 365 Copilot Chat.
7: The number of months since Microsoft first became aware of the issue (January to October 2023).
365: The number of days in a year, underlining the need for continuous vigilance when using AI tools.

Definitions and jargon buster

Microsoft 365 Copilot Chat: A generative AI chatbot integrated into Microsoft applications like Outlook and Teams to assist users in managing emails and tasks.
Confidential Label: A designation applied to email content that indicates it should be treated as sensitive and not shared with unauthorised individuals.
Data Loss Prevention (DLP): Security measures that prevent the accidental sharing of sensitive information outside of an organisation.

How to think about the next steps

Near term (0–4 weeks)

In the immediate future, organisations using Microsoft 365 Copilot Chat should monitor updates from Microsoft regarding security patches and ensure they are applied promptly. Training for employees on safe usage of AI tools should also be reinforced.

Medium term (1–6 months)

Companies should consider conducting comprehensive audits of their data protection policies and the tools they currently employ. This may involve reassessing AI tool usage and implementing stricter governance frameworks to safeguard sensitive information.

Signals to watch

Updates from Microsoft regarding further improvements to Copilot Chat's security features.
Changes in data privacy regulations that may impact the use of AI tools in businesses.
Emerging best practices from industry leaders on managing generative AI tools safely.

Practical guidance

Do

Regularly update software to ensure all security patches are applied.
Train staff on data protection policies and safe AI tool usage.
Monitor communications from Microsoft regarding any changes or updates to their AI products.

Don’t

Neglect to review your company’s data protection policies in light of new AI features.
Assume that all AI tools are secure without due diligence.
Overlook the importance of feedback mechanisms for reporting AI-related issues.

Checklist

Verify that all employees are trained on data security practices.
Ensure that software updates are regularly scheduled and applied.
Conduct regular audits of AI tool usage within your organisation.
Establish clear protocols for managing sensitive information.
Develop a response plan for potential data breaches involving AI tools.

Risks, caveats, and uncertainties

The recent incident with Microsoft 365 Copilot Chat is a stark reminder of the potential risks associated with adopting AI tools in business environments. While Microsoft asserts that no unauthorised access occurred, the rapid development of generative AI features raises the likelihood of similar issues in the future. As companies rush to integrate new technologies, there may be gaps in data protection measures that could be exploited. Vigilance and robust governance are essential to mitigate these risks.

Bottom line

The recent error involving Microsoft 365 Copilot Chat underscores the need for businesses to tread carefully as they adopt generative AI tools. While Microsoft has addressed the bug, the incident serves as a wake-up call for organisations to reassess their data protection policies and ensure robust safeguards are in place. As the use of AI continues to grow, maintaining vigilance will be critical to preventing future data breaches.

FAQs

What is Microsoft 365 Copilot Chat?

Microsoft 365 Copilot Chat is an AI-powered assistant integrated into Microsoft applications that helps users manage tasks such as summarising emails and answering questions.

How did the error occur?

The error occurred due to a code issue that caused Copilot Chat to process users' confidential emails incorrectly, leading to the summarisation of sensitive content from drafts and sent items.

What should organisations do to protect data when using AI tools?

Organisations should regularly update their software, train employees on data protection policies, and conduct audits of their AI tool usage to ensure sensitive information is safeguarded.