OpenAI did not respect Canada's privacy laws when launching ChatGPT, investigators find

OTTAWA — OpenAI did not respect Canada’s privacy laws when it launched and trained ChatGPT, a joint investigation from the federal and three provincial privacy commissioners has concluded.

Philippe Dufresne, Canada’s privacy commissioner, says the way the company collected Canadians’ data and information to train its early models was too wide-ranging and did not properly factor in concerns over individuals’ privacy, according to findings from the probe that the privacy bodies launched three years ago.

“W e find that the nature and scale of OpenAI’s collection and use of personal information from publicly accessible websites and licensed datasets, at the time of training its GPT-3.5 and 4 models, was overbroad and therefore not necessary and proportional,” the report read.

Dufresne and his provincial counterparts in Quebec, Alberta and British Columbia nevertheless said the company has since enacted measures to address how OpenAI collects Canadians’ personal information to train new models by limiting the scope of what is collected.

He says the company has also pledged further action in the form of more clearly communicating with Canadians, including giving notice to users that the company may uses exchanges with ChatGPT to train its models.

Dufresne told reporters that he believes OpenAI has “conditionally resolved” their concerns and that he and his provincial counterparts would be monitoring its progress.

He noted that the investigation found a “lack of accountability” on the part of the company and pointed to previous statements made by its leadership about feeling that they “had to move” when it came to releasing ChatGPT in November 2022 as there were other models that were preparing to be launched into the market.

“We found that problematic,” he said.

“We found that there were things that they could have done, and indeed that they have now done, that should have taken place before this was launched.”

He said that was the main message sent to the company.

Following the report’s release, a spokesman from OpenAI pointed to a blog post outlining how the company collects its information and the steps it takes to reduce the amount of personal information it gathers.

“People are using ChatGPT in increasingly personal ways, including for questions and tasks that can touch sensitive parts of their lives,” the post read. “We recognize the deep responsibility that comes with that trust.”

The release of the results come amid calls from children’s health organizations and online safety advocates for Prime Minister Mark Carney’s government to introduce regulations targeting AI chatbots, citing growing concerns over the technology’s impact on children.

Canadian Heritage Minister Marc Miller has committed that a bill would be introduced to better protect Canadians against online harms, adding that the issue of AI chatbots was being examined by a group of academics and other experts as it was a more complicated technology than social media platforms. The government has provided no timeline for that bill.

OpenAI, the company behind ChatGPT, has come under heavy scrutiny by leaders in Canada after acknowledging that it did not alert Canadian police to exchanges a shooter who opened fire on school children and family members in Tumbler Ridge, B.C., had months earlier with the chatbot, which the company deemed warranted an account suspension.

Federal Artificial Intelligence Minister Evan Solomon and other ministers met with company representatives to discuss concerns around their safety protocols in the wake of that revelation.

OpenAI CEO Sam Altman also released a letter of apology for its decision not to alert police to those exchanges.

Solomon in a statement said Wednesday’s report underscores “the importance of protecting Canadians’ personal information in age of AI.”

“Every company operating in Canada must respect Canadian law, protect Canadians’ personal information, and be accountable for how their systems affect people in the real world.”

When it comes to the joint investigation, the report says it was launched in 2023 in “ response to a complaint alleging that the organization had collected, used and disclosed the complainant’s personal information without consent.”

It focused exclusively on the two early models that OpenAI used to train its chatbot, which is largely used to generate answers and information to questions asked by users.

The report says the company, which is based in the U.S., took issue with the jurisdiction of the privacy bodies investigating its practices.

“ OpenAI challenged the Offices’ jurisdiction (both in whole and in part) on the grounds that ChatGPT was not released in Canada until November 30, 2022, and that OpenAI had no establishment or employees in Canada prior to that date,” it read.

It also said the company “did not grant our request to access and examine its systems,” as it cited “security, confidentiality and operational considerations.”

“However, in its representations to the offices, OpenAI acknowledged that, in the course of its activities, it collects, uses and discloses personal information.”

It outlined how ChatGPT collects its information from sources including publicly available websites, such as Wikipedia, as well as from content licensed through third parties, as well as exchanges it has with users and “conversations generated by human AI trainers.”

“OpenAI stated that the inclusion of personal information in its training datasets is incidental to the broader goal of obtaining a large and varied body of text necessary to effectively train its models,” according to the report, adding that the company pointed to “mitigation measures” it employs that seeks to limit the amount of personal information used to train its models.

“We do not accept OpenAI’s assertion that its collection of personal information is merely “incidental,” the report says.

“Rather, we find that OpenAI collects significant amounts of personal information for the purpose of training its AI models.”

The investigation concluded that because of what it called “limited privacy-protective measures in place at the time,” the company’s training datasets included “significant amounts of personal information,” ranging from an individual’s medical history to information related to minors.

National Post

Our website is the place for the latest breaking news, exclusive scoops, longreads and provocative commentary. Please bookmark nationalpost.com and sign up for our politics newsletter, First Reading, here.