This Bay Area city is using AI to detect homeless camps. Will others follow suit?

The pilot program is thought to be the first of its kind nationwide.

Across the country, cities have begun experimenting with artificial intelligence to map potholes, reduce traffic and fight wildfires. In San Jose, officials are now harnessing the rapidly evolving technology with another goal in mind: detecting homeless encampments.

Three times since December, a white city-owned Toyota sedan affixed with a half-dozen small cameras has cruised through South San Jose to collect footage of parked cars and RVs. The images were then fed into different AI systems developed by four private companies to determine whether people were living inside the vehicles.

The open-ended pilot program, thought to be the first of its kind nationwide, may soon also seek to identify tent encampments and could one day expand to a permanent fleet of vehicles that crisscross the city.

While homeless advocates fear the effort could lead to more encampment sweeps and impounded lived-in RVs, city officials say they are optimistic it will help connect homeless people with needed services and shelter or housing. The program is not designed to collect identifying footage of license plates or people’s faces, officials stressed.

“We’re not interested in the individual identities of people who are living outside,” said San Jose Mayor Matt Mahan. “But we do need to know where all the lived-in vehicles in the city are so that we can manage them.”

The program, first made public by The Guardian, comes as Mahan pushes to “end the era of encampments” as residents have grown increasingly frustrated with street homelessness.

At the mayor’s urging, the City Council agreed earlier this year to develop policies to ban RVs near schools, restrict oversized-vehicle parking across the city and establish new tow-away zones. The city estimates it has more than 800 lived-in RVs.

At the same time, city officials are devising plans to move around 1,000 homeless people from local waterways and into shelters. San Jose has an estimated 6,300 homeless residents, about 70% of whom live outdoors or in vehicles. The rest stay in shelters.

In addition to responding to encampments, San Jose’s pilot program aims to help identify trash, graffiti, potholes and parking violations. Other cities already use AI for those purposes, but San Jose appears to be the first to employ the technology to spot RVs or tents with people living inside, according to AI experts and national homeless advocates.

Tami Rule moved into an RV parked in a residential area near Highway 87 and Capitol Expressway last year after a fire destroyed the apartment she shared with her husband in San Jose’s West San Carlos neighborhood. Rule, 56, said her biggest concern about the program is people gawking at footage of the plastic bins filled with clothes and shoes, small potted plants and other belongings the couple keeps stacked outside their vehicle.

“But for somebody to understand what it’s like to live in a camper,” she said. “I wouldn’t mind that.”

Elsewhere in the Bay Area, San Francisco, Oakland, Fremont and Mountain View officials said they had no plans to use AI to track encampments.

Still, experts said it’s not hard to imagine cities following San Jose’s lead. Last month, Mahan hosted officials from the White House and local governments across the country for a virtual forum on using AI to improve public services.

Vishnu Pendyala, an AI professor and researcher at San Jose State University, said the technology has “huge potential” for detecting homeless camps. But he also noted the privacy concerns surrounding the pilot program and other fledgling AI efforts.

“We have already seen many things being hacked, and many things being misused,” Pendyala said. He pointed to reports of Apple contractors allegedly listening to recordings of iPhone users’ queries to Siri, a program powered by AI.

Officials from the city and the companies working on the pilot said that protecting people’s privacy is a top priority — in part by explicitly instructing AI systems to ignore faces and license plate numbers.

“Whether we like it or not, AI is going to be a dominant technology,” said Khaled Tawfik, San Jose’s chief information officer. “We want to be the leader in discovering the risk and finding ways to mitigate it.”

Tawfik said his department is not sharing any information with other local agencies or the police department during the pilot, and any data or footage distributed in the future would obscure identifying details.

San Jose started the program, which is currently limited to South San Jose’s Council District 10, in response to the thousands of 311 calls it receives each year to report vehicles that appear abandoned, Tawfik said. He said the city aims to proactively identify which vehicles have people living in them so officials know where to send homeless outreach teams.

So far, the pilot has identified lived-in RVs with about 70% accuracy. It’s recognized lived-in cars correctly only about 10% to 15% of the time.

Masaf Dawood, a vice president with San Mateo-based Xloop Digital, one of the companies involved in the pilot, expects those numbers to improve as the AI analyzes thousands more images. He said this is the first time a city has asked the company to identify lived-in vehicles. Xloop Digital’s software looks for markers including litter on the roadway, lines of parked RVs and unexpected objects, such as a coffee maker on a vehicle dashboard.

Despite the technology’s potential, Dawood still anticipates a margin of error. “I don’t think we can say, or that we should say, ‘Oh, we can be 100%,’” he said.

The other companies that have taken part in the pilot are Sensen.AI, CityRover and Mountain View-based Blue Dome Technologies

Many residents, meanwhile, want the city to take a tougher stance on vehicle and tent encampments. Students at KIPP San Jose Collegiate, a high school in East San Jose, pressured city officials to move forward with the RV ban near schools, saying homeless people have broken into school buildings and left needles on lunch tables.

As San Jose officials ramp up enforcement, they say they want to ensure homeless people have safe places to go. They point to recent efforts to build hundreds of tiny home shelters and open safe overnight parking lots with supportive services.

Even so, those solutions aren’t coming fast enough, Langton said. The city still lacks enough beds for everyone on the street, and with a severe shortage of affordable housing, most people who move through local shelters don’t find lasting homes.

“It’s just a drop in the bucket of what needs to be done,” Langton said.

Staff writer Kate Talerico contributed to this report.