A 13-year-old North Carolina student formed a relationship with Google’s Gemini chatbot on a school-issued Chromebook and came close to taking her own life, an advocate told a state task force that looks into the causes of child deaths.

Mick Tobin, co-founder and advocacy director of the Young People’s Alliance, was referencing a lawsuit now in federal court in California that is still playing out. It is one of several similar cases nationwide, and more than 25 deaths have been linked to AI chatbots, Tobin said. “So even in North Carolina, we’re seeing these harms, and although these companies claim they’ve improved their products and have added safeguards, they’re not. They’re still harming young people,” Tobin told the state’s Child Fatality Task Force on Tuesday.

Chatbots have also encouraged young people to harm others, he said, and even when kids aren’t physically hurt, they can come to depend on chatbots for emotional support and validation.

Following Tuesday’s presentation, the task force, a legislative study commission that examines the causes of child deaths and makes recommendations to lawmakers, adopted a new committee recommendation. It endorsed the Young People’s Alliance’s efforts to protect young people from the harms of AI chatbots and companions through legislation.

The Young People’s Alliance was founded by North Carolina high school students and now works in several states and on Capitol Hill. It’s helped shape social media and AI chatbot legislation federally and in various states.

What lawmakers could do

Under the recommendation, legislation would need to “protect young people from emotional dependence by design (restricting human-like features of chatbots to use by adults only); cover chatbots that young people use the most; and provide families real recourse, including financial penalties and/or a duty of care.”

Human-like features can include chatbots claiming sentience, simulating friendship or romance, or nudging kids to come back for companionship, Tobin said.

Under legislation sponsored by the Young People’s Alliance, real recourse means the attorney general could seek civil penalties of up to $7,500 per intentional violation, and families would have a private right of action, letting parents take companies to court themselves and seek $750 per user, with no cap on total damages.

A duty of care is a legal obligation to act reasonably to avoid harming others, according to Cornell Law School’s Legal Information Institute.

The task force also approved a recommendation endorsing legislation to address addictive social media algorithms that harm children. That recommendation also supports the Young People’s Alliance’s efforts, and the task force also endorsed it in 2025.

In 2025, facing the rapid rise of AI, the task force voted only to keep studying how chatbots and companions affect young people, including their design features. With Tuesday’s vote, it is now formally calling on lawmakers to act.

Interest from two top senators

There could be some bipartisan interest in bills dealing with AI, if debate at the task force is any indication.

Debate about the language around holding companies accountable ultimately led to the addition of the phrase “including financial penalties and/or a duty of care,” to the recommendation approved by the task force.

While the Young People’s Alliance bill already includes such language, task force members wanted it spelled out in their endorsed recommendation.

Senate Minority Leader Sydney Batch, a Wake County Democrat and a task force member, said stronger penalties were needed.

“When you’re a multibillion-dollar company or a trillion-dollar company, a $750 fine is absolutely nothing,” Batch said.

Democrats will have a lot more say next year on legislation passed.

Democrats cut the Republican majority in the Senate to 26-24 in last week’s election, with one race for a GOP-held seat still within the margin for a recount. House Democrats took control of the chamber from Republicans.

Batch said a strong recommendation from the task force gives lawmakers something to point to. “When you guys pass things, it is easier for us to fall back on the fact that the task force says, ‘Hey, this is what we think.’”

“Whatever iteration gets passed needs to at least have an eye on the enforcement aspect,” she said.

Tobin said because there is no limit on the $750-per-user damages, a class action could cost companies significant sums. He said he was open to raising the penalties “because I’m all for big tech accountability.”

Sen. Jim Burgin, an Angier Republican who is also a task force member, and who is seeking to lead the Senate next year after Senate leader Phil Berger lost his primary, has filed chatbot legislation in the form of Senate Bill 963, which would put new licensing, safety and privacy rules on AI chatbots and allow the state’s attorney general and people to sue. He said he developed that bill with NC State and UNC students and plans to reintroduce it.

Burgin said companies are developing chatbots to provide elderly adults with companionship and medication reminders and that a group has pitched a number of lawmakers on using rural federal funding on such initiatives.

“It’s kind of on both ends of the extreme — early and late — because they say if we live long enough we become children again,” he told task force members. “So that’s something we need to be thinking about.”

_____

©2026 Raleigh News & Observer. Visit newsobserver.com. Distributed by Tribune Content Agency, LLC.