As
AI
chatbots
become
more
popular
in
mental
healthcare,
the
American
Medical
Association
is
urging
Congress
to
strengthen
safeguards.
The
organization
sent
letters
to
the
Congressional
Artificial
Intelligence
Caucus,
the
Congressional
Digital
Health
Caucus
and
the
Senate
Artificial
Intelligence
Caucus.
The
letters
follow
numerous
reports
of
AI
chatbots
encouraging
suicide
and
self-harm
among
vulnerable
populations.
Congress
held
hearings
on
the
role
of
AI
in
mental
health
last
year,
which
“emphasized
several
critical
mental
health
concerns,
including
emotional
dependency
on
AI
systems,
the
potential
distortion
of
reality
through
prolonged
interaction
with
chatbots,
and
the
current
lack
of
consistent
safety
protocols,”
the
AMA
said
in
the
letters.
These
hearings
showed
the
need
for
“immediate
attention”
to
ensure
AI
tools
don’t
harm
those
seeking
mental
health
support,
the
letter
added.
That
said,
the
AMA
acknowledged
that
AI
tools
could
be
potentially
valuable
in
mental
health
care
if
used
safely.
“Across
the
country,
patients
persistently
struggle
to
access
mental
health
care,
either
for
reasons
of
access
or
affordability,”
the
AMA
said.
“Well-designed
AI-enabled
tools
may
serve
as
supportive
resources
that
expand
access
to
evidence-based
information,
facilitate
early
identification
of
mental
health
concerns,
and
connect
individuals
with
appropriate
clinical
services.
When
developed
and
deployed
within
clear
regulatory
guardrails,
these
technologies
have
the
potential
to
complement,
not
replace,
clinicians
and
help
mitigate
persistent
workforce
shortages
and
other
access
issues.”
The
AMA
provided
several
recommendations
for
AI
chatbot
safeguards,
including:
-
Improve
transparency:
Require
chatbots
to
clearly
disclose
that
users
are
communicating
with
AI,
as
well
as
ban
systems
from
presenting
themselves
as
licensed
clinicians. -
Create
clear
regulatory
boundaries:
Prevent
chatbots
from
diagnosing
or
treating
mental
health
conditions
without
the
right
regulatory
review.
The
AMA
calls
on
Congress
to
direct
agencies
to
create
a
“modern,
risk-based
oversight
framework
and
clarify
when
AI
tools
qualify
as
medical
devices.” -
Improve
oversight:
Require
ongoing
safety
monitoring,
reporting
of
adverse
events
and
strict
standards
for
technology
used
by
children
and
adolescents. -
Protect
privacy
and
security:
The
AMA
called
for
rigid
data
protection
standards,
such
as
limits
on
data
collection
and
retention
and
clear
user
consent
for
data
use. -
Limit
commercial
use:
Ban
advertising
on
mental
health
chatbots,
especially
for
minors.
“AI-enabled
tools
may
help
expand
access
to
mental
health
resources
and
support
innovation
in
health
care
delivery,
but
they
lack
consistent
safeguards
against
serious
risks,
including
emotional
dependency,
misinformation,
and
inadequate
crisis
response,”
said
Dr.
John
Whyte,
AMA
CEO,
in
a
statement.
“With
thoughtful
oversight
and
accountability,
policymakers
can
support
innovation
and
ensure
technologies
prioritize
patient
safety,
strengthen
public
trust,
and
responsibly
complement—not
replace—clinical
care.”
Photo:
Witthaya
Prasongsin,
Getty
Images