via
ChatGPT)
If
you’re
a
Pennsylvania
lawyer
wondering
whether
you
need
to
disclose
AI
use
in
your
court
filings,
Google’s
AI
summary
has
an
authoritative
answer
for
you.
It’s
a
wrong
answer,
mind
you.
But
authoritative!
As
Christine
Lemmer-Webber
put
it,
it’s
“Mansplaining
as
a
Service”
—
unwavering
confidence,
no
matter
how
incorrect.
If
one
opens
an
incognito
window
and
searches
Google
for
“Pennsylvania
AI
disclosure
lawyers,”
the
AI-generated
summary
will
explain
that
“Key
developments
include
mandatory
disclosure
of
Generative
AI
(GAI)
in
court
filings.”
Throw
in
“August
2024”
because
you
vaguely
remember
seeing
something
about
AI
on
that
date
and
the
result
reads
“As
of
August
2024,
Pennsylvania
mandates
explicit
disclosure
of
AI
use
in
all
court
submissions,
making
transparency
a
mandatory
filing
requirement.”
None
of
that
is
true.
The
Legal
AI
Governance
tracker,
an
invaluable
tool
maintained
by
Brian
Alenduff
of
Desired
Path
Consulting,
provides
a
comprehensive
rundown
of
Pennsylvania’s
AI
rules.
There
are
standing
orders
in
some
courtrooms,
and
the
state
supreme
court
issued
a
rule
governing
court
personnel
only,
but
as
for
the
state
of
Pennsylvania
writ
large,
there
is
no
statewide
rule
as
of
now.
The
tracker
notes
that
what
Pennsylvania
does
have is
Joint
Formal
Opinion
2024-200,
a
2024
advisory
ethics
opinion
from
the
Pennsylvania
Bar
Association
and
the
Philadelphia
Bar
Association
flagging
AI
as
a
competence
issue
under
existing
rules.
But
that
opinion
explicitly
states
that
it
is
“advisory
only
and
is
not
binding.”
The
ABA’s
own 50-state
survey classifies
Pennsylvania
as
“court
dependent.”
Scrolling
down
to
the
search
results
would
reveal
to
anyone
interested
that
there’s
not
really
a
mandatory
disclosure
policy.
So
how
did
it
end
up
in
the
Google
summary?
Alenduff
investigated
and
believes
it
first
emerged
from
a
vendor
blog
post.
Google
flags
a
Paxton.ai
post
titled
“2025
State
Bar
Guidance
on
Legal
AI”
among
its
sources.
That
post
includes
a
bullet
point
that
reads:
Pennsylvania:
Mandates
explicit
disclosure
of
AI
use
in
all
court
submissions.
Transparency
isn’t
optional;
it’s
a
filing
requirement.
Implementation:
August
2024.
From
there
it
seems
that
Google’s
AI
Overview
inhaled
that
sentence
and
started
serving
it
back
to
lawyers
as
the
law
of
the
Commonwealth.
Hallucinations
are
all
the
rage
right
now,
but
over
the
long
haul
the
greater
AI
risk
will
be
an
unfailingly
credulous
bot
elevating
and
validating
mistakes
until
the
error
gets
picked
up
as
reality.
It’s
a
chicken
and
the
egg
problem,
but
another
post
by
Twinladder.ai
repeats
Paxton’s
language.
Whether
they
got
it
from
Google
AI
or
their
mistaken
citation
led
Google
to
consider
the
mistake
independently
confirmed
doesn’t
really
matter
—
it’s
out
there
now.
This
is
the hot
dog
champion
problem for
lawyers.
Journalist
Thomas
Germain
recently
seeded
a
few
corners
of
the
web
with
the
claim
that
he
was
a
competitive
hot-dog-eating
world
champion,
and
watched
ChatGPT
and
Google
Overviews
dutifully
repeat
it.
Funny
when
it’s
hot
dogs,
but
decidedly
less
funny
when
it’s
“the
ethical
rules
binding
the
legal
profession.”
This
is
the
compression
problem
that
AI
brings.
As
those
developing
this
technology
push
to
condense
the
legal
workflow
into
a
series
of
“agentic”
steps
flowing
from
input
to
deliverable,
it’s
both
obfuscating
the
process
and
robbing
lawyers
of
the
slow-thinking
opportunities
that
produce
optimal
work.
Lawyers
become
—
even
against
their
conscious
instincts
—
more
trusting
of
output
that
looks
like
a
polished
finished
product.
They
also
become
incentivized
to
move
on
from
a
project
faster,
sacrificing
the
down
time
between
turning
drafts
that
used
to
birth
epiphanies.
And
miss
me
with
the
“human
in
the
loop”
platitudes.
How
often
has
Google’s
AI
produced
a
summary
that
you
took
as
gospel
without
bothering
to
scroll
down?
I’ll
go
you
one
better…
Google
includes
links
to
its
sources,
but
how
many
times
have
you
actually
clicked
on
one
of
them?
I’m
willing
to
wager
you
—
at
best
—
registered
that
it
cited
something
and
counted
it
as
a
win
without
bothering
to
consider
what
it
cited
or
whether
the
underlying
source
was
right.
Human
in
the
loop
is
good
ad
copy,
but
in
practice
it’s
a
glorified
Google
source
panel:
an
invitation
to
check
that
lawyers
will
never
click
on.
As
long
as
the
bot
purports
to
be
showing
its
work,
then
it
must
be
correct,
right?
We
risk
a
tech-enabled
speedrun
of
qualified
immunity.
The doctrine
of
qualified
immunity rests
on
a
literal
scrivener’s
error,
as
an
1874
federal
compilation
simply
deleted
language
that
Congress
wrote
into
Section
1983
specifically
rejecting
any
state
law
limitations
on
the
new
cause
of
action.
From
then
on,
courts
acted
upon
the
error.
AI
can
absorb
and
uncritically
repeat
false
claims
at
scale
and
every
time
someone
falls
for
the
error,
the
bot
interprets
that
as
more
proof
that
it’s
right.
The
redux
of
the
qualified
immunity
problem
now
accelerated
by
technology
instead
of
just
racism.
Speaking
of
racism…
imagine
what
AI
will
do
for
originalism.
There’s
a
growing
industry
of
amateurs
cosplaying
as
historians
compiling
disingenuous
claims
into
secondary
sources.
Judges
already
misapply
precedent
to
smooth
the
way
for
entrenching
this
revisionism
into
the
law.
Create
enough
of
an
echo
chamber
of
half-truths
and
all
of
a
sudden
it’s
“fact”
by
the
bot’s
reckoning.
The
answer,
as
always,
is
that
lawyers
need
to
remain
highly
skeptical
of
AI
conclusions.
It
can
be
a
powerful
tool
for
pointing
users
in
the
right
direction,
but
no
matter
how
authoritative
it
sounds,
it’s
only
an
invitation
to
deeper
human
research
and
not
a
replacement.
In
other
news,
did
you
know
that
I’m
also
a
former
world
hot
dog
eating
champion?
If
you
don’t
believe
me,
check
Google
in
a
few
weeks.
Joe
Patrice is
a
senior
editor
at
Above
the
Law
and
co-host
of
Thinking
Like
A
Lawyer.
Feel
free
to email
any
tips,
questions,
or
comments.
Follow
him
on Twitter or
Bluesky
if
you’re
interested
in
law,
politics,
and
a
healthy
dose
of
college
sports
news.
Joe
also
serves
as
a
Managing
Director
at
RPN
Executive
Search.