Another
week.
Another
law
firm
caught
citing
cases
that
didn’t
exist
and
for
incorrect
propositions.
All
allegedly
due
to
reliance
on
AI.
But
this
was
not
just
any
law
firm.
It
was
Sullivan
&
Cromwell,
one
of
the
largest
and
most
influential
law
firms
in
the
United
States
and,
for
that
matter,
the
world.
It’s
emblematic
of
the
threat
of
misplaced
reliance
on
artificial
intelligence
not
only
to
the
lawyers
and
law
firms
but
also
to
our
own
rule
of
law
itself.
The
Sullivan
&
Cromwell
Error
The
firm
reportedly
recently
filed
a
motion
in
bankruptcy
court
that
had
numerous
errors
and
incorrect
citations.
Once
they
were
discovered,
Sullivan
&
Cromwell
of
course
immediately
apologized
to
the
court
and
even
to
the
opposing
law
firm
that
discovered
the
mistakes.
(It’s
funny
how
apologetic
you
get
once
caught.)
And
these
were
more
than
a
few
errors.
There
were
reportedly
around
36
errors
that
took
up
three
pages
to
describe
and
included
imagined
passages
from
real
cases.
When
informing
the
court
of
the
“errors,”
the
firm
said
that
its
policies
for
using
AI
were
not
followed
and
that
the
errors
occurred
even
though
it
had
a
training
course
designed
to
prevent
just
this
sort
of
thing
from
happening.
To
call
this
an
error
is
like
saying
the
Titanic
running
into
an
iceberg
was
a
slight
miscalculation.
It’s
not
just
a
nanny
nanny
boo
boo.
Why
This
Is
So
Bad
So,
we
can
add
Sullivan
&
Cromwell
to
the
ever-growing
list
of
firms
that
are
making
these
kinds
of
mistakes
and
miscalculations.
In
fact,
it’s
reported
that
it’s
happened
over
1,300
times.
And
those
are
just
the
ones
that
have
been
caught.
The
problem
has
gone
well
beyond
just
all
inadvertence
and
strikes
at
our
fundamental
system.
I
recently
attended
an
American
Bar
Association
conference.
I
listened
to
a
panel
discussion
about
the
role
of
lawyers
not
only
in
the
development
of
our
legal
system
but
as
guardians
of
the
rule
of
law
and
a
fair
and
impartial
system
of
justice.
The
panel
was
composed
of
Michelle
Behnke,
the
current
ABA
President,
Hon.
Bernes
Aldana,
who
was
until
recently
the
President
of
the
National
Judicial
College,
and
Mario
Sullivan,
the
President
of
the
ABA’s
Civil
Rights
and
Social
Justice
Section.
The
panel
made
the
important
point
that
the
legal
profession
has
an
obligation
to
teach
and
educate
the
public
about
the
rule
of
law
and
why
it
is
so
important.
And
why
it’s
critical
that
disputes
can
get
resolved
fairly
and
impartially.
Our
system
of
justice
hinges
on
the
public’s
trust
and
confidence
in
our
legal
system
and
the
bedrock
concepts
on
which
it’s
based.
That’s
why
we
need
to
take
responsibility
to
make
sure
that
the
public
understands
what
these
concepts
mean
and
why
they
are
so
important.
So,
what
does
this
have
to
do
with
the
Sullivan
&
Cromwell
error?
Aldana
made
a
very
perceptive
point.
In
order
for
our
system
to
function,
lawyers
have
to
be
candid
with
the
courts
and
judges.
And
they
have
to
conduct
themselves
in
ways
that
bolster
public
confidence
in
them
and
our
system.
When
we
cite
cases
that
don’t
exist
or
cite
cases
for
the
wrong
proposition,
it
undermines
that
trust
and
confidence.
When
judges
can’t
have
some
degree
of
trust
in
what
both
sides
tell
them,
it
disrupts
their
ability
to
be
fair
and
impartial.
Not
to
mention
what
happens
if
you
can’t
at
least
have
a
modicum
of
trust
in
what
your
adversary
says.
And
when
the
public
hears
about
lawyers
citing
cases
and
laws
that
don’t
exist,
they
conclude
the
whole
system
is
a
sham.
It
falls
right
in
line
with
the
inability
to
trust
any
video
or
audio
recording
and
with
the
shameless
lies
that
permeate
social
media.
It
makes
it
easy
to
conclude
our
system
of
justice
is
just
more
of
the
same.
As
Aldana
put
it,
“the
legitimacy
of
our
courts
depends
on
the
public’s
trust
and
confidence.”
This
is
serious
stuff.
It’s
a
further
erosion
of
trust
in
the
rule
of
law.
I
was
trying
to
explain
the
hallucination
problem
to
someone
recently
who
is
not
a
lawyer
and
has
no
legal
training.
Their
first
reaction:
“Oh,
so
the
lawyer
just
made
up
cases
to
try
to
win
their
lawsuit.”
Of
course,
that’s
not
what
happened.
But
the
fact
that
it
was
the
immediate
and
initial
reaction
says
a
lot.
It
demonstrates
why
we
need
to
scrupulously
avoid
even
the
appearance
of
impropriety.
The
Devil
Made
Me
Do
It
AI
can
do
wondrous
things.
It
can
help
lawyers
work
better
and
faster.
But
it’s
well
known
(or
should
be
well
known
by
now)
that
it
has
a
dark
side:
it
makes
things
up.
When
accuracy
is
critical,
that’s
a
real
problem.
We
can
talk
a
lot
about
why
hallucinations
and
inaccuracies
still
show
up
frequently
in
court
filings.
Time
pressure,
the
way
that
we
have
historically
gotten
work
done
as
we
have
discussed,
the
time
necessary
to
check
citations
that
often
negates
the
AI
advantage
in
the
first
place
as
we
have
also
written.
Add
to
all
this
the
fact
that
it’s
easy
and
tempting
to
use
AI
to
get
information
quickly
and
it’s
easy
to
see
why
we
have
a
problem.
But
that
doesn’t
solve
the
problem
and
may
in
fact
compound
it.
Look
at
it
this
way:
if
an
associate
in
a
law
firm
or
a
judicial
clerk
pre-GenAI
cited
a
case
that
didn’t
exist,
they’d
be
fired
and
likely
reported
to
the
bar.
We
seem
to
think
that
citing
a
fictitious
case
that
GenAI
provides
somehow
provides
a
bit
of
excuse
for
what
happened.
Yes,
courts
are
getting
serious
about
fining
lawyers.
But
the
lawyers
offer
the
old
“the
devil
made
me
do
it”
excuse.
So,
we
still
see
the
same
thing
happen
over
and
over.
The
public?
It’s
rolling
its
eyes
at
us.
Our
stature
is
falling.
And
by
undermining
our
stature,
it
undermines
what
we
do,
our
system
of
justice,
and
the
rule
of
law
concept
on
which
it’s
all
based.
When
a
leading
law
firm
in
the
world
with
all
its
training
and
policy
bells
and
whistles
cites
a
nonexistent
case,
it’s
a
problem
for
all
of
us.
What
Can
We
Do?
So,
what
do
we
do?
Of
course
it
starts
with
education.
Been
doing
that.
We
need
guidelines
and
policies.
Done
that.
We
need
to
fine
the
reprobate
lawyers
that
do
this.
Doing
that.
We
need
to
get
stricter.
Lawyers
that
cite
hallucinated
cases
perhaps
need
to
be
reported
to
the
bar
and
disciplined.
When
a
lawyer
steals
clients
funds,
the
bar
comes
down
hard.
Why?
It
undermines
the
client
relationship
and
trust.
It’s
bad
for
the
client
and
it’s
bad
for
the
profession.
The
same
is
true
here.
We
need
to
mandate
tools
that
enable
citation
checking
quickly
and
easily,
as
we
have
written.
It
may
even
mean
limiting
the
use
of
AI
in
certain
circumstances.
It
means
doing
things
like
the
ABA
is
doing:
focusing
on
our
collective
responsibility
to
protect
the
integrity
of
the
rule
of
law.
We
need
to
understand
that
improper
reliance
on
AI
isn’t
just
bad
for
the
lawyers
that
do
it.
It’s
bad
for
the
profession
and
our
system
of
justice.
If
we
don’t
grasp
the
relationship
between
the
rule
of
law
and
citing
fictitious
cases,
one
day
we
may
wake
up
in
a
land
where
citing
fictitious
cases
won’t
matter
because
there
is
no
rule
of
law.
Stephen
Embry
is
a
lawyer,
speaker,
blogger,
and
writer.
He
publishes TechLaw
Crossroads,
a
blog
devoted
to
the
examination
of
the
tension
between
technology,
the
law,
and
the
practice
of
law.