Something
jumped
out
at
me
early
at
the
recent
LegalTech
Connect
Law
Firm
Research
&
Innovation
Conference
with
the
legal
research
community. Greg
Lambert
asked
how
many
attendees
have
encountered
partners
who
don’t
know
their
Westlaw
or
Lexis
passwords. That
question
resonated
throughout
the
room
as
most
had
experienced
this.
It’s
general
knowledge
that
partners
rarely
conduct
legal
research,
but
if
they
don’t
know
how
to
log
in
to
a
legal
research
platform,
what
process
are
firms
using
for
partners
to
access
the
underlying
sources
in
a
brief?
Over
the
past
year,
we’ve
seen
a
steady
stream
of
headlines
about
AI
hallucinations
where
fake
cases,
fabricated
quotes,
and
misstated
facts
made
it
through
a
review
process
and
into
court
filings.
Technology
Isn’t
The
Issue;
It’s
A
Process
Problem
While
lawyers
are
responsible
for
their
work,
there
is
a
tendency
to
blame
the
breakdown
on
the
introduction
of
AI
technology
into
the
process. I
think
a
better
explanation
is
that
AI
exposed
a
flaw
that
already
existed
in
most
processes.
For
decades,
law
firm
drafting
was
built
on
partner
guidance
and
associate
writing,
with
a
senior
associate
taking
a
pass
and
then
the
partner
reviewing
in
a
supervisory
role.
Citation
tools
such
as
Shepard’s
or
KeyCite
are
used
by
the
associate
along
the
way.
But
in
practice,
how,
what,
and
when
does
the
supervising
attorney
review
the
work?
If
a
case
was
reviewed
earlier
in
the
process,
is
it
checked
again
at
the
end
of
the
process,
including
the
quotation
pulled
from
the
case?
Before
AI,
a
mistake
that
squeaked
through
a
gap
in
the
process
didn’t
matter
as
much
because
potential
errors
were
minor
and
could
be
corrected
as
errata
if
necessary.
But
that
doesn’t
work
anymore.
Human
authorship
is
being
augmented
by
AI-generated
content
and
editing
that
can
alter
facts,
fabricate
quotes,
and
invent
citations
to
nonexistent
cases.
AI
Exposes
The
Gaps
That
Were
Always
There
Anyone
who
uses
AI
grammar-checking
software
or
Microsoft
Copilot
on
a
regular
basis
has
had
the
experience
of
“uncorrecting”
its
revisions
that
are
more
grammatically
correct
but
also
change
the
meaning
of
a
statement.
Associates
are
still
learning
and
may
miss
subtle
errors
introduced
by
these
powerful
new
tools.
What
was
correct
at
an
earlier
stage
of
a
document
may
very
well
be
changed
by
AI
in
a
later
version.
Errata
Versus
Fabrication
The
central
issue
here
is
that
an
AI
hallucination
is
a
fabrication.
Fake
cases
are
one
issue,
but
more
recent
issues
in
filings
include
fake
quotations.
A
complete
review
of
everything,
including
foundational
facts,
needs
to
be
conducted
at
the
end
of
the
process
before
filing.
The
tried-and-true
workflows
that
once
assumed
human
authorship
must
change.
Drafting
And
Review
Should
Be
Inseparable
Remember
your
high
school
science
class?
There
are
protons,
neutrons,
and
electrons
within
an
atom,
but
the
atom
itself
cannot
be
separated.
AI
highlights
this
in
document
creation.
The
drafting
and
review
processes
are
atomic.
They
can’t
be
separated
without
some
kind
of
nuclear
reaction.
AI
creates
efficiencies
in
drafting,
but
it
imposes
new
burdens
on
the
review
process
and
highlights
assumptions
about
who
reviews
what
and
when.
The
process
now
needs
to
track
who
verifies
citations,
when
verification
occurs,
and
how
it
is
communicated
across
the
team
drafting
and
reviewing
the
document.
The
same
should
happen
for
quotations
and
fact-checking.
Zero
Trust
principles
In
Cybersecurity,
there
is
a
concept
called
Zero
Trust.
It’s
basically
“never
trust,
always
verify”.
Unfortunately,
AI
systems
are
probabilistic.
They
are
literally
guessing
the
next
part
of
a
word
that
fits
a
pattern.
This
means
that
hallucinations
are
a
feature
of
the
system’s
design
rather
than
a
bug.
There
are
techniques
to
minimize
hallucinations,
but
they
can’t
be
eliminated.
“Back
to
Basics”
review
processes
The
consequences
of
AI
incorporated
into
drafting
are
that
errors
can
become
fabrications
that
are
well
beyond
the
scope
of
errata.
Before
the
Internet,
it
was
common
to
print
out
all
authorities
and
keep
them
in
binders
for
access
and
review.
But
the
digital
equivalent
to
that
physical
binder
has
been
lost
in
some
firms.
There
are
solutions
available
to
help
organize
and
improve
processes.
Creating
hyperlinks
within
documents
to
a
repository
of
referenced
authoritative
sources
can
help
expedite
review.
If
a
link
doesn’t
exist
to
a
case,
then
it’s
a
red
flag
that
it
may
not
exist.
Clicking
on
a
link
can
help
review
and
verify
that
quotes
or
facts
are
correct,
too.
Firms
should
take
action
now
AI
Policy
is
not
a
process.
Firms
need
to
operationalize
how
attorneys
can
comply
with
the
policy
and
provide
the
processes
and
tools
needed
to
make
it
easier
to
meet
the
standards
of
Rule
11
Ethical
responsibility.
The
California
Bar
recently
solicited
comments
on
changes
to
the
rules
of
professional
conduct
as
they
pertain
to
AI.
And
there
is
some
discussion
proposing
an
actual
Hyperlink
Rule
to
ensure
sources
are
linked
in
filings.
Firms
shouldn’t
wait
for
regulators
to
tell
them
what
to
do.
Conclusion
AI
may
have
introduced
gross
fabrication
into
legal
work,
but
it
also
removed
the
illusion
that
review
processes
were
ever
airtight.
There
are
more
advanced
solutions
possible,
including
some
I
have
suggested
in
prior
writings
on
hallucinations.
But
before
the
industry
considers
more
advanced
solutions,
it
should
ensure
it
addresses
the
basics
first.
The
firms
that
recognize
and
rebuild
their
review
processes
and
tightly
couple
them
to
the
drafting
process
will
be
in
a
much
better
position
than
those
that
treat
this
as
a
problem
caused
by
technology
or
a
problem
to
be
solved
with
policy
alone.
AI
was
used
in
the
review
of
this
article.
Ken
Crutchfield
has
over
forty
years
of
experience
in
legal,
tax,
and
other
industries.
Throughout
his
career,
he
has
focused
on
growth,
innovation,
and
business
transformation. His
consulting
practice
advises
investors,
legal
tech
startups
and
others.
As
a
strategic
thinker
who
understands
markets
and
creating
products
to
meet
customer
needs,
he
has
worked
in
start-ups
and
large
enterprises.
He
has
served
in
General
Management
capacities
in
six
businesses.
Ken
has
a
pulse
on
the
trends
affecting
the
market.
Whether
it
was
the
Internet
in
the
1980s
or
Generative
AI,
he
understands
technology
and
how
it
can
impact
business.
Crutchfield
started
his
career
as
an
intern
with
LexisNexis
and
has
worked
at
Thomson
Reuters,
Bloomberg,
Dun
&
Bradstreet,
and
Wolters
Kluwer.
Ken
has
an
MBA
and
holds
a
B.S.
in
Electrical
Engineering
from
The
Ohio
State
University.