In
our
enthusiasm
to
adopt
and
use
GenAI,
are
we
ignoring
a
critical
problem:
what
are
the
tools
doing
to
our
critical
thinking
skills?
And
just
why
are
experienced
partners
better
at
using
GenAI
than
younger
lawyers?
I
attended
an
interesting
panel
discussion
on
Monday
at
the
American
Association
of
Law
Librarians
(AALL)
conference
in
Portland.
It
was
entitled
I
Hate
Hallucinations:
How
to
Combat
Misconceptions
Around
GenAI
Accuracy.
It
was
advertised
to
be
a
discussion
on
the
misconception
of
GenAI
inaccuracies,
how
to
reduce
hallucinations,
how
to
ask
GenAI
questions
the
right
way
to
reduce
risk,
and
what
the
future
holds.
The
panelists
included
Andrea
Guldalian,
Director
of
Library
Services
at
Duane
Morris,
Saskia
Mehlhorn,
Director
of
Knowledge
Services
at
Norton
Rose
Fulbright,
and
Patrick
Parsons,
the
Director
of
the
Law
Library
at
the
University
of
Pittsburgh
School
of
Law.
The
Critical
Discussion
But
despite
the
session
description,
the
conversation
quickly
pivoted
to
something
more
fundamental:
the
implicit
dangers
of
GenAI
and
the
problems
the
profession
faces
in
the
training
of
young
lawyers.
(I
recently
discussed
this
issue
in
an
article
on
the
development
of
gut
instinct
in
the
days
of
GenAI.)
This
part
of
the
discussion
raised
some
troublesome
and
challenging
issues.
These
issues
can
be
clearly
seen
in
the
context
of
legal
research.
Here’s
the
reasoning
of
the
panel.
First,
the
use
of
GenAI
for
everyone
but
particularly
law
students
and
young
lawyers
is
tempting
because
it’s
so
easy.
Parsons
summed
it
up
this
way:
“It’s
too
easy
and
really
tempting
to
just
rely
on
what
it
gives
you.”
The
other
problem
is
that
GenAI
is
really
good
at
doing
rote
tasks,
summarizing
and
manipulating
data,
and
easing
the
everyday
burdens.
But
legal
research
is
different.
It’s
the
Process,
Stupid
Parsons
believes,
and
so
do
I,
that
legal
research
is
a
process.
“Legal
research
is
not
finding
the
law.
It’s
not
a
one-sentence
answer.”
Legal
research
involves
finding
a
case,
reading
a
case,
reading
similar
cases,
reading
cases
that
come
to
an
opposite
conclusion,
and
understanding
not
just
the
holding
of
the
case
but
the
context
and
nuance.
None
of
the
GenAI
tools
can
do
this,
at
least
not
yet.
The
danger,
of
course,
is
that
the
more
younger
lawyers
—
and
for
that
matter,
the
rest
of
us
rely
—
on
GenAI
for
the
rote
tasks,
the
more
we
are
tempted
to
use
it
for
more
complicated
things
like
legal
research
to
get
quick
answers.
“Legal
research
is
not
just
finding
a
quote,”
said
Parsons.
Certainly,
in
my
years
of
practice,
I
saw
plenty
of
lawyers
who
tried
to
take
the
easy
way
out
with
research
pre-GenAI.
These
were
the
ones
who
relied
on
headnotes
instead
of
reading
the
cases.
This
often
led
to
disastrous
results
when
their
opposition
(me)
actually
read
their
cases.
But
today,
it’s
even
easier
and
more
tempting
than
ever.
The
Problem
is
the
Problem
Which
raises
another
more
fundamental
concern.
Sound
legal
research
(and
solving
many
other
complicated
problems)
involves
thinking
through
the
problem
on
the
front
end.
There
is
a
well-known
adage
that
has
been
applied
to
things
from
therapy
to
business
management:
“the
problem…is
the
problem.”
One
of
my
mentors
used
to
always
ask
me:
what’s
the
first
thing
you
do
in
any
case?
It’s
not
reading
cases
or,
in
today’s
parlance,
asking
ChatGPT
for
an
answer.
It’s
sitting
back
and
looking
out
at
the
Ohio
River
(our
offices
overlooked
the
River).
What
he
meant
of
course
was
thinking
through
the
problem
was
the
key.
Or
as
Albert
Einstein
put
it,
“If
I
had
an
hour
to
solve
a
problem,
I’d
spend
55
minutes
thinking
about
the
problem
and
5
minutes
thinking
about
solutions.”
But
here’s
what’s
going
on
now,
according
to
the
panel:
as
Mehlhorn
put
it,
“Critical
thinking
has
left
the
house.
Too
often
younger
lawyers
don’t
think
through
the
problem,
they
just
want
information
quickly.”
Senior
Lawyers
Better
at
Using
GenAI?
What?
The
erosion
of
critical
thinking
skills
makes
another
panelist’s
observation
particularly
telling:
an
observation
that
on
first
blush
seems
surprising
but
really
shouldn’t
be.
Mehlhorn
observed
that
it
was
the
more
senior
lawyers
who
are
the
best
users
of
AI
products.
Why?
Mehlhorn
says
it’s
because
they
know
what
they
are
looking
for.
In
other
words,
they
have
a
better
grip
on
the
problem.
They
also
know
when
an
answer
looks
right.
And
when
it
looks
wrong.
How
do
they
know
these
things?
I
think
it’s
because
of
their
experience
with
the
research
process
that
Parsons
talked
about,
over
and
over
again,
so
that
they
perhaps
subconsciously
see
patterns,
as
I
discussed
in
my
article.
It’s
this
pattern
that
triggers
what
I
referred
to
as
“gut
instinct.”
But
as
someone
smarter
than
me
observed
after
reading
my
article,
it’s
not
gut
instinct
but
wisdom.
The
Real
Challenge
But
here
is
the
danger
and
the
challenge
we
as
a
profession
are
facing.
As
we
rely
more
and
more
on
GenAI
tools,
we
risk
losing
the
process
that
develops
the
critical
thinking
skills,
the
wisdom
that
more
experienced
lawyers
have.
We
have
to
find
some
ways
to
first
instill
among
our
young
lawyers
the
respect
for
critical
thinking
and
the
process
necessary
to
solve
legal
problems.
We
have
to
develop
ways
to
ensure
that
they
develop
and
use
those
skills
and
processes.
We
have
to
make
sure
that
with
all
the
benefits
of
GenAI,
we
don’t
come
to
expect
and
even
demand
that
quick
answer.
That
isolated
and
seemingly
reliant
quote.
We
need
to
demand
that
they
show
us
that
they
have
thought
through
the
problem,
not
asked
ChatGPT
for
an
easy
answer.
The
solution
isn’t
to
abandon
GenAI.
But
we
need
to
be
intentional
about
preserving
and
encouraging
the
critical
thinking
skills
that
make
good
lawyers.
This
means
building
training
programs
that
emphasize
process
over
speed,
creating
mentorship
opportunities
where
senior
lawyers
can
pass
on
their
pattern
recognition
skills,
and
perhaps
most
importantly,
giving
young
lawyers
permission
to
take
the
time
to
think.
Let’s
Not
Kid
Ourselves
As
with
everything,
reliance
and
use
of
GenAI
comes
with
a
price.
We
are
only
starting
to
see
that
price.
We
have
to
find
ways
to
minimize
and
control
that
price
else
our
future
lawyers
won’t
be
humans.
They
will
be
robots
with
simplistic
non-answers
to
complicated
problems
that
go
nowhere.
Just
as
it
has
always
been,
the
future
of
law
depends
on
lawyers
who
can
think.
A
GenAI
tool
that
spends
55
minutes
thinking
about
a
problem
would
be
a
laughingstock
today.
Let’s
not
give
our
profession
the
same
treatment.
Stephen
Embry
is
a
lawyer,
speaker,
blogger
and
writer.
He
publishes TechLaw
Crossroads (Opens
in
a
new
window),
a
blog
devoted
to
the
examination
of
the
tension
between
technology,
the
law,
and
the
practice
of
law.