Law
firms
often
assume
that
classrooms
trail
practice.
The
thinking
is
familiar.
Students
learn
theory.
Lawyers
learn
reality.
Training
catches
up
later,
shaped
by
client
demands
and
live
matters.
The
empirical
evidence
from
AI-supported
classrooms
suggests
the
opposite.
Classrooms
are
not
behind
practice.
They
are
stress
tests
for
legal
AI
design,
and
they
surface
failures
long
before
those
failures
become
visible
inside
firms.
This
inversion
became
clear
during
a
series
of
empirical
classroom
pilots
run
through
Product
Law
Hub
using
an
AI-based
legal
coach
called
Frankie.
The
pilots
were
designed
to
observe
how
users
interact
with
AI
while
learning
judgment-based
legal
skills.
The
findings
draw
on
quantitative
engagement
data
and
qualitative
interviews
conducted
throughout
the
course.
What
the
classroom
revealed
should
matter
to
any
firm
investing
in
AI
for
training,
knowledge
management,
or
decision
support.
Classrooms
Remove
The
Incentives
That
Hide
Failure
In
practice,
lawyers
are
remarkably
good
at
adapting
around
broken
tools.
They
learn
workarounds.
They
ignore
features
that
get
in
the
way.
They
keep
using
systems
long
after
they
have
stopped
trusting
them
because
abandoning
them
feels
riskier
than
tolerating
them.
Classrooms
strip
those
incentives
away.
Students
do
not
have
billable
pressure.
They
do
not
have
clients
waiting.
If
a
tool
feels
unhelpful,
they
disengage
immediately.
If
it
undermines
confidence
or
clarity,
they
say
so.
That
blunt
feedback
loop
makes
classrooms
unusually
good
at
exposing
design
flaws.
During
the
pilot,
disengagement
showed
up
quickly
when
the
AI
behaved
poorly.
Sessions
shortened.
Follow-up
interactions
declined.
Interview
feedback
became
more
critical.
In
a
firm,
the
same
tool
might
limp
along
for
months
before
anyone
admitted
it
was
not
working.
Disengagement
Is
An
Early
Warning
Signal
One
of
the
most
valuable
signals
from
the
classroom
data
was
disengagement.
Not
failure
to
complete
an
assignment.
Not
incorrect
answers.
Disengagement.
When
students
stopped
asking
follow-up
questions
or
abandoned
sessions
early,
it
was
a
sign
that
the
AI
was
not
supporting
their
reasoning.
That
signal
emerged
far
earlier
than
any
formal
evaluation
would
have.
In
firms,
disengagement
often
goes
unnoticed.
Lawyers
stop
using
a
tool
quietly.
Adoption
metrics
flatten.
Leaders
attribute
the
problem
to
change
management
instead
of
design.
The
classroom
made
it
clear
that
disengagement
is
not
a
user
problem.
It
is
a
system
problem,
and
it
appears
long
before
productivity
metrics
move.
Feedback
Loops
Are
Faster
And
More
Honest
Another
advantage
of
classrooms
is
speed.
Feedback
loops
are
short.
Students
interact,
react,
and
reflect
within
days,
not
quarters.
Interviews
conducted
shortly
after
use
capture
impressions
before
rationalization
sets
in.
In
the
pilot,
qualitative
interviews
surfaced
nuanced
reactions
that
would
be
difficult
to
extract
from
practicing
lawyers.
Students
articulated
when
the
AI
felt
helpful,
when
it
felt
condescending,
and
when
it
felt
inattentive.
They
described
confidence
erosion
and
trust-building
moments
in
real
time.
In
firms,
those
conversations
happen
later,
if
at
all.
By
then,
the
cost
of
change
is
higher
and
the
opportunity
to
redesign
is
smaller.
What
Practice
Hides,
Classrooms
Reveal
Many
of
the
failure
modes
observed
in
the
classroom
map
directly
onto
problems
firms
experience
with
AI,
but
more
quietly.
Overly
directive
systems
discourage
thinking.
Repetition
undermines
trust.
One-size-fits-all
interactions
frustrate
users
at
different
experience
levels.
These
issues
surfaced
immediately
in
the
classroom
because
there
was
no
reason
to
pretend
otherwise.
In
practice,
those
same
issues
show
up
as
stalled
adoption,
uneven
use
across
seniority
levels,
and
skepticism
disguised
as
compliance.
By
the
time
leadership
notices,
the
system
has
already
shaped
behavior.
Classrooms
make
these
dynamics
visible
early
enough
to
fix.
Training
Environments
Are
Safer
Places
To
Fail
There
is
another
reason
classrooms
matter.
They
are
safer
places
to
fail.
Testing
AI
in
live
matters
carries
reputational
and
client
risk.
Testing
AI
in
classrooms
carries
learning
risk.
That
distinction
should
encourage
more
experimentation,
not
less.
The
Product
Law
Hub
pilot
demonstrated
that
training
environments
can
be
used
to
probe
how
AI
affects
judgment,
confidence,
and
reasoning
without
exposing
clients
to
harm.
Design
choices
can
be
stress-tested
before
they
harden
into
workflows.
Firms
that
ignore
this
opportunity
are
missing
a
low-cost,
high-signal
testing
ground.
Why
Firms
Underestimate
Classroom
Insights
Despite
these
advantages,
firms
often
discount
classroom
findings
as
academic
or
theoretical.
That
dismissal
is
a
mistake.
The
classroom
data
was
not
about
doctrine.
It
was
about
behavior.
How
long
users
stayed
engaged.
Whether
they
asked
better
questions.
When
they
trusted
the
system.
Those
behaviors
are
directly
relevant
to
practice.
What
differs
is
not
the
psychology,
but
the
incentives.
Classrooms
remove
incentives
that
mask
problems.
That
makes
their
insights
more
predictive,
not
less.
Seeing
Around
Corners
Requires
Paying
Attention
Early
The
most
strategic
insight
from
the
pilot
is
that
AI
design
failures
are
detectable
early
if
firms
know
where
to
look.
Disengagement,
confidence
erosion,
and
trust
breakdowns
appear
first
in
learning
environments.
Waiting
for
client
complaints
or
adoption
metrics
to
surface
problems
is
reactive.
Using
classrooms
as
observatories
is
proactive.
Firms
that
pay
attention
to
these
early
signals
can
redesign
tools
before
they
shape
bad
habits.
Firms
that
do
not
will
keep
wondering
why
expensive
systems
never
quite
deliver.
The
Uncomfortable
Implication
For
Training
Leaders
The
uncomfortable
implication
is
that
law
firm
training
leaders
should
be
paying
closer
attention
to
classrooms
than
to
vendor
demos.
Classrooms
reveal
how
AI
actually
interacts
with
human
reasoning.
Demos
show
what
tools
can
do.
Classrooms
show
what
tools
do
to
people.
That
distinction
matters
as
AI
becomes
embedded
in
how
lawyers
learn
to
think.
The
Takeaway
Firms
Should
Not
Ignore
The
takeaway
from
the
empirical
classroom
work
is
not
that
education
should
drive
practice.
It
is
that
learning
environments
provide
early,
honest
feedback
about
AI
design.
Classrooms
are
not
behind
the
profession.
They
are
ahead
of
it,
precisely
because
they
expose
problems
before
incentives
smooth
them
over.
Firms
that
want
AI
to
support
judgment
rather
than
undermine
it
should
treat
classrooms
as
diagnostic
tools,
not
afterthoughts.
The
future
of
legal
AI
will
be
shaped
by
those
who
are
willing
to
listen
early,
before
the
warning
signs
become
too
expensive
to
ignore.
Olga
V.
Mack
is
the
CEO
of
TermScout,
where
she
builds
legal
systems
that
make
contracts
faster
to
understand,
easier
to
operate,
and
more
trustworthy
in
real
business
conditions.
Her
work
focuses
on
how
legal
rules
allocate
power,
manage
risk,
and
shape
decisions
under
uncertainty.
A
serial
CEO
and
former
General
Counsel,
Olga
previously
led
a
legal
technology
company
through
acquisition
by
LexisNexis.
She
teaches
at
Berkeley
Law
and
is
a
Fellow
at
CodeX,
the
Stanford
Center
for
Legal
Informatics.
She
has
authored
several
books
on
legal
innovation
and
technology,
delivered
six
TEDx
talks,
and
her
insights
regularly
appear
in
Forbes,
Bloomberg
Law,
VentureBeat,
TechCrunch,
and
Above
the
Law.
Her
work
treats
law
as
essential
infrastructure,
designed
for
how
organizations
actually
operate.