Reviewed by Susan Graham
Middle School Family and Consumer Science
Stafford County, VA

Creating Meaning from Chaos in Order to Move Mountains

If my review subtitle has the ring of a sermon title, it may
be because Reeves has taken on something of an Old Testament
prophetic role in telling hard truths about accountability
based on objective analysis of real, rather than perceived
performances. While the Prophets leaned heavily on signs and
wonders to back up their positions, Reeves depends on data
and numbers to add veracity to his pronouncements. He offers
a vision of moving statistical mountains and includes appendixes
of tools to help in the excavation of good practice and the
construction of data walls that help measure progress.

For Reeves, objective analysis is inherently tied to data collection
and statistical scrutiny. At a time when many casually and
selectively access research to justify pet policies, Reeves
demands rigor that does not tolerate the sloppy slip that
allows a correlation between two indicators to be promoted
to the level of causation.

His system node analysis acknowledges that school and student
performance must take into consideration an enormous range
of unrelated but intertwined multi-variables that arise from
diverse educational circumstances, practitioners, participants,
and outside factors. Reeves concedes that pure research that
identifies absolutes is not possible to do with real live
people. What he does is to analyze chaotic masses of data
to look for recurring commonalities. Through this process
he has identified consistent factors in schools that are achieving
academic excellence as measured by multiple criteria.

Successful schools are alike in that they:

Include consistent non-fiction writing assessment in all subjects

Use frequent common assessments

Provide immediate and decisive intervention

Apply analysis of data for constructive changes

Have staffs that accept responsibility for outcomes and hold
students to high expectations

If your first thought is "Well, that won't work at my school
because..." then you just fell into the trap of accepting
myths that Reeves maintains constrain education reform. Just
as students fail when they are willing to believe that "demographics
are destiny," teachers too often excuse their own failure
by buying into the myth that they, too, are victims who have
no control over their practice or its impact on their students.

Administrators and Central Office leaders put faith in their instructional
design decisions (the right reading program or scheduling
plan) as the silver bullet that "fixes" school. And Reeves
presents a study that supports the educator mindset that central-office
level decisions control instruction. But when the same respondents
were asked to identify who makes a series of specific instructional
decisions, the teacher in the classroom emerges as the true
maker of critical decisions, with the most real control.

Good research design requires a hypothesis based on something other
than random "what ifs," so while Reeves acknowledges that
new approaches have no guarantee of success, program decisions
should be grounded in research that predicts probability of
success rather than based on gut feelings and fervent hope
for a positive outcome. Reeves holds that the lack of experimental
purity does not excuse us from implementing change; but, instead,
requires us to carefully monitor the implementation of programs,
to closely and continuously examine progress by multiple measurements,
and then to make that information publicly and prominently
available.

Still skeptical? That's okay because Reeves maintains that "behavior
precedes belief." That means that change must begin even though
not everyone has bought into the plan. But he warns against
halfway, unsupported initiatives because teachers have seen
many too plans come and go or simply come and stagnate. As
a result, teachers are, with good reason, often skeptical
about a new, untested weapon to combat an "America at Risk."
Comments like "If you would just get on board with the new
plan the kids would all be successful!" from an administrator
are likely to result in a defensive "I didn't break these
kids! Don't blame me and my practice!" from the teachers.

We have considered "deficit model" education reform in our Teacher
Leaders Network discussions. Reeves addresses that in his
approach to analyzing school performance. He suggests that
rather than looking at what is wrong with teaching and learning,
leaders should begin with a treasure hunt to identify "islands
of excellence" where positive things are happening within
a school. By carefully analyzing those islands of excellence
for common practices or attributes, patterns begin to emerge
from the chaos of what may appear to be unrelated instructional
disciplines and strategies.

Once chaos becomes patterns, through the analysis of data, Reeves
suggests school leaders must take the initiative to align
school operation and instruction with the identified successful
practices. But continued data collection, information tracking,
professional development and training, and program implementation
represent a massive amount of work. Change isn't easy and
most of us fear and avoid it; but Reeves maintains that to
resist making changes in the face of student failure is "belligerent
indifference."

He tasks administrators to set priorities and initiate change
but warns that success is dependent on the administrators'
willingness to delegate and share responsibility and control,
provide sufficient opportunity for collaboration, and to be
transparent in the ongoing evaluation of outcomes. He also
warns that to instigate new programs without evaluating the
efficacy of continuing existing initiatives and limiting the
number of concurrent initiatives is to invite failure.

But doesn't moving mountains sound like an invitation to failure?
Once again, Reeves returns to his statistical roots and one
its earliest lessons—the bell curve—which he tosses
out on its ear. Just as he refuses to accept demographics
as destiny, he refuses to accept the bell curve as a predictor
and justifier of failure. Reeves holds that a bell curve represents
the naturally occurring distribution of student performance
without intervention. Therefore, if teaching produces no significant
change in the curve, then there is no indication that the
teaching made any real difference beyond progress students
might have made on their own.

Reeves asserts that effective teaching can and should reshape that
normal distribution curve to look more like a mountain than
a bell—with individual student achievement data points
skewed to the right and clustered close together resulting,
in a peak rather than a curve. Effective instruction may not
eliminate the outliers of failure, but it can, and should
make a difference. It occurs to me that this aligns with the
parental and student expectations of "above average" as the
new "average." Teachers can feel reassured that it is possible
to meet that expectation with academic integrity intact.

For data collection to be useful, Reeves cautions, consistent
shared evaluation is necessary. He maintains that evaluation
must avoid the well intended but misguided practice of allowing
compliant completion to be confused with proficient performance,
and that grading policies (yes, the old zero debate) need
to be logical and consistent.

The last paragraph of The Learning Leader gives insight
into Reeves' uncommon combination of dispassionate statistical
analysis and missionary zeal about education practice.

Many people live their lives aspiring to make a difference
and to live a life that matters. There need be no such
uncertainty in the life of an educator or school leader.
Every decision we make, from daily interactions with students
to the most consequential policies at every level of government,
will influence leadership and learning. After all these
words, statistical analyses, and graphs, the preceding
pages come down to this: What we do matters.