A review of the Dragon Naturally Speaking voice recognition software

Reviewed by Karen Van Duyn
High School English & Drama (IN)
Teacher Leaders Network

What does a teacher do when she starts to feel like an “old” teacher? Why, she finds something new to learn! Lately my lessons have mostly been in the area of technology, as that seems to be an area of constant development and change. In the past couple years, I’ve become dependent upon my LCD projector, gone wireless at home (with some tech support), and scratched the surface of Moodle and online learning (great potential and lots of upfront prep time). Very recently I started my relationship with a Smart Notebook and its many wonders. While I’m adept with most of these devices and do so enjoy using them, I have come to the realization that I truly prefer to be a USER rather than a TECHNICIAN.
   
Coinciding with this revelation came the opportunity to review a new software program. Of course I couldn’t resist seeing what I could or couldn’t do with it. Dragon Naturally Speaking is voice-recognition software, and I’ll admit, I approached it with some trepidation. I had used other such programs in the past that required hours of “practice” with my speech patterns in order to be able to give dictation and came away with less than satisfactory results. Naturally Speaking, however, produced great results with a minimum of set up and training time.

I’m still learning all the available commands, but the basics can be picked up very quickly. Admittedly, some of the commands are quite techy – like “Listen to me” and “Stop listening.” I know, I know, pretty difficult. The real bonus is that this software does not require its own special word processing software, but works with Microsoft Office products and other such commonly used programs. It can be used not only to dictate text or data, but it also allows the user to navigate the software itself. This requires the use of other high-tech commands such as “File,” “Open,” “Save,” and “Close.”

Kidding aside, I can perform almost any task with the microphone that I can with a mouse, and the dictation is amazingly accurate. My students even asked me to challenge the software with that old favorite, Supercalifragilisticexpialidocious. Naturally Speaking won that challenge.

It's the best I've tried - but a wireless mic would be wonderful

While I found this dictation software to be the best I’ve ever used, there were a couple drawbacks. The microphone that comes with the product is hardwired to the computer and is sensitive enough to provide excellent transcripts; however, both these features also cause some problems. Extraneous sounds will cause errors in translation (so don’t be working with the TV on or while herds of students are tromping down the halls), and being tethered to the computer by a 4-foot cord limits possible uses. What I couldn’t do with a wireless version of this microphone!

I have had brief love affairs with a great many technological products, especially while the representative is presenting all their features and their ease of use. Often the honeymoon has been over quickly. Either the demands of my classroom prevented further experimentation or the lack of “up-front” time kept me from getting the product to work as well as in the demonstration. This has brought me to my two ultimate litmus tests for technology: Can a USER rather than a TECHNICIAN operate it easily? and How quickly can it be available for educational purposes in my classroom? In other words, would it make my life easier enough to use it?

Skepticism nurtured by experience caused me to take what seemed like an inordinate amount of time experimenting with Naturally Speaking before finalizing my evaluation. In that time, however, I discovered real educational uses that help and excite me as well as my students.

Capturing instructions and teaching note-taking

I type rather quickly so simple dictation features don’t really appeal to me. However, when that dictation is made while I’m actually addressing a class with important information about requirements for an upcoming paper – and I’m able to create an exact transcript of what that class was told, saved to a Word document—then I do get a little excited! Without having to exit from the program or use other peripherals, I can upload the transcript to the class web page for later reference by students. And when one of them says, “You didn’t tell us that,” we have a routine way to check the accuracy of their claim. If, right before the due date, there is a need to recheck the criteria to be met, it’s available to everyone. All this without any extra bit of time to create!

And how about this: the dictation utility can be used to teach note-taking skills. Imagine I’m introducing new information and I expect students to take notes. Using the LCD projector, I can have the Naturally Speaking software capture my words as I present the lesson. I can even organize what’s captured, using bullets or tabs, in real time. I’m modeling note-taking skills to the students without any extra preparation. (Warning: today’s cellphone-obsessed students love to watch text appear and could be distracted watching the technology.)

As students improve on their note-taking, lessons can be dictated without the projector (see warning above) and students can later check their work against the transcribed notes posted on the webpage. Those posted notes can be especially helpful with inclusion students. They can use the real-time visual of what is being said and/or benefit because their learning center teachers now have ready access to the lecture and can enhance their tutoring. A wireless microphone would allow me to move around the classroom and check their notes as I present them.

Mixed student reviews on paper-grading

With recent cuts in staff, my paper load as an English teacher has greatly increased—along with the number of students in each class and the number who need extra help. Finding time for conferencing with students on their papers is much more difficult. With Naturally Speaking, I worked on making verbal comments on a student’s paper as I read through it, as if they were sitting with me. Those comments were then saved as a Word document and sent to their student folder on our network for their retrieval at any time. This process isn’t as comfortable as I originally thought it might be because tone of voice isn’t always obvious in the transcribed text—especially when text is the product of an impromptu, stream-of-consciousness type of assessment. The students have given this strategy mixed reviews. They felt the comments are sometimes clearer and more thorough than the paragraphs describing the strengths and needs of their paper that I write on each title page, yet not as effective as one-on-one, face to face discussions.

Overall, this USER is pretty impressed with Dragon Naturally Speaking. It has multiple uses, including replacing keyboards and mice for those who are dexterity-challenged. I have two senior boys who already have this software on their graduation wish list so they won’t have to type their college papers (Mom won’t be there to help).

You might discover other uses for Naturally Speaking that fit your particular teaching needs or style, but the bottom line for me is that it can both enhance my educational strategies easily and keep this “old teacher” learning new things. Now, if I could only find the proper wireless microphone, I could try to...

Karen Van Duyn, a 34-year veteran, teaches English/LA and drama at a rural high school in Indiana, where she also sponsors the Student Council and the National Honor Society.  The awards and honors she has received for her teaching are always overshadowed by the experience of seeing the “AHA” in the eyes of her students.

[NOTE: Karen tested Dragon Naturally Speaking (Standard edition) for the PC. The company has recently acquired MacSpeech for Macintosh OS X, but we haven't tested that one. Karen received a review copy of DNS at no cost but was free to judge it completely on its merits.]

Metaphors and Analogies: Power Tools for Teaching in any Subject
by Rick Wormeli
(Stenhouse, 2009)

Reviewed by Marsha Ratzel, NBCT
Middle Grades Math and Science (KS)
Teacher Leaders Network

Metaphors and Analogies: Power Tools for Teaching in any Subject taught me more than any education book I’ve read in the last 10 years.

This book opens doorways into using metaphors and analogies as instructional tools that I never dreamed of. Rick Wormeli brings both his intellectual insight and his pragmatic understanding of daily classroom instruction to what is an often-overlooked but critical part of effective teaching.

In Chapter 1, Wormeli opens by saying “Little in education has as much influence….as metaphors and analogies to make unfamiliar concepts clear.” He urges teachers not to limit themselves to those “teachable moments” that arise unexpectedly in class – but to make deliberate and purposeful use of stories and examples that help students experience “aha” moments and build understanding.

Chapters 2 & 3 sketch out a plan for teaching students the format of metaphors and metaphorical thinking. Wormeli offers a way to critically evaluate metaphors for their usefulness and to engage students in that process. By internalizing the structure of this kind of analogical thinking, students gain a framework for looking at these comparisons and ultimately finding comparisons of their own.

Wormeli stresses that background knowledge is critical to understanding metaphors. This section of the book is loaded with ways to improve vocabulary and background for all students, including those in ELL programs.

I hadn’t considered how incorporating physical movement amplifies metaphors. It only makes sense now that I’ve read Chapter 6. Wormeli provides excellent examples to illustrate how to power-up critical thinking by adding dance moves, song or art. The following chapter adds visual metaphors. Wormeli explains how to attach an image to the concept you are introducing to make it long-lasting. I hadn’t seen a rebus since my childhood days of reading Highlights magazine, but Wormeli quickly helped me remember how powerfully those images were in making my own learning “sticky.”

Chapter 8 exposes the reader to scaffolds that students can use to create their own metaphors. At the same time, Wormeli reminds us that the kind of conversation that pushes deep thinking happens as students promote and defend their ideas. The chapter called Incubation Stage shows how to deepen and extend metaphors. One of the things I learned was that, at some point, a metaphor breaks down. As it breaks down students come to realize the comparison isn’t literal. It becomes a model they can use for thinking. Once they assimilate the model they will start connecting the “likeness” in unlike things between and within disciplines.

Finally, in understanding all this, students and teachers sharpen their analytical skills. They embrace abstract kinds of thinking and students are able to take their concrete knowledge to the next level –which is where every teacher wants to journey with our kids.

I heartily recommend this book for you and for a book club study group. So much practical advice. It’s worth every penny and more.

Marsha Ratzel teaches middle grades math and science in the Blue Valley (KS) School District, where she has also served as an instructional and technology coach. She blogs at Reflections of a Techie. Marsha participated recently in some discussion of Metaphors and Analogies at the Stenhouse Publishers’ Ning site.

Bill Ferriter’s participation in the Teacher Leaders Network began in March 2003, on the very first day of TLN’s existence. We kicked off TLN with an email conversation among 200 accomplished teachers, mostly from states in the southeast. The idea was to stage a series of 3-day chats exploring some key professional issues and then see who wanted to stick around and become the nucleus of a national virtual community of classroom educators.

By the end of the six-week startup, Bill’s provocative questioning and his willingness to reflect on his own professional practice — tarnish and all — made him a leader in our fledgling community. He’s gone on to serve as a TLN Senior Fellow, co-author of a major TLN TeacherSolutions report, and the internationally respected TLN blogger known as The Tempered Radical, whose lively mix of classroom practice, policy and politics, and digital learning topics attracts a broad audience and recently led to a monthly column, Digitally Speaking, in Educational Leadership magazine.

As it turns out, Bill’s willingness—compulsion, even—to bare his teaching soul makes him one heck of a book author, too. Building a Professional Learning Community at Work: A Guide to the First Year, co-written with principal Parry Graham, grows out of a several-year struggle by Bill and a group of teacher colleagues at his North Carolina middle school to learn to collaborate in powerful ways. In this interview, Bill recounts some of the pain and gain of that trial by fire, and how he and Graham went about translating their own experiences into a practical primer for other educators who might be willing to set forth on a similar journey.

As a bonus, I've asked Bill to speak to fellow teachers who harbor their own desire to write a book from their teaching lives. Bill generously shares his secret to publishing success, but I’ll warn you in advance. It involves a lot of writing. – John Norton, co-founder and moderator, Teacher Leaders Network.

----------

What created the burn to write a book for teachers just beginning to explore PLCs? Give us some history.

Bill Ferriter: I think the burn to write a book for teachers just beginning their work with professional learning communities is a product of my own early struggles and successes with collaboration. About five years ago, I had the opportunity to open a new middle school being built as a PLC from the ground up—and I was paired on a learning team with five of the most intelligent and capable language arts and social studies teachers that I’d ever met.

The problem was that we had no real idea what it was that we were supposed to be doing with each other, and that led to early frustrations. Our initial meetings were rambling, unfocused affairs and we often felt like failures. We fought through personal conflicts and professional disagreements, though, and eventually found a working rhythm and synergy that made all of us better teachers. I wanted to try to make those early struggles transparent to other teachers, hoping that the lessons my team learned might sustain others when the inevitable challenges of collaboration arise.

Collaborative work can make any teacher more productive and professionally satisfied, but only when early efforts at collaboration are structured and meaningful. Parry and I believe that Building a Professional Learning Community at Work can provide some of that structure for learning teams at any stage of their professional work together.

Your book is filled with action-oriented subheads, reproducibles, practical recommendations — there's even an annotated chapter guide divided into seasons of the year. It has the feel of authors who have been there and done that. How much easier might things be for teacher teams who take your advice?

Bill Ferriter: I think the real strength of Building a Professional Learning Community at Work is that Parry and I are both full-time practitioners. I still meet with a learning team every week, trying to identify instructional practices that work and ensure the success of every child. Parry still works as a building principal, trying to create the systems that enable teachers and students to thrive.

That first-hand experience with the real work of learning teams is evident in every chapter, handout and subtitle in our text. Our suggestions, strategies, and materials are suggestions, strategies and materials that we’ve used successfully in our work—and the text is really nothing more than a public reflection of the learning that we’ve done as we’ve tried to make professional learning communities happen in our own schools.

 If you had to come up with 4-5 key insights represented in the book, what would they be? Tempt us.

Bill Ferriter: Picking out 4-5 key insights is close to impossible because our readers will all be working on learning teams that have their own unique personalities and challenges. What may look like a key insight or solution for one team may be a strategy, practice or behavior that another team is unprepared for.

Teams new to collaboration are likely to find the strategies for structuring productive meetings to be the most valuable, while highly productive teams might embrace our suggestions for useful data conversations. Administrators—regardless of how long their teachers have been working in learning teams—will love the surveys that we’ve developed to gather information about the overall health of their professional learning teams.

In some ways, that’s the beauty of our book—it is designed to provide customized support for any learning team, regardless of their current circumstances, and is written with all the players in mind.

The "Winter" section of your book is titled "Weathering the Challenges." Is this the stage — 4 or 5 months in — where even the best-intentioned PLCs are most vulnerable to inertia or break-down?

Bill Ferriter: To be honest, I’m not sure that learning teams are ever fully invulnerable. I know that our team has cycled in and out of moments of inertia and breakdown over the past five years.

Sometimes it’s because we try to tackle too many well-intentioned initiatives all at once—simultaneously designing remediation and enrichment programs, collecting and analyzing data, creating a warehouse of best instructional practices. Sometimes it’s because our team composition changes, and sometimes it’s because other priorities take precedence in our personal or professional lives. We’ve found ourselves wandering off the PLC path more than once and for a range of reasons that is almost mind boggling!

I think the key to our success has been our faith in one another and the professional satisfaction that we’ve gained from working together. The levels of trust on our team are high because we’ve got an extensive base of shared experiences with one another. When conflict comes, it’s productive, built on the belief that everyone on our team is working towards a shared mission even when we see alternative routes to the same end point. We’ve learned to listen to one another, to approach collaborative work as an experiment, and to embrace struggles as learning experiences.

You've been at this for some years now. Is there another crisis point several years into the work, when it begins to feel "ordinary" and less stimulating? If so, how did you and your team confront that?

Bill Ferriter: The work of my professional learning teams never feels ordinary to me! After all, I’ve been given the opportunity to reflect on my craft with other like-minded peers. Every year, we find new practices that we’d like to explore, new trends in student learning data that leave us confused, and new structures or processes that might just make our own work more efficient. Our student population changes, bringing new challenges that we’ve got to find solutions for, and new teachers are hired, bringing different perspectives to our conversations.

It’s the process that I’m motivated by—we investigate, we implement, we reflect, we explore, and we learn no matter how long we’ve been together as a group, and investigation, implementation, reflection, exploration and learning are always motivating.

Professional learning teams only become ordinary when we stifle teachers—when school and/or district leaders place an inordinate emphasis on products instead of processes. Districts that create system-wide pacing guides, lesson planning templates, meeting requirements and common assessments in an attempt to make things easier for—or to monitor the work of—teachers ruin the most rewarding aspects of the professional learning community process. We’ve got to give learning teams room to create and to innovate in order to keep the work exciting.

You developed a relationship with PLC experts Rick and Becky DuFour several years ago. How did that come about and what did you learn from that association that helped you and your colleagues back in your own school?

Bill Ferriter: My relationship with Rick and Becky DuFour actually started long before we’d ever met in person. Preparing for my new position at a PLC school, I chewed through Professional Learning Communities at Work—the seminal book on PLCs that Rick coauthored with Bob Eaker—in about 12 hours one weekend, and it was a vision for teaching that resonated deeply. I loved thinking that “someone official” believed in the potential of classroom teachers—and was willing to argue that schools couldn’t succeed until groups of teachers worked together to generate a body of knowledge about what worked in their classrooms and with their students. It was one of the first times that I felt empowered as a professional.

Our first personal contact came a little over a year later, after the National Staff Development Council published a reflection that I had written about the impact that my own professional learning team had on my instruction. Rick read my article and dropped me an email praising the piece. It was pretty amazing to me that a guy whose thinking I respected greatly saw value in something that I had written. We crossed paths in person for the first time at a dinner with the State Board of Education here in North Carolina. “Are you the same Bill Ferriter who wrote a piece about PLCs for NSDC?” Becky asked during our introductions. “Rick and I loved that article!”

Since then, Rick and Becky have been cheering for me—impressed enough by my work to recommend that Solution Tree hire me a PLC Associate. They also provided constant feedback as Parry and I worked through the initial drafts of our manuscript and served as a sounding board in a thousand situations. They are two of the most approachable experts that I’ve ever met, willing to give their time and attention to help others to succeed.

In the end, it’s humbling to know that they believe in me. In the eyes of a lot of people, I’m still “just a classroom teacher.” To Rick and Becky, I’m a classroom teacher with practical experiences to share and a level of expertise that should be respected and admired. I’ll be forever grateful for their confidence in who I am—as a teacher, writer and professional development provider.

In their foreword, the DuFours single out the accessible "conversational tone" of the book. How hard was that to achieve? In fact, how did you and Perry Graham meet the challenge of co-authoring a book — and your first book at that?

Bill Ferriter: To tell you the truth, writing Building a Professional Learning Community at Work was an amazing experience. Parry is someone who I’ve always had a synergy with—he’s brilliant, and we both like to read anything that we can get our hands on that’s connected to organizational theory and human nature. We’ve spent thousands of hours mentally wrestling with the challenges of making professional learning communities work. It was only natural for us to try to turn those conversations into a text that others could learn from.

And we both brought a different set of writing skills to the project. Parry is a meticulous writer who is skilled at organizing thoughts. He was almost singlehandedly responsible for the general structure and outline of our book, and he did a great job churning out chapter after chapter. I’m more of a wordsmith, so after we’d brainstormed together and Parry organized our thinking into a first draft, I’d add the spit and polish. What’s beautiful to me is that by the time we were done writing, neither of us could tell who had written what. The book had become truly “ours.”

Many teachers have a secret desire to write a professional book. What guidance would you offer? What did you discover along the way that you might not have anticipated?

Bill Ferriter: I think the most important advice that I can give to any teacher interested in writing professionally is start your own blog and start it now! Blogging regularly about your professional passions can help you to polish your voice and to practice articulating key concepts in writing—a process that can be difficult for accomplished teachers who often act on intuition. What’s more, blogging makes your thoughts transparent—your audience can push back at your core beliefs, pointing you in new directions or forcing you to find the flaws in your logic. While public challenge may not feel comfortable at first, your thinking will become more nuanced and sophisticated over time.

Better yet, publishers are constantly scouting collections of teacher blogs for potential writers. Almost every professional writing opportunity that I’ve had in the past six years—writing for NSDC and ASCD, having articles published in Educational Leadership and the Journal for Staff Development, landing a contract with Solution Tree—started after someone spotted something that I’d posted online. Blogging is the great equalizer, giving everyone the chance to be recognized and to cultivate an audience.

What havoc did writing a book wreak on your teaching and personal life? How much writing time, how much research time, how much time with editors? How long did it all take?

Bill Ferriter: Building a Professional Learning Community at Work was an 18 month project, John. That’s something I don’t think most teachers interested in writing realize. Parry and I signed a contract with Solution Tree after submitting a proposal and having finished one chapter from start to finish. That was enough to convince Solution Tree that our project was worth pursuing.

From that point, the grind began—and at times, it really felt like a grind! We read everything that we could read, searching for research that supported our key points. While we were confident in our core beliefs—we work in learning communities full time, after all—we knew that readers would respect our opinions more if we could back them up with the conclusions drawn by other recognized experts.

Simultaneously, we were drafting and revising chapters, meeting with one another to bounce ideas around, and changing directions. Typically, Parry would produce a first draft of a chapter and then send it to me. My job was to reorganize and/or reinforce his initial attempts to put our thinking into writing. I’d add language, polish bits that I thought needed polishing, add new recommendations based on my experiences as a classroom teacher, push against points that I wasn’t completely sure of, and create handouts and tools that would support the content of our chapter. Then Parry would review what I’d written before we’d begin a new chapter.

Only when we’d finished our entire manuscript did BPLC make it to Solution Tree’s editors. They sent our text out to three independent reviewers—as well as to Rick and Becky DuFour—for initial comment. All of those people sent extensive feedback that Parry and I were asked to try to incorporate into the final product. After about two months of systematic revising, Parry and I handed off our final copy to the editors at Solution Tree, who worked through the piece to make sure that our language was consistent and articulate.

I probably spent somewhere between 10 and 20 hours a week writing BPLC. It was essentially a part time job! That’s why teachers need to find the right time in their professional and personal lives to tackle book projects. If you haven’t got the time to invest in a book, you’re going to end up frustrated and overwhelmed. And frustrated, overwhelmed writers rarely turn out quality work.

If you’ve had a new baby, started in a new position, or have family responsibilities that you just can’t be away from, it might not be the time to write. Wait for the cycles of your life where professional and personal responsibilities leave an open window for extra work.

Is there another book in the works? You're well known in the edu-blogsphere and beyond for your work integrating Web 2.0 tools and ideas into your everyday teaching. Is that the next project?

Bill Ferriter: I’m actually in the middle of my second book as we speak—co-authored with Adam Garry, another professional colleague and friend. It’s tentatively titled Plug Us In: Five Digital Projects that can be Tackled Today — it's an attempt to create a series of practical activities that teachers can use to introduce 21st Century skills into their classrooms.

What makes Plug Us In unique is that the focus of our writing is on the kinds of skills that today’s students need to master in order to succeed—persuasion, collaboration, communication, information management. While we introduce extensive sets of tips and tricks for using digital tools—things like blogs, wikis, RSS feeds, Voicethread, social bookmarking and shared annotation applications—in our text, they all serve to strengthen good instructional practices.

I’ve got to tip my hat to long-time TLN member Sheryl Nussbaum-Beach for that orientation! Years ago, she convinced me that technology conversations needed to be focused on good teaching instead of new tools. Tools are simply a vehicle for making good teaching—and efficient learning—possible. Those ideas have driven my own thinking about teaching with technology ever since, and they play a prominent role in Plug Us In.

The manuscript is due in the spring—and I’m keeping my fingers crossed that the text will be published in the fall of next year.

Making the Grades: My Misadventures in the Standardized Testing Industry
By Todd Farley
(PoliPoint Press, 2009)


Reviewed by Kenneth Bernstein, NBCT

High School Government & Social Studies (MD)

Teacher Leaders Network

As the use of tests created external to schools and classrooms has exploded, one issue has always been the question of whether to rely merely upon selected response (a.k.a. multiple choice) items, or to also include constructed response items (paragraphs and essays).

Selected responses are cheap to administer; they can be scored solely by machine and the results obtained quickly. It is even possible, utilizing item response theory, to administer the test on a computer and use early responses to vary the items offered to the test taker, thereby determining the level of performance more quickly and accurately.

But many think we need more: after all, life does not ask us to choose one out of four or five pre-selected choices. Thus many colleges and universities, employers, and classroom teachers prefer that the tests include constructed items — “essays” if you will.

While it is possible to machine-score such items, that technology is still in its relative infancy, which is why companies that produce tests have need of human scorers. And it is because of this need that we get Todd Farley’s book Making the Grades.

Farley spent 15 years in a variety of positions involved with the scoring of such constructed responses. He worked for a number of America’s most important assessment companies, often doing the work on contract for various states, including Virginia, where I live. 

I am not a trained psychometrician, although during my now-abandoned doctoral studies I did seriously study issues of assessment. I am a school teacher today, in my 15th year of teaching. Each year but one, I have had to prepare students to sit for external tests – which may or may not have met the criteria to properly be labeled “standardized” — that included constructed responses. These tests have included the Maryland School Performance Assessment Program, the Maryland High School Assessments, and The College Board’s Advanced Placement examinations. During my one year of teaching in Virginia, known for its Standards of Learning (SOL) assessments, the middle school American History test was made up entirely of selected response items. I also bring to this review some experience that parallels Farley: in 2009, I served as a Reader for the Advanced Placement examination in U S Politics and Government and scored one of the four Free Response Questions on that year’s examination.

As I glance at my copy of the book, I have more than 40 sticky notes that I have affixed to pages containing passages I thought might possibly be worth quoting. Some I obviously will forego. Farley offers explanations of terms like reliability and validity, and explains how in the case of reliability, the term was often misused by those supervising the scoring process. Simply put, scoring companies are often satisfied if those scoring agree 80% of the time, even if that to which they agree is erroneous. It is like a scale that consistently reports your weight as 20 pounds less than reality. The information you obtain is reliable — but it is NOT valid.

What educational measurement should provide is the ability to draw valid inferences from the information analyzed. If nothing else, reading this book will raise questions in your mind about whether many of the tests being used to evaluate students, teachers and schools meet that standard.

Farley demonstrates that reliability is not necessarily something we can rely on. Allow me to quote an entire paragraph from pp. 55-56 to illustrate:
But you want to talk about a sliding scale? The scale we used to score writing flopped about like a puppy on a frozen pond, going every which way, keeling over and standing up and falling down. In scoring writing, for instance, an essay that had a good development of ideas could earn a 6, a 5, a 4, maybe even a 3. An essay that was troubled on the sentence level in terms of grammar, usage, and mechanics could earn a 1, a 2, a 3, perhaps even a 4, 5, or 6. (I don’t dispute the idea: Gertrude Stein said of F. Scott Fitzgerald that she’d never met anyone who was such a poor speller, yet he still managed to produce a decent text or two.) The point is that essays with identical levels of ability in certain areas could end up (due to other considerations on the rubric) with significantly different scores. In scoring writing, we were far from having hard and fast rules to live by. It all seemed a little untenable, rather mystifying, and the easiest thing to do was to hand your essay off to your neighbor or plead with your supervisor for help.
That passage references the idea of the rubric, the standard by which the grader is supposed to evaluate the essay. If a rubric is sufficiently clear to give guidance, it also may be too rigid for the occasional creatively written paper. A rigid application of the rubric might, as Farley illustrates on more than one occasion, result in a good piece of writing being undervalued and a poor piece of writing receiving a high score.

And the scoring companies often have little control over the rubric and how it is applied. They are scoring under contracts issued by states that may leave them little flexibility. Allow me to illustrate using an example of a scoring team examining an anchor paper.

Anchor papers are supplied to scorers to give examples of the work expected at each scoring level of a rubric. Farley provides the four-point rubric for an 8th grade writing assessment (descriptive mode). The rubric, provided by the state in question, expects the student to use a five-paragraph format. According to the rubric, for the scorer to assign all 4 points, the organization, focus and development, style and sentence fluency, and grammar-usage-mechanics should be considered “excellent.” For  3 points, they should be considered “good.”

Farley describes how table leaders were trained to lead the scoring of this 8th grade assessment. After reviewing the anchor paper for a score of 3 (which Farley reprints in the book), all of the table leaders were scratching their heads, describing the paper as “lame.” One seventh grade teacher in the group argued that she would not consider this essay good work by her students. Another pointed out that it consisted of only simple sentences, and Farley (also a table leader) noted it had no voice and no style. The response of their trainer Maria is telling:
Maria looked down at the essay. “I’m not saying I’d give this a 3 in my classroom, either, but that’s how we have to score it based on this ‘focused holistic” rubric. Most importantly, in this state’s Department of Education, the essay has a five-paragraph format, with introductory, body, and concluding paragraphs, and an introductory sentence in all five of them.”
As shocking as that is, the reader might not be prepared for what comes next. It’s an anchor paper that contains simply brilliant writing (and would be so for a high school student) but earns a score of 2. Here is Farley’s account of the conversation that ensued among the scorers:
   Greg scoffed. “This kid needs a publisher, not a score from us."
   Maria looked guilty. “I know,” she said. “I certainly wouldn’t give this a 2, either. The writing may be sentimental, but it’s first-draft work from an eighth grader. It’s a damn good response, I agree.”
    “So?” Harlan said.
    “Well, what’s the important person or thing in the essay? It’s her favorite spot, a fact we don’t know until the last sentence. That’s not five-paragraph format, is it? There’s no introductory paragraph, no introductory sentence --”
   “No,” Greg said, “it’s way more artful than that, building up the suspense nicely and using some beautiful descriptive language.”
   “Yup,” Maria agreed shrugging. “I know. But this is how they want us to score them”
   “Really?” I asked. “Rather a tedious five-paragraph essay than a beautifully done three or four paragraphs?”
   “It seems that way,” Maria answered. She looked at us, resigned. We looked back at her defeated.
   “All we care about is the formatting?” Pete asked.
   “That’s not the only thing, “ Maria answered, “but it is the first thing.”
   “Wow,” I said, “it almost seems a kid could get a 3 for turning in an outline.”
   Maria thought about it. “Not quite,” she said.
The book is a good read, such a good read that I hesitate to go into too much detail, so that I don’t spoil the enjoyment – and the shock – you will experience as you read it. But I’ll share a few other samples.

At the time of the passage cited above, the scorers were earning $10/hour or less. They were not required to be content matter knowledgeable, something that was a persistent issue in the experiences Farley cites. Scorers were trained, and had to meet a certain standard of scoring accuracy in order to be allowed to score. But the need for scorers was so great that the standards of accuracy were often bent, and scores were changed and manipulated to maintain acceptable levels of accuracy.

Please note that term — acceptable. Farley cites examples of where too great a level of accuracy could cause problems, and this was truly scary, because the examples involved the scoring of the National Assessment of Educational Progress (NAEP), which is supposed to be the ‘gold standard” by which all other educational assessment in this nation is measured. Farley was told he could not have a higher degree of accuracy than was recorded in previous scoring cycles lest the comparability of scores from cycle to cycle be lost. Ponder that for a while.

I do want to offer some cautions. Farley paints with a very broad brush. What he says is certainly widely applicable, but not universally so.

I teach in Maryland, which until May 2009 included two kinds of Constructed Responses (Brief and Extended) as part of the High School Assessments required in four subjects (Biology, Algebra, English, and Government) for graduation from high school. Each constructed response was scored by the same 4–point rubric, a copy of which students had during the exam. In the scoring process, each response was read by two scorers. Inter-rater reliability required only that the scorers gave adjacent scores, not identical scores. If the scores were not identical, the student received the higher score. I am not sure how accurate a measurement that was, but at least the students got the benefit of the doubt, unlike the scenario above that Farley described.

Cost control is another factor that influences the quality of the scoring process. Farley’s account focuses on scoring companies that paid relatively low wages to individuals who often lacked the necessary professional background to make accurate, independent judgments about the work they were scoring. As a result, a highly controlled system of scoring was imposed. But this method of assessing writing samples is not universal.

Here I speak from my experience as a reader of free response questions on the Advanced Placement exam for US Government and Politics. To score this exam, an individual must teach the subject in a post-secondary institution or have at least three years experience teaching the AP course in a high school. We were certainly qualified as to content. We were also paid substantially more than the $10 an hour Farley cites for the incident above, plus expenses for transportation, food and lodging.

We were thoroughly trained. We had our work closely examined at first, until we demonstrated our competence. We were spot-checked regularly by table leaders and by question leaders. The range of our scores was monitored by computer, and if we showed any scoring patterns that raised questions, our work would be reexamined. But once we got going, the scoring was limited to a single reader because we had over 100,000 exams (four questions each) to be scored in less than a week after training.

I know how seriously the Advanced Placement officials took this process because of my own experience. I read very quickly, and I was so much faster than others that, in the beginning, my work was checked very closely until the question leader determined that I was scoring accurately. When I had any doubt about a response, I would on my own initiative check with one of my fellows and/or with the table leader. That pattern was widespread among my fellow scorers.

I would argue that the AP people have demonstrated that reasonably accurate and consistent scoring of constructed response by properly trained people is possible, if one is willing to accept the concomitant costs.

Still, despite the caveat I offer based on my AP experience, I think Farley’s book is a valuable read with much to tell us about the often poorly understood processes and implications of large-scale high stakes testing. He ends the book with these blunt words:
If I had to take any standardized test today that was important to my future and would be assessed by the scoring processes I have long been a part of, I promise you I would protest; I would fight; I would sue; I would go on a hunger strike or march on Washington. I might even punch someone in the nose, but I would never allow that massive and ridiculous business to have any say in my future without battling it to the bitter, better end.

Do what you want, America, but at least you have been warned.

 
[click to enlarge image]

Teachers in the TLN Forum had lots to say about a recent New York Times article reporting on the growing tendency of teachers to sell lesson plans and other teaching materials they’ve developed via the Internet. A selection of comments from our discussion appeared in a recent post at Teacher Magazine, titled “Should Teachers Sell Their Class Materials?” If you’re interested in this topic, be sure to read this story.

We had more insightful comments than we could possibly fit into our Teacher space, so we thought we’d share more of them here. We do recommend reading the Teacher story first.

Several TLN members were taken aback by this paragraph in the NYT article:

“To the extent that school district resources are used, then I think it’s fair to ask whether the district should share in the proceeds,” said Robert N. Lowry, deputy director of the New York State Council of School Superintendents.

Like many teachers, Ken creates his lesson plans during uncompensated time:

If the school system wants to claim ownership of my lessons, which I create on weekends and evenings and during vacation, during the many extra hours for which I do not receive additional compensation, first they will have to pay me for that time. If they are not going to pay me additional for the hours I put in, which include searchingfor materials,reviewing them, thinking about how to use them, designing lessons and units, and laying out the materials (which are often NOT provided by the school system), then they have no claim of ownership and whatever I choose to do with them is my business.

As it happens, I design my lessons for my own use, and I am not sure how effective they would be for other people. I modify them based on my knowledge of my students individually and my classes each as a separate collective.

I am usually more than happy to share ideas and materials with other teachers -- in my building or in my system -- that I know perhaps only electronically.

But it is MY decision to share gratis the results of my efforts, and if I chose to charge -- assuming anyone would want to pay -- I fail to see how my school system has any claim upon the income I would generate.

A school system which wishes to take that approach might suddenly find it has so alienated its teaching staff that they will work to rule, which would mean no planning nor correcting/grading of student papers/tests outside the hours of the school day. Oh, and if you have two or three preps, one planning period a day will not get it done.

Sorry, but I am getting irritated at the small minds running some school systems who cannot see the forest for the trees, or whatever other image you care to use.

I am tired at having to take on an outside responsibility this evening for three hours plus travel time to make $100 to help pay my bills. If schools are so concerned about economic equity, they should start by paying us for all the hours we actually do work.

Mary added:

I agree with Ken. I am absolutely appalled at a school system's suggestion that they might own what a teacher creates. Since we have invested our own time and often our own money to educate ourselves, I believe the public school system owns none of it. (It's interesting that we are sometimes treated like blue collar workers and then other times like in-house talent.) In addition, I am currently getting my own professional development through various online NINGs, blogs, etc. that have nothing to do with my system's sanctioned program of development - which appears to be nil.

Nancy wrote:

As for the bureaucrat who works for New York's superintendents organization and his remark about the school district owning teachers' lesson plans and materials--wouldn't it be great if schools paid teachers extra for creating curriculum and materials, custom-tailored to their student population? In that case, teachers would be fairly compensated for their expertise, and the school would have some ownership rights. Schools don't balk at paying the big bucks for packaged curricula--and all of those materials were written by someone. Why shouldn't ground-level expertise be rewarded by both recognition and remuneration?

Vicky noted:

Imagine districts that might go beyond the familiar "publish or perish" mentality of higher education to a "produce or perish" attitude, requiring teachers to have a certain amount of financial intake each year. In addition, districts might consider the money teachers make on their lesson plans as stipends or part of the expected budget, so teachers would lose out on any classroom or departmental assistance they may currently receive.

Kathie offered her veteran’s perspective:

How much times have changed since I first entered the classroom in 1970! In my first year as a fourth grade teacher, an assistant principal told me the district follow-up worksheets were "crap" and I should make my own. How many hours I worked that first year to create my own materials, knowing I knew virtually nothing about how to do it, much less how to create an improved product! However, that was also the first step in acquiring a great affinity for creating my own lessons and units.

Now, however, I can go to the Internet and search for new ideas to supplement my own. I have yet to pay for a lesson -- I search under "free.” But teachers have been looking for pay for their ideas for a long time. A few years back, a company purchased my graph art ideas for two workbooks. I made $1,200 on the deal, but now I'm wondering how much more I might've made in the era of the Internet!

I see nothing wrong with teachers profiting from their ideas. I would try it myself, except that after so many years in the classroom, I recognize the fact that I have no clarity as to what is solely my own ideas and what I "borrowed" from others over the years. I may have to think more on that one!

Elizabeth wrote:

My first response to this article was..."What?! Are you kidding me?!"

Putting a price on sharing--in garage sale manner mode--just seems strange to me. As I read on it came to me. It's not about the sharing. And it's not about the money. It's about the underlying issue that many teachers feel undervalued. So, they're seeking this "long-awaited recognition of their worth." In the article, teacher Erica Bohrer says, "Teaching can be a thankless job.....I put my hard-earned time and effort into creating these things, and I just would like the credit."

I'm just not convinced that this is the way to fill the void of validation. I mean if they created great lessons that were effective for their students, and no one buys them, does that mean they're not great lessons--or valuable teachers? Of course not.

And if a teacher sells lessons, does that sale really make the teacher more valued or provide the credit she or he craves? I don't think so.

It can't be all about the money. There are so many other, more professional ways to share and achieve a sense of validation.

Joe disagreed:

In my opinion, the article was not about teachers feeling a sense of validation by having the ability to publish. Instead, it was whether teachers should be allowed to publish their intellectual property and profit from it. Those are two entirely different issues.

Elizabeth replied:

Yes, but what's underneath? I agree...on the surface. It's about teachers having the freedom to profit from their creations. Yet, I can't help but ask...why in this manner?

And what is the result of this profit...is it only for the money? Or does the profit also include a more intangible form? Teachers in the article speak to the monetary benefits (that's clear)..but some get the sense that "hey, my lessons are valued...someone out there likes my thinking." And the ego can rest.

 Sherry also disagreed about validation:

For me, the issue is not validation; rather, it's being compensated for the time spent designing lessons. Selling my lessons isn't an ego-driven endeavor. Instead, it justifies (in a monetary sense) all of the hours spent away from my family -- countless unpaid hours. Yes, the "real" reward is creating meaningful learning experiences for my students. Handing out free lessons to my colleagues does nothing to nurture their practice; rather, it gives them the space and time to hang out with their families. If I'm going to work for my colleagues, then I don't think it's unreasonable to earn compensation for it.


Bob had a similar view:

I agree with Sherry's points. If teachers are merely using their colleagues because they do not want to do the extra work, they are missing out on the power of designing, planning and learning. There may be a need for slightly uncomfortable conversations about what an optimal collegial relationship looks like.
 
Other math teachers have told their students to check my website for reviews or solutions. The extra attention keeps me on my toes because I need to keep all errors to a minimum since distribution is much wider than I expect.

I probably am less sensitive to the proprietary nature of my lessons and work since I do not have a patent on the point slope form of a line. When I find out that many teachers from other schools have used my materials, I feel like I am paying it forward because I am still so grateful for the guidance, resources, and ideas that a colleague shared with me when I first started teaching A.P. Calculus. It also comes back to help me. A teacher used my resources from last year, improved them, and shared them back with me this year. We now are at the same school and share resources all the time. The relationship, conversations, and sharing is priceless.
 
The capacity of teachers to learn and do new things varies throughout the year. I have seen teachers learn to use technology and do things that I never expected. As a math teacher, I do not really care about products. I care about the discussions we have as colleagues about student work, the assessments, and how students are learning. I have received a lot of recognition this year, so I care that my colleagues see that I am still doing the work, sharing, and happy to have them as colleagues.
 
I believe in the power of collaboration and the power of opening up our practice via transparency. I wish they both got a lot more attention.
 
Anthony wondered about collaborative products:

Many research studies have shown that the schools that are best equipped to make improvements in student outcomes are those where the staff collaborates and learns together. When we do this, we organize ourselves into teams, we plan units together, and develop common assessments. Schools that are really great at this have interdisciplinary teams collaborating on projects that allow students to delve deeply into an issue or a theme, and develop their skills in math, science, social studies, writing, and art all at the same time.

If a creative team has come up with an outstanding set of resources, to whom should that product belong? These lessons usually take a great deal of time to create, time beyond the teachers’ compensated day. That makes me think they should be entitled to some reward for their outstanding work.

Anne noted that there’s nothing new about teachers marketing what they know:

I've thought a lot about this article. Teachers have always published, you know. I remember regularly buying books by a teacher who wrote earth science activities. I met a number of teacher-authors and bought their materials at conferences throughout my career. I actually started writing my own book as a teacher who was eager to make teamwork successful. Is the situation described in this article different? Actually, I think it is.

First of all, "publishing" today has taken on new meaning. Everyone with access to the Internet can publish information - or misinformation for that matter. I think that the philosophical discussions about whether teachers should publish their lesson plans is a moot point. It's going to happen. So, maybe we can better focus our energies on guiding principles to use when considering buying lesson plans.

 First, who's to say that the information in the lesson plan is correct? I remember getting some lesson plans from a veteran teacher during my first year of teaching. The lesson was about gravitational force and supposedly demonstrated that the angle of the earth allowed you to balance raw eggs on end during the spring Equinox. In other words, this teacher was unwittingly putting forth a popular urban legend as credible science. Scary. Un-vetted lesson plans have the potential to spread misinformation and misconceptions more quickly than ever, and to actually sabotage student learning.

Next, we need to guard against teachers re-retreating into isolationism and guarding their best ideas, lest someone else take them and publish them. It may sound far-fetched, and I hope it is. However, teachers haven't been out of the competitive mindset for long . . . if they are now.

The New York Times article seemed to look at this issue from the standpoint of a teachers "right" to publish his/her lesson plans. I'm not concerned at all about that -- it's going to happen. It's happening now. As educators, however, we might approach this wave of lesson plans for sale with thoughtful skepticism concerning the quality, accuracy, and effectiveness of the teaching materials that will be put out there.

Marsha wondered “Who owns what?”

Very little of what I create is original. Most of what I create is a synthesis of curriculum standards, the instructional support materials I have (both the textbook and anything supplemental I've purchased over the years) and the multiple times I've tried it out in class.

What is original is my "mix" of the ideas. It's the match between my  students' needs and my teaching abilities/capabiliites. If what I've re-created is mostly like the places I got the original idea, then it probably doesn't belong to me. If what I've re-created is mostly dissimilar, then it's probably mine.

Practically speaking, I find it difficult to think I have much to add to the body of knowledge about something like adding, subtracting, multiplying and dividing fractions, for example. Anything I would have to offer wouldn't be substantially different than what you could probably find somewhere. The most recent work that I've been doing with reading heavily relies on the work of Stephanie Harvey and Ardith Cole. So is the mix of their techniques and ideas with my curriculum and textbook unique and something that is my intellectual property? Sure I use stuff and then modify it in response to how well my lessons went and what I saw was ways I could improve the responsiveness to my students. But I didn't generate the original idea.

To sum it up, I have no problem with anyone selling anything that is theirs. Where I feel like we have slippery slopes is in who created the idea and who owns it. It seems to me that I would need to much more reading about remixing and mixing of old ideas with new approaches before I could understand who owns a lesson plan I create.

I think this discussion springs from the creation of new industry as more and more commoners learn how to commericalize the internet. What paradigms of thinking I've created are probably based on old notions of publishing and copyright. It is super exciting to hear, think and consider the ways in which economics have change in response.

Mark sees vestiges of the idea that "teaching is missionary work":

Thought One: I think part of the tone of this article is a result of the idea of "teaching as missionary work." Missionaries aren't out there to make money and there are religious orders that take a vow of poverty. Although I didn't expect to become a millionaire as a teacher, I also didn't agree to place my career before all else.

Thought Two: What happened to the "let the market decide" attitude toward school improvement. I guess it's OK to let capitalism decide the fate of public education in the various forms of school choice but its not OK when teachers gain from the free market. Another point -- no one is forcing teachers to buy these lesson plans. A teacher can find many resources for free on the web so this boom in lesson plan sales must be filling some kind of unmet need (supply and demand - capitalism at work again)!

As for quality, teachers have always pulled resources into the classroom from multiple places that weren't necessarily reviewed for content. Is there a difference between a poor lesson plan that is free and a poor lesson plan that someone paid for? I think that part of my professional expertise as a teacher is being able to make that judgement call.

A personal story: the first year I taught, I was hired to teach chemistry in a brand new math and science academy that was being formed in my district. I taught with a veteran chemistry teacher. One of our charters was to make the chemistry instruction unique in the district so we would sit and either find and modify or create new new labs for students. One of the labs we created was a quantitative study of the reactants and products of a chemical reaction. We came up with the procedure on our own. About 7 year later, this lab appeared as a lab kit in a catalog for one of the major science supply companies. I can't tell you how much that irritated me (the other teacher had moved to a different district so I don't know what her response was). This was something I shared with colleagues for free and now some supply company was making money off of it.

I think the flap about this is overblown. There are many, greater issues in education to worry about.

Bill has a vague sense of unease:

I find myself agreeing with most everything here... whose intellectual property a lesson plan is, whether teachers should have the right to sell them, that teachers are looking for respect as much as money, that money is nonetheless a factor, and so on and so on. Yet I was plagued by a vague sense of unease for which I just didn't have the words.

Maybe one reason for this is that I tend not only to eschew Internet-based lesson plans and activities, but also to toss my own more or less as soon as they're done. Case in point: my kids are currently doing a student-designed unit on "raiseURvoice" to combat racism and sexism. I designed an activity for them today to teach them the concepts of overt and covert racism and sexism. I think it worked reasonably well. But it was based on an NPR story on Samoan football players which came out recently, and on a blogged commentary on that story. By the time I might do a similar activity again, who knows if these resources will even still be available, or if they are, if they will still be at all relevant to my students? Why save that plan in the first place?

So if the factory model of schooling finally crumbles to dust and rubble (as well it should), I wonder what will become of teacher-created lesson plans and activities. Will we take them online to a huge database of resources upon which students as well as teachers can draw, and if so will that be pay-per-use?

Will we offer online courses that allow us direct access to students around the world, no longer needing other teachers to be our intermediaries? How much instruction will need to be completely individually designed for each student and how much will transfer easily from kid to kid? And for that matter, to what extent is that last statement true even within the factory model of schooling?

I find myself with far more questions than answers.
 
Nancy concluded:

Just another thought on selling "lesson plans." Several people have noted that lesson plans are only as good as the teacher and the currency of the materials used (Bill explained that beautifully, above). WE understand that. But do most people (including most policy-makers) get that? I'm thinking here about Bill Gates' idea that he could tape the most skillful teachers' "lectures" (Bill's word) and use them to teach large groups of kids--or show other teachers how to improve their practice.

There are lots of things wrong with that concept, of course. What's missing is the active, dynamic relationship between teacher, student and material. While we share ideas, strategies, materials, prompts and techniques, we don't really share lesson plans, because we don't really know each other's students. And that's something you can't buy and sell.