Standardized Testing

Two Posts Not To Miss by My TLN Colleagues

I've been on a bit of a writing hiatus this month while I charge through the first month of school.  It's been a great year so far, and I'll be writing more about why in subsequent posts.  I first want to shout out two amazing posts by my TLN colleagues.

ONE. If you haven't checked out Jose Vilson's post, How Jay-Z Can Help Us Remix Education, run to Edweek to check it out.  As someone who has followed Jay-Z's music and career since his debut almub, Reasonable Doubt, I too have noticed that, wow, he's the only guy from that era still really in the hiphop game as a current artist.  What has he done to remain current, influential, and a leader in his field? Jose analyzes that, and points out that he has become more than "just" a rapper.  He's become successful and influential in several other arenas, and at the same time, he hasn't stopped being a rap artist.  Jose makes an apt comparison to the concept of the teacherpreneur that the co-authors of Teaching 2030 and I put forth in the book and other places. 

TWO. Education Nation came and went this past Sunday and I was busy grading, planning, and catching my breath.  I did not even tune in at all, partly because of how busy I was, and partly because of what a disappointment it had been for me last year.  I was there in person at Rockefeller Center last year and just left feeling used (blogged about it here and here).  This year, it sounds like NBC put on a more balanced program, with space for more thoughtful contributions by teachers and less dramatics.  But that's not the end of the story.  If you have not read Anthony Cody's post at Living in Dialogue analyzing the circular logic of Melinda Gates' statements about her organization's research, you absolutely must. 

Cody points out that, while Melinda Gates asserts that we need multiple measures of effective teaching, she goes on to explain that her research is looking at only those measures (forms of peer observation and student feedback, to name a few) that correlate to higher test scores at the end of the year.  In other words, what do teachers whose students get high test scores do, and how other than testing, can we measure this same outcome?  The problem is that with this model of "multiple measures" all roads are leading toward a single measure--high scores on a narrow and imperfect test, which may have little to do with success in the world students must navigate as adults. As Cody points out, the notion that standardized tests measure the skills that will matter for students in their adult lives remains unproven. Again, you must read his piece on this and decide for yourself whether we are moving in a good direction.

 

[image credits: Cody at blogs.edweek.org  Vilson at november-group.com]

Tracking Skills in Isolation: Missing the Forest For the Trees

Today one of my tasks was to create an assessment calendar for the year.  This is a plan for mandatory benchmark and interim assessments to track my students' abilities to perform on distinct sections of the NY State ELA exam throughout the year.  There are three major sections of the test, which we track separately through interims and all together in benchmarks: (1) reading & multiple choice, (2) listening and written response, and (3) reading and written response.  

I don't really have a problem with giving any one of these assessments.  After all, my school has a mandate to raise students' profiency levels, as measured by this test, and we need to do all we can to make sure our students are able to be successful on the it.  The standards being assessed are all important--supporting an answer to a question with evidence, identifying literary devices and so forth.

My problem comes when I look at this schedule of assessments as a whole: its effect on my teaching and what it suggests to students about what matters in their learning.  Between a September diagnostic benchmark, quarterly interims on each section, and a winter and spring benchmark, at least a day of class almost every month must be devoted to test practice, and hours of time to creating the tests, grading them, and analyzing the data.  

Since my time is not unlimited, this means that I do less grading and assessment of other types of tasks and skills.  In fact, these standardized-style assessments are the only ones I'm responsible for collecting data on regularly.  Of course, I can maintain my usual classroom assessment practices, just with a bit less time and encouragement to focus on them. 

So... what's wrong with that?  What's to say that standardized assessments aren't sufficient or even superior to teacher-created authentic assessments, being somewhat more objective and performed without any help from the teacher or other students?  Aren't we getting a more accurate reading of what students have learned in a standardized assessment?  That depends what we are interested in measuring. 

Standardized test questions assess students' skills out of context. This is what makes them more objective and simpler to grade.  However, in life, the ability to apply skills to a specific context is extremely valuable--more so today, it turns out, than one's ability to perform a task in isolation. Daniel Pink makes this point in his landmark book, A Whole New Mind, which warns that machines and cheaper labor forces overseas are taking over tasks that can be done out of context, through standardized methods. The jobs that remain for Americans demand that we apply our skills through the filters of sound principles, careful judgements and decision-making, empathy, cross-cultural competence, creativity and ever-increasing self-awareness.  

It's a complicated world out there, and changing every day.  "Mastering" skills in isolation is sort of like learning to sail a boat without going out on the water.  I don't have a problem with any one day on my official assessment calendar or any one skill assessed. I usually find it interesting to see how kids do and track their progress. But in privileging standardized-type assessments over authentic ones--which is happening all over the country, wherever the stakes are high for the scores--I suspect we are missing the forest for the trees. And, unfortunately, it's the kids who will lose. 

 

[image credit: nasa.gov]

Superintendent John Kuhn Turns Failure On Its Head

Some may be put off by Superintendant John Kuhn of Texas calling out politicians directly, and flipping the notion of "failure" on its head. But he is right, and his conviction is inspiring.  (See the VIDEO of his speech BELOW.) His points reveal in a timely way an inconvenient truth in education and politics right now. NCLB, Race to the Top, and other policies that use high stakes tests to assign value to students, teachers and administrators do one thing really well: they create an even stronger disincentive for teaching in high needs schools than do the difficult working conditions that have always existed in underresourced schools--the imminent threat of being labelled unacceptable or ineffective by one narrow standradized test given on one day in a year, the results of which correspond more closely nation-wide to socio-economic status than any other factor. They create the same disincentive to learn for such students.  

These are tests that ELL's in NY State, who have been in the country for only one year, must take and pass, or be labelled failures, along with their teachers and schools. If your job was on the line, would you choose to work with ELL's? (See the end of this recent post for my own story on this.) If you were an ELL, how would you feel about studying for and taking that test?  Would you want to work with students who come to middle school unable to read--if you knew that even if they improved by 3 "grade levels" (which are arbitrary designations as it is), their progress would likely not even show up on a seventh grade test, because they moved from a "kindergarten" to a "2nd grade" reading level, and this progress would be labelled unacceptable?  If you were that child, would you want to come to school to prepare for that test?  

The heroes, Kuhn says, are the teachers who continually choose to teach these students, despite the threats and labels (and do not take the low road of cheating).  The failures, he argues, are politicians who allow the poverty to continue.

Kuhn is right when he says that we are replacing real educational opportunity with the idea of "accountability."  It's not that teachers do not want to be held accountable for the job of teaching.  It's that this particular system of top-down acountability serves to systematically label the poorest children and those who teach them failures rather than building them up in response to their actual needs.  These policies also take the attention away from the politicians whose job it is to address poverty and equality, placing the onus entirely on teachers, who make an easy target.  Furthermore, through increasing teacher turnover in high poverty communities by favoring untrained itinerant teachers over experienced career teachers, these policies contribute to the instability in poor communities.  I have seen this with my own eyes so many times here in NYC. 

Watch Kuhn's speech at the Save Our School March and see what you think.

 

 

[image credit: under30ceo.com]

Listen: Anthony Cody (of SOS March) On the Effects of Current Reforms on Students & Teachers on NPR--and My Response

Check out this interview by California NPR news station KalNews, Anthony Cody, one the founding organizers of the SOS March taking place in Washington DC tomorrow.  Here he breaks down the negative consequences of the education reforms began by No Child Left Behind and continued, actually "intensifed" he argues, by the current U.S. Department of Education.  It is a very worthwhile 10 minute listen.

I appreciate his clarity and detail in describing how exactly, over around 10 years' time, conditions in schools have changed, the content and ways children learn have changed, and the conditions of the teaching profession and teacher turnover rates have worsened.  He describes the narrowing of the curriculum, which has become a buzzword and is starting to lose its meaning.  However, as a high school science teacher he makes it concrete: kids came to him with less and less science knowledge because schools had prioritized ELA and Math. 

At the end of the interview, he describes an initiaitive he was a part of years ago and a high needs Oakland public school, where teachers were given time to collaborate, conduct lesson studies together, read books together, and generally learn and support one another as professional teachers.  He says during these years, his school retained 100% of its teachers--something that is pretty rare in high need urban schools, and something that is even more important there, where there is so much instability in students' lives and their communities due to poverty and its effects. When the punitive measures of NCLB came, this school was labelled failing for reasons associated with test scores.  Schools were encouraged to replace teachers, and the thriving teaching community and its progess came to a halt.  

I can offer an almost identical story.  In my first school in East Harlem, a high need middle school serving a large population if ELL's and students with special needs, we were developing a teaching community similar to the one Cody describes. Through a partnership with Bank Street College, teachers in our bilingual academy were given time to work on intredisciplinary curriculum, read and discussed relevant child development research together, and began to support one another in our common work with students.  (This was in contrast with the usual top-down "professional development" mandates.)

We saw a huge improvement in teacher retention during those years, and in our floor, which was a subset of the larger school, we actually saw a significant spike in student test scores.  However, the entire school was still labelled failing.  Our principal needed to take measures to improve test scores across the board, due to the pressure from punitive NCLB policies.  

One of the measures was to close the bilingual academy.  ELL's were too "costly"--not financially, but in terms of how their test scores usually looked and the consequences these posed for the school as a whole. The laws had recently changed and ELL's were expected to pass the state ELA and Math tests after just one year in the country. Because of this change, the school stopped offering its transitional bilingual program and with this change replaced many staff members.  Teacher turnover that year was huge, and I was among the leavers.  It was sad.  I had loved working there with those students and colleagues, and the closing of the academy remains a huge loss. 

 

[image credits: marketplace.publicradio.org, schoolsmatter.info]

Making Cheating Irrelevant

The truth is: We could provide modern teaching, learning, and comprehensive assessment opportunities that make cheating not just difficult, but irrelevant.

Guest View: Teachers’ Report Points in Right Direction

Here's the response to my last blog post from my collegue, Susan "Ernie" Rambo. She has been a guest blogger here before.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hi Renee,

I've read the report on teacher evaluation several times now and find myself comparing it to what my school district might have in store for us in the near future. Both the governor of our state and our school district's new superintendent are recommending that we adopt similar measures to evaluate our teachers. What I like most about the report, is that it includes teachers in the decision-making process of creating and implementing the four distinct areas of teacher evaluations. Bill Ferriter stated it quite well in his blog when he mentioned that the work of the Denver New Millennium Initiative (DNMI) extended the work of policy-makers so that teacher evaluation reform can become a reality instead of a "half-baked plan!"

Rather than dismiss any weaknesses within the Colorado policy, the DNMI made recommended several reality-based ways to strengthen the policy.

    The group focused on four areas within their state's new policy:

  1. Developing meaningful measures of student growth (including in non-tested areas) to comprise 50 percent of a teacher's evaluation.
  2. Defining qualifications and training for evaluators.
  3. Determining how to account for school conditions and student factors in a teacher's evaluation.
  4. Designing an evaluation system that informs both employment decisions and professional growth and learning.

You mentioned how the recommendations for the qualifications and training of peer evaluators reminded you of the training for scoring National Board of Professional Teaching Standards portfolios. I was also reminded of that process as I read the suggestion that teachers create individual growth goals related to their schools' growth plans, and develop the tools to measure their progress toward meeting those goals. Such work requires teachers to reflect on the needs of their students and why they choose specific strategies to support those needs, much like the work required in attaining National Board Certification.

I am especially interested in the DNMI's recommendations for using teachers in hybrid roles – part time teaching and part time evaluating their peers – to improve the evaluation process of teachers. A national professional development organization, Learning Forward, recommends that every teacher, as well as every student, learns every day. A well-designed peer-evaluation system would generate an atmosphere where all educators could improve their practice through the proven application of reflection and action. The DNMI suggests using teachers in hybrid roles to first be trained, then master, and then train others to apply an evaluation process. Being evaluated by a peer, experiencing the same challenges as I do in the classroom, would maintain much more credibility than the brief visits of an over-burdened administrator who might or might not have time to visit the classroom during the year.

I commend the DNMI for putting forth the suggestion that teachers' evaluations should include a professional development component. The recommendation of using a professional guild to design personal professional development activities for teachers based on their evaluations would be a welcome improvement to the traditional "one-size-fits-all" and "sit and get" professional development provided at many schools. I would also like to see a component that includes each teacher using student achievement data and peer evaluation results to plan a personal professional development plan that utilizes the offerings of both the professional guild and external resources.

In my work with other teachers at the Center for Teaching Quality in studying Teacher Working Conditions, we recommended that teachers should be provided with High Quality Professional Development that focuses on meeting the unique needs of their students.The DNMI report extends this concept when they describe using teacher-evaluation rubrics that are created by the state but also include locally-based options for criteria that are based on local goals and concerns. Additionally, the component that encourages teachers to conduct action research toward goal attainment, as you mentioned in your letter, is a most powerful tool that allows teachers to substantiate the choices they make in their classrooms that impact student learning. What works in one classroom might work in another, but each teacher needs time and support to reflect upon and evaluate chosen strategies to determine their effectiveness.

Despite the concerns with using test scores to evaluate teachers, the Denver group has published realistic suggestions for how our schools can move toward continuous learning and higher achievement of our students.I look forward to more news from this group and from you, as we experience the changes in store for us.

Regards,

Ernie

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Anybody else think the teachers are pointing us in the right directon on developing effective teacher evaluation systems? What do we need to do to get these ideas into policy?

Read This: Connect the Dots

Add these to the summer required reading list for members of Congress, U.S. Dept. of Education staff, and other policymakers attempting to make critical decisions about education quality and reform in America (care of my friends at Carnegie Foundation for the Advancement of Teaching):

1. Report from the National Center on Education and the Economy showing that the direction of popular U.S. trends in education reform are headed in the wrong direction and may further reduce the performance of American students against that of their peers in competing nations.

2. Nearly a decade of America's test-based accountability systems, from "adequate yearly progress" to high school exit exams, has shown little to no positive effect overall on learning and insufficient safeguards against gaming the system, a blue-ribbon committee of the National Academies of Science concludes in a new report.

Yes, this material will be on the test.

Teaching & Learning Are Messy

On my last post, My Kind of Teaching, on constructivist pedagogy, I received this thoughtful response from a veteran teacher, who chose to remain anonymous. I appreciate it so much, because it offers some validation some of my thinking and intuition regarding the thorniest of issues  when it comes to teaching methodology right now. It seems like in the push to be urgently focused on short-term objectives and obsessive tracking of student learning toward a measurable goal (performance on standardized tests), we miss some of the point--how students actually learn. Often messily.  

Thank you, anonymous teacher, for sharing your thoughts here:

I've been teaching for about 12 years now. When I graduated from TC, my advisor told me one thing that I took with me and that guides me still: If the kids aren't doing something, the kids aren't learning.  He meant that the kids should never just be sitting there listening to me talk, they should instead be working on meaningful tasks used to facilitate learning and later demonstrate what they've learned.  So I always plan like this: What am I doing?What are they doing? The reform movement, unfortunately, has been very bad for progressive teaching because it is all about standardized testing or "accountability."  It is not always easy to control for what a student has learned.  Sometimes students learn something completely different than what I intended.  Sometimes, a student learns what I wanted to teach days after everyone else.  Does that make me a "bad" teacher?  This is a phrase that gets bandied about a lot lately.  Many times I learn from the kids.  It's an interactive process.  Teaching is messy - that's the beauty of it.  The way kids learn is messy.  I also want to point out that alternative certification programs have brought into the teaching field new teachers who are much more traditionalists overall.  I think this is because there is so much emphasis on classroom management.  There's less focus in these programs on philosophical issues of teaching and learning as well as developing rich curriculum because the teachers have to be up and running in just short weeks.  So at TC we had lots of discussions about what the actual role of the teacher was and what the student brought with them into the classroom.  We were encouraged to let the curriculum take care of the classroom management issues. 

Also, make sure to read Alfie Kohn's new article, "Poor Teaching for Poor Children," in which he argues that "it’s possible for the accountability movement to simultaneously narrow the test-score gap and widen the learning gap." Scary, but possibly true. 

[image credit: sophiemazingarbe.com]

 

Student Thoughts on Accountability for Teachers

"Ms. Sacks, where are you going tomorrow?" a student in my small group tutorial asked.  I had told them earlier that day I would be out at a conference tomorrow.  (See this post for more information.)

"I'm going to be at a conference for education writers about teacher effectiveness," I said, realizing how garbled that probably sounded to my students.  I went on to explain that there were many big arguments going on about how school systems can tell if teachers are doing a good job.  

I now had all ten of my students' total attention. They looked pensive or confused, trying to wrap their minds around the idea.  It was still pretty abstract for them. 

"Do you know one way that many people in education are deciding if a teacher is good or not?"

"How?!" they asked, excited to know.

"By how their students do on state tests," I said.  

Immediately, the entire group of students let out groans, actually surprising me by their unified negative reaction.  

"Oh, man, really?!" one student asked, wrinkling up his face in disgust.

"Why would they do that?" another student asked, incredulously.

I was, at this point, a little puzzled by the strong reaction.  "Well, I said, can you explain why you don't agree with this?"  I started taking notes on my laptop.  

One boy answered, "Because if a kid doesn't have a good score on the test, you can't really blame the teacher."

"Yeah," another boy added, "Like if a kid doesn't care, it's not the teacher's fault."  They were turning on all their seventh-grade-passion-for-fairness, and evidently none of their seventh grade tendency to place responsibility on others.

"If you get a low score on the test, and then you have to be in a lower group, you feel left out, so you really don't want to do the work," another boy said.  He was commenting on the way that tests are used to rank students and the effects this has on student motivation. 

"Teachers can't do much about kids' effort.  The teacher could be teaching you a lot of stuff, and you are learning, but on the test you just decide not to do 100%.  I know people who just guess on multiple choice tests. That wouldn’t be the teacher's fault."

"But doesn't a good teacher motivate students to do well?" I asked.

"Yes, but that's only a little part of it.  Kids aren't always putting full attention on the teacher. When a teacher does good, some kids still just don't want to pay attention. There are other things, distractions..."  

They were really drilling this point, and puzzling me in the process.  I believe a teacher is a huge influence on student motivation.  I'd even wielded my influence recently with students as they took a practice standardized reading test, encouraging them to beat the test and show what they know. The results were very good.  But my students were passionate about their argument.  What were they trying to tell me?

One of the messages I took from their comments is that in students' minds, testing is about a lot more than the teacher.  (And students are correct in thinking this way).  Students have been acquainted with standardized tests since early elementary school. They already have a relationship with these blunt instruments of assessment.  Their motivation is colored by past experiences, as well as the boring, often inane, nature of the tests in relation to their vibrant development and struggles as individuals and thinkers.

Students are also keenly aware that that the standardized test is not the same animal as a teacher-made classroom test--one that directly connects to their current learning experience--and they don't treat it as such. That is why, I believe, they were fairly shocked and disappointed that their teachers would be judged based on their performance on these tests, that are alien in a way their teachers are not.

[This reminds me of a conversation with another class about testing itself.  One student commented that the test often seems to have little to do with the class itself or things they care about.  Another student suggested, "Teachers should be able to input their curriculum to, like, a central system.  Then the system could create a test based on the curriculum the individual teacher says the class is learning." With a little tweaking, this idea could be an interesting direction!]

"So," I asked, "if not by test scores, how should people judge whether a teacher is doing a good job or not?"  Hands shot up.

"They could determine who's a good teacher if they come up to the school and maybe listen to your lesson plan or--" one student started.

"They should send people to the school and view them teaching to see if they're doing the job they got hired to do, if they're doing the job like they should be," another added.

"But they should, like, surprise-come, because the teacher might just be good on one day if they know they're coming," the first student qualified.    

"Yeah." 

Finally, I asked, "And what makes a good teacher?  What should they look for?" 

"Elements of fun," one student, who'd been quiet thus far, contributed.

"Yeah.  Humorous, and a little bit strict."  Nods of agreement.

"They understand your problems.  They don't just assume you don't wanna do your best if you don't do well."

Hm... here was an admission that a good teacher believes he or she can help students do well, doesn't judge students superficially or give up on them, does not see student achievement or effort as out of his or her locus of control.  BUT--this admission came from this student in the context of judging teachers through classroom visits, that is, by the daily practice of learning that happens inside the classroom. A teacher's assessment of this practice contrasts dramatically with an abstract test created by outsiders, which students understand is for the purposes of ranking them, and now their teachers.

I'm not arguing that there is no value to standardized tests and that students should never encounter them. I am pointing out that they are often used in a way that is not productive. 

"Good teachers--they are nice and strict and funny."

"They have patience."

In my school, though students don't explicitly know this, they do have a voice in assessing the quality of their teachers through annual anonymous student surveys.  Test scores and other student achievement data gathered by the teacher also play a critical role, as do classroom observations and debriefs, parent surveys and peer surveys. Teachers play a role in our own evaluation, by setting goals for our professional development, against which we are assessed at key points throughout the year.  

Grading itself is very tricky, and any system has its flaws.  (Teachers know this.) But if we need to have a system for grading teachers, then this seems like a pretty fair way.  The important thing is that every one of these pieces of feedback from all of the key stakeholders (360 degrees, it's called) is valuable and should have weight.  When we rely on one source of information exclusively, the picture we get is just that limited, and the side effects of blowing one voice out of proportion are damaging to everyone involved.  Talk to students for five minutes about this, and quickly get schooled. 

 

[image credit: secure5.softcomca.com]

 

 

 

 

 

 

 

 

 

 

 

Story Idea For Education Journalists

Yesterday, I had the pleasure of being at a conference on hosted by the Education Writer's Association and the Carnegie Foundation here in NYC.  It brought together an interesting group of education reporters, teacher bloggers, and education "experts" (=policy people & researchers) to discuss the topic of teacher effectiveness.  The event was quite unique and engaging, though I would have liked to see teachers featured on panels as "education experts" as well.  

At the end of the day we broke into small groups and talked more directly with the reporters about ideas for stories.  Here one I'd like to suggest:

Does innovative teaching lead to better test scores?  

On the one hand, everyone from principals to President Obama is saying that it's bad to "teach to the test."  On the other hand, when it comes time to look at data and evaluate students, teachers and schools, test scores are the measure.  

Many people, including Judy Zimny of ASCD who was a great panel presenter yesterday, have asserted that innovative teaching that excites students around content leads naturally to higher test scores.

In other words, there is no need to teach to the test.  Is that true?  It is a question I've been trying to answer myself.  I see some evidence that it is true, and some evidence it might not be universally true.  What are the factors at play here?

Let's hear from innovative teachers who see big gains in their students' test scores but do not seem to "teach to the test".  What populations do they work with?  What type of schools do they work in?  What do they focus their curriculum on, and to what do they attribute the success of their students on the test?  Are there things these teachers think are important to teach, but leave out, because they aren't tested skills or content?  Where do "soft" skills like collaboration, self-reflection, creativity, and empathy figure into their classrooms and curriculum?  

Let's also hear from teachers who refuse to teach to the test and who may not see huge gains on test scores, but who have been deemed excellent, innovative teachers by other measures, such as National Board Certification, feedback by their colleagues, school leaders, students and parents.  What is their rationale for the choices they make regarding curriculum and teaching style?  What growth do they see in their students, and why don't they think it's being measured accurately or at all or by the standardized test?  I have read blog posts from Bill Ferriter, an expert teacher for sure, who incorporates digital media into his classroom in extraordinary ways and who teaches his students to be global citizens--he has at times mentioned that his students' test scores are lower than those of the other teachers on his hall, because he focuses so much energy on skills he believes are of extreme importance, but do not get tested.  

Finally, I'd like to hear from teachers who do teach to the test, and who would not describe themselves as particularly innovative--both the ones who see their students make gains on the test and those who try and do everything they're told to do, but don't.  How do they conceptualize what they are doing?  How do conditions at their schools factor into their decisions and outcomes?  What does success on the test mean to their students and them? 

Also, what do parents want for their kids?  How do they feel about their child's educational experience in relation to their perception about whether their child is being taught to the test or not?  

 

[image credit: news.xinhuanet.com]

 

123next ›last »
Syndicate content