Sixteen-year-old Kiri Davis took a black doll and a white doll to a Harlem daycare center and asked the children there, all of whom were black, which doll they liked better. Fifteen of the 21 children picked the white doll.
Davis' experiment, which echoes the work of Kenneth Clarke that led to the famous Supreme Court desegregation ruling of 1954, is part of her film, entitled "A Girl Like Me," which was shown at the Tribeca Film Festival. It was also screened at the Urban Academy, a New York City public school, to fulfill Davis’s requirements for graduation.
Later this month, thousands of other New York City high school students will also try to move toward graduation. They will take state Regents examinations, answering multiple-choice questions and writing brief essays. Unlike most New York students, who have to pass five Regents exams, Davis and students at Urban Academy and other schools in the Performance Assessment Consortium do projects instead of some of the tests in history, art, science, math and literature.
The difference between Davis and the students filling in their bubble sheets points up a central dispute in the debate over how to judge students. Should standardized, quantitative measures be key, or should teachers, parents and others rely more on observations and judgments on the quality of work being done and instruction being offered?
|
Now this debate has extended to how to judge schools and the entire New York City public school system. Since the advent of mayoral control in 2002, the Department of Education has increasingly relied on standardized exams as an indication of a school’s performance and of the state of the nation’s largest school system. And it has sought to find other measures to gauge school strengths and weaknesses.
Starting next school year, the Department of Education will conduct more tests (or “assessments”) of students and offer new ways to report results of the existing annual tests. The city’s new plan, dubbed the Accountability Initiative, will add more measures of school performance, incorporating visits to schools and parent surveys. The department says it will use the results to improve instruction and determine the future of schools and their principals.
“We hold our students accountable everyday. Starting now, we are holding our schools and ourselves more accountable as well,” Chancellor Joel Klein said in announcing the move.
But critics say the new program will make students slaves to testing, adept at filling in answer sheets but deficient in reasoning, writing and the arts.
THE NEW WAY OF MEASURING SCHOOLS
New York City's new Accountability Initiative comes during a national boom in school testing and other measurements of government services, from police to child welfare. The $25 million program, set to go into effect in September, includes a number of new ways to assess schools and use those measurements. “What we’re trying to do,” said James Liebman, the education department’s chief accountability officer, “is create an integrated, comprehensive accountability system for measuring the contribution that schools make to the learning of children.”
Comparing Scores
Now, when educators look at test scores, they consider how the scores of fourth graders in a given school this year compare with fourth graders in that same school last year. Such an analysis does not consider differences among fourth grade classes – any teacher will tell you that, by the luck of the draw, some classes are smarter than other.
To address that, the new program calls for tracking how the same student’s scores change from one year to another. Even a low score could be good news – if it shows improvement. Explaining the system, Klein told reporters, “No longer will you get a snapshot....You’ll get a moving picture.”
Robert Tobias, a former director of assessment for the city schools who is now a professor at the NYU education school, said, “Principals have been asking for this for years.” But he said, like all test data, the “value added” results, as they are called, have to be evaluated carefully. For example, he said one year’s test might be far easier than the test given the same student the previous year.
High school students do not take annual achievement rates. To evaluate them, the department will look at other factors such as Regents tests and graduation rates, and is considering administering the PSAT, now a voluntary test for college-bound students, to all high school kids.
More Testing
In addition to yearly tests, starting next year, elementary and middle school students will also be given assessments every marking period.
Liebman said these tests will not determine promotion or be used to evaluate a school. Instead teachers, principals and parents will look at the results to see whether a student has mastered a certain skill – comprehension, say, or recognizing words. The teacher can use that information to zero in on the areas where students need help. “The best way to get improvement is to focus on each child,” Leibman said. “So what you want to do is figure out how each child is doing and where the strengths and weaknesses of each lie and then you want to come up with a plan for how to get that child to where he or she needs to be.”
Liebman said such tests boost achievement and cites statistics on so-called Beat the Odds schools -- schools that perform far better than those with similar student bodies. A study (in pdf format) of predominantly Latino schools in Arizona singles out frequent assessments as one of the factors that helped some schools do better than others.
But Tobias expresses skepticism as to whether such tests provide useful information. ”Everything we need to know about student levels of achievement in English language arts, you can get from the annual assessments,” he said.
School visits
Department officials plan to visit schools to evaluate how they address student needs. To accomplish this, the district has hired Cambridge Education, a British consulting firm, at a cost of $500,000, mostly from private donors. Cambridge is develop evaluations called Quality Reviews that will consider such issues as how well the school tailors its curriculum and teaching methods to the needs of every child in the schools.
Reviews
Parents, teachers and students will be asked to rate their schools on Satisfaction Surveys. The surveys will also pose such questions as, Do you feel safe at this school? Do you feel your child is safe at this school? How engaged are you in the school? And so on.
Report cards
Following all these evaluations, schools will receive a letter grade (A, B, C, etc.). The school's report card will incorporate statistics on safety, and attendance, information from the surveys and the analysis of the scores on yearly tests.
Schools that do poorly could be restructured or closed. And the principals in charge of those schools could find themselves out of a job, although that would require changes in the principals' contract. (The contract expired almost three years ago, and talks are deadlocked.) In return for these sticks, Klein says, principals would gain increased autonomy and schools that do well could get more money.
The school will also be judged in their performance on the Quality Reviews, given a score of “well developed,” “proficient,” or underdeveloped.”
THE TESTING EXPLOSION
The Accountability Initiative is only the latest in a series of other programs at the federal, state and city level that have increased the emphasis on testing. Some of this has been spurred by the U.S. No Child Left Behind Law. It requires that all students meet proficiency levels in reading, language arts, math and science by 2014. And schools must make progress toward that goal every year. That progress is determined by standardized tests.
The federal effort is administered by the states. And even before No Child Left Behind, New York set standards for what students should master in every grade, tested to determine whether students met those standards and published lists of schools that were not succeeding.
At the beginning of this academic year, the state found 331 city schools to be “in need of improvement” and another 93 “requiring academic progress,” a similar designation. It placed 40 New York City schools on its list of most troubled schools: Schools Under Registration Review.
Students in failing schools are eligible for outside help and can even transfer out. At the same time, the schools receive additional aid in the hopes that they can improve. But if that does not work, the school might be restructured or even closed.
The emphasis on testing underlying all this has dismayed some experts. “The substance of education -- history, science, literature, art, music -- is being sacrificed to narrowly focused demands to produce high test scores in reading and math,” educators Diane Ravitch and Deborah Meier, who disagree on many education issues, wrote in Education Week.
Pther critics questions whether the tests are the most reliable indicator of how a student is doing. â€The best assessor of a student's progress is the classroom teacher. Anything else is just a waste of taxpayer's money,” a reader wrote to a dissident education blog.
But Klein dismisses such arguments. “Our teachers should teach students the ideas and skills they must master in order to pass their tests,” he said in a recent speech.
Reflecting that attitude, Klein has upped the ante on the tests. Students need to score above a certain level to be promoted. Their performance on fourth grade tests helps determine what middle school they will attend, and the seventh grade test is key to high school admissions. And a new test will figure out admission to elementary school gifted and talented programs.
The test scores carry high stakes for individual schools and the Department of Education as well. Parents use the scores to help them select schools for their children.
The scores play a role in the department’s evaluations of principals. In extreme cases, bad test scores can mean death to a school. Low Regents test scores helped convince Klein to shut down some of the city’s large high schools, such as William H. Taft and Martin Luther King.
And voters use test scores to judge politicians. When Bloomberg took control of the city schools, he told voters to hold him accountable for the results. And so last spring’s increase in test scores for city fourth graders probably played a role in Bloomberg’s landslide victory.
Some wondered about the validity of the rise. “It’s anyone’s guess why the fourth-grade scores rose so sharply this year at the same time that eighth-grade reading and social studies scores went from bad to worse,” wrote the Manhattan Institute’s Sol Stern. “No reputable researcher would rely on a one-year bump in some test scores to judge the efficacy of a new program.”
But voters are not necessarily researchers. ''The spike in fourth-grade test scores immunized the mayor against criticism from his opponents and has taken the education issue off the table,'' Diane Ravitch told the New York Times during the campaign.
ALTERNATIVES TO TESTING
There is a pervasive sense among those concerned about testing that it has taken the joy out of learning. “What do kids need to go out into the world, participate in a democratic society?” Ann Cook of the Performance Standards Consortium asked a meeting of parents concerned about testing. “What do they need to have some excitement, interests and curiosity so when they get to be 30, they have something to be passionate about and sustain them as human beings?”
Certainly the type of assessments done at the Urban Academy, where Cook is co-director, generate more enthusiasm than a Regents test.
The work that students do to indicate proficiency springs from something that came up during class and sparked the students' interest. In his science class, Bobby Haas became interested in the mechanics of hearing. So, to prove proficiency in science, he devised an experiment to find out whether people can determine what direction a sound is coming from. He tested 40 subjects but then used a statistical method known as a chi square to evaluate his results.
And Xiomara Pedraza’s work in history sprung from her strong feelings about Latin America. “I thought countries were being exploited,” she says.
But the people assessing her work, including a Queen College professor, press her to examine her conclusions. What has the neo-liberal model meant for Bolivia, asks one. Where does land reform fit in to President Ivo Morales’ plan for his country? Will Morales’ policies drive corporations away, leaving Bolivia even poorer than it is now?
Critics say, as appealing as such projects might be, they can be subjective and hide shortcomings in basic knowledge and skills. The program of tests, says State Education Commissioner Richard Mills, who has spearheaded New York’s standards and testing program, came out of a real need. “The people of this state demanded we raise standards, create a reliable way to measure and help more students to succeed,” he has written. The state responded and now, he says, “students who used to be neglected got help. Student achievement improved.”
AND AFTER THE TESTS...
But beyond the testing, lies the issue of what, if anything, parents and students can do with all the data. The city’s experience with No Child Left Behind is sobering.
Many city students attend failing schools and so, under the rules, can seek transfers. But in many neighborhoods there are not enough good schools to accommodate all the students eligible for transfers. Parents seeking new schools for their childen learned they could send them to another failing school nearby -- or a better school an hour or so away. Similarly, parents reviewing possible middle or high schools for their children often find the schools reports and scores tell them one thing: Decent schools are scarce.
Liebman says the information is valuable nonetheless. “What we are dong here is a very potentially risky thing,” he said. “We are going to make transparent to the public how well the schools where their children go are doing and we are doing that with full recognition and indeed a hope that where the results are not satisfactory to parents, in particular, they will use either their voices or their choices to change what’s taking place.”
But some parents wonder. Learning about plans to provide more data, Lisa Donlan of the District 1 Education Council told the Times, “The reaction of parents right away was, â€Do you understand that we go to zoned schools...and this is not a competitive market?’”
