Taking the Public Pulse |
The Department of Education is just getting started with surveys. We see this effort as handing the school system back to parents so that they have information to voice their concerns about their schools or move from one school to another.
We are putting into place five new measures. We are going to take periodic assessments of where the kids are during the year. We are going to do an on-site quality review of every school. And we are going to look at school environment. The key to the school environment measure is going to be surveys of parents, students and teachers starting about 10 months from now. From middle school up, students will be included. At all schools, parents and teachers will be included.
We are going to ask them about a number of things. First, do parents feel that their children are safe at school and on the way to school? Do children feel that they are safe?
Secondly what about engagement? This affects everyone -- teacher, student and parent. Do they understand what the mission of that school is and what their role is in that mission? Students and parents are going to be asked about this in a very focused way: Imagine your child’s – or your -- third period class, and answer these questions about that class. Do know what’s taking place there? If you feel your child has problems in that class, do you know who you should go to?
Very shortly, we are going to put out a request for proposals and ask people both to help us design those surveys and conduct then. It is very important that the designers of the surveys work not only with us but with parents, teachers and students. We want to be learning what people think of the system even before the surveys are built so we can build them the right way.
James Liebman is chief accountability officer for the New York City Department of Education
Â
This spring, the city's 1,400 principals got a choice: Do you want to stay with your current boss? Or do you think you could run your school better on your own, without supervision?
In a far-reaching experiment that represents the second radical reorganization of the school system since Mayor Michael Bloomberg took office in 2002, nearly one-fifth of the city's principals have been invited to have their school leave their current districts and become what are called "empowerment schools." Starting this fall, if all goes as planned, these 331 schools will be able to decide everything from the starting time of the school day to the vendors from whom they will buy books -- as long as they follow state laws and existing union contracts.
More Money, More Accountability
What this means for parents, teachers and students is still unclear. In theory, the principal will have more discretionary money -- roughly $150,000 per year -- to spend on projects that the school community deems important, such as a new music teacher or extra library books. Principals will no longer be required to attend meetings at their district or regional office, potentially freeing up time to work directly with students and teachers. In exchange for the new freedom, schools must prove that their students are making progress. Empowerment schools will be required to conduct "interim assessments" of all students every six to eight weeks. They will undergo annual "quality reviews" and receive grades of A to F from inspectors who spend two or three days evaluating them. (For more on the new assessment program, see “Grading Students, Grading Schools.”)
"We believe schools are most effective when decisions are made by people closest to the children," principals and teachers, rather than regional administrators, said Eric Nadelstern, a seasoned and respected school leader who has been named chief executive officer of the empowerment schools. "Principals will no longer be able to say 'my hands are tied,’'' he said.
The Department of Education hopes the number of empowerment schools will grow each year, Nadelstern said. And, according to its plan, by the 2007-2008 school – whether or not they become empowerment schools -- will be required to conduct interim assessments every six to eight weeks and undergo quality reviews.
The Response
The initiative has created lots of controversy. The leadership of the principals' union, which has been without a contract for three years, is livid. The union, the Council of Supervisors and Administrators, plans to sue the city for violating state law that governs school districts, its president Jill Levy said in a telephone interview. In a memo sent to principals, she said the initiative is "an attempt to break the back of this union."
Recently, three well-respected senior administrators -- Carmen Farina, deputy chancellor for instruction, Lori Mei, chief accountability officer, and Michelle Fratti, superintendent of Region 7 in Brooklyn and Staten Island -- have announced their retirements. While they have not publicly criticized the initiative, some colleagues speculate that the three did not want to stay on to carry it out.
Teachers are divided, particularly on the issue of interim assessments. While one Manhattan kindergarten teacher told Inside Schools that interim assessments, or frequent testing, will help her gauge how her students are doing, a fifth grade teacher said "It will drive me bonkers." The teachers' union seems cautiously supportive. "For many schools, the greater flexibility may be a good deal since it will allow educators to get away from some of the micromanaging that has been the bane of teachers for the last few years," according to New York Teacher, the newspaper of the United Federation of Teachers.
Some parents are opposed, while some are taking a wait-and-see attitude. "People believe it's just going to result in even more testing," Sam Greenberg, a Queens parent, told The New York Post. But Tim Johnson, chair of the Chancellor's Parent Advisory Council, a panel of parent leaders, said the new assessments are "potentially a good thing" that may prove to be as benign as a weekly spelling quiz and be a useful way to learn how a child is doing before the end of the semester or year. "I want to know how my daughter is doing and I don't want to wait for three months," he said.
A 180-Degree Turn
But for better or worse, the initiative does represent a dramatic shift in the way the Department of Education has done business since Bloomberg took office in 2002. Some are calling this latest change a mid-course correction.
After Bloomberg persuaded the state legislature to give him control of the schools in 2002, he dismantled the discredited Board of Education and consolidated 32 districts into 10 regions. In Bloomberg's highly centralized system, most decisions about curriculum were made at the top. Some principals and teachers complained that every detail of school life -- from the placement of a rocking chair in a kindergarten class to displays on bulletin boards -- was dictated from on high.
But now he empowerment schools seem to represent a move from radical centralization to radical decentralization. "It's a complete 180, a correction of the top-down stuff," said Johnson of the Chancellor's Parent Advisory Council,. "But you won't get them to admit that centralization didn't work."
Nadelstern, who opened several dozen small high schools in the Bronx, said the empowerment schools merely expand on the "autonomy zone," a network of schools that have operated outside the control of the regional offices for several years. He said centralization was necessary in the early days of the Bloomberg administration to bring order to a chaotic and unequal system, but that greater autonomy will liberate principals and teachers to do their best work.
Principals are divided. Some agree with Nadelstern, while others fear the new system will leave them worse off than before.
Why It Might Work
Carol Stock, an experienced principal of the successful PS 199 on Manhattan’s Upper West Side, said she decided 199 should become an empowerment school in part because she believed the "staff development" or training she was able to offer her teachers in-house was superior to that offered by her region. She said the fewer meetings she had to attend outside the school, the better. And Stock said she believes the interim assessments and the quality review (the annual report card in which schools are graded from A to F) will be useful for teachers, parents and administrators.
Empowerment schools will be able to design their own interim assessments she said, as long as they can show students are making progress. The assessments may take the form of computerized bubble tests. But they could also include a one-on-one interview of the child by the teacher, or a test, similar to a pop quiz, devised by teachers. With the help of such assessments, Stock says, teachers will be able to how their students are doing as the year progresses and will be able to adjust their techniques before it's too late,
PS 199 was one of a small group of schools selected this past year for a pilot quality review. Inspectors contracted by the education department spent two and a half days in the school, talking to everyone from cafeteria workers to parents. Their report included descriptions of the school's tone and climate that went far beyond a recitation of test scores, citing, for example, the school's dance program for children in wheelchairs. "I think it is a good faith effort to measure how schools are doing," Stock said. "How it will actually play out, I don't know."
And What the Risks Are
For principals who have chosen to stay within their regions, however, the risks outweigh the possible benefits. For them, their current supervisors, called "local instructional superintendents,” are not oppressive bureaucrats but trusted colleagues; their regional offices are a source of support and good advice.
Sharon Hill, principal of the well-regarded Manhattan New School, who chose not to become an empowerment school, said her local instructional superintendent, Anna Marie Carillo, is “fabulous….She used to be a principal. She understands instruction, and she understands the rules and regulations. I take care of the school, and she takes care of me. She is a shoulder to lean on. I have a great support system at the regional office. Why would I give that up?"
Hill also suggested that the amount of money offered, which varies according to the size of the school, might not be enough to compensate for the services the empowerment schools would have to take on, such as staff development.
Levy, of the principals' union, said the new initiative is part of a "corporate model" of education. She predicted that principals of empowerment schools would spend an excessive amount of time "concentrating on the management and business [aspects] of schools" and would have less time to focus on instruction. She said details are now handled by the regional offices, such as negotiating with vendors, would now be handled by principals.
The 10 regional education department offices still have some authority over empowerment schools. They will continue to determine student placement for special education, gifted and talented programs, and services for English language learners, Nadelstern said. But Nadelstern's office -- not the regional office – will have the power to hire principals.
-- Clara Hemphill, is project director of InsideSchools.org project director and author of “New York City's Best Public Elementary Schools: a Parents' Guide,” “Public Middle Schools: New York City's Best, “ and “New York City's Best Public High Schools.” This topic page has been adapted from an article that originally appeared on the pages of our partner, Insideschools.org, an independent guide to New York City public schools sponsored by Advocates for Children.
Â
Sixteen-year-old Kiri Davis took a black doll and a white doll to a Harlem daycare center and asked the children there, all of whom were black, which doll they liked better. Fifteen of the 21 children picked the white doll.
Davis' experiment, which echoes the work of Kenneth Clarke that led to the famous Supreme Court desegregation ruling of 1954, is part of her film, entitled "A Girl Like Me," which was shown at the Tribeca Film Festival. It was also screened at the Urban Academy, a New York City public school, to fulfill Davis’s requirements for graduation.
Later this month, thousands of other New York City high school students will also try to move toward graduation. They will take state Regents examinations, answering multiple-choice questions and writing brief essays. Unlike most New York students, who have to pass five Regents exams, Davis and students at Urban Academy and other schools in the Performance Assessment Consortium do projects instead of some of the tests in history, art, science, math and literature.
The difference between Davis and the students filling in their bubble sheets points up a central dispute in the debate over how to judge students. Should standardized, quantitative measures be key, or should teachers, parents and others rely more on observations and judgments on the quality of work being done and instruction being offered?
Now this debate has extended to how to judge schools and the entire New York City public school system. Since the advent of mayoral control in 2002, the Department of Education has increasingly relied on standardized exams as an indication of a school’s performance and of the state of the nation’s largest school system. And it has sought to find other measures to gauge school strengths and weaknesses.
Starting next school year, the Department of Education will conduct more tests (or “assessments”) of students and offer new ways to report results of the existing annual tests. The city’s new plan, dubbed the Accountability Initiative, will add more measures of school performance, incorporating visits to schools and parent surveys. The department says it will use the results to improve instruction and determine the future of schools and their principals.
“We hold our students accountable everyday. Starting now, we are holding our schools and ourselves more accountable as well,” Chancellor Joel Klein said in announcing the move.
But critics say the new program will make students slaves to testing, adept at filling in answer sheets but deficient in reasoning, writing and the arts.
New York City's new Accountability Initiative comes during a national boom in school testing and other measurements of government services, from police to child welfare. The $25 million program, set to go into effect in September, includes a number of new ways to assess schools and use those measurements. “What we’re trying to do,” said James Liebman, the education department’s chief accountability officer, “is create an integrated, comprehensive accountability system for measuring the contribution that schools make to the learning of children.”
Comparing Scores
Now, when educators look at test scores, they consider how the scores of fourth graders in a given school this year compare with fourth graders in that same school last year. Such an analysis does not consider differences among fourth grade classes – any teacher will tell you that, by the luck of the draw, some classes are smarter than other.
To address that, the new program calls for tracking how the same student’s scores change from one year to another. Even a low score could be good news – if it shows improvement. Explaining the system, Klein told reporters, “No longer will you get a snapshot....You’ll get a moving picture.”
Robert Tobias, a former director of assessment for the city schools who is now a professor at the NYU education school, said, “Principals have been asking for this for years.” But he said, like all test data, the “value added” results, as they are called, have to be evaluated carefully. For example, he said one year’s test might be far easier than the test given the same student the previous year.
High school students do not take annual achievement rates. To evaluate them, the department will look at other factors such as Regents tests and graduation rates, and is considering administering the PSAT, now a voluntary test for college-bound students, to all high school kids.
More Testing
In addition to yearly tests, starting next year, elementary and middle school students will also be given assessments every marking period.
Liebman said these tests will not determine promotion or be used to evaluate a school. Instead teachers, principals and parents will look at the results to see whether a student has mastered a certain skill – comprehension, say, or recognizing words. The teacher can use that information to zero in on the areas where students need help. “The best way to get improvement is to focus on each child,” Leibman said. “So what you want to do is figure out how each child is doing and where the strengths and weaknesses of each lie and then you want to come up with a plan for how to get that child to where he or she needs to be.”
Liebman said such tests boost achievement and cites statistics on so-called Beat the Odds schools -- schools that perform far better than those with similar student bodies. A study (in pdf format) of predominantly Latino schools in Arizona singles out frequent assessments as one of the factors that helped some schools do better than others.
But Tobias expresses skepticism as to whether such tests provide useful information. ”Everything we need to know about student levels of achievement in English language arts, you can get from the annual assessments,” he said.
School visits
Department officials plan to visit schools to evaluate how they address student needs. To accomplish this, the district has hired Cambridge Education, a British consulting firm, at a cost of $500,000, mostly from private donors. Cambridge is develop evaluations called Quality Reviews that will consider such issues as how well the school tailors its curriculum and teaching methods to the needs of every child in the schools.
Reviews
Parents, teachers and students will be asked to rate their schools on Satisfaction Surveys. The surveys will also pose such questions as, Do you feel safe at this school? Do you feel your child is safe at this school? How engaged are you in the school? And so on.
Report cards
Following all these evaluations, schools will receive a letter grade (A, B, C, etc.). The school's report card will incorporate statistics on safety, and attendance, information from the surveys and the analysis of the scores on yearly tests.
Schools that do poorly could be restructured or closed. And the principals in charge of those schools could find themselves out of a job, although that would require changes in the principals' contract. (The contract expired almost three years ago, and talks are deadlocked.) In return for these sticks, Klein says, principals would gain increased autonomy and schools that do well could get more money.
The school will also be judged in their performance on the Quality Reviews, given a score of “well developed,” “proficient,” or underdeveloped.”
The Accountability Initiative is only the latest in a series of other programs at the federal, state and city level that have increased the emphasis on testing. Some of this has been spurred by the U.S. No Child Left Behind Law. It requires that all students meet proficiency levels in reading, language arts, math and science by 2014. And schools must make progress toward that goal every year. That progress is determined by standardized tests.
The federal effort is administered by the states. And even before No Child Left Behind, New York set standards for what students should master in every grade, tested to determine whether students met those standards and published lists of schools that were not succeeding.
At the beginning of this academic year, the state found 331 city schools to be “in need of improvement” and another 93 “requiring academic progress,” a similar designation. It placed 40 New York City schools on its list of most troubled schools: Schools Under Registration Review.
Students in failing schools are eligible for outside help and can even transfer out. At the same time, the schools receive additional aid in the hopes that they can improve. But if that does not work, the school might be restructured or even closed.
The emphasis on testing underlying all this has dismayed some experts. “The substance of education -- history, science, literature, art, music -- is being sacrificed to narrowly focused demands to produce high test scores in reading and math,” educators Diane Ravitch and Deborah Meier, who disagree on many education issues, wrote in Education Week.
Pther critics questions whether the tests are the most reliable indicator of how a student is doing. â€The best assessor of a student's progress is the classroom teacher. Anything else is just a waste of taxpayer's money,” a reader wrote to a dissident education blog.
But Klein dismisses such arguments. “Our teachers should teach students the ideas and skills they must master in order to pass their tests,” he said in a recent speech.
Reflecting that attitude, Klein has upped the ante on the tests. Students need to score above a certain level to be promoted. Their performance on fourth grade tests helps determine what middle school they will attend, and the seventh grade test is key to high school admissions. And a new test will figure out admission to elementary school gifted and talented programs.
The test scores carry high stakes for individual schools and the Department of Education as well. Parents use the scores to help them select schools for their children.
The scores play a role in the department’s evaluations of principals. In extreme cases, bad test scores can mean death to a school. Low Regents test scores helped convince Klein to shut down some of the city’s large high schools, such as William H. Taft and Martin Luther King.
And voters use test scores to judge politicians. When Bloomberg took control of the city schools, he told voters to hold him accountable for the results. And so last spring’s increase in test scores for city fourth graders probably played a role in Bloomberg’s landslide victory.
Some wondered about the validity of the rise. “It’s anyone’s guess why the fourth-grade scores rose so sharply this year at the same time that eighth-grade reading and social studies scores went from bad to worse,” wrote the Manhattan Institute’s Sol Stern. “No reputable researcher would rely on a one-year bump in some test scores to judge the efficacy of a new program.”
But voters are not necessarily researchers. ''The spike in fourth-grade test scores immunized the mayor against criticism from his opponents and has taken the education issue off the table,'' Diane Ravitch told the New York Times during the campaign.
There is a pervasive sense among those concerned about testing that it has taken the joy out of learning. “What do kids need to go out into the world, participate in a democratic society?” Ann Cook of the Performance Standards Consortium asked a meeting of parents concerned about testing. “What do they need to have some excitement, interests and curiosity so when they get to be 30, they have something to be passionate about and sustain them as human beings?”
Certainly the type of assessments done at the Urban Academy, where Cook is co-director, generate more enthusiasm than a Regents test.
The work that students do to indicate proficiency springs from something that came up during class and sparked the students' interest. In his science class, Bobby Haas became interested in the mechanics of hearing. So, to prove proficiency in science, he devised an experiment to find out whether people can determine what direction a sound is coming from. He tested 40 subjects but then used a statistical method known as a chi square to evaluate his results.
And Xiomara Pedraza’s work in history sprung from her strong feelings about Latin America. “I thought countries were being exploited,” she says.
But the people assessing her work, including a Queen College professor, press her to examine her conclusions. What has the neo-liberal model meant for Bolivia, asks one. Where does land reform fit in to President Ivo Morales’ plan for his country? Will Morales’ policies drive corporations away, leaving Bolivia even poorer than it is now?
Critics say, as appealing as such projects might be, they can be subjective and hide shortcomings in basic knowledge and skills. The program of tests, says State Education Commissioner Richard Mills, who has spearheaded New York’s standards and testing program, came out of a real need. “The people of this state demanded we raise standards, create a reliable way to measure and help more students to succeed,” he has written. The state responded and now, he says, “students who used to be neglected got help. Student achievement improved.”
But beyond the testing, lies the issue of what, if anything, parents and students can do with all the data. The city’s experience with No Child Left Behind is sobering.
Many city students attend failing schools and so, under the rules, can seek transfers. But in many neighborhoods there are not enough good schools to accommodate all the students eligible for transfers. Parents seeking new schools for their childen learned they could send them to another failing school nearby -- or a better school an hour or so away. Similarly, parents reviewing possible middle or high schools for their children often find the schools reports and scores tell them one thing: Decent schools are scarce.
Liebman says the information is valuable nonetheless. “What we are dong here is a very potentially risky thing,” he said. “We are going to make transparent to the public how well the schools where their children go are doing and we are doing that with full recognition and indeed a hope that where the results are not satisfactory to parents, in particular, they will use either their voices or their choices to change what’s taking place.”
But some parents wonder. Learning about plans to provide more data, Lisa Donlan of the District 1 Education Council told the Times, “The reaction of parents right away was, â€Do you understand that we go to zoned schools...and this is not a competitive market?’”