As statistics go from tick marks to laptops, police struggle to make sense of trends
The San Francisco Police Department cannot precisely measure the number of domestic violence cases it handled before 2011, because investigators in the Special Victims Unit hand-tallied monthly records, and used changing and inconsistently understood categories of crimes. This story appeared as part of a special report on domestic violence in the Fall 2012 print edition of the San Francisco Public Press.
As statistics go from tick marks to laptops, police struggle to make sense of trends
The San Francisco Police Department cannot precisely measure the number of domestic violence cases it handled before 2011, because investigators in the Special Victims Unit hand-tallied monthly records, and used changing and inconsistently understood categories of crimes.
The published statistics suggest that from 2004 through 2011, the number of domestic violence investigations fell by more than 10 percent. If accurate, that might indicate that the police have prevented crimes by supporting early-intervention programs — or that the department is simply following up on fewer cases.
But because of record-keeping inconsistencies, the Police Department and agencies that for 10 years have overseen the handling of family abuse crimes cannot be sure. Less than a year after installing a new unified crime database, officials said they could only guarantee accuracy of the older case counts to within a few hundred cases per year.
The lack of reliable statistics makes it difficult for agencies across the city that strive to prevent domestic violence to evaluate the success of a decade of reforms, including an additional $10 million for violence prevention, community centers and victim-outreach programs, said Kenneth Theisen, a longtime member of the Justice and Courage Oversight Panel, established in 2002 after a high-profile domestic violence murder exposed gaps in the criminal justice system’s response.
“How can the city determine how many police inspectors and other law enforcement personnel are needed to be assigned,” Theisen said, “if we do not know the true numbers of arrests, the number of cases investigated or the numbers of cases charged?”
The police say that current efforts to overhaul information technology will prevent such errors in future reports. This summer, the department rolled out a new database to enable officers on the street to better respond to incidents by accessing suspects’ case histories on laptop computers mounted in patrol cars. But this new technology cannot easily unscramble poor records from the past, meaning the department will be stuck with a fuzzy understanding of recent citywide and neighborhood trends.
In 2011 the Police Department reported 1,470 domestic violence felony cases. As best the police can make out, that is an 11 percent drop from 2004. Michelle Jean, the police lieutenant who took over domestic violence cases at the Special Victims Unit a year and a half ago, said the published figures were “not completely off,” but acknowledged the real annual counts could be anywhere between 1,400 and 1,700 cases per year.
When she took charge in April 2011, she said, investigators were still tallying monthly counts on photocopied sheets of paper by hand using tick marks, then entering them into a spreadsheet. The categories changed from year to year, making comparisons difficult. For instance, one column labeled “filed” was changed to “contact” in 2011. And since these terms were not precisely defined, investigators from month to month assumed different meanings for each.
“It was ridiculous that we were hand-counting stuff,” Jean said.
But if the recorded decline in cases handled by investigators is accurate, she argued, that would indicate that the city’s decade-long coordinated response to domestic violence was working, because improved public education and training for first responders led to earlier intervention, preventing escalation to ever-more serious crimes.
Another measure of the city’s overall performance, calls to crisis lines and community groups, increased steadily during this time. Jean said this also shows that the city’s outreach and education initiatives have paid off: “If you are informed that you cannot batter your spouse, and you know about it, you are not going to do it.”
This year, 31 organizations fighting domestic violence in San Francisco received a total of $2.7 million from the city, a 37 percent increase from 2000. This pays for housing, clinics, emergency phone lines and education for both victims and known batterers.
Emily Murase, who heads the Department on the Status of Women , has been working with the Justice and Courage Oversight Panel since 2008 to oversee the city’s response to domestic violence.
The police, the 911 call center and other departments are required to make semiannual reports summarizing the number and disposition of cases. A 2002 audit by the panel urged the police to track statistics for cases in several stages, including 911 calls, investigations and cases referred to the District Attorney’s Office. The oversight panel said at the time that these numbers would be used to assess the success of policing and social service programs.
But Theisen said the group has “never really been confident” in the police tallies. It was not until this summer that the oversight committee first discussed the data discrepancies publicly. Police Lt. Art Stellini told the panel on Aug. 1 that official counts of cases dating back to 2004 were inconsistently kept, and that his personal files differed significantly from official records.
Murase often cites nonprofit organizations’ annual reports regarding the number of people accessing services, which has increased by 28 percent since 2007 (the first year for which records are available) to about 29,000 this year. So have domestic violence calls through the city’s 911 center — a rise of 17 percent since 2005, to 7,700 calls this year.
But these numbers give little indication of how many suspects and victims interact with the criminal justice system, or how frequently.
When shown the official police statistics suggesting that the Special Victims Unit was handling fewer cases, Murase said she was “stunned,” because if more victims are seeking help, more ought to be going to the police as well. She said outreach efforts and the training of 435 officers in techniques to evaluate what constitutes domestic violence should have led to more criminal investigations.
Murase said she thinks police record-keeping practices were “conscientious,” adding that the department has long suffered from a “lack of adequate information technology infrastructure.”
A series of reforms in the way the city handled domestic violence cases followed the particularly high-profile murder of a woman named Claire Joyce Tempongko in 2000. After ex-boyfriend Tari Ramirez was charged with the murder, San Francisco embarked on a thorough overhaul of domestic violence response infrastructure and policy to figure out why he was not removed from the home after repeated domestic violence calls to the police.
Tempongko, a mother of two, had desperately tried to get out of the relationship. She contacted the police on six occasions and won a restraining order. But after she was stabbed to death in front of her children in October 2000, Ramirez fled to Mexico, and he was on the lam for six years, but was arrested and brought back to San Francisco for trial. He was convicted of murder in 2008 and sentenced to 16 years to life. But the case is not yet settled: His conviction was overturned in 2011 after a state appeals court found the trial judge had misled jurors about the option of a manslaughter verdict. Ramirez remains behind bars and his case is before the state Supreme Court.
At the request of then-Supervisor Michael Yaki, the Department on the Status of Women launched an investigation into the city’s domestic violence policies. The Justice and Courage Oversight Panel, made up of community advocates, convened to evaluate police and community efforts and identify problems. Louise Renne, then city attorney, oversaw the probe. In 2002 the panel published a blueprint for sweeping changes throughout the city. The primary concern was communication among departments.
The goal was to create a “seamless criminal justice response to domestic violence,” reducing costs and decreasing the number of cases introduced to the criminal justice system.
“The Justice and Courage Panel has always asked for accurate and up-to-date stats,” said Theisen, who is also communications director for Bay Area Legal Aid, which provides legal advice and representation to low-income people. “We wanted reliable, useful data. We have often asked about this at our meetings.”
Yet the panel never established any data-driven benchmarks to evaluate progress. The panel made similar recommendations for the District Attorney, Adult Probation Department and the San Francisco Superior Court.
The police have racked up numerous qualitative successes, including changing departmental protocols: developing a script for 911 operators and providing confidential investigation rooms for domestic violence detectives to interview victims.
Police Chief Greg Suhr and Chief Information Officer Susan Giffin say modernizing the department’s technological capacity is a priority.
A new system, the Crime Database Warehouse, went live in January. The department has trained 1,500 officers to use a new mobile application giving them instant access to criminal histories and complaints, making for speedier and more accurate assessments responding to family violence incidents.
Jean said the new system will allow the Special Victims Unit to produce better statistics. “We will get clearer data,” she said.
But errors from the past may not be recoverable. Giffin, who spearheaded the new database, said the case file “hub” may help officers solve new crimes, but cannot by itself reconstruct statistics on past trends.
The digital filing cabinet allows police to store all incident and investigation records, Giffin said. In the last two years, she scanned 10 years of investigative case files. Under the old system, if a batterer changed his or her name or address or started abusing someone new, the Special Victims Unit had a hard time making connections among cases unless they pulled the paper case files.
Now detectives perform Google-style searches to find information about suspects — an address with a history of complaints by neighbors, a victim’s name or a suspect’s unique tattoo or hair color.
“There are a lot of different ways to connect cases together,” Giffin said.
Domestic violence investigators used to sum monthly summary records by hand on loose sheets of paper, with two investigators cycling through record-keeping duty each month. Investigators entered data inconsistently, and the categories for crimes and how they were handled changed over the years.
The differences among the past official numbers vary by as many as 300 cases per year, Jean estimated in an interview in late August.
The Crime Database Warehouse fits into a broader citywide system called JUSTIS, accessed by other departments, such as the District Attorney’s Office.
Nationally, many of the largest police departments have embarked on extensive data-driven policing initiatives. New York, Washington, D.C. and Los Angeles were among the first to use CompStat, a system that tracks crimes by location, allowing analysts to plot crimes on maps and timelines to more efficiently deploy street patrols. In recent years, as more and more departments move to digital records, third-party services such as CrimeMapping.com have begun to aggregate neighborhood trends for reporting to the public, in collaboration with police departments.
In San Francisco, accurate trend data on domestic violence cases could be useful for determining which neighborhoods are long-term hot spots for domestic abuse and crimes.
The published statistics indicate that across the city, some of the highest pockets of domestic violence are in low-income neighborhoods.
The highest concentration appears to be in the jurisdiction of the Tenderloin Police Station, with 173.1 cases per 10,000 residents. Other police districts with high domestic violence rates are Southern Station located south of Market Street, at 138.9, and Bayview Station, responsible for most of the city’s southeast corner, at 99.4.
The lowest concentration of domestic violence cases, according to the official data, is under Richmond Station.
These neighborhood differences should become more clear under the new computer tracking methods established by Giffin and Jean.
“Is there a better way that we can be doing tracking? Yeah,” Jean said. “I think it’s a given. SFPD has admitted to it. We have old computer systems. Nothing is updated. We are working on it. Is it as fast as we would like it to be? No, but we are trying.”