Spring 2000

• "It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories instead of theories to suit facts," Sir Arthur Conan Doyle
  
  
• "To write it, it took three months; to conceive it, three minutes; to collect the data in it - all my life," F. Scott Fitzgerald on This Side of Paradise

Maybe it only seems that way but the staff of local Urban Health Initiative campaigns spend all their lives collecting data, too. After all, using a data-driven process is a core value of the UHI. Good data help focus campaigns on areas of need, and assure that the most effective strategies are employed and that baselines are established in order to properly measure impact over time.

Too often data gathering becomes a necessary but monumental, time consuming effort. "Surely," says the naive one, uninitiated in the world of public health and community development, "this important piece of information exists and is readily available in someone's desk drawer."

Jaded (sorry, experienced) professionals scoff, knowing that it's rarely that easy. Often, multiple jurisdictions are involved in collecting data, and each reports it differently. Or, some agencies are hesitant to release data due to confidentiality or other concerns. Or, there is simply politics afoot.

How can organizations -- particularly those seeking broad systemic changes that cross geographic and political jurisdictions -- gather and use valuable data that is not readily available?

The following "lessons learned" were compiled from interviews with:

• Greg Hodge, executive director of UHI campaign Safe Passages: Oakland Child Health and Safety Initiative;
  
  
• Lisa Specter, deputy director of UHI campaign Youth Matters, an initiative of the Greater Richmond Chamber of Commerce;
  
  
• Janice Ford Griffin, deputy director of Fighting Back: Community Initiatives to Reduce Demand for Illegal Drugs and Alcohol;
  
  
• Carol Stone, executive director for Regional Drug Initiative (RDI);
  
  
• Audrey Jordan, Senior Research Associate, Survey and Evaluation Research Lab, Virginia Commonwealth University.

Build relationships

Developing relationships with individuals at the upper and lower ends of agencies that compile data is perhaps the single most important way to find out where data exists and to compile it. Line-level staff are generally more facile than senior staff with the data. However, they are sometimes hesitant, or forbidden, to release it without an okay (or push) from supervisors, so relationships throughout the agency are important.
Relationships can be formal, such as a data committee that includes members from data-gathering agencies, or informal, such as working relationships with evaluators from universities knowledgeable about data sources.

Provide resources

Sometimes organizations have to build reporting mechanisms from scratch in order to assure timely and accurate reporting of data. Those expected to gather and report data may need training, especially if doing so is outside their normal areas of responsibility (e.g., teachers reporting data vital to a new tutoring program).

Follow the accountability trail

Data from police departments is relatively easy to get, because they feel accountable to the city. On the other hand, getting data from the state can be very difficult to obtain because it is often gathered through funding from block grants. Accountability is hard to trace. At any rate, these agencies do not feel accountable to cities or community groups and therefore may not be forthcoming with information.
Knowing this helps organizations understand when to use other methods to get the information, such as involving state legislators.

Involve state legislators

State legislators can get information from state agencies when local groups can't. Doing so not only makes them look good to local constituents, it also gets them informed about issues that they might have otherwise missed. Organizations can simply "cc" state legislators on letters they send to state agencies, or call the legislators' offices for help.

Know your politics

Politics can hinder the effort to gather data. This is especially true when, for example, governors and mayors are from different political parties. Another example is a situation like Chicago - its data for a host of topics dominate that of the entire state of Illinois, but the power over the data mostly comes from rural and other non-Chicago legislators.
Not much can be done about situations such as these, but being aware of the politics can help organizations be savvy in requesting data and avoid political land mines that could really hinder their efforts.

Agitate

Community groups can demand that a government entity start collecting data on a certain topic, or release existing data, or look at existing data in a new way. Groups should be clear about the need for the data and the policy implications.

Clarity/security

Being clear and specific about how data will be used will go a long way toward easing concerns that agencies may have about releasing data. Also, agencies may need to be convinced that mechanisms are in place to maintain confidentiality of individuals in the database.

Make it useful to many

Those with data may be more willing to share it if there is something in it for them. For example, individual schools may report statistics in different ways, but if one entity can gather it, streamline the process of reporting it, and make the presentation of the information uniform, the schools may be more willing to participate. In short, the whole is greater than the sum of the parts, and everyone benefits.

Focus/don't cover everything

Don't try to cover or measure everything. Have a thoughtful process to determine which indicators to include, and make sure your final document is easy to read and use and not unwieldy, overly dense and hard to use.

Gather from credible sources, regularly

Data that is suspect or comes from a source that is not perceived to be credible will taint your data as well. Does the source have quality control checks and validate its database through periodic reviews? Make sure the source is credible - and gather it regularly.

Remember process data

In addition to benchmark and outcome data, you may need to collect process data to establish that the intervention strategy was actually carried out (e.g., did students really attend the new tutoring program for the proscribed length of time?). The process data will help explain outcome data and make them more credible.

Thank yous

Thank those who've shared and compiled data, and prominently give credit to all involved.

An example of many of these "lessons learned" successfully put into practice involves Youth Matters' America Reads-Richmond, Chesterfield and Henrico (ARCH) initiative.

To evaluate the success of the effort, Youth Matters and its partners needed report card data on the kids in ARCH from the public schools. Previously, the data had to be gathered by having someone visit each school and retrieve the data, with some schools better at reporting it than others.

"We pursued getting the information electronically," says Lisa Specter. "We had to convince the schools that the data will be kept confidential and go only to the university research lab with client identification numbers rather than names. It turns out that many of the public schools' evaluation departments need help developing their own uniform data reporting capacities. So there is something in it for them, and most of them have been happy to help."

Now Youth Matters has the data needed to evaluate ARCH. They'll be able to analyze it in a variety of ways, such as determining whether students with more tutoring sessions did better than those with less and what the optimal hours of tutoring are.

Not only does data help public policy decision-making, it also can be a valuable aspect of an organization's public relations endeavor and improve the public discourse of a topic.

"We release our data to the media, and they tell us that they use it throughout the year as background for articles," says Carol Stone of the RDI. "We believe that partly as a result of this, coverage of alcohol and drug issues by the media has increased a great deal, and that the coverage is very responsible."

Data gathering and analysis, in spite of the complexities, is invaluable. But just remember Finagle's Third Law: "In any collection of data, the figure most obviously correct, beyond all need of checking, is the mistake. Corollary 1: Nobody whom you ask for help will see it. Corollary 2: The first person who stops by, whose advice you really don't want to hear, will see it immediately."

Editor's note: Fighting Back, funded by the Robert Wood Johnson Foundation, requires communities, through a single unified initiative, to address their substance abuse problems by orchestrating the efforts of their many public, private and volunteer organizations. Visit www.fightingback.org

The Regional Drug Initiative (RDI) is a coalition of concerned policy makers from business, education, government, health care, law enforcement, treatment providers and community groups. Located in Portland, OR, RDI is committed to establishing a drug-free community. Visit
www.regionaldruginitiative.org