One of the underlying -- and motivating -- assumptions here at the IAJ is that having good data is a prerequisite to doing good analysis.  And the analytic journalist needs to know first what data is available before any questions can be raised about the quality and defintion of that data.  All this means we are talking about transparency in government and, when possible, the private sector.  (It also applies to transparency in journalism.  More on that here and in days to come.)

One of our favorite and most reliable sources is The Scout Report.  It informs us today:

"Development Gateway: Public Sector Transparency
http://topics.developmentgateway.org/special/transparency

The Scout Report has profiled various offerings from the Development Gateway in the past several years, but one of the  group's latest creations is both
thought-provoking and helpful for policy-makers and persons generally interested in the subject of governance. This particular site casts an eye on the question of transparency in governmental transactions through interviews with leaders from a broad range of sectors, along with allowing space for individual feedback. The "Points of View" section is a good place to start, as it includes commentary from government officials from Bolivia, Guatemala, and Tanzania about the question of public sector transparency.  Other sections on the site address such thorny questions as "What tools help sustain public sector transparency?" and "What practices promote public-private partnerships?" Those visual learners coming to visit the site may appreciate the gallery of charts that offer indicators of levels ofgovernance and transparency for more than 209 countries. [KMG]"

On that site you will find:
"This Special Report on Public Sector Transparency illustrates current international trends in advancing transparency through civil society, government and the media. Through extensive interviews with leaders across a range of sectors as well as survey feedback from Development Gateway users, this Report explores the practical issues of ensuring openness in governments around the world."

In the IAJ's on-going search for new methodologies, reinforcing lessons often come through.  On Saturday, April 30, NPR's Scott Simon taked with John Lentini about analyzing fires.  Lentini's comments emphasize the need for questioning assumptions and pressing hard to clarify definitions.

From NPR's "Weekend Edition Saturday:"
"John Lentini, an arson expert, tells Scott Simon about changes that have brought into question many convictions based on outdated methods of determining arson. One of this convictions resulted in the execution of a Texas man in 2004."  To listen, click here.

Derek Willis -- who might be considered an analytic journalist's analytic journalist -- reminded us today that there already is an abundance of resources in every newsroom in the world.  The problem is, journalists don't understand the concept of synergy, and that one piece of your information and one piece of my information can total the three pieces required to produce an uncommonly good story.

See:
Derek Willis' The Scoop
http://www.thescoop.org/thefix/

http://www.sfexaminer.com
Today's SF Examiner has an innovative full-scale cheerlead for a project with the potential to significantly impact San Francisco. The Examiner swerves from the unusual placement of a front-page editorial, to a news story with several relevant sidebars, to a columnist's unusual  perspective, to an editorial appeal by the Mayor. Click on the above link and then click on the link for today's newspaper.

The mission of the IAJ is to find better ways of doing journalism.  That rarely means turning to others practicing – and writing about how to practice -- the same old classic journalism.  Instead, we in the profession must make a greater effort to seek other methods to know more about a variety of phenomena.  All professions and academic disciplines have journals and trade magazines.  These can go a long way to teaching how those practitioners think and what they think about, along with the methods of those disciplines.

Education is surely among the most data-rich of the professions.  [The flock of newspapers publishing scorecards of school performance and salaries is evidence of how we can use this.]  And a good many educational administrators spend a lot of time trying to figure out what that data means.  Data mining is one of their tools.

The current issue of Technology & Learning carries the first of a two-part series on data mining in education.  The story, of course, is aimed at educators, so the enterprising journalist will have to do some translation and seek opportunities for intellectual-technology transfer.  Be sure to read the sidebars.

Data: Maximize Your Mining
By Todd McIntire
 http://www.techlearning.com/story/showArticle.jhtml?articleID=160400818

Abandoning the News

by Merrill Brown

What's the future of the news business? This report to Carnegie Corporation of New York offers some provocative ideas.

http://www.carnegie.org/reporter/10/news/index.html

Today's New York Times Science Section carries a column by Richard Friendman, M.D. about the challenge of explaining statistical concepts and probability to anyone.  (Even journalists?)  His conclusion: "The truth is that random events can make or break us.  It is more comforting to believe in the power of hard work and merit than to think that probability reigns not only in the casino but in daily life."

So there.

See: "Mix Math and Medicine and Create Confusion."

A different viewpoint about granting reporters too many privileges.

Shield laws can cover too much

Dan Gillmor picks up a story from California's Contra Costa Times about a Republican operative who has been sending phony letters-to-the-editor bashing Demos, more than 200 letters for the past 10 years.

 
According to the CC Times story....

""Bogus letters have a tremendous effect on the readers," Times Editorial Page editor Dan Hatfield said. "People need to be able to know that the letters to the editor are real people, writing about real issues. They need to be able to believe what they read in the newspaper. The discovery of false letters makes the reader wonder about the veracity of the opinions on our pages....
"Hatfield said the paper has tightened its policy, but there is no way to screen writers intent on breaking the rules....

"The Times, [San Francisco] Chronicle and [Tri-Valley] Herald have similar letter to the editor verification policies. A writer must provide his or her resident city and phone number. A newspaper employee then calls the writer to verify that they sent it in.

"Unfortunately, there is not a fail-safe way that I have found. No matter how elaborate the system one designs, there is always some knucklehead out there who wants to ruin it for everyone by proving that he or she can beat it."

Maybe not a "fail-safe" way to stop this Astro-turfing, but stronger controls would be possible if an organization like the American Press Institute or Newspaper Association of American would create an online data base that all newspapers could have access to.  The calling-to-check approach is pretty standard in the business.  Each letters editor could enter the pertinent info on the writers they decide to publish into the data base.  It wouldn't take much programming to do some automated data mining on phone numbers and/or cities or addresses or spelling patterns of names for flags to be raised.  Sure, someone could always have a couple phone numbers and even a couple mailing addresses.  But 200? 

As to text analysis that could be applied to the language of the letters-to-the-editors, see the IAJ link lower right to Don Foster's book, Author Unknown.

Hardly a week seems to go by but one of those folks who calls himself/herself a journalist is busted for either "creative harvesting", i.e. plagiarism, or engaging of the writing of fiction under the guise of journalism.  The latest is a case involving Technology Review.  According to Boston.com and AP:

"Review raises doubts about reporter's accuracy
By Ken Maguire, Associated Press Writer  |  April 21, 2005

"...Technology Review, which is affiliated with the Massachusetts Institute of Technology, last month retracted two Delio stories about the dismissal of Carly Fiorina as chief executive of Hewlett-Packard Co. after officials told Pontin they couldn't identify a Delio source.

The magazine pulled all 10 of Delio's articles from its Web site -- including the two that were retracted -- and hired Susan Rasky, a journalism instructor at the University of California, Berkeley, to investigate their
accuracy."

Here at the IAJ, we think this removal is, at best, unprincipled and unprofessional on the part of the magazine.  At worst, it is rewriting history. Either way, the action compounds any errors the author is alleged to have made.  Librarians have wrestled with these digital issues for almost 20 years now.  The consensus seems to be that, no, you don’t remove the offending documents, but you can/should flag the stories with a prominent note explaining the evolution of the case and offering, essentially, a “Reader Beware” notice.  It wouldn’t hurt either if publications developed something akin to the “ethical guidelines” policies in place in many publications that would explain to readers that, “No, we do not alter the historic record.”

That said, check out J's Scratchpad and the entries for "Journalism Errors."  Jessica Baumgart, Information Resources Specialist in Harvard University's Office of News and Public Affairs, is taking the time to track these sorry events.