Institute for Analytic Journalism

Three Tuesdays workshop on data and the political campaigns at the Santa Fe Complex

Tom Johnson — 2008-09-27T13:50:19-06:00

Handicapping the Horserace

Published by Don Begley at 10:09 pm under Complex News, event

September 30, 2008
6:30 pm	to	8:00 pm
October 7, 2008
6:30 pm	to	8:00 pm
October 14, 2008
6:30 pm	to	8:00 pm

It’s human nature: Elections and disinformation go hand-in-hand. We idealize the competition of ideas and the process of debate while we listen to the whisper campaigns telling us of the skeletons in the other candidate’s closet. Or, we can learn from serious journalism to tap into the growing number of digital tools at hand and see what is really going on ...

If you're really serious about searching....

Tom Johnson — 2007-12-04T18:29:37-07:00

Deep Web Research 2008

http://www.llrx.com/features/deepweb2008.htm

By Marcus P. Zillman, Published on November 24, 2007

Printer-Friendly Version

Bots, Blogs and News Aggregators is a keynote presentation that I have been delivering over the last several years, and much of my information comes from the extensive research that I have completed over the years into the "invisible" or what I like to call the "deep" web. The Deep Web covers somewhere in the vicinity of 900 billion pages of information located through the world wide web in various files and formats that the current search engines on the Internet either cannot find or have difficulty accessing. Search engines currently locate approximately 20 billion pages.

In the last several years, some of the more comprehensive search engines have written algorithms to search the deeper portions of the world wide web by attempting to find files such as .pdf, .doc, .xls, ppt, .ps. and others. These files are predominately used by businesses to communicate their information within their organization or to disseminate information to the external world from their organization. Searching for this information using deeper search techniques and the latest algorithms allows researchers to obtain a vast amount of corporate information that was previously unavailable or inaccessible. Research has also shown that even deeper information can be obtained from these files by searching and accessing the "properties" information on these files.

This article and guide is designed to give you the resources you need to better understand the history of the deep web research, as well as various classified resources that allow you to search through the currently available web to find those key sources of information nuggets only found by understanding how to search the "deep web".

This Deep Web Research 2008 article is divided into the following sections:

Articles, Papers, Forums, Audios and Videos	Cross Database Search Services	Peer to Peer, File Sharing, Grid/Matrix Search Engines	Resources - Deep Web Research
Cross Database Articles	Cross Database Search Tools	Presentations	Bot Research Resources and Sites

ARTICLES, PAPERS, FORUMS, AUDIOS AND VIDEOS (Current and Historical)

Tracking the bucks all the way to court

JTJ — 2006-10-01T19:10:54-06:00

Another unique investigation by The New York Times gets A1 play in this Sunday's edition (1 Oct. 2006) under the hed "Campaign Cash Mirrors a High Court's Rulings." Adam Liptak and Janet Roberts (who probably did the heavy lifting on the data analysis) took a long-term look at who contributed to the campaigns of Ohio's Supreme Court justices. It ain't a pretty picture if one believes the justices should be above lining their own pockets, whether it's a campaign fund or otherwise.

In any event, there seems to be a clear correlation between contributions -- and the sources -- and the outcome to too many cases. A sidebar, "Case Studies: West Virginia and Illinois," would suggest there is much to be harvested by reporters in other states.

There is, thankfully, a fine description of how the data for the study was collected and analyzed. See "How Information Was Collected"

There are two accompanying infographics, one ("Ruling on Contributors' Cases" ) is much more informative than the other ("While the Case Is Being Heard, Money Rolls In" ), which is a good, but confusing, attempt to illustrate difficult concepts and relationships.

At the end of the day, though, we are grateful for the investigation, data crunching and stories.

Ver 1.0 -- The beat goes on

JTJ — 2006-04-18T14:14:32-06:00

We're pulling together the final pieces following the Ver 1.0 workshop in Santa Fe last week. Twenty journalists, social scientists, computer scientists, educators, public administrators and GIS specialists met in Santa Fe April 9-12 to consider the question, "How can we verify data in public records databases?"

The papers, PowerPoint slides and some initial results of three breakout groups are now posted for the public on the Ver1point0 group site at Yahoo. Check it out.

SJ Mercury-News Series: "Tainted Trials, Stolen Justice."

JTJ — 2006-01-23T16:18:58-07:00

Friend-of-IAJ Griff Palmer alerts us to an impressive series this week that examines the conduct of the DA's office in Santa Clara County, California. If nothing else, the series illustrates why good, vital-to-the-community journalism takes time and is expensive. Rick Tulsky, Griff and other colleagues spent three years -- not not three days, but YEARS -- on the story. Griff writes:

"I invite you all to take a look at "Tainted Trials, Stolen Justice." This five-day series was three years in the making. It starts in today's Mercury News:

http://www.mercurynews.com/mld/mercurynews/news/special_packages/stolenjustice/

Free registration is required to view the ...

Resources related to Crime Mapping

Tom Johnson — 2005-12-06T20:04:45-07:00

We don't know if there has as yet been any empirical research done on how interested media consumers are in online crime mapping -- and how good the coverage is -- but there is a body of literature debating readers' interest in crime per se. It would seem to be a pretty good bet, though, that if people are interested in crime AND if more and more are going online via broadband, that some dynamic crime maps would get some hits.

Remember that crime mapping is not just about pushing digital push-pins on a map, GoogleMap or otherwise. "Journey to Crime" maps or maps showing where a car was stolen and when it was recovered can provide interesting insights.

Here are some links recently posted to the CrimeMapping listserv that could be of value to journalists:

Journey-after-crime: How Far and to Which Direction DO They Go?
http://www.ojp.usdoj.gov/nij/maps/boston2004/papers/Lu.ppt

Linking Offender Residence Probability Surfaces to a Specific Incident Location
http://www.ojp.usdoj.gov/nij/maps/dallas2001/Gore.doc

Journey to Crime Estimation
http://www.icpsr.umich.edu/CRIMESTAT/files/CrimeStatChapter.10.pdf

Applications for Examining the Journey-to-Crime Using Incident-Based Offender Residence Probability Surfaces
http://pqx.sagepub.com/cgi/content/refs/7/4/457

The Geography of Transit Crime:
http://www.uctc.net/papers/550.pdf

See, too: Paulsen, Derek J. "WRONG SIDE OF THE TRACKS: EXPLORING THE ROLE OF NEWSPAPER COVERAGE OF HOMICIDE IN SOCIALLY CONSTRUCTING DANGEROUS PLACES." Journal of Criminal Justice and Popular Culture, 9(3) (2002) 113-127

Yes, Virginia, methodology DOES matter

JTJ — 2005-11-10T12:38:43-07:00

A piece on calling the elections in Detroit:

MAKING A FORECAST: A secret formula helps producer call the election right

BY CHRIS CHRISTOFF
FREE PRESS LANSING BUREAU CHIEF

November 10, 2005

What was a viewer to believe?

As polls closed Tuesday, WDIV-TV (Channel 4) declared Freman Hendrix winner of Detroit's mayoral race by 10 percentage points.

WXYZ-TV (Channel 7) showed Hendrix ahead by 4 percentage points, statistically too close to call.

But WJBK-TV (Channel 2) got it right, declaring just after 9 p.m. that Mayor Kwame Kilpatrick was ahead, 52% to 48%, which turned out to be almost exactly the ...

Digital detectives

JTJ — 2005-11-03T13:30:45-07:00

For those interested in the forensic process -- and in this case, computer forensics -- be sure to check out this fine, fine piece of digital detective work by Mark Russinovich, a computer security expert with Sysinternals. He discovered evidence of a "rootkit" on his Windows PC.

We don't think journalists need to know how to DO this kind of deep-diving probing, but we should be aware that it is possible and, broadly speaking, the methods if only to know the appropriate search terms.

Through heroic forensic work, he traced the code to First 4 Internet, a British provider of copy-restriction technology that has a deal with Sony to put digital rights management on its CDs. It turns out Russinovich was infected with the software when he played the Sony BMG CD Get Right With the Man by the Van Zant brothers.

Here's WIRED Magazine's take on the story, "The Cover-Up Is the Crime"

And here's what Dan Gillmor had to say about it, with additional links.

We should be talking to -- and learning from -- each other

JTJ — 2005-11-03T10:57:47-07:00

Another example of how journalists can learn from other disciplines comes to the surface in the form of an LA Press Club meeting Nov. 9.

"Digging deep: What reporters can learn from and about private investigators," is the topic, and the panel of speakers, though large, seems rich with potential.

Here at the IAJ we also value the well done blog, "PI News Link," run by Tamara Thompson. Check it out; enter it in your blog harvester.

What's behind the curtain? "Private Warriors"

JTJ — 2005-07-07T12:30:07-06:00

We're pleased that the PBS program "Frontline" is keeping up the good fight to produce important journalism. And thanks to the Librarian's Index to the Internet for pointing us to:

Private Warriors

This Public Broadcasting Service (PBS) Frontline program looks "at private contractors servicing U.S. military supply lines, running U.S. military bases, and protecting U.S. diplomats and generals" in Kuwait and Iraq. Website features discussions of the appropriateness of outsourcing, whether privatization saves taxpayer money, and the role of contractors. Includes contractor profiles, interviews, a FAQ, video of the program, and related links.

http://www.pbs.org/wgbh/pages/frontline/shows/warriors/
Subjects: Government contractors -- United States | Public contracts -- United States | Private security services | United States -- Armed Forces -- Management | New this week
Created by je - last updated Jul 6, 2005

Be sure to drill down to the section, "Does Privatization Save Money." A nice example of a reporter asking the right questions.