The Problem of Merging Databases

Bookmark and Share

Peter Manning, criminologist

More news on the growing convergence of the military and police in the U.S. has come my way from Boston criminologist, author and professor, Peter Manning, whose skepticism about the ability of police to take full advantage of IT was reported in a previous blog.

First of all, Manning says that the efforts to merge proprietary military and policing databases have failed and he has seen press reports that efforts to continue research in this area have been abandoned by the parties involved.

Also, this is a problem across the board in the U.S. federal government where agencies like the department of Homeland Security, the Pentagon, Immigration and Customs and the FBI have their own unique database systems.

Manning reminds us that this is a common problem facing public and private organizations which seek to internally or externally share database information sitting on different software or various platforms.

"It is the way that the data is encoded and the way that the encoding can't be erased when it is dropped into another database. There are intrinsic limitations to collapsing these large databases."

It is apparently not uncommon for separate databases to be kept after a corporate merger. The professor cites the example of how Delta and Northwest Airline passenger lists were kept separately out of technical necessity after the two airlines were combined.

Manning believes that the major defense contractors such as Lockheed Martin have not given up working to solve the problem of marrying military and police databases for their clients in the Department of Defense in Washington. But he doubts that we will hear about a software solution anytime soon.

"Defense contractors want to be able to merge various databases including commercial and alike for monitoring and tracking because it is a fundamental problem in the military world."
In the post-9/11 hysteria of the Bush administration, all sorts of information has been collected by U.S. government agencies on airline passengers, bank records, donations to Islamic charities and political activity.

A number of media outlets reported back in 2005 that something called the Counter-Intelligence Field Activity had been conducting surveillance and generating reports on more than 20 legitimate activist and anti-war groups around the US.

The revelations called into question the usefulness of the CIFA data mining tools which had been developed to allow intelligence experts to search vast troves of information, including reports of investigation, collection reports, statements of individuals, affidavits, correspondence and other documentation for possible illegal and criminal acts
Originally, CIFA was established in 2002 by former deputy secretary of defense Paul Wolfowitz, to consolidate databases on suspicious activity around and inside military installations by the U.S. Army, Navy, Air Force, and Marine Corps.

Since then, we have been told by U.S. government officials that the surveillance of peaceful dissenters by the U.S. military has ended and the CFIA itself was closed down last year.
However, some criminologists have pointed to a deeper issue of law enforcement organizations being given money by the U.S. government to invest in technologies like new databases or programs that have little to do with the solving of local crimes and more to do with the growing militarization of the police since Ronald Reagan.

Of the $4-billion which the Obama administration has invested in stimulus money going to police, what percentage of the purchases being made on new guns, computers, cruisers and new hires are entirely justified?

That's a difficult question but one that I will be exploring in further details in my upcoming blogs. 

Leave a comment