The rise and rise of Palantir and its ‘deep domain knowledge’

On the fourth floor of an office building on Northbourne Avenue, in what passes for Canberra’s CBD, is an outpost of a much talked-about company that has so far gone under the radar in Australia. It is, however, unlikely that many Australians have avoided the company’s forensic gaze.

Palantir Technologies was established in 2002 by a clutch of US information analysts to explore the potential of datamining tools developed for Paypal. The CIA was a foundation investor, providing $2 million, and for several years its only customer. However, unusually for a company that has become a key vendor to the US military-industrial complex, its senior ranks are almost entirely men (and they’re pretty much all men) with Silicon Valley-style IT or financial backgrounds; the revolving door to the US military and foreign policy establishments that typifies most defence and intelligence companies doesn’t appear to be in full operation (yet).

Palantir does datamining, and does it very, very well. So well, in fact, that the US government and major companies have hungrily devoured its data search tools (for an account of what exactly its products can do, try this). As we’ve since learnt courtesy of Edward Snowden, agencies like the NSA are compiling vast amounts of personal information on most of the planet’s internet users. Palantir’s products help agencies effectively search through huge amounts of different information and collate them with other agencies’ data. It has rapidly become a key player in the establishment of the US surveillance state and a poster boy for what smart people and lots of computing power can do to strip away privacy and garner intelligence down to the individual level. And it has rapidly become an attractive investment: two weeks ago the company, now estimated to be worth $8 billion, announced it had raised nearly $200 million in capital.

And behind a unicorns-and-rainbows façade (Palantir is a Lord of the Rings reference; its California headquarters is called “the Shire”) is a ruthless player in cybersecurity. In 2011, as Crikey reported at the time, the company joined with Berico Technologies and HBGary Federal to develop a multi-million dollar plan to disrupt WikiLeaks and discredit journalist Glenn Greenwald. The plan, only revealed when Anonymous hacked into the IT system of HBGary Federal’s Aaron Barr, involved proposals to feed false information to WikiLeaks, break into its servers and wage a media campaign against it and Greenwald.

“Combating this threat requires advanced subject matter [sic], expertise in cybersecurity, insider threats, counter cyber-fraud, targeting analysis, social media exploitation. Palantir Technologies, HBGary Federal, and Berico Technologies represent deep domain knowledge in each of these areas. They can be deployed tomorrow against this threat as a uni?ed and cohesive investigative analysis cell.”

When the plan was revealed, Palantir backpedalled as quickly as it could. Chief executive Alex Karp released an apology saying:

“Personally and on behalf of the entire company, I want to publicly apologize to progressive organisations in general, and Mr. Greenwald in particular, for any involvement that we may have had in these matters.”

Just how sorry was Karp? It was initially reported that the Palantir employee involved with the plan, “Forward Deployed Engineer” Matthew Steckman (Palantir likes to use that military-sounding title) had been sacked; it turned out he had merely been placed on leave. A few months later, after the issue had dropped out of the media, Steckman was secretly reinstated (the journalist who revealed Palantir’s involvement, Barrett Brown, remains in jail awaiting trial for, inter alia, link-sharing).

There are also questions about whether data-mining actually helps catch terrorists or identify and stop them any better than, or even as well as, more traditional intelligence-gathering methods that don’t involve governments. The American Civil Liberties Union called it ”monitoring the activities of innocent Americans on a mass scale/collecting the records of those activities and leaving them open for suspicionless exploration by government analysts”.

Palantir doesn’t merely work for the US government; indeed, more than half of its income now comes from the private sector, including major banks and media companies. If you ever see a News Corporation story sourced from tweets, for example, you’ll see they’ve licensed Palantir Torch, a social media search engine.

The company’s arrival in Australia is relatively recent. The Department of Defence began using some of its software in 2011 via third-party providers, but this year has seen the company grow rapidly. Former energy sector engineer Robert Minson was brought from Europe to oversee the company’s Australian operations at the end of 2012. Top-flight lobbying firm Government Relations Australia was hired to represent them in Canberra and state capitals. In the last few weeks, the company has secured multi-year contracts with the Department of Defence’s Intelligence and Security branch worth nearly $2 million, all secured via limited tender. While the company has staff in Melbourne and Sydney, it has a number of “Forward Deployed Engineers” in offices on Northbourne Ave and is trying to recruit more?—?only last week it was advertising for engineering interns.

Those of course are the contracts we know about. Palantir’s size and growth in Australia?—?it has 14 publicly identified staff on LinkedIn?—?suggests it has more work than that provided by the defence contracts. Security agencies don’t reveal their external contracts (for “operational reasons”).

Palantir’s growth isn’t merely testimony to the perceived quality of its products or the unevidenced claim that datamining is an effective national security tool, but also to the rude health that the cybersecurity and IT intelligence sector is in even as traditional defence sectors suffer from funding cuts. That’s the reason why, in recent years, traditional defence companies have been gobbling up cybersecurity companies, aware that governments, persistently beating the “cyberwar” drum, continue to ramp up their spending on online defensive and offensive products. As Palantir’s burgeoning operations here demonstrate, the Australian government is no different.

This article first appeared on Crikey.