Open source intelligence and the internet detectives

British spy thriller TV series, Killing Eve follows the story of an intelligence officer in her hunt for a psychopathic killer.

Aside from these central characters, the audience meets self-trained investigator Kenny Stowton. He becomes the young man MI6 agent Eve Polastri relies on time and time again. He locates the face of the wanted killer through a database of known convicts. He finds the precise location of a person using nothing more than an email address. In magical TV seconds, Stowton gets it right every time.

Embedded in this fictitious tale of quick investigators is the very real story of how much information about the world and its people is publicly accessible online, much of it carrying the potential to become intelligence that will ultimately determine a course of action.

Investigative experts in Canada’s corporate security landscape do not speak of magical seconds in open source intelligence, or OSINT, but rather describe a process that requires a fine balance between human analytical ability and machine-led artificial intelligence to be successful.

The human and the machine
Mike Lantz, vice-president at Paladin Risk Solutions, says securing the right balance between a good analyst and AI has been the biggest evolution in OSINT since its inception. According to Lantz, a private investigator working out of British Columbia for more than 27 years, so many tools have emerged from OSINT. The term itself has transformed to more modern descriptions like “intelligence risk monitoring.”

“There’s no really good AI tool out there that will do it for you. And as a single analyst, it is difficult to do it by yourself. That’s why it’s a good pairing,” says Lantz.

The technology is there to collect the data. As Lantz explains, the more data collected, the higher the possibility of locating potential threats. The analyst, or team of analysts, is there to assess the data and determine whether a real threat to their client is represented in the information.

Successful analysts possess technical prowess and are social media savvy. They have an investigative mindset but maintain an open mind that can piece together data from various sources. They are familiar with different databases and know the best places to look for certain types of information.

Existing OSINT-related tools serve to narrow down searches for the analyst. Some tools monitor and produce alerts while others serve more as investigative tools to collect data.
In his role as VP at Paladin, Lantz oversees the company’s North American risk operations. He says the company invests at least half a million dollars in annual subscriptions towards data mining tools.

“There’s so much information out there that’s not really relevant to what we need, but somebody needs to sift through it to make sure that it’s not, in fact, relevant. Having those tools is really imperative to be able to get through that information.”

Steering through the data universe
One such tool comes from digital threat intelligence company, Halifax-based Liferaft.

Navigator is a platform powered by Liferaft. The company’s manager of market strategy, Neil Spencer, describes it as a system that is designed to make the analyst’s life easier in the old needle in the haystack scenario.

“It’s multifaceted. It is one big collection and aggregation of that data, so you need the machines to go and find the right content,” says Spencer.

Navigator has machine learning and AI built into it. Its functionality allows users to pick and choose the information that is most pertinent to their investigations, and some of this is done using keyword terms. Once the platform sees that specified information come through, it will elevate that content of interest for the user.

As Spencer explains, the human is there to validate the information brought before them. At this stage, the security professional would assess the information before highlighting the findings to stakeholders within the organization. This could be done through a report or mass communication, if time is of the essence.

One of the greatest changes in OSINT is the availability of open source data that could become intelligence. Five years ago, people in the field only had what he called a “smattering” of sources centred around social media like Reddit, Facebook and Twitter.

“Now, there is a much more broad array of areas that contain information that can identify threats,” said Spencer.

Since the digital era has been plagued by misinformation, validation of the data collected is a process that requires yet another set of skills.

As security veteran Bob Riddell explains, analysts cannot take any of the information they find at face value.

OSINT and misinformation
Riddell, whose security career spans more than 30 years in financial institutions and property management, said security analysts leveraging OSINT need to grow their analytical abilities to counter the rising tide of erroneous data online.

“Despite being so plentiful, social media already heavily integrates misinformation,” he says. “Not only do they get the information, but you have to go through a process of discerning as to what is fact versus fiction. There’s a lot of sophisticated analysis that has to be done.”

Riddell has contributing to the security industry in multiple capacities, including as the founding chair of the Building Owners & Managers Association Toronto – Security Risk Management Advisory Council. Riddell established his own advisory practice, Riddell Risk Management, in 2020 and is also the director of Consilium Public Sector Services.

Throughout his career, Riddell used OSINT to identify threats in the form of potentially violent protests where protection of private property was concerned. In the area of finance, OSINT became a security tool to identify potential scams.

“The overarching rule for any analysts moving forward, when they’re reviewing information, is they’ve got to be cautious in their assessments of the data and take steps necessary to cross verify against other sources, so they can make sure of the veracity of that information,” says Riddell, adding that this is of even greater importance if they’re going to be distributing their findings or making recommendations based on their research.

Despite the security challenges that inevitably arise as technology advances, it should be noted that the expansion of OSINT has also meant opportunities for growth.

The development of Edmonton-based AI and big data company Samdesk is almost a direct correlation of this expansion in open source information.

The earliest signs of a threat
Samdesk founder and CEO James Neufeld describes the green stages of his career in newsrooms, when he held a technical, behind-the-scenes role assisting journalists with breaking news by scouring social media for the earliest signs of an event.

In the years that followed, Neufeld would build a company similar to that role, however, his company would use AI to monitor data at the global level and help identify the earliest signs of a threat to different stakeholders. Currently, Samdesk’s portfolio includes corporate clients, NGOs and public sector organizations, among others.

The company’s sources have expanded to include reports from community-based journalism, satellite imagery, audio sensors, live streaming data, textuals and footage uploaded from the mobile devices of bystanders.

Neufeld says the goal of Samdesk is to give its users the most robust and raw data from multiple angles so that they have the richest possible view of events as they transpire on the ground.

“We’re now in a unique position from an information and situational awareness standpoint that we’ve never really had in human history, where everything is being documented from numerous angles in near real-time, or often in real-time,” says Neufeld.

Of late, the company has integrated its technology with travel management tools to provide travel managers and security managers with alerts. Samdesk has also integrated with enterprise resource planning software to assist companies with their supply chain management.

For Neufeld, human intelligence is not only a critical component in the analytical stage but also in the actual building of the platforms. Neufeld explains that data scientists and engineers are responsible for the creation of systems that can identify items of significance using AI.

Currently, Samdesk is also using its platform to collect data on events in Ukraine to supply Amnesty International’s investigations and documentation of the conflict for human rights abuses.

“It becomes a really powerful data set where we don’t have to rely on official government bodies as much as we once did. We don’t have to rely on the assessments of individual analysts and their view of the world.”