Sharper focus: the evolution of video analytics

While adoption so far has been more pronounced in some markets — such as retail, transportation, education and health care — they are finding their way into many different sectors. As they become a more mainstream part of the security market, end users are still discovering how to use them and how effective they are, and still trying to assess exactly what value they bring to their organization.

A year and half ago, the City of Calgary, together with Calgary Transit, began testing two enterprise-level analytics systems. So far, security uses have included applications such as breached perimeters, direction of travel, basic motion, left objects and loitering. They’ve also used analytics for heat mapping to help monitor the number of people at public events.

“An analytic that can see a crowd has grown will set off an alarm. All of a sudden, there are 200 people here; a lot of activity. Somebody needs to know about that,” says Sean Bolli, team lead, technical operations and support, corporate security at the City of Calgary.

Bolli also believes the data from analytics can be used to deliver city services more effectively. For example, at Calgary Transit’s LRT platforms last year they tested an analytic designed to alarm when someone crossed the yellow line on the platform when there was no train in the station, indicating the person may have jumped onto the track. The analytic had to de-activate when a train came into the station, when people must cross the line to board the train.

The business strategy group then asked if they could produce data on when trains enter and leave stations, he says. “We were able to get a report by day, or by hour, showing all train times — when they entered the station, when they left, how long they stayed — that we could provide to our business groups. It didn’t require us giving them any video, so we weren’t breaking any privacy laws.”

Bolli sees other opportunities for analytics, including traffic management (to adjust traffic signals) and in city parks where a heat mapping analytic can be used to determine attendance numbers.

However, they have found some difficulties. For one, analytics do not work well with PTZ cameras. Another concern is the false alarm rate. During phase one of testing, they wanted to see how many false alarms the system produced.
“Analytics can introduce a whole new dynamic of false alarms on top of what we already get for false alarms,” Bolli says. “There’s a whole other level now. Where do you put them? What cameras do you use? Are they up to date? Are they generating good image quality? What are the lighting conditions like? What are the activities being captured on that camera already normally?”

During testing of left-object capability, for example, when they would add an object to an area to see if the system captured it, they found sometimes it did and sometimes it didn’t. When it didn’t, they would tweak the rules to try to improve the system’s ability to capture the object.

“And sometimes, when a person stood still for five seconds, the system thought, there’s an object left behind and set off an alarm. You can’t teach the system that kind of intelligence. You have to do that through the rules. There are many different factors to adjust.”

The current adoption of analytics is being driven by a number of factors, says Nick Ingelbrecht, research director at Stamford Conn.-based Gartner. One factor is technology and improved systems: better platforms and cameras, 3D systems, multiple sensors, displays, better algorithms — which have increased the accuracy of systems and produce fewer false positives — along with, of course, advances in the analytics capabilities themselves.

“They’re all part of the system that delivers that capability. It’s what the analytics actually does at the end of the day, and what it enables you to do in terms of such things as response, facial recognition or managing environmental events,” he says.

Also key was the introduction of machine learning capabilities into the lower-tier products, says Ingelbrecht.
“These are systems that can train themselves, so you move more towards the plug-and-play environment. The system learns about the environment and determines over a period of time what is routine activity and what are the exceptions that you want to trigger an alarm.”

Another trend behind the adoption of analytics, he adds, is the movement of analytics-related storage into the Cloud. The use of mega-pixel cameras and HD video, in particular, creates enormous storage requirements.
Lastly, with the proliferation of cameras and the enormous amount of video that now exists, organizations face the issue of what to do with it and how to automate video processing.

“In a control room, where an operator is watching a series of monitors, there’s a problem of maintaining attention. You have to automate that vigilance so that you just respond to the anomalies and automate the routine monitoring of images,” Ingelbrecht says.

“And of course, if you have thousands of cameras in an airport, thousands of people moving through, and you’re trying to identify individuals, you need the capability to manage the amount of data you’re getting and to generate useful alerts.”

After the big rush to adopt analytics about six years ago (which led to huge disappointment due to dashed expectations) the market is moving into a “real-life version” of video analytics, says Erez Goldstein, senior product marketing manager Paramus, N.J.-based Qognify. As the technology was taken up in areas such as customer service and retail, it created more confidence and led to the next generation of analytics.

As an example, Goldstein points to his company’s Suspect Search, which allows a user to search recorded or live video for a specific person in an area. This next generation, he says, does not rely on binary, yes-or-no matches — did the object cross the line or not? Is there a line longer than this number of people or not? — and so provides more advanced insight.

“In Suspect Search, it doesn’t matter whether it’s a crowded place like an airport or city, the person’s image will be found based on sophisticated algorithms that compare the way they look and the way they are presented to the camera,” he says.

“It represents the way the industry is going, towards more human-like needs. I need to find you, so I need an application that finds you and not a random person with the same colour shirt.”

Adam Curtis, senior director of corporate investigations at Milton, Ont.-based AFIMAC, says they often use video analytics for client companies that want to stop theft from construction sites. The covert cameras are equipped with motion detection. When the system detects motion, it records it.

Another major use is for internal theft. Again, they set up covert cameras in an office or retail store. “From the analytics produced, we can hone in and target those specific areas we want to look at.”

Curtis says analytics, such as the facial recognition systems they are working with, are smart enough now to trust with critical decisions. Instead of key card access, for instance, which requires a person to swipe their card on a reader, the system recognizes a worker based on a photo.

“As you’re walking up to the door, it recognizes your face, understands you are authorized to enter the facility and, then, automatically opens the door without your having to do anything.”

The system also recognizes a face even when the person has changed their appearance; for example, hair style or colour, he adds. “It recognizes the changes of the individual, so it’s actually the computer program thinking.”
Curtis says analytics do not completely replace people: you need people to analyze video and ascertain what information is being captured and whether it is relevant to an investigation. But, he adds, analytics do replace people in some situations. AFIMAC has guards at 21 cargo facilitates across Canada with extensive CCTV with analytics. “That is substantially reducing our physical feet on the ground because cameras can cover a lot more.”

Bolli agrees that analytics do not not necessarily replace people. Rather, they help make staff more efficient and effective and able to respond to a wider variety of situations earlier, smarter and with more information.
Goldstein also agrees, saying the next generation of analytics will take advantage of both the human and the algorithm. With Suspect Search, for example, the algorithm presents an operator with several potential suspects, all similar to the actual person. The operator then, using the human eye and brain, which are much better at distinguishing the suspect from similar people, eliminates some of the results and tells the algorithm to search again based on the most relevant images.

“There is an iterative process of refining the search again and again based not just on the algorithm but on the human input,” he says.

“This creates a strong and very accurate process that happens within seconds to find a person out of hundreds of hours of video. You have the strength of the algorithm finding a match and then the human refining the search and sending it to another iteration of researching.”

Although Calgary is still testing different systems, Bolli says he is confident the analytics will make security more valuable to the city. In addition to using it to improve the ability of security staff to monitor and respond to situations earlier and better, they also plan to use it across all their business units.

“That increases the value tenfold of the security program, security infrastructure and the expertise that security professionals have when it comes to CCTV, to now deliver business-enabled services,” he says. “What is basically a cost centre can add value to other areas that make it more valuable to incorporate into future planning.”