Reliable data is the lifeblood of any company. The better the data, the better your business decisions. It’s a maxim that holds true across every industry, including out-of-home. Effective out-of-home decisions need to be fueled by dependable data, and dependable data requires a solid collection methodology.

Historically, out-of-home has fallen behind other media, making decisions based on estimated basic impressions. But the last five years have changed that. New data sources and new data science analysis methods have led to a surge of innovative techniques to measure the results of an OOH campaign or the audience for a specific OOH location.

Working with our data scientists, we’ve assembled the five major hurdles facing any OOH measurement framework. Evaluating how a methodology overcomes these can help you determine the reliability of its data.

Hurdle 1: How specific are your business objectives?

It’s easy to skip this first hurdle, but the effectiveness of everything else you do will depend on the clarity of your objectives. Your goals and KPIs directly determine the type of data that you’re trying to collect.

Picture it like ordering in a coffeeshop and telling a barista to, “Make me a hot drink.” The cup they hand you could have anything from a super sweet tea drink to an eye-opening latte with ten espresso shots to hot water. The poor barista has to make their best guess, and odds are good that what comes out won’t be what you wanted. This is exactly what companies do when asking for generic results. Data science teams are left to guess whether they’d like the number of people who saw the ad, how many cars drove by, or how long the ad ran during peak hours.

If you want a medium latte with whole milk and three pumps of hazelnut syrup, then you better tell that to the barista. The same is true for OOH, whether you want to increase app downloads among millennials, push coupon activations for stay-at-home mothers, or figure out how many middle-aged men visited a sports bar after seeing your ad.

Getting dependable results from your measurements starts with specific KPI objectives that precisely define what qualifies as a result.

Hurdle 2: Are you making the right assumptions?

By the time an OOH campaign goes live, the team is ready to make an impact and you’re eager to get a return on your investment. But that enthusiasm for results has a habit of hamstringing measurement by assuming that any uptick is caused by the campaign.

Imagine that a clothing designer sees a 27% increase in sales during the first two weeks of an OOH campaign. It’s tempting to attribute that to OOH, but it could also be because a star randomly decided to wear that designer to a premiere. Similarly, a surge of diners in April at a restaurant known for patio views could be from their bus shelter ads or because of a warmer-than-average Spring.

Measurement studies should start with the assumption that the campaign had no effect. Any recorded data can then be analyzed with a view to the surrounding context and focus on eliminating other possible sources. Once these contextual variables are removed from the equation, you’re left with reliable data that can be precisely and exactly attributed to the OOH campaign.

A good methodology starts with a scientific mindset devoted to discovering the results that are there instead of the results you want to see.

Hurdle 3: Is your data rich and pure enough?

The recent surge of data sources has made it easier than ever to gather mind-boggling amounts of data. It’s tempting to get swept away by the hundreds of gigabytes of data, but all of that means nothing if the data is biased or inaccurate.

We’ve all seen enough cops and lawyers on TV or in movies to know about the chain of custody for evidence. You can’t build a case on a key fingerprint if twenty different people smudged it or added their own fingerprints to the mix. The same thing is true with your data. You need to know the provenance of your data, or its chain of custody, if you’re going to base business decisions on it.

Almost all of the data you get will have something done to it, even if it’s just stripping identification information. To ensure data accuracy, get it as close to the raw source as possible. At the same time, your team needs to know the original source of the data, how it was gathered, and exactly what happened before it ended up in your hands.

If you want reliable data, then make sure solid data provenance is integrated into your methodology.

Hurdle 4: Do you account for spatial autocorrelation?

Correlation is when two data points are related and change at the same time. When these data points are compared on a graph, a clear trend will emerge between them that can be positive or negative. Ice cream sales, for example, might go up when the temperature goes up, and graphing these would show a positive correlation between how much ice cream people buy and how hot it is.

Starting with that concept, we can view the map as a graph showing position, with each data point showing the location of someone at various times. Spatial autocorrelation is used to chart how inherently related these movements are due to the progression of time and their journeys. As an illustration, picture a group of one hundred cars on a freeway that’s the main route downtown. If you had positions for the cars every 10 minutes, it follows that they would drive directly between those points. Spatial autocorrelation measures just how related paths like this are to account for their presence in data sets.

When measuring OOH, spatial autocorrelation is vital for removing false positives from the equation. Take that freeway example. If there’s a billboard along the way for a store downtown, then their data could indicate that the cars passed the billboard and then drove to a location within a half-mile radius of that store. It’s tempting to attribute that behavior as a result, but spatial autocorrelation tells us that their path was probably going to take them downtown anyway.

Compensating for spatial autocorrelation is vital if a methodology is going to produce real, actionable data.

Hurdle 5: What is your model telling you?

There’s a legend in the data science world about an army study in the early days of machine learning. The story goes that they built a program to identify tanks from aerial photos. After teaching the program, they got back 100% success on the tests, but then the program didn’t work in the field. Looking back through the tests, they found that every picture with tanks was taken on a sunny day, while the ones without tanks were taken on a cloudy day. So their model didn’t measure whether or not there were tanks — it measured whether or not there were clouds.

Remember this illustration when looking at your measurement models. Are you actually measuring the people who saw your billboard and downloaded your app, or are you measuring how many commuters are downloading your app? If your methodology is constructed correctly, it will balance the data to separate commuters from people who responded to your OOH campaign.

As a related point, it’s also important to know the certainty of your model, which is usually shown with a margin of error percentage. Also known as a p-value, the standard acceptable margin for science is 5%, but the lower the better. For example, at Boohma, we always aim for a 3% margin of error.

Ensure that your model is carefully designed to measure exactly what you want with as little error potential as possible.

With the right methodology, you can accomplish the incredible

It takes a lot of work to build a dependable measurement methodology, but it’s worth the effort. Once you have reliable OOH results data, you can refine your future campaigns or figure out how to improve the impact of a specific OOH unit. Every time you do, you’ll feed more data into your attribution models, making them stronger. The result is more effective OOH campaigns, better KPI results per placement and an ROI that just keeps getting better.

Invest the time and effort to develop a reliable measurement framework today and you’re creating the high quality data your business needs to grow and thrive. Sound good? Reach out today to learn more and get help from our team of experts!