In Audience Measurement

Recently I participated in a panel discussion at OTTxSCIENCE; and it proved to be one of the most interesting onstage experiences I’ve had all year.

All about Audience Measurement

The purpose of the conference was to explore key questions around audience preferences and habits. By 2022, more than 58 percent of the US population will subscribe to OTT (over-the-top) video content services, according to eMarketer. The event explored such key questions as: 

  • How and why do viewers choose platforms and networks? 
  • How do viewers discover new programs? 
  • What influences viewer program selections?

The topic of our panel discussion, hosted by Jane Clarke – formerly of CIMM (Coalition of Innovative Media Measurement), which is now part of the Advertising Research Foundation (ARF)—was “SVOD Services – Who’s Watching Which Content?” As I shared onstage, I personally feel the measurement industry lags behind  consumer behavior. It’s very difficult for tech companies to keep pace with the evolution around new screens, solutions, apps, and content fragmentation etc. Measurement methodologies and technologies develop much slower by comparison. That means it’s imperative we look to the future and anticipate challenges on the horizon.

About the Panelists

The selection of panelists offered an intriguing look at the issue of audience measurement, from a variety of expert perspectives. 

  • Among all the panelists, Verto Analytics is one of the more active market research companies, working for our customers with our behavioral research platform. 
  • Luke McGuinness, representing Tvision, shared his company has built its own household panel, with living room-based technologies to detect the faces of consumers in front of the screen; the company’s technology can also track what viewers watch on  big-screen TVs. 
  • Becky Vu from Luth Research shared that her company is working a lot with brands around online clickstream data.

The panel quickly got into a discussion about measuring streaming apps, including industry challenges and proposed solutions. We discussed different methodological approaches. I mentioned how Verto has been investing heavily in both mobile and PC based metering in our consumer-centric Verto Smart panel, which means we can more easily detect content that people are watching within selected apps or web browsers on their devices. 

Also, some of our new tech can even help us do that for apps like Facebook, by validating the in-app video time spent. We are committed to using an approach that allows us to validate the type of service/app visible on the screen, regardless of the device being used (especially when it’s a personal mobile device). We’ll continue investing in approaches to detect the content, too, while counting subscribers vs. free users, etc.

When it comes to the measurement of streaming apps on the big screen (which is a much more complex question), we’ve spent five years exploring the best ways to do cross-platform media measurement. Because most of our business is based on syndicated research solutions, trackers, and data feeds, we’ve only done custom studies within the cross-platform space. I’ll be the first to admit this hasn’t been easy. Our company’s DNA is all about software; hence we’ve tried to find methods that are SW-based in our own research and development.

The Future of Audience Measurement: Three Key Takeaways

Here’s what I shared with the audience at the event about the future of audience measurement.

#1: To get great audience measurement, we’ll need bigger panels.

The value of traditional audience ratings data, as reflected in the best possible way with the old-fashioned rankers as we know them, is in decline. 

Bigger panels will help us attribute campaigns to real outcomes, build segmentation models, and target segments which can be projected to the bigger universe. Also, to profile audiences at scale with very granular segments or conduct any proper custom research like second screen studies, we’ll need bigger panels. Of course, building big, scalable panels won’t be easy, because there are unit costs for the hardware (in addition to building and operating costs). However, the advantage of hardware-based setups like TVision may be their high quality, and the way we can  control the measurement. Hardware-based setups have a permanent location in the household, which means we can gather use cases around attention and emotion-tracking.

#2: We must address Automatic Content Recognition (ACR) challenges.

Our panel discussion side-stepped the automatic content recognition (ACR) space, which is all about detecting content being watched on the big screen, using audio samples and fingerprinting of video content—like what Shazam is doing in the music domain for consumer-facing apps. 

At Verto Analytics, we’ve explored the use of ACR with a mobile SW meter. We’ve invested tens of millions in our Verto Smart panel, and the longitudinal nature and representation are very important for us. 

We’ve tried three vendors over the last five years, and while there is some promise, significant challenges remain. When we tried ACR with many of these vendors, we were faced with many technical problems. Many SDKs didn’t work. And, if they did work, we came up against other problems with the panelist experience (for example, the famous red bar visible on iOS devices, or problems making calls or otherwise using the microphone). 

In our case, the solution was to invest in extra research and development to create a streamlined experience in which  all components (including the microphone and power saver features) worked well. But we still faced challenges. Not all consumers position their devices optimally for using the microphone (and indeed it matters if the device is in a purse or pocket); and the coverage of mobile SW-based ACR use was limited. 

At best, SW-based ACR use could be interesting for custom studies or research where the use case is not based on having accurate market-level measurement of TV/OTT. The upside, however, is that it is capable of running in today’s smartphones, based on just downloadable SW, and can measure also out-of-home usage. 

That said, if one missed an entire iOS population, how can you go over the statements made about your panel coverage? We do not want to be a custom research house, so it would be difficult to live with this type of support/coverage. We also found varying results in covering SVOD services—it used to be better, to be frank, than today—mostly due to some of the copyright items and the vast amounts of metadata today’s content mapping requires. For these reasons, we’ve decided to downgrade the priority of these meter developments and also haven’t maintained that ACR-based meter lately.

Finally, we have been active in doing panel overlays, mapping our panels to panels/audiences where there is smart TV-based measurement of linear TV, advertising, and some OTT. We’ve used ACR to do this. Because we own and have the rights to connect our panelists to other datasets, these type of partnerships can actually result in pretty high-quality data without additional investment into HW or new SW, merely doing panel overlays.

#3: Companies will need to collaborate successfully to meet the needs of the mobile market.

Mobile measurement was a big topic during our panel discussion. No surprise there. Mobile is such an important piece of today’s digital service usage—including for video. 

Verto’s approach has always been based on cross-device SW metering—with downloadable apps and very sophisticated meter technologies that capture what is being done with the device. In terms of our approach, it is equally important to measure desktop PCs, laptops, and lightweight notebook computers, as well as smartphones and tablets. We see many positives in today’s measurement of mobile compared to other devices or time periods.

Today’s mobile universe is more concentrated and ruled by just two platforms: Android and iOS. If we go back 10 years, we had a more volatile and fragmented market (remember Symbian; multiple proprietary platforms; RIM; and of course early versions of iPhone and Android). Today two platforms make development easier.

At the same time, apps have increased in importance with this new mobile world, and mobile web browsing—while significant—has been growing mostly due to non-video related reasons. 

Due to apps, native video experiences, consumption taking place within the app, all of which  brings complexity and fragmentation, no two apps are similar. Based on the work we have done with clients like Yume, for example, we’ve validated how liquid today’s consumers are in moving from one service to another, across the day-in-the-life experience, and how they are exposed to video content in different ways, using different services. There are dedicated video apps and video services, like Youtube, Hulu, and Amazon Prime, but people also consume content within other apps like Twitter, Facebook, and Snapchat.

Security has also been getting better, so we’ve built new technologies to track in-app usage and validate time spent on video within apps like Facebook. This isn’t working perfectly, but we’re proud of the features we currently have in place. The form factors and touchscreens of today’s mobile devices, and the fact that people still typically use one app at a time, also make the measurement of mobile easier from PCs and bigger screens. This is particularly true where multi-tasking or multi-window UIs are evident. One trend, of course, is  increased security and encryption, which makes measurement more difficult.

One good thing in mobile is the fact that the dominant big Internet companies tend to control most of our engagement within mobile, and they have more control in the PC-centric world. This means that if one figures out how the big apps work, you’ve gone a long way toward measuring mobile properly without needing to map the whole world.

At the end of the panel, Jane asked about the best ways for tech companies to move ahead with audience measurement. I believe cross-platform OTT or video measurement is not easy to pull off by any single player. For new companies, there is simply no practical financial or technical means to build a single service that measures it all. 

That’s why the panelists agreed collaboration is inevitable; and much needed. Combining forces, various smaller companies could enable powerful solutions by working together. I echoed that we welcome discussions with both clients and vendors to come up with the best solutions to make this happen in the future!  Email me at if you want to learn more about our takes on this topic.

Recommended Posts
Sign up for our newsletter: