Farewell to Nielsen Diaries?

The recent news that Nielsen intends to eliminate paper diaries from the 140 ratings markets in which they are currently being used was met with relief in some quarters and with incredulity in others that such antediluvian methods were still being deployed to generate TV ratings this late into the 21^st Century. For me, however, it brought back memories from more than a decade ago, when Nielsen first announced its intention to phase them out by 2011.

Paper diaries harken back to the dawn of the television age when there were only a handful of TV networks and viewers could generally be counted on to remember what channels they watched throughout the day.  But even back then, Nielsen founder Art Nielsen was always searching for a better electric measurement system that would record what people actually watched instead of what they said they watched.

Fast forward to June 2006.  Coming off a bruising fight with News Corp over the introduction of the Local People Meters that replaced these same paper diaries in the top ten local markets, Nielsen faced an even more implacable foe: the Internet.  People were beginning to find new ways to watch TV online and Nielsen needed a plan to adapt.

The result was an initiative called Anywhere Anytime Media Measurement, aka A2M2.  Looking back on that plan ten years later is a reminder of how visionary corporate aspirations are often restrained by more mundane considerations such as cost, technology and clients.  For example, it eventually turned out that clients weren’t interested in paying Nielsen to measure viewing outside the home (although that is apparently back on the table again.)  And when it further turned out that iPods (remember them?) wouldn’t become the primary vehicle for mobile viewing, Nielsen dropped the quest to develop a “go meter”.  On the plus side, Nielsen did eventually integrate Internet viewing into its ratings, and it did expand the number of local markets measured with People Meters.

The nut it couldn’t crack was the complete elimination of diaries.  Nielsen’s ratings CEO Susan Whiting had pushed for a deadline of introducing electronic measurement in all local markets by 2011, reasoning that the Nielsen staff would be motivated into action by an aggressive but firm deadline.  I was the PR representative on the A2M2 team so I can testify that they were motivated, and yet the 2011 deadline came and went with diaries as firmly entrenched as ever.

It wasn’t for lack of trying.  For at least a couple of years, the A2M2 team met every Friday morning via a conference call between the project managers in the company’s business headquarters in New York City and its technology headquarters in Oldsmar Florida.  They considered and tested a number of options, including a “mailable meter” that would be sent to viewers in lieu of a diary, placed next to a TV for a month where it would record what shows were being watched, and then returned to Nielsen for transcription.

A lot of work went into developing and testing this mailable meter, including research into packaging and shipping, but in the end it wasn’t good enough or cheap enough to replace diaries.  Because here’s the thing about diaries: they are really cheap. Nielsen lost money measuring many of these markets but it lost less than it would have with electronic measurement.

Here’s the other thing about diaries: they hung around so long because a lot of the people who pay the bills (i.e., the local stations) didn’t really want to change if it meant lower ratings.  TV viewers are most likely to mark down the big-name shows they “usually” watch and those are typically network shows, which inflates their ratings, so there was an incentive to maintain the status quo.

Or at least there used to be.  In a world of streaming, video playback, on-demand, channel-switching, premium cable stations, and limited attention spans, viewers are less likely to remember anything they watch, even the six o’clock news, and the diaries have finally lost any credibility among advertisers.   The incentive for abandoning diaries is finally larger than it is for keeping them.

What’s coming instead is something that’s been in Nielsen’s toolbox for years – fusing together different datasets, including return path data from set top boxes and data from other electronic sources like the National People Meter sample.   The flaws of set top box data are numerous, including a lack of representativeness and no information on which person in the house is watching, but at this point anything would be better than paper diaries.

The adoption of fusion data based on modeling different datasets is a significant development.  Until now, almost all TV ratings have been based on quantifiable data from scientifically selected panels that can be double-checked.  You can go back and see how many people actually pushed People Meter buttons in a market or wrote in particular programs in their diaries.  There’s even a room at Nielsen’s Oldsmar facility where station managers can go to review the individual diaries from their markets and confirm that, yes, this 53-year-old white woman really did watch “The Wheel of Fortune” instead of “Jeopardy.”  You can’t do that when data from set-top boxes are funneled into a computer, fused with other data, and modeled using an algorithm that only a handful of data scientists can understand.

Welcome to the 21^st Century.  I’m sure my former Nielsen colleagues are thrilled they’ve finally convinced the market to phase out diaries, which symbolized the old, analog Nielsen in a digital age.  That’s an image no one wanted.

The new target date for eliminating diaries is now 2018, seven years after the first deadline.  But I’d bet on Nielsen making this one.  Not only is the technology there but so is the marketplace.

 

 

 

• A2M2
• Go Meter
• Nielsen Diaries
• Susan Whiting