Measuring Subscriber Retention Is Harder Than it Looks

To win over subscribers, publishers have leaned into a myriad of offerings, from apps to Substack to print packages. But that diversification introduces a new challenge: accurately measuring subscriber retention.

As page views, email open rates and clicks lose relevance in this new age of media, publishers need better ways to understand what keeps subscribers engaged—and, ultimately, what keeps them paying. Yet measuring retention, and identifying what actually drives it, remains more art than science.

A survey from mobile app platform Pugpig found that publishers regard subscriber retention as a very important metric in understanding app performance—including in-app and direct subs. Of the 50 publishers surveyed, 32% said they measured retention ad hoc, while 16% had no capability for this measurement.Subscriber and registration flows topped the list of app capabilities that publishers most wanted to improve on this year.

“Most publishers can calculate basic retention and churn, but turning that into a nuanced, cross‑channel understanding of why subscribers stay or go is still a widespread challenge,” James Kember, head of consulting at Pugpig, told AMO in an email. “The increasing importance of apps, newsletters and off‑platform discovery means subscriber journeys are more fragmented and many organisations are held back by siloed data and inconsistent measurement frameworks.”

A dashboard from nonprofit organization LION, which supports local and independent media organizations, shows that of 438 media organizations, only 22% have the means to track loyal users. This increases to 91% when the data focuses on the “sustaining” businesses, which are those that have reached a level of viability across LION’s three pillars of sustainability (operational resilience, financial health and journalistic impact). However, those “sustaining” businesses only make up 2.5% of their data, or about 11.

Greg Piechota, researcher-in-residence at the International News Media Association (INMA), has predominantly worked with large commercial publishers with substantial digital subscription businesses and notes it’s not the measurement of retention that trips up major publishers, but rather how they approach improving it. The biggest mistake is treating retention as a lever rather than as an outcome, he said.

“You cannot directly manipulate an outcome; you can only influence the inputs that produce it, such as subscriber engagement, onboarding, payment success or customer experience,” he told AMO in an email.

How Do Publishers Measure Retention?

Subscriber retention is a data point that the business side of the Wall Street Journal measures, said Tess Jeffers, the Wall Street Journal’s head of newsroom AI and data. Then her team breaks down the points into smaller building blocks that are within the newsroom’s control.

“People will cancel their Netflix and HBO accounts because things are getting expensive out there—that’s just one of a myriad of reasons for why retention numbers might shift over time,” Jeffers told AMO. “And that doesn’t necessarily have all that much to do with anything we can control as a business or certainly not in the news.”

In the Journal’s newsroom, the north star is being audience-first with a focus on time spent, which is increasingly pivoting toward “time well spent.”

“Audiences have a cap—they’re not going to spend 10 hours a day with us,” Jeffers said. “People turn to the Wall Street Journal to make them smarter, to get them in the know, to tell them something exclusive or distinctive. And so rather than trying to push time spent with Wall Street Journal further, further, further — well, obviously we’d like that — now we’re thinking about those macro strategic initiatives where we can make people smarter faster.”

The Journal is also focused on measuring visits, particularly where readers haven’t engaged with an additional article, video or podcast, with the hypothesis that it’s a less valuable visit for that user, Jeffers said. “We want to move the needle on that metric so that they’re always finding something [additional] every time they come to us that proves the value of that Wall Street Journal experience,” she said.

Across the pond, UK publisher Times Media, which includes The Times, The Sunday Times and Times Radio, is measuring subscriber retention several ways because it’s so complex, with no one metric providing the full picture, Tracy Yaverbaun, general manager of Times Media, told AMO in an email.

“We look at retention curves, percentage of a cohort still subscribed after month one, three, six, 12, which is more useful than a single churn figure because it shows when people drop off, not just how many,” Yaverbaun said. “We also do a lot of cohort analysis, so comparing retention across acquisition channels or sign-up periods.”

Measuring retention by subscriber cohorts — which are groups that share common characteristics like “millennials” or “super users,” — acquisition source, offer, tenure, or customer segment—is fairly common among larger publishers, INMA’s Piechota said. The challenge is understanding the causality, he said, like why do some cohorts retain better than others? Which operational changes actually improve retention?

What’s a Good Retention Level?

At Dow Jones, subscription retention varies based on factors such as subscription, tenure, economy and news cycle, a company spokesperson said. Retention rates are expected to remain equal to or better than forecast levels, they said. And as data insights have improved, so too has retention, they said.

This has been the case for local news publisher Lookout Local, which has found that a good retention strategy requires tracking and knowledge, Ken Doctor, founder of Lookout Local, told AMO in an email. It uses a customized reader revenue system and reporting tools, with the data then exported and run through custom tools built in-house, he said.

One of the reasons publishers tend to struggle with measuring retention within apps is that the technical and data infrastructure required is quite involved, Pugpig’s Kember said. “App data needs to be matched to subscriber data, then retention needs to be modeled in a meaningful way,” he said. “That requires reliable IDs across platforms, a joined-up data architecture and often a single customer view.”

The single customer view tends to be the enabling step to make retention analysis more actionable, and then publishers can understand what app behaviors, content patterns and engagement signals correlate with better retention outcomes, he said.

Lookout’s newest publication, the Eugene-Springfield launched some 16 months ago, has retention of 92% with annual members and retention of 97% with monthlies, Doctor said. That was helped with Newspack, which created membership programs with prices that made sense.

“Pricing is hugely important and underused by digital news publishers,” Doctor said.

Times Media didn’t disclose a singular retention figure due to the variation between segments, but noted that getting a subscriber past the first year is where built habits settle, which stabilizes retention rates.

Focusing on the churn rate of the headline figure can be easy to misread in isolation, Yaverbaun said. “You could have the best subscriber retention rates because everyone is dormant and not realizing they have a subscription,” she said.

Low churn can sometimes indicate that a publisher isn’t acquiring enough new subscribers, as new subscribers churn at higher rates than long-standing ones, Piechota said. Businesses growing aggressively often accept somewhat higher churn as a consequence of healthy acquisition, he said.

That doesn’t mean publishers should forget about retention because improving that increases subscriber lifetime value and profitability, Piechota said. It’s just that acquisition and retention should be balanced rather than looked at in isolation.

Improving Retention

The Journal is improving retention by doing more initiatives around free registration, which warm up the pipeline of prospective audiences and help with tracking of reader experiences, Jeffers said. It’s also a strategy that Fortune has been implementing.

Times Media has increased its offerings beyond the core product to drive retention—including adding “bonus accounts,” which enable a subscriber to share the Times with three additional friends and family members, new puzzles and games, and enhanced audio narration of articles, Yaverbaun said.

“We categorize these as High Value Actions (“HVA”) which correlate best with retention and LTV—these form the core of our in life engagement and retention activities,” Yaverbaun said. “In particular, we focus on a subscriber’s early life (first month, and even first 7 days) as when to drive habit—we know if we get that right early it compounds in terms of benefit over time.”

These types of changes can play directly into leading indicators—those create the opportunity to intervene before subscribers become disengaged and are typically monitored weekly or daily, Piechota said. Churn and acquisition rates are typically reviewed monthly, but for smaller publishers, ad hoc reviews can be reasonable because the numbers move slowly and resources are limited, he said.

Engagement-based leading indicators are incredibly important to Times Media, which obsesses over the subscriber daily return rate and cohort analysis. The Journal is also getting smarter about, and is prioritizing, cohorts.

“That’s a really foundational shift for us as well, especially as it comes to retention and growth of understanding the diversity and variety of what our different audiences need from us,” Vivyan Tran, the Journal’s head of audience strategy, told AMO. “That’s what we’re hoping will really move the needle on some of these things.”

Story updated to reflect Kember’s new title.

The post appeared first on A Media Operator.