CHAD OTTO: How good measures become corruptible targets
Avoid giving agencies targets and ultimatums over something as corruptible as a digital metric
30 March 2023 - 04:58
byChad Otto
Support our award-winning journalism. The Premium package (digital only) is R30 for the first month and thereafter you pay R129 p/m now ad-free for all subscribers.
When the British government arrived in its new colony of India, it was uneasy about the number of venomous cobras slithering about.
So they offered locals a bounty for every dead cobra turned in. This strategy turned out to be very successful as they were bringing in large numbers of snakes in exchange for money.
However, the government started to wonder why the wild cobra population wasn’t diminishing. A bit of detective work revealed that enterprising locals were breeding cobras for profit. So the programme was scrapped, and the now worthless snakes were released into the wild — which worsened the cobra problem.
This anecdote encapsulates Goodhart’s Law. Named after the British economist, Charles Goodhart, the law states: “When a measure becomes a target, it ceases to be a good measure.” In other words, people are likely to game the system when a metric (a measurement for success) becomes a target — rendering the metric useless.
It’s like a health insurance company rewarding members for going to the gym rather than exercising, people just end up swiping their membership cards on the way to the pub.
Perhaps you’ve even seen this happening in your agency when you start prioritising certain digital metrics at the expense of others.
One of the strengths of digital media, as the digital evangelists will tell you ad nauseam, is the ability to track every possible action with a corresponding metric. Being able to scrutinise the minutia of the target market’s behaviour when a new campaign goes live, gives marketers the chance to peek behind the curtain and better understand what may or may not resonate with their online audience. Something that can’t be done with a billboard on the side of a highway.
But what happens when these helpful metrics go from useful guardrails to enforced targets? Metrics are indicators of performance, they can keep campaigns on track or help rectify an issue before it gets too late. Metrics highlight which tactics are good and which ones are bad. And when combined, they should all help marketers meet their overarching campaign objectives.
Giving agencies set-in-stone targets for the array of digital metrics, such as cost per mille, cost per click, and cost per reach (reach, not resuscitation — though sometimes it does feel like the latter would help) doesn’t lead to greater campaign success. All it does is shift the focus, forcing agencies to prioritise metrics over objectives (practical, overarching targets).
Thus, striving for short-term wins at the expense of long-term progress. And most disturbingly, it encourages corrupt behaviour such as ad fraud and data manipulation. Cue the cobras!
Goodhart’s law has played out in many forms since the rise of digital media. Opportune entrepreneurs simply swapped the cobras for clicks but they were farming them for profit just the same.
Though not as prominent any more, click farms (businesses that exchange social media engagement for money) illustrate how far some digital agencies will go to reach a target.
Under pressure from their clients, agencies would simply outsource part of the required clicks, views and comments to a click farm in Bangladesh, and guess what? They met, if not exceeded, their targets — every time. When last did you see a report where an agency did not exceed the target?
Happy client, happy agency. Win-win, right? Well, no. Not if the objective of the campaign was to tangibly move the needle by recruiting new consumers and generating more revenue, not just getting clicks from Asia.
Luckily, click farms are out of vogue and don’t play a part in digital marketing any more. Instead, the humans were replaced by bots and the fraud was automated for a fraction of the price.
It’s irrelevant whether the fraud is an unintentional product of human negligence or if it’s a result of an unscrupulous sector wanting more profit. What matters is that it’s widespread. A whitepaper released by Ad Age in 2020 estimates that ad fraud is stealing 20% of global ad spending. That equates to $66bn disappearing annually.
Yes, this is still happening, despite the digital media industry preaching about all the amazing anti-fraud tools they are using and charging for.
A recent instance of widespread ad fraud was published in Wall Street Journal in May 2022. Titled “Ad-Tech firms didn’t sound alarm on false information in Gannett’s Ad Auctions”, the article claims that “publisher Gannett Co. (owner of multiple news sites) provided inaccurate data to advertisers for more than nine months”.
In doing so, the “publisher misrepresented where billions of ads were placed”. Brands affected include Nike, Adidas and Ford with some ads running on a biweekly news site in Mexico rather than the flagship site, USA Today, as intended.
This was all going on despite the implementation of brand safety tools such as Integral Ad Science and DoubleVerify. Both firms were paid to provide web traffic insights and validate the ads appeared where they were meant to. However, they declared nothing to the publisher or the advertisers. Sort of like the US credit ratings agencies that failed to warn investors and thus prevent the 2008 financial collapse.
So, what’s the solution? As Stephen Covey, author of 7 Habits of Highly Effective People, says: “Keep the main thing the main thing.”
• Use your metrics to measure how effective your tactics are. Reference historical campaigns as a benchmark so the number in front of the metric has context. This way you can appropriately interrogate or congratulate your agency.
• With the primary objective in mind, see how the various digital tactics are contributing to the bigger picture. The strength of digital media is the ability to change tack and iterate regularly. This flexibility should be exploited for the betterment of the campaign, rather than doubling down on a weak tactic to reach a target.
• Understand the relationship between long-term and short-term marketing activities. According to Les Binet and Peter Field in their influential book, The Long and The Short of it, digital KPIs have become popular because they are easy to gather and affect. It’s more difficult to draw a correlation between different touch points and overall market share growth. This usually requires complex econometrics — which is why the former is preferred: it’s easier, and lazier.
• Avoid giving agencies targets and ultimatums over something as corruptible as a digital metric. Or the snake might just come back to bite you.
• Otto is digital strategy lead at Grey Advertising.
Support our award-winning journalism. The Premium package (digital only) is R30 for the first month and thereafter you pay R129 p/m now ad-free for all subscribers.
cobras TO CLICK FARMS
CHAD OTTO: How good measures become corruptible targets
Avoid giving agencies targets and ultimatums over something as corruptible as a digital metric
When the British government arrived in its new colony of India, it was uneasy about the number of venomous cobras slithering about.
So they offered locals a bounty for every dead cobra turned in. This strategy turned out to be very successful as they were bringing in large numbers of snakes in exchange for money.
However, the government started to wonder why the wild cobra population wasn’t diminishing. A bit of detective work revealed that enterprising locals were breeding cobras for profit. So the programme was scrapped, and the now worthless snakes were released into the wild — which worsened the cobra problem.
This anecdote encapsulates Goodhart’s Law. Named after the British economist, Charles Goodhart, the law states: “When a measure becomes a target, it ceases to be a good measure.” In other words, people are likely to game the system when a metric (a measurement for success) becomes a target — rendering the metric useless.
It’s like a health insurance company rewarding members for going to the gym rather than exercising, people just end up swiping their membership cards on the way to the pub.
Perhaps you’ve even seen this happening in your agency when you start prioritising certain digital metrics at the expense of others.
One of the strengths of digital media, as the digital evangelists will tell you ad nauseam, is the ability to track every possible action with a corresponding metric. Being able to scrutinise the minutia of the target market’s behaviour when a new campaign goes live, gives marketers the chance to peek behind the curtain and better understand what may or may not resonate with their online audience. Something that can’t be done with a billboard on the side of a highway.
But what happens when these helpful metrics go from useful guardrails to enforced targets? Metrics are indicators of performance, they can keep campaigns on track or help rectify an issue before it gets too late. Metrics highlight which tactics are good and which ones are bad. And when combined, they should all help marketers meet their overarching campaign objectives.
Giving agencies set-in-stone targets for the array of digital metrics, such as cost per mille, cost per click, and cost per reach (reach, not resuscitation — though sometimes it does feel like the latter would help) doesn’t lead to greater campaign success. All it does is shift the focus, forcing agencies to prioritise metrics over objectives (practical, overarching targets).
Thus, striving for short-term wins at the expense of long-term progress. And most disturbingly, it encourages corrupt behaviour such as ad fraud and data manipulation. Cue the cobras!
Goodhart’s law has played out in many forms since the rise of digital media. Opportune entrepreneurs simply swapped the cobras for clicks but they were farming them for profit just the same.
Though not as prominent any more, click farms (businesses that exchange social media engagement for money) illustrate how far some digital agencies will go to reach a target.
Under pressure from their clients, agencies would simply outsource part of the required clicks, views and comments to a click farm in Bangladesh, and guess what? They met, if not exceeded, their targets — every time. When last did you see a report where an agency did not exceed the target?
Happy client, happy agency. Win-win, right? Well, no. Not if the objective of the campaign was to tangibly move the needle by recruiting new consumers and generating more revenue, not just getting clicks from Asia.
Luckily, click farms are out of vogue and don’t play a part in digital marketing any more. Instead, the humans were replaced by bots and the fraud was automated for a fraction of the price.
It’s irrelevant whether the fraud is an unintentional product of human negligence or if it’s a result of an unscrupulous sector wanting more profit. What matters is that it’s widespread. A whitepaper released by Ad Age in 2020 estimates that ad fraud is stealing 20% of global ad spending. That equates to $66bn disappearing annually.
Yes, this is still happening, despite the digital media industry preaching about all the amazing anti-fraud tools they are using and charging for.
A recent instance of widespread ad fraud was published in Wall Street Journal in May 2022. Titled “Ad-Tech firms didn’t sound alarm on false information in Gannett’s Ad Auctions”, the article claims that “publisher Gannett Co. (owner of multiple news sites) provided inaccurate data to advertisers for more than nine months”.
In doing so, the “publisher misrepresented where billions of ads were placed”. Brands affected include Nike, Adidas and Ford with some ads running on a biweekly news site in Mexico rather than the flagship site, USA Today, as intended.
This was all going on despite the implementation of brand safety tools such as Integral Ad Science and DoubleVerify. Both firms were paid to provide web traffic insights and validate the ads appeared where they were meant to. However, they declared nothing to the publisher or the advertisers. Sort of like the US credit ratings agencies that failed to warn investors and thus prevent the 2008 financial collapse.
So, what’s the solution? As Stephen Covey, author of 7 Habits of Highly Effective People, says: “Keep the main thing the main thing.”
• Use your metrics to measure how effective your tactics are. Reference historical campaigns as a benchmark so the number in front of the metric has context. This way you can appropriately interrogate or congratulate your agency.
• With the primary objective in mind, see how the various digital tactics are contributing to the bigger picture. The strength of digital media is the ability to change tack and iterate regularly. This flexibility should be exploited for the betterment of the campaign, rather than doubling down on a weak tactic to reach a target.
• Understand the relationship between long-term and short-term marketing activities. According to Les Binet and Peter Field in their influential book, The Long and The Short of it, digital KPIs have become popular because they are easy to gather and affect. It’s more difficult to draw a correlation between different touch points and overall market share growth. This usually requires complex econometrics — which is why the former is preferred: it’s easier, and lazier.
• Avoid giving agencies targets and ultimatums over something as corruptible as a digital metric. Or the snake might just come back to bite you.
• Otto is digital strategy lead at Grey Advertising.
Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.
Please read our Comment Policy before commenting.
Most Read
Related Articles
Will ChatGPT kill the search engine?
Retail media networks: the next big thing in digital advertising
Women in their right place
Published by Arena Holdings and distributed with the Financial Mail on the last Thursday of every month except December and January.