SWEDISH PODCAST MEASUREMENT STANDARDS
The Swedish Podcast Measurement Standard is created by a working group including Kantar, Sveriges Annonsörer, Sveriges Radio, Acast, Bauer Media, Nordic Entertainment Group and Podspace.
The purpose of this standardization is to introduce transparent and consistent podcast audience metrics across podcasts, platforms, publishers and service providers. The standardization offers all podcast stakeholders a defined set of metrics making it possible to track listening consistently over time and to equally compare podcasts and podcast networks.
The standardized methodology is implemented by the group's members in the technical committee and audited by the third-party Kantar Media Audit.
The Swedish Podcast Measurement Standards are harmonizing metrics with IAB Podcast Measurement Technical Guidelines 2.0.
The methodology includes two metrics making it possible to evaluate audience reach and activity across podcasts, platforms and publishers. All filters applying to Listens/Downloads also apply to Reach/Listeners.
The group is in contact with podcast standardization initiatives globally, mainly IAB Tech Lab (US) through IAB Sweden. Technology is in constant development and digital distribution is global, thus the Swedish working group has all intentions to participate globally and if needed adjust these standards over time.
Mattias Björkman - Bauer Media, Chairman of this committee
Fredrik Hermansson - Acast
Robin Calmegård - Nent
Anna Johansson - Sveriges Radio
Edward Jewson - Podspace / Bowtie
Karolina Roslund - Perfect Day Media
Peter Mackhé - Sveriges Annonsörer
Maria Grip - Kantar
Lars Björkman - Kantar
Measurement Standard versions
Version 1 was released 2017-11-15 when Poddindex.se was launched.
Version 2 was released 2018-01-17, in which clarification regarding all filters applying Listens also apply Reach was added. As well as change in duration filtering from 10 sec to 60 seconds harmonizing Reach metric with IAB Podcast Measurement Technical Guideliness 2.0.
For the purpose of this paper Podcast is defined as all editorial audio content that can be consumed On Demand, including audio productions that have been distributed via radio as well as productions exclusively distributed On Demand.
How Listeners Access Podcasts
Podcasts are accessed through a wide range of applications, websites and devices, in this paper called ‘clients’. These clients distribute podcasts directly or through RSS feeds. The RSS protocol makes it possible for a client to subscribe on a podcast updating and consuming it from publishers without further technical integration.
This has led to superior accessibility where publishers can distribute podcasts through a range of platforms, not limited to proprietary clients. As a result apple iTunes-podcaster is the biggest client for podcasts listening in Sweden, pre installed in all iPhones with most podcasts and publishers represented. Listening is however fragmented on various websites, applications and digital appliances, including proprietary publisher platforms such as Sveriges Radio Play (SR), Acast (Acast), RadioPlay (Bauer Media) and I Like Radio (MTG).
Server Side Measurement
The fragmented distribution across multiple clients has led to the development of new measurement technologies where both audience and ad serving is measured through servers streaming the podcast rather than clients consuming it. Server side technology has huge advantages in cross platform measurement and monetization but poses new challenges when it comes to data quality.
Like all digital measurements there is always a risk that content or ads can be requested by non-human traffic. Counting the number of raw requests to the server could thus generate far higher numbers than actual requests made by true listeners. It can easily overstate delivery by a factor of 4x or more. This can be solved using filters making sure traffic is as human as possible.
This methodology include two metrics making it possible to evaluate audience reach and activity across podcasts, platforms and publishers.
Measurement of time consumed is not included in this methodology and not yet a widely used metric for podcast audience measurement. Podcast providers are working on an estimate for time consumed and the inclusion of such a metric must be clearly labeled to differentiate from reach and listens.
2. Measurement of Listens
A listen is a started or downloaded podcast. While the metric aims to capture that a podcast is heard it can't be guaranteed. Compared to technical viewability used in display and video advertising one could say that a podcast downloaded while listening has 100% technical hearability, while a downloaded podcast for later listening has unknown technical hearability. We estimate that 70% of all podcast consumption in Sweden is downloaded during listening. On proprietary publisher platforms over 90% and on iTunes Podcaster between 40-60%.
When measuring podcast downloads and/or starts one cannot look at the raw number of request to the server. Like all digital measurements there is always a risk that request are done by non-human traffic. In server side measurement this traffic is not only created by bots, but also by clients creating several requests for one single listener and play.
In this methodology we address this issue by several filter types. The combination of these filters are making sure traffic is as human as possible.
Filters for progressive download (http GET code 206)
The most common form of accessing a podcast in Sweden is downloading the file during listening. This is called progressive download, and in daily speech streaming. In strict technical terms it’s not true streaming since the connection to the server is not necessarily persistent, but most clients do use progressive download instead of true streaming since it provides the best user experience balancing fast and stable playback.
Duration of 10 seconds in bytes
1 440 000
2 400 000
In addition to duration filtering the methodology must take into account that progressive downloads might produce several requests in one single session. This should be avoided by either only include requests if the range starts at 0, or by making sure only one single request per session is counted.
Podcasts is mainly distributed in the format of MP3-files which may contain additional non audio data referred to as metadata. This additional data include basic information such as title, publisher, and publishing date but are versatile and can also include larger seized information such as images. The size of such metadata might disturb the duration filter and should be excluded when calculating duration from bytes.
Filter for Download (http GET code 200)
Another way of accessing podcasts is by downloading the file for later listening, simply called downloading. Downloading is widely used in countries with limited or expensive mobile data, since users want to download content before they go out of home. This behaviour has been in steady decline in Sweden which probably derives from the commoditization of mobile data.
It is true that some downloaded episodes will go unplayed, while some are played multiple times. It is however important to note that most clients with subscription features (iTunes Podcaster in particular) won’t download new episodes from a subscribed podcast if the user haven’t played that podcast recently. Most clients learn the user behaviour since it saves data costs for users as well as clients.
Filter for Frequency (http GET code 206 and http GET code 200)
Besides having filters for duration or amount of data downloaded, there should also be a frequency filter limiting the number of listens being generated from a unique identifier over a given time on a single file.
The purpose of the frequency filter is to create an accurate limitation of listens where bots, auto downloading, and spamming are excluded. The limitation is a balance between gaining as accurate data as possible while not blocking legitimate requests from listeners, especially listener sharing public IP addresses which do affect the frequency when the unique identifier is calculated from unique combinations of IP and User Agent.
3. Measurement of Reach
Reach is an estimate of the number of unique listeners during a given time on a given podcast or a given set of podcasts. While the metric aims to capture unique individuals it is, with the exception of user id, limited to unique clients.
The reach metric is calculated as a unique listen. As such all filters applied to listens also apply to reach.
Due to the fragmented consumption of podcasts the identification of uniqueness vary depending on client. These identifiers can be used:
Unique identification through UserID, DeviceID, and Cookie is similar to most digital client side measurements, for example KIA-index. The unique combination of IP and User Agent is a server side technology and is in this methodology the minimum requirement for unique identification.
Technologies for unique identification is constantly evolving and podcast providers are working on an improved server side identification using new attributes and technologies. Any improvements and/or changes to this should be transparently communicated and accounted for making sure it is not overstating the unique reach in relation to above standards.
4. Measurement of Ad Delivery
The scope of this standardisation is limited to metrics for audience measurement. In it’s essence the methodology for podcast audience measurement do however also apply for podcast audio advertising, although some differences are worth noting.
Podcast audio advertising is delivered and measured through two main technologies.
Ads integrated in Content
These ads are integrated in the episode file and thus measured through the audience metrics described in this paper.
This technique is now seldom used for ad delivery, but more often used in editorial partnerships and native content.
Ads Dynamically Inserted in Content
These ads are inserted when the request is made and thus delivered similar to other streamed media in the form of pre-, mid- or post-rolls. They are delivered either by the server or the client stitching ads to the requested file.
In this technique advertising- and campaign reach should be measured through the same methodology as described in this paper. A campaign's podcast reach should thus be defined as the number of unique identifiers the podcast ad has been served to during a given time.
The main metric and often basis for pricing (CPM) of dynamic ad delivery is impressions. Although impressions are similar to listens they have somewhat different attributes, mainly that several impressions can be delivered in one listen (e.g. one pre-roll and mid-roll). To achieve accuracy the Impression tracking should include duration filtering as well as frequency filtering described in this document. Details and values in such filtering can however differ due to ad duration, capping- and frequency rules set by the publisher.
In similarity to all digital advertising it’s recommended to ask the podcast provider or publisher which ad system is used and to include third party ad tracking.
Following publishers and/or service providers have implemented the standardized methodology in the following manners;
NORDIC ENTERTAINMENT GROUP
In order to achieve higher transparency and quality assurance, a Sample Audit model is implemented for Poddindex-connected service providers.
The standardized methodology described in this paper is based on server data and thus integrated with podcast distribution. Podcast platforms and publishers use different systems for distribution which means that there might be slight variations in how the methodology is implemented between service providers.
The very purpose of the standardization is to introduce transparent and consistent podcast audience metrics. Therefore, each certified service provider specifies what system is being used and how the standardization is implemented, see appendix 1. Each system is audited by Kantar Media Audit and are only approved if the results are in line with the predicted outcome. Acceptable discrepency is 10%.
The audit concept is called Sample Audit since it is based on random sampling of log files, provided to Kantar Media Audit by the service providers.
Process of audit
The service providers deliver server logs according to established requirements specification, aggregated per week, to Kantar Sifo AB, on request 3 to 4 times per year.
The log files form the basis for a selection of podcasts to review. When a selection of podcasts has been reviewed, the service provider will be notified about the result of the audit. The log files are delivered through an SFTP owned by Kantar Sweden. Analysis of the log files are made in collaboration between Kantar Sweden and Kantar Norway. Results are shared with the service providers individually and are not to be published by Kantar.
Kantar Media Audit will request data for a specific week for all podcasts per service provider. The weeks thus become random over the year, but the same for all service providers.
Approximately 8 - 20 podcasts per service provider and year will be selected by Kantar Media Audit for sample audit.
The service providers deliver server logs according to this requirements specification.
Log file requiered fields and format
To comply with the measurement functionality the log file needs to contain the following fields in the exact order as stated:
|Timestamp (incl. date)||http status code||http method||IP-address||user agent||URL||bytes served||RSS||bitrate|
Functions of each field:
Example of content in logfile:
|Timestamp||http status code||http method||IP-address||user agent||URL||bytes||RSS||bitrate|
|2020-10-13T21:07:02+00:00||206||GET||872348273489||AppleCoreMedia/18.104.22.168A393 (iPhone; U; CPU OS 14_0_1 like Mac OS X; nb_no)||/suzykonnyaltelleringenting/suzy-konny-10-10-20-klipp-v1.mp3||1100||https://feed.pod.space/suzykonnyaltelleringenting||128|
|2020-10-13 21:07:02+00:00||206||GET||387484814198||CastBox//8.5 (fm.castbox.audiobook.radio.podcast; build:15; iOS 12.4.1)||1406/trailer_domkyrkokuppen_b7f06149_normal.mp3||1100||https://rss.podplaystudio.com/1406.xml||128|
Appendix 1 - Implementation until Spring 2020
Since Sveriges Radio is a non-commercial organization, an exception was given to deviate from the methodology as long as deviations are described and actions are taken to create comparable results in the audit process.