Generative AI comes to Google Search
On May 14, 2024, Google introduced new Artificial Intelligence (AI) features into their search engine for users in the US. We analyzed the network of Parse.ly data and concluded that the impact for publishers isn’t the Doomsday event that some had predicted. At least, not yet.
Under the banner “Let Google do the searching for you,” AI Overviews is a new feature that produces AI-generated summaries for some—but not all—search queries.
For publishers, it can be a source of uncertainty. If users answer their question with the Overview, will they ever click through to the sites? If they don’t, what happens to the valuable traffic that’s the backbone of these publishers’ businesses?
The significance of this potential problem cannot be overstated—for many sites, Google is the source of more than 35% of their web traffic.
To get clarity for digital publishers, the Parse.ly team looked at the past 20 months of data for unique patterns in traffic and Google referrals. This analysis included thousands of sites, and limited analysis to ones that have received steady traffic over the entire 20-month period.
The investigation unearthed very key findings in traffic patterns overall, despite not finding any major impacts in the AI Search question that prompted the inquiry.
“The reports of my death are greatly exaggerated”
SEO
Google maintains a slow and steady climb
For publishers, Parse.ly data showed very little impact from AI search. Instead, Parse.ly data revealed gradual, long-term trends that predated the May 14 search changes and a continuation of those trends after that date. In fact, we saw very little effect due to specific Google changes.
We specifically looked at Google referral traffic changes surrounding three key time frames: May 14, 2024, when AI changes were introduced; March 5 to April 19, 2024, when Google rolled out the March “Helpful Content” core update; and Oct. 5 to 19, 2023, when Google implemented the October core update.
We divided sites in various ways—by high and low traffic, by content area, and by publisher. We looked at overall traffic and Google referral traffic, both as raw numbers and as a percent of total traffic. We compared Google referral traffic to referral traffic from other sources.
We found preexisting trends over a 20-month period. We didn’t see activity associated with key dates when Google implemented important changes. When there was any variability, it was consistent with the variability over the 20-month period.
The trends Parse.ly data did show were eye-opening to us. Here’s a summary.
Overall traffic has declined: Busiest sites feel brunt, largest publishers insulated
Parse.ly data showed that overall traffic across the network declined 14% over the past 20 months. It was worse for high-traffic sites, which saw a 15% decline in traffic.
Traffic went from a high of 502K average weekly pageviews per site in the last week of 2022 to 432K pageviews by August 2024.
Then we grouped sites into quartiles, four equal-sized groups, based on traffic volume. The highest traffic sites went from a high of 1.9M to 1.6M average weekly pageviews.
We also looked at our largest publishers, who tend to own multiple sites, often with similar or complementary content.
Here, we didn’t see the same drop in traffic in the highest-traffic group of publishers. Instead, we saw stable numbers across the entire period.
Our largest publishers are also frequently the customers with the highest traffic sites.
The discrepancy between our highest traffic sites being hit the hardest by traffic declines, while also belonging to publisher networks that have shown the most resilience in their traffic numbers, perhaps follows consolidation trends occurring across the media industry.
In other words, these businesses may have acquired new sites to bolster overall network numbers, or created robust strategies to share traffic across their network from one site to another. In any case, they’ve managed to insulate themselves against falling traffic.
Google referral traffic rises
While traffic dipped, the percentage of traffic across the Parse.ly network from Google referrals increased by 8%.
Google referral traffic started out at 720M pageviews and increased to 776M pageviews by the second week in August.
In terms of overall network traffic, Parse.ly saw a slow, steady trend across 20 months, with the average site receiving an extra 11K pageviews per week from Google.
Looking only at the quartile of sites with the highest traffic, we saw an extra 39k average weekly pageviews delivered by Google.
We also looked to see if the trend held for our publishers who own multiple sites. And it did for our highest-traffic publishers. They saw increased Google referral traffic in raw numbers while also seeing stable overall traffic.
For our second-highest group of publishers, things looked different. These are publishers with fewer sites or sites with moderate traffic. They saw a decline in overall traffic by 7%, including a decline in Google-referred traffic by about 12%. This was part of a long-term, stable trend and didn’t seem associated with changes at specific dates.
Different search patterns for different content
Parse.ly looked for Google AI-related changes in traffic based on the type of content publishers produced. Here, we didn’t find major changes.
We grouped sites by publisher and then assigned them to a group based on the main type of content they produced.
The groups: non-media corporate branding sites, news, business, sports, culture (a bucket that includes entertainment, general technology, lifestyle, and similar publications), and industry-specific publications. The last group includes things like news organizations dedicated specifically to the legal profession, to the real estate industry, to agriculture, or other similar niche areas aimed specifically at professionals within the corresponding industry.
We didn’t see large changes related to traffic or Google referrals around May 14, when Google implemented AI changes. Instead, we saw the continuation of slow, steady evolution.
Average traffic was steady across most sectors, but Google-referred traffic increased slightly, both in raw numbers and as a percent of total traffic.
Sports traffic was the exception, as it appeared to be more driven by extreme, external, seasonal events.
We looked at the average Google-referred traffic per publisher in terms of total pageviews, as well as a percentage of total traffic.
The sectors showed different traffic volumes, and these were fairly flat (except for sports).
The highest average volume of Google-referred traffic went to publishers with lifestyle and culture content (3.0M pageviews per publisher per week). The lowest was for corporate branding sites (0.2M pageviews per publisher per week).
The content sectors relied on Google referrals to differing degrees.
The highest reliance on Google was for culture and lifestyle sites (47% of total traffic), and the lowest was for business news sites (21% of total traffic).
The percentage of Google-referred traffic increased slightly across most sectors, which is consistent with data we saw for sites overall.
Methodology
Upon a visit to a Parse.ly-tagged page, a code bundle is downloaded from Parse.ly’s global content delivery network. This code bundle retrieves information about the visit to that page, such as pageview and time spent. This data is recorded in our logs.
We looked at the daily logs for all the sites that used Parse.ly over the past 20 months (between Dec. 26, 2022 and Aug. 18, 2024), excluding sites that opted out of our Network Data studies. We also excluded clients with incomplete data over the past six months. This includes clients who left the platform and data from test sites. Parse.ly automatically removes all bot and spider traffic.
We tried different groupings to look for trends. We grouped sites into quartiles based on traffic volume. We grouped sites by publisher and publishers by quartile, again based on traffic. Finally, we grouped publishers based on their main content areas: news, business news, sports, lifestyle and culture, industry-specific news, and corporate branding sites.
Our study includes weeks when Google made important changes to search.
2023
- March. 15: Google March 2023 core update
- Aug. 22-Sept. 7: Google August 2023 core update
- Sept. 14-28: Google September 2023 helpful content update
- Oct. 5-19: Google October 2023 core update
- Nov. 2-19: Google November 2023 core update
2024
- March 5-April 19, 2024: Google March 2024 core update
- May 14, 2024: Google rolls out AI Overviews in the US
The AI changes in May were our focus.
Bing now more relevant than Twitter
Parse.ly expected to see a slow decline in referrals from Twitter. But we found a steep decline—with Microsoft Bing now referring more site traffic than the troubled social media platform.
X, formerly Twitter, has been undergoing deep platform changes (see below) since being acquired in April 2022. These changes were accompanied by a marked decrease in referral traffic from X, especially for news sites.
Our data shows the same pattern, with all X referral traffic dropping from a high of 26M to 14M over the past year and a half. If we look at news publishers, the drop is just as dramatic, from an average of 141K pageviews per publisher per week to 88K. The largest drop was for industry-specific publications who saw a 62% drop, from an average 6K to 2K pageviews per week.
Since 2022, global usage of X has declined by 23%. X has also instituted changes that de-emphasize referrals generally, and news coverage specifically. In October 2023, X removed headlines from the platform. They also removed the “verified” blue check mark from journalists who don’t pay for it, another policy change.
Meanwhile, Bing continues to deliver steady traffic, and since January 2024, more than X. With Bing traffic holding steady and X traffic declining, we expect the gap to widen.
Social media referral traffic declines by half
The decline in traffic referrals isn’t limited to X. It’s across other social media platforms as well.
In October, the New York Times described a new rift between social media platforms and publishers, reporting a decline in the percentage of traffic referrals from social media platforms that have changed their business models. Social media-referred traffic went from 12% of total web traffic to about 7% over the previous three years.
Parse.ly data showed a similar trend on a condensed 20-month timeline.
We saw the percentage of Facebook-referred traffic decline from a high of 14% in spring 2023 to about 6% in August.
As reported above, referrals from X were already low, but by August, driving just 0.6% of traffic, down from 1% at the end of 2022. This is also a drop of almost half.
Other social networks (not shown) provided even less traffic.
A final word for publishers
What is the status of search, and how can Parse.ly data help publishers navigate the current landscape?
We examined traffic across the entire Parse.ly network over the past 20 months, looking for changes in overall traffic and Google referral traffic after Google implemented AI Overviews.
We didn’t find changes in traffic associated with AI Overviews. Instead, we found preexisting, long-term trends.
In summary, here are the key takeaways from our research:
- There’s been very little impact from AI search so far.
- Overall traffic continues to decline.
- Large publishers are successfully insulating themselves from this decline in traffic.
- Google-referred traffic continues to increase, in raw numbers and as a percent of total traffic.
- Facebook and Twitter traffic continue to decline.
With all this in mind, Parse.ly suggests that publishers may want to emphasize direct relationships with their readers, building a platform-agnostic business that is resilient against changes in algorithms from any of the major referral platforms.