If you have an agent ready to rewrite product titles and descriptions across the catalog, that's a good use of one. Before it runs, save what your search traffic looks like today, because that traffic is shifting on its own.
Researchers at the Indian School of Business and Carnegie Mellon ran a randomized field experiment early this year, covered by Search Engine Journal in April. On queries where Google showed an AI Overview, organic clicks fell 38% compared with users who had the overviews hidden. The paper is a draft and the sample was about a thousand desktop users, so treat the size of the effect with care.
So if your organic clicks drop after an agent rewrites your product pages, was that the rewrite or the results page? Without a baseline and a control group, you won't know.
Search Console blends AI traffic in
Google added AI Mode to Search Console in June 2025. Its clicks, impressions and positions count toward the Web totals in the Performance report, and the report doesn't show them separately. A change in your numbers could come from classic results, AI Overviews or AI Mode, and the report won't say which, so keep your own records.
What to save
Export, by page and by query, the last sixteen months of Search Console performance data. Search Console's Performance report holds sixteen months of data, and a year-on-year comparison for the holidays needs last November in hand.
Group the pages the way you'll make changes, usually by product category. For each group, keep clicks, impressions, click-through rate and average position by week.
From analytics, keep conversions and revenue from organic search for the same groups. A rewrite can change traffic without changing sales, or the other way round.
Hold some pages back
Pick a group of pages that you won't change: similar products in the same categories, with similar traffic. This is your control.
When the agent's changes go live on the rest, compare both groups week by week and watch the gap between them, which is the rewrite's effect.
Change in batches and keep a log
Release changes one category at a time, with a date for each batch, and keep a log of exactly what changed on which page. Have the agent write that log as it goes, with the old and new text for each field and the date it went live.
With the log you can tie a movement in the numbers to a specific batch, and roll back a batch that hurts from the record.
Leave room before the holidays
Give each batch a few weeks to show up in the numbers. If the rewrite goes live in late September, you'll have a few weeks of data before holiday traffic arrives in November. That's enough to see whether the first batches helped, and to pause the rest if they didn't.
Frequently asked questions
Why record a search baseline before changing product pages?
Organic clicks are shifting for reasons outside your control, including AI answers on the results page. Without a saved baseline and a control group, you can't separate the effect of your changes from the market's movement.
Can Search Console separate AI Mode traffic from regular search?
No. Since June 2025, AI Mode clicks, impressions and positions count toward the Web totals in the Performance report, which doesn't show them separately.
How long does Search Console keep data?
Sixteen months for the Performance report, which Google announced when it introduced the current Search Console. Export what you need before it ages out, because a year-on-year comparison needs last year's numbers.
What is a holdout group for SEO changes?
A set of comparable pages you deliberately leave unchanged while you edit the rest. Compare the two groups week by week, and the difference between them is what your changes did.