What the data tells us about the death of exact match and its impact

On March 17, Google announced an adjustment to the functionality of our beloved match types, effectively changing exact match into semantic match. For example, [new york flights] could potentially show for flights to new york, since the presumed intent is the same.

The change affects two major parts of what we advertisers have historically loved about exact match: preserving word order and keeping out filler, or “function,” words.

I won’t go in-depth as to how the change works, as it’s been outlined by a dozen folks already (reading list to follow). Rather, I’ll dive into the data to assess exactly what’s going on. I looked into a trio of accounts in varying industries that should’ve been affected by the change and outlined the findings below.

For some context, the three accounts we looked at were:

• a home services company. A huge account, broadly built out including geospecific terms and every possible name for each service.
• a national media company. Another massive one, including names of artists for the media that’s being sold and every filler term (e.g., All Out of Love by Air Supply).
• a luxury travel company. A smaller account that has a rock-solid foundation, but plagued by the dreaded Low Search Volume tag when exact match builds are extended.

Here are the four fears I had surrounding the change — let’s see if the data dispels them:

Fear #1: Exact match is dead — long live semantic match

Reality: False. The proportion of close variant impressions is decreasing for extensive accounts.

I reviewed a variety of accounts across industries and sizes, ranging from huge to tiny, from local-focused to national. The proportion of volume for exact match queries vs. close variants actually improved post-change. Woo hoo!

The charts below highlight impression volume only, but the metrics held true for click, spend and conversion volume as well.

Now, before I start the parade, it’s important to remember this change wasn’t necessarily meant for agencies like Elite, who are notorious for insanely complex builds. We already have most of the expected close variants covered based on the style of our accounts, so we may be sheltered from the change. We found that smaller accounts like the travel account noted above were affected more than more robust accounts.

Fear #2: Performance will differ for semantic match

Reality: Yup. Not much worse than close variants used to, but this is worth keeping an eye on.

Performance was always different for close variants vs. pure exact match. Occasionally for the better, but usually… not. Much like the section above, we broke out the same three accounts before and after to see how dramatic the difference was between new and old, and if it’s worth being concerned about.

The two larger accounts saw little to no effect — slightly poorer performance post-change for the large media account and slightly better performance for the large home services account. Our smaller luxury travel account is seeing a pretty big swing here to the tune of… well, really bad. I don’t believe the data is even close to statistical significance quite yet, but you should be starting to see a theme here.

Fear #3: Behavior is more than semantics

Reality: To be continued…

This is the area that I’ve always struggled with the most for close variants. First, there are the obvious areas where pluralities or shifting word order mean something drastically different. Let’s say you’re a good PPC manager, and you structured robust brand campaigns including your site URL. There is a pretty big difference between [www soap com] and [www soaps com], but that’s not something Google will pick up. Same goes for [office trailer], where the person is likely searching for a video and [office trailers] where they’re searching for a temp building.

What we’re starting to see with this new shift is an increase in the types of misspellings that are coming up.

For our little travel company, we target a few fancy-pants hotels around the world — and some interesting behavior is coming up. Targeting [Les Bains Hotel] for example, never used to have any close variants. We’re starting to see it show for what we’ll call accent queries like Le Bain Hotel & Le Ban Hotel – basically typing how they’d be pronounced. Seems like a small enough change, but we’re willing to bet the person spelling like that is far less likely to book a trip to the Louvre. Yes they’re semantically the same, but they could indicate a drastically different customer. A person could be less educated, have lower (or higher) income, be a casual observer or window shopper — the list goes on, but you get the point.

We haven’t seen a dramatic shift in behavior yet, but I’m starting to suspect these are the types of queries that are causing our travel friend to struggle. It makes me pleased to see the final fear has been debunked.

Fear #4: Hyper-segmented accounts will be ignored

Reality: False (at least so far).

With this change, I expected to see a lift in average word count for close variant queries, since the old iteration wouldn’t include filler words like to, from, in and so on. Based on the prior two observations, I think you know what’s coming.

I observed more or less perfect stability in the average query length across larger accounts. Since they were already so robust, there was next to no change in matching behavior. If anything, there was an improvement. Since the smaller account hadn’t been fleshed out as dramatically, the updated exact algorithm had to go fishing a little more and was finding new queries that the existing exact terms weren’t already hitting.

So, is this bad?

It depends.

From my observations, the update is doing what it’s supposed to do. The reach of exact match terms has been extended, granting new volume you weren’t reaching before. That can be good or bad, depending on the industry, linguistic meanings, double entendres and so on. You may find there are a lot more positive terms out there you weren’t covering with your existing build.

For smaller accounts, you may have to do a bit of work on structure and negatives, as the change is clearly having an effect. For larger accounts, years of solid maintenance and best practices mean we’re in great shape.

Bigger picture, this represents a shift away from keywords and toward user-driven marketing, in line with what we’ve seen over the past year. I’ve always been a staunch advocate for rigorous segmentation by intent + performance — the future will be no different. However from my observations, Google’s semantic and intent-driven matching still has a ways to go before I’ll turn the keys over 100 percent.

Further reading

• Inside AdWords: Close variants now connects more people with what they’re looking for
• PPC Hero: Your First Look at the Impact of Google’s Exact Match Update
• Search Engine Land: Seriously, Google, can you just make exact match exact?
• Adalysis: How to Analyze the Exact Match Change

The post What the data tells us about the death of exact match and its impact appeared first on Search Engine Land.