We turned to Econsultancy’s storied roster of search experts to ask: what do MUM and multimodal search mean for our relationship to search and SEO? How could they change the game for marketers? And what potential opportunities and challenges do they present?
At Google’s 2021 Search On event, the search giant shared an early look at the future of multimodal search – search that combines different methods to get more relevant results.
Search on the web has historically been mostly text-based, with searchers using words to relay what they wanted to find. Even when technology advanced enough to make innovations like visual search possible, searchers would still have to pick a single method to carry out their search: either visual input, or text input, which limits the amount of nuance that can be conveyed.
But thanks to Google’s Multitask Unified Model (or MUM) technology, it will soon be possible to combine different modes of search to obtain a more relevant result. For example, Google’s VP of Search, Pandu Nayak, showed how searchers could use an image of someone wearing a colourfully patterned shirt and combine it with the words, ‘Socks with this pattern,’ to bring up socks with a similar look. This is something that would not be possible with text or visuals alone.
Google’s VP of Search Pandu Nayak demonstrates Google’s new multimodal search capabilities at Search On 2021. Image: Google
Google has also shared other ways that MUM will enhance the search experience, such as by applying understanding of how users typically interact with a topic to display “deeper insights” – such as surfacing how-to guides for painting with acrylics in response to the query ‘acrylic painting’, even if the words ‘how to’ weren’t included. Other upcoming features include the ability to “zoom in and out of a topic” with suggestions to broaden or refine a search. The acrylic painting searcher could ‘zoom in’ on specific painting techniques or ‘zoom out’ and learn about other types of painting.
Many commentators have compared MUM to BERT, the machine learning algorithm unveiled by Google in late 2019, which also represented a huge change in how Google parses search intent and displays results to users. The announcement of BERT was accompanied by many fears that SEO as we know it would become obsolete – and these fears have only intensified with the advent of MUM. But is there reason to panic?
Let’s ask some experts…
What do MUM and multimodal search mean for the way we think about search and SEO?
A new relationship between users and search engines
“[The advent of MUM and multimodal search] changes the relationship between users and search engines,” says Clark Boyd, Head of Strategy at Cadeera, a provider of multimodal search for retailers. “Users have been trained to use text online, but if we take a step back we can see that this an anomaly. Shopping is visual; so are people. We have used text as a blunt, imperfect tool to find what we are looking for. It served its purpose, but it’s time to move on.”
Brent Ramos, Director of Product, Search at Adswerve, adds that MUM and multimodal search will introduce “a more visual and omnichannel experience, changing up the traditional ways consumers and marketers think of search. The look and feel [of Google search] is slowly evolving to what appears to be an image, video, text, and Search-esque mashup experience where the user is returned relevant information across all channel formats.”
“Broadly, MUM just extends the trend Google have been pursuing for some time, moving beyond standard ad copy and text practices into integrating broader multimedia. Video, audio and imagery is playing a significantly larger role across PPC and SEO channels,” says James Ross, Head of Planning at digital experience agency SYZYGY. “This makes sense; even though we’re used to Google being a text-based search engine, it has been evolving over the last decade. Now we’re seeing the incorporation of more imagery, voice using Hey Google, using maps and looking at video as a source of information.”
“It will take time for users to establish the muscle memory to search in these new ways,” Boyd adds. “But as users learn to search more flexibly, they will find new ways to express intent. Brands that can capture that demand effectively will be able to respond with personalised experiences.”
Taking ‘intent’ to a more genuine place
“For starters, I think MUM solidifies the already strong relationship between formal “SEO” and content strategy,” says Olga Andrienko, VP Brand Marketing at SEMrush. “What MUM aims to do is to not only parse the query, but to strip away the latent layers of intent embedded within it. In this sense, MUM is honed in on predictive need. It’s trying to understand the layers of a query so that it can produce content that predicts the various needs of the user.
“Doing SEO in the era of MUM is going to be about you doing the same. It will be about SEOs, site owners, and content creators being able to predict the various needs of a diverse set of users. So while there’s already a large amount of chatter around “intent” within the SEO world, MUM will take it to another (and far more genuine) place.”
Andrienko also acknowledges that, “Google being able to provide a layered and predictive content experience can, all things being equal, change the way users search. A significant change in search behaviour will intrinsically alter what content ends up in front of the eyes of users.”
How can marketers optimise for search in the age of MUM?
Empathising with users
“MUM will force SEOs looking to rank to consider all of the various layers of “need” embedded within the query or topic,” says Andrienko. “SEOs will have to consider all of the latent needs that are needed to satisfactorily and comprehensively answer the user’s question. The only way to do this, fundamentally, is by understanding and empathizing with the target audience (also known as people).”
Boyd agrees: “Marketers need to get closer to customers to understand their true intent, beyond keywords. This will entail a new approach to strategy and measurement, with digital marketers taking more responsibility for the full customer journey.”
Rethinking how content serves a need
Andrienko explains that MUM will inevitably increase the competition surrounding each query, as Google aims to serve a variety of needs in one, and different sites compete for the same real estate. “Google is aiming to predict the various latent intents (or user needs) that are embedded and alluded to within the query. That would logically mean that the scope of the competition increases as the user has a variety of pathways they can now follow.
“For example, in recent years Google has understood that transactional queries (buy a laptop) can also be interpreted as informational or commercial queries (how to buy a laptop). As a result, more and more commerce sites are competing with informational sites (such as product review sites) for the same real estate. MUM will take this dynamic to new places as it presents an even more diverse understanding of both what’s behind and related to a given query.”
Andrienko cautions marketers against attempting to spread themselves too thin in order to cater to this. “If Google is showing a diverse set of information so as to predict the various needs and wants of users, it’s unrealistic to think that your content will fill every such need. Meaning, instead of looking to cast a wide net and capture all traffic, focus more on capturing the right traffic. That is, traffic that speaks to a specific pathway that’s placed in front of the user on the “MUM SERP”.”
However, she acknowledges that marketers can still cater to different, related types of user need. “It’s definitely a good idea to pivot and to create as much content around a topic as possible from multiple angles,” Andrienko says. “The idea being to align your site with various layers of user need related to a given topic or product, etc. This is similar to how many insurance companies have created more informational content so as to align with Google showing both transactional and informational content for what would [historically] be considered a strictly transactional keyword.
“The danger here is in straying too far from your core identity. It’s very much possible to create content around a topic that your site deals with but that does not in fact relate to the purpose of your site. It’s a fine balance and a dynamic that is again not new. Over the past few years Google has been doing a lot to encourage sites to “stay in their lanes” and to do less “net casting” and more audience targeting with their content.”
Optimising for multimodal search
What about the new challenge presented by optimising for search queries that span completely different modes? Cadeera’s Boyd has some advice for marketers here.
“Multimodal search is not about optimising at the periphery, but rather rethinking how we create, store, and serve data to customers,” he says. For retailers in particular, it should lead to a “wholesale change” in how the ecommerce experience is imagined. “Instead of taking a text query and pointing a user to the best-fit category page, we can take a text + image search and use it to pinpoint individual user needs.
“To serve this demand, marketers should work to unify images and text within a central knowledge repository,” Boyd goes on. “This can be used to shape personalised customer journeys, from inspiration to purchase.”
“Optimisation [for multimodal search] means brands need to have user-centric, meaningful creative across all channels,” adds Brent Ramos. He also advises that when it comes to paid search, marketers should “utilise first-party data to enrich auction-time bids and ensure that the content is served appropriately to the right users, at the time, responsibly.”
Experiment!
The advice from SYZYGY’s Ross is to take things gradually and experiment. “The approach to optimising this at this stage should be experiment-led, to determine whether this new medium provides a better customer interaction or improves a return on investment for brands, as it develops and properly beds in.”
What MUM means for retailers
An evolution in how we market products
“For starters, [MUM] can create new opportunities for retailers selling online,” says Olga Andrienko. “If Google is going to try its best to predict the various needs of users, those needs might strongly correlate with your products. Moreover, the connection between the user’s intent and your product could be quite strong and come with a high probability of conversion.”
Andrienko also imagines that MUM’s inference of user intent could change the way retailers market products, in order to cater to these increasingly specific search results. “Take a very simple example of hiking boots. If the user mentions hiking near a river in the query then it’s entirely feasible for Google to use MUM to show only those boots that are meant for “river hiking” (whatever those boots may be, I’m not a big hiker).
“Establishing product profiles for yourself might be an effective way to appear in front of an audience with a high probability of converting. If the set of dining room chairs you’re selling are extremely sturdy, then it might pay to profile the product (not exclusively, of course) as being great for a family with young and rambunctious kids. That can mean creating a whole piece of content around that profile or by simply writing a line or two on the product page (or within an FAQ on the product page).”
Serving the full ecommerce journey within the SERP
“Google had a slide at Search On that said, “Inspire, Explore, Purchase” and that is an apt summary of their ambitions for multimodal search,” says Boyd. “It does not simply want to understand queries in more detail with the aim of channelling users to websites. Google wants to serve the full ecommerce journey within the SERP. Within the same announcement, Google showcased “shoppable images” for SEO as well as paid search.
“Google wants to be more like Amazon. And Instagram. And Pinterest.”
While Boyd ultimately sees multimodal search as a wholly positive development for retailers, he cautions that retailers need to move with the times. “[Retailers’] dependence on Google for sales will only increase if they do not overhaul their own AI capabilities,” he says. “If we look at today’s retail websites, most are structured with text-driven Google search in mind. The value exchange was clear in the past: retailers optimise for Google and Google “rewards” them by sending traffic to their websites.
“That dynamic is shifting and retailers should be wary of a potential imbalance between their on-site experience and Google’s experience. This will lead to even more valuable user data residing with Google alone, with retailers paying for access.”
The solution, in Boyd’s eyes, is for retailers to invest in multimodal search on their own sites, rather than restricting searchers to text. “This puts them in a better position to benefit from Google’s latest developments, as they can provide the assets Google needs to serve user queries. More importantly, it gives them more control over their own customer experience and the resulting data.”
Pioneering new frontiers
Jordan Koene, SEO Strategist and Advisor at Searchmetrics and former Head of SEO and Content Development at eBay, believes that retailers will need to pioneer new frontiers in order to win search traffic. “Taking commercial keywords head-on is a failing strategy, as the likes of Amazon, eBay, Walmart and global giants around the world control most of this search traffic,” he says.
“In order to be effective in online retail selling, you must identify and win within non-commercial searches to gain a foothold on the market. A great example of this is how Wayfair has generated an entire catalogue of content to help consumers recycle or dispose of old furniture. This content is no coincidence; they are making a clear play at connecting with consumers post-purchase and maintaining a brand connection.”
What are the challenges presented by MUM?
Less traffic to websites?
A key concern expressed by publishers about the rollout of MUM has been the potential for Google to answer even more questions within the SERP – instead of directing to websites. Koene echoes this concern: “MUM can improve Google’s ability to use individual paragraphs, lists, images, video etc from a web page and combine this with material from other pages in order to provide a comprehensive and contextualized answer.
“One drawback of this is that Google may no longer pass traffic from search results, while still answering the open question. And as a search engine it will have to work much harder to manage the expectations of the brands that provide content and data while no longer providing the same level of traffic from search.”
An even bigger ‘black box’
Adswerve’s Ramos summed up what has concerned many SEOs about trying to cater to Google’s ever-more insightful machine learning algorithms – going back to RankBrain, BERT and now MUM.
“The advancement of the algorithms continuously makes it harder for marketers to influence what seems to be a ‘black box’,” says Ramos. “This is not inherently a drawback, but it does mean brands and marketers need to fully understand that to win in search marketing – they are going to have to activate first party data within the auction to move the needle.”
Not bad, just different
Andrienko is philosophical about the changed landscape that MUM will present for marketers. “Drawbacks are usually a matter of perspective,” she says. “If the changes that come with MUM benefit you, there are no drawbacks. If not, the entire construct is a drawback. There are going to be losers and winners and changes that are going to have to be made. As great as we are at predicting how things will go, a whole new suite of concerns will arise as MUM hits the SERP.”
“Fundamentally speaking, MUM is just different. It’s an entirely different approach to what a search engine is from a traditional point of view. MUM turns the SERP more into a portal of information than a set of resources per se. In many ways comparing the SERP as it is now to what Google envisions it to become with MUM would be like comparing apples and oranges (to an extent).”
Search Engine Optimisation (SEO) Best Practice Guide
