Wednesday, December 2, 2009

E-Commerce Customer Experience 1: Search Relevance - Ranking

Post series written by Matthew Lynch (Senior Consultant at Charteris specialising in eCommerce). Before joining Charteris Matthew worked for IBM on various eCommerce enablement projects across multiple industries from the late 1990’s through to the present day. 

This series of blogs is about how product managers in an eCommerce business can influence and optimise the customer search experience on their website by tuning the on-site search engine.

Get your search engine to understand your customers’ language
Is your site conversion in need of improvement? Do too many of your customers drop out of the funnel early on? Do they consistently find what they’re searching for on the first page or two of search results? Do you have feedback from customers saying they find it difficult to find what they want on your website?

If the answer to any of the above is yes, you may need to make it easier for your customers to find product. In other words, you may need to actively tune your search engine to suit the way your customers search for product.

Search results - What’s relevant?

Search is search is search isn’t it? That’s what some people think; “surely, if we implement the ‘best’ internet search application, that’ll solve the problems we have with our poor search relevance won’t it?”On-site search for an eCommerce website is not an exact science and is certainly not best served by a generic internet search tool. The main reason for this is that what a generic search tool considers ‘relevant’ will almost certainly not be close to what you or your customers would consider relevant search results for your eCommerce website.

For example, what would product managers think if the products appearing at the top of your search results page were just those with the most matches to the search keywords, sorted by price alone? It is unlikely that this idea of ‘relevance’ would meet their or your customers’ sophisticated expectations.

The answer lies in defining what ‘relevance’ means for your customers and getting your search engine to return results that get as close to that definition of relevance as possible. Relevance may well
vary in your business according to the type of customer or type product / service that is being sold. For example, relevance could mean ‘popular’ or ‘most recently released’, ‘capable of being delivered soonest’, ‘most highly rated’, ‘highest converting’ or any number of other things depending on the nature of your business, product or customer base.

Taking a simplistic view to illustrate the point, let’s consider that there may be three types of visitors to a retail website : a) customers who know what they want and have a specific item in mind b) customers who have an idea what they want but want to compare products before making a choice and c) customers who have no real idea and are just browsing.

Relevance for customer type (a) is likely to be very focussed on as close a match as possible to their search term in the product title. Relevance for customer types (b) is likely to be product or product content whose title and category have the best match to words in the search term entered. Relevance for customer types (c) is likely to be product or content whose categorisations have the best match to words in the search term entered.

For a retailer of DVD’s, therefore, customer (a) might enter a search term such as ‘Temple of Doom’, customer (b) might enter ‘Indiana Jones’ and customer (c) might enter ‘Adventure movie’.
If your search engine hasn’t been configured with appropriate relevance ranking strategies, the results may not be what each type of customer expects. Taking the simplest model of ‘relevance’ where purely the number of matching search words determine the relevance, you could end up with something like this :
“Temple of Doom” ->
1. The Little Princess: Shirley Temple DVD - the word Temple appears lots of times in
the product description
2. Paul Temple (DVD)
3. Doom the Movie – Doom appears lots of times in the product record
4. Indiana Jones : The Temple of Doom (ought to be at the top of the list)

“Indiana Jones” ->
1. The World’s Fastest Indian - title + lots of matches to Jones in the cast list
2. History of Indiana Basketball -lots of references to Indiana !
3. Indiana Jones : Raiders of the Lost Ark (1st film, so appears 1st)
4. Indiana Jones : Temple of Doom (2nd film, so appears next)
5. Indiana Jones : Last Crusade (3rd film, so appears next)

“Action Adventure DVD” ->
1. The Adventure of the Action Hunters

2. Lara Croft Tomb Raider: The Action Adventure
3. Operation Dalmatian: The Big Adventure
Here, matches are on the product record in preference to the categorisation.

As you may see from the above examples, a search relevance ranking strategy that assigns different weight to product category and title in the product record over other ‘less relevant’ parts of the product record, ought to improve the relevance of results for the customer. In addition, a commercial weighting could improve the weighting for the business. For example, it could boost the relevance ranking of the more popular 'coming soon' or 'recently released' film releases e.g. putting the 2008 film Indiana Jones and the Kingdom of the Crystal Skull at the top of the list). Bounce rates should be reduced by making sure that the latest releases are always shown.

As well as weighting the elements of product records, tunable search engines allow you to determine the priority of search ranking schemes (algorithms) to affect the results. For example, the search engine might have a method A that scores results based on the number of search keyword matches per field and scheme B that scores results by the total number of matches across the whole product record. In many circumstances, a retailer would want the scores allocated by scheme A to have more weight that scores allocated by scheme B.

A methodical approach to improving your search relevance is recommended. This involves the following high level steps :

1. Define what relevance means for which customer types and product types
2. Measure your current site search results against the desired outcomes defined in step 1
3. Prototype and test different relevance ranking strategies in your search tool
4. Periodically measure and review your relevance ranking strategy to maintain the optimum outcome for the customer and your business

In the next blog entry, we will explore what you can do to tune a search engine to handle problems with specific words.


Contact -