E-Commerce Customer Experience 3: Search Tuning - People and Process

Post series written by Matthew Lynch (Senior Consultant at Charteris specialising in eCommerce). Before joining Charteris Matthew worked for IBM on various eCommerce enablement projects across multiple industries from the late 1990's through to the present day.
This series of blogs is about how product managers in an eCommerce business can influence and optimise the customer search experience on their website by tuning the on-site search engine.

Tuning search is like servicing your car; omit it and you risk losing value and breaking down. Get the habit and the right people
In the last blog entry, we covered what search tuning was and why it was important to do it in order to optimise the customer search experience on eCommerce websites. In this blog entry, we will explore the process that businesses should consider implementing in order to keep up-to-date with their search tuning.

Servicing your car is a chore, but most of us do it. Why? For peace of mind; perhaps to ensure you optimise its resale value, to prevent the inconvenience of breakdowns and to keep it safe. The same goes for search tuning. If you don’t tune your search regularly, the value of the search experience (and hence your eCommerce site) will reduce or may even breakdown (if customers can’t find product). Also, like skipping maintenance on your car, skipping maintenance on search means that you won’t know if anything is wrong and you won’t be able to predict when the reduction in performance or breakdowns might occur. Search tuning doesn’t need to be an onerous process either, as long as it’s done regularly, it shouldn’t need to take long.

Search tuning isn’t everone's forte.

Given the apparent technical nature of the task, you might be inclined to give it to your help desk, your web designers or your admin staff. Whilst their skillsets might fulfil two of the criteria required of the search tuning process : ‘attention to detail’ and ‘discipline’, they may not be able to adequately fulfil the key requirements of ‘product knowledge’ and ‘good language skills’.

Attention to detail is a desirable trait because, whatever tuning you do, if you do it wrong, it can be worse than not doing it at all. For example, if you mis-spell a synonym (e.g. you enter bart->kart instead of cart->kart, not only have you not corrected the customer’s problem that the synonym was supposed to resolve, but you have potentially created a new problem for the customer as well (people searching for Mario Kart won’t be satisfied AND those searching for Bart Simpson will be shown irrelevant products relating to Mario Kart as well).

Like attention to detail, discipline is important to ensure that search tuning is done regularly enough to catch problems before too many customers have experienced them and to ensure that any changes to search tuning are tested to make sure that they have the desired effect. The regularity of tuning will depend on the size and regularity of changes to your product / content catalogue and also the number of customers hitting your website. Once under way, you’ll get a feeling for how often you should be search tuning from the number and frequency of problems identified in the search reports after the initial set of search tuning sessions. You will have a bunch of things to fix when you start, then the number should settle down. Omitting to test changes means that you don’t know if you’ve actually made the experience any better and, just as importantly, you don’t know if you’ve inadvertently made it any worse. Testing is straightforward: check the search results on your website before making a change and compare it after tuning.

With the general skills sets out of the way, consider the two key specific skills your search tuning staff ought to have...Product knowledge is a must have. Without it, you won’t be able to pre-empt some problems let alone deal with ones that are identified in the analytics from your search tool.
For example, if you’re not into video games, it unlikely you would know that people using ‘wow’ or ‘cod’ as a search terms on the games retailer’s website were searching for the game titles ‘World of Warfare’ and ‘Call of Duty’.

Choosing someone to perform search tuning who has good language skills is probably a no-brainer, but should be kept in mind all the same. Knowing the difference between a noun, verb, adverb, adjective etc is useful as is the ability to spell and recognise the likely intended meaning behind search phrases that customers are using. If your website is represented in more than one language, if you can, get native speakers doing the search tuning, because they will have a much better feel for how their language is currently being used.

Prevent and react to search issues
Depending on the nature of searchable content your product or content teams are adding to the website, you may be able to prevent some search problems. There’s no harm, for example, in adding synonyms for words that are in the dictionary or synonyms between unique product acronyms and the full product title. With practice and analysis of your product set, you may find others as well.

That said, beware of doing search tuning for the sake of it. As mentioned before, it can be quite easy to make things worse than they were by being overzealous tuning search before you have seen any problems in your search reports.

Business-as-usual tuning

You should analyse your search analytics frequently for search terms that return zero results and search terms that return large numbers of results. Analysis of the zero results report should reveal acronyms, misspellings and typos that you should account for in your synonym database. It may also reveal searches for products that you don’t have for sale.

Search terms that result in too many results could be an indication of the term containing a word that occurs so frequently, it should be ignored or an indication that you should instruct your search engine to ensure an exact match ‘enforcing a phrase match’, thereby excluding partial matches from the list of results.

Consider tuning as new products are added to your catalogue. You may prevent problems occurring, for example, by adding known acronyms as synonyms. Again, beware of being over-zealous because adding unnecessary synonyms to your thesaurus just means there’s more work for the search engine, more to clean up later and more potential for incorrect entries. Let the analytics of real customer behaviour drive your main tuning activities.

As mentioned before, testing needs to be an integral part of the tuning process so that you know the effect of changes you make.

Search tuning housekeeping

Finally, bear in mind that, just because a synonym or enforced phrase worked at one time, doesn’t mean it will stay useful forever. This year you may setup a synonym of touch -> “ipod touch” because the “ipod touch” is popular right now and you know that there’s a lot of searches just using the word “touch”. However, at some point in the future the search term “touch” could be used in a completely different context by customers, hence the need for periodic reviews of your search tuning.


This concludes this set of blogs on search tuning.
---------------------------------------------------
Contact – matthew.lynch@charteris.com

E-Commerce Customer Experience 2: Search Tuning - Techniques

Post series written by Matthew Lynch (Senior Consultant at Charteris specialising in eCommerce). Before joining Charteris Matthew worked for IBM on various eCommerce enablement projects across multiple industries from the late 1990’s through to the present day. 

This series of blogs is about how product managers in an eCommerce business can influence and optimise the customer search experience on their website by tuning the on-site search engine.

The search engine doesn’t know what it doesn’t know

You may already have a facility that helps you improve on-site search for your customers.
It should have been setup with an initial “dictionary” to make sure that it works well at go live.
But, after go-live, how can you make sure it continues to work for you? After all, you’ve got new products coming on-line all the time and you want to ensure your customers can find them.

Search engines only know what they’re told

Without manual intervention the search engine is only as good as the in-built auto-correction function (to deal with typos, mis-spellings), the default dictionary, any content that it knows about and ‘domain-specific’ dictionaries which, if vendor supplied, are likely to be quite generic.

Search tools are not necessarily aware of common mis-spellings, typo’s, domain-specific acronyms / phrases, what customers will actually search for in real life or other common language in the industry sector. They will also not be aware of specific key phrases or made-up product names that exist in the product catalogue that have specific relevance.

Also, language use evolves over time, sometimes quite quickly (often as a result of creative marketing or journalistic minds), so there’s no way that a search engine is going to keep up-to-date without human intervention.

Regular search tuning improves the bottom line

Search tuning enhances the relevancy of results, reduces the number of zero results found and helps identify application configuration and design changes that may be required. The process of search tuning is also another way of keeping you in touch with customer trends. All of these things help to improve the customer experience ultimately leading to improved conversion, basket size and customer loyalty.

The business, not IT is best placed to perform search tuning

If your business is not tuning its on-site search engine, then it may not know how many customers are having trouble searching for your products on your site. Many people assume that the search engine belongs to and is maintained by IT. You might think that your SEO strategy and keyword-promotions ensure that customers can find your products using Google, Yahoo etc, but if you’ve got any direct-loaders (people who go straight to your site and then use your site’s search facility) then they need their searches to be looked after by the business, the guys who understand the products and how they are marketed and sold to the customer.

Consider who owns the customer relationship. The business doesn’t get IT guys to write the product titles, descriptions, retail reviews or marketing copy, so why should the business rely on IT, when search engines deal with words that customers are looking for in the very text that has been created by the business?

Use analytics to identify search tuning needs

In eCommerce, many people will have come across a “top-search-terms” report from the web-analytics tool. This, generated regularly, can form the basis for search tuning. For best use, it needs to report the popularity of search terms together with the average number of results returned and needs to show search terms with zero results. In addition, it is useful to have a report showing you search terms that have been autocorrected.

Tuning affects which words and phrases the search engine considers or ignores

Search tuning involves telling the search engine how to handle words and phrases that it may not ‘understand’ or that it may assign more or less relevance to than desired by the business.

Stop words
In most circumstances, there’s no point in a search engine searching for words that are so common that they appear in large numbers of product titles or descriptions. It results in too many, often irrelevant, results for the customer and wastes the search engine’s resources in matching them. Typically these words include ‘a’, ‘an’, ‘of’, ‘the’ ‘i’, ‘is’ etc but the business really ought to work out for itself which words are so common that the search engine should ignore them. This may include some words that are core to the product set being sold e.g. a entertainment retailer might get the search engine to ignore DVD, CD etc. Be wary of adopting the standard set provided by the search tool vendor. For example, a ‘stop word list’ provided by a search tool vendor may include the word ‘that’, but for a CD retailer, this would not be an appropriate stop word if it meant that it made searches for CDs by ‘Take That’ of lower relevance than results that don’t contain the word ‘that’. Try it, you’ll find there’s a band called “Take”.

Synonyms
It is estimated that 20-30% of all search terms used on the internet contain a mis-spelling or a typo. If that is the case and your search engine doesn’t manage to autocorrect them or the typo is on a made-up word such as a brand or product name, then your customers will get zero results, the dreaded dead-end.

The solution; find out the terms that customers are using that return zero or few results and setup a synonym to the term that does match products.

Some examples where you would use synonyms include :
- Numbers – equate 2 with ii and two
- Acronyms – to equate product-acronyms with the full product title
e.g. LOTR -> Lord of the Rings
- Common mis-spellings e.g. cart <-> kart
- Made-up product titles with spaces taken out e.g. (wii) motion plus = motionplus
- UK / US spellings (if not auto-corrected) e.g. color <-> colour, metre <-> meter,

Enforced or Automatic Phrasing
If your customers are getting too many results for certain search phrases, of which you find that many are irrelevant, then you should instruct your search engine to only consider results where there is an exact match to the phrase (words and their order).

e.g. "32gb ipod touch" - assuming you didn’t want the search engine to return 4/8/16gb ipods or ipod nanos in the results set)

Stemming
Good search engines take verbs, adjectives and adverbs that are used in a search term and condense them to their ‘root’ form or ‘stem’ (e.g. running -> run, richer -> rich, strongly -> strong) before comparing them to the content to be searched which has also been ‘stemmed’. Search engines often come with stemming built-in but if problems are identified using the search analytics, then additional stemming ‘synonyms’ or rules will need to be added. The process of getting to a word stem is language dependent and may require the skills of an experienced linguist if adjustments need to be made.

Search keyword / phrase Redirects
Sometimes it is desirable to redirect a customer to a hub or landing page that makes the best representation of a product or product set that the customer is looking for. This gives the customer a better experience than the presentation of a product list (which may well not show the full range of potential product type matches on page 1).
It is a common occurrence for customers to search for types of products e.g. Romance films, so a redirect for such types of search would be a good idea.

It is good practice to monitor for new occurrences of such searches and setup a search term redirect for the phrases used. Good search engines provide a facility for recording such redirect phrases with the desired destination such that the search engine provides redirect URL destinations to the eCommerce website when such phrases are matched.

In my next post we will look at what’s desirable to look for in people who will perform search tuning and how the tuning exercise can be organised
---------------------------------------------------
Contact – matthew.lynch@charteris.com