Solving the Problems of Enterprise Search

Organizations generally recognize that an enterprise-wide search platform can benefit employee and staff productivity by making the full digital knowledge of the enterprise available. Sadly, the reality of broad search and findability initiatives often falls short of expectations, which leads to employee frustration: people want search work ‘more like Google’.

As an enterprise search consultant, I’ve seen this vicious cycle a number of times: search sucks, so replace it with a better platform. The new one sucks; replace it. I’ve seen this happen even when the platform to be replaced was the Google Search Appliance!

Why does this happen? We sometimes see cases where the platform that best meets the business requirements was not selected. Other times, the business team was not involved in the decision process; and sometimes there are decisions made for non-business reasons.

Endeca was remarkably successful as an eCommerce platform because it had fine-tuned analytics and a business console that let product line managers set special offers easily and monitor how those offers were performing. Selecting Endeca where eCommerce search was the primary need usually made a lot of sense. Selecting a different platform in these cases often led to disappointment, but it still happened.

Nonetheless, most modern search platforms are capable of pretty good generalized results across enterprise content. The trick of going from ‘pretty good’ to ‘great’ takes time and effort, and ongoing maintenance. The annual search and findability survey of enterprise search owners and users done by Findwise spotlights the problem: the average enterprise search implementation has less than one full employee responsible for ongoing tuning and maintenance of enterprise search.

Can you think of any enterprise-wide software product that is ‘fire and forget’? Relational and NoSQL database systems tend to have armies of experts monitoring and tuning full time. How many FTE actively manage your enterprise search platform?

It costs less to maintain a software product than it does to replace it every few years. Over the coming weeks leading up to the Fall enterprise search conferences in London and Washington, I’ll be blogging about the small things you can to do to get your search platform in shape and save your corporation upwards of $500K in replacement software you’d otherwise need to spend every other year.

The Audit

Before you start changing anything, you’ll want to audit how you’re doing now.

Some basics:

Know your content: See how many documents your search platform should have indexed. Identify how many different repositories you are searching, and determine about how many documents you’d expect to find in each. Then sit down and search each. Not all platforms support the “*” wildcard syntax, but by working with other employees who are familiar with the repositories, explore and make an educated guess.

If some of the content is secure and requires specific permissions, you’ll want to conduct searches in the context of both a non-privileged user and a ‘super user’ to confirm the content count of each. For right now, don’t be too concerned about multiple levels of security and different counts for each level; focus on the total unsecured and secured count.

While you’re addressing security, test your security by searching for secure content when you’re using non-privileged credentials. Each organization uses different terms and tags, but look for terms like ‘Managers Only’ or ‘Limited Distribution’ using an account that should not have corresponding access.

Next, discover your top queries.

You’ll want to see what people are looking for when they search. Most modern search platforms include at least the basics of reporting. If your platform does not include it, work with your IT staff to see if you can extract search queries from the web server log. At a minimum, you’ll want to see the top overall queries, and those queries that produce no results. You may want to break out these metrics for secure and non-secure logins, or by repository.

The top queries tell you what content is in greatest demand, and it’s a nice touch to let the authors of valuable content know that their work is popular. Depending on the nature of your content, you can learn of much more:

  • Unexpected terms that offer a clue for missing or incomplete content
  • Queries for policies that may indicate inappropriate behavior in a department or office
  • On public sites, unusual queries may be indicators of product flaws
  • Potential ‘hack attempts’ to locate secure content using insecure credentials

The ‘zero hits’ query is a potential goldmine for you. From this, you can determine a number of things:

  • Hot topics for your users that may not exist, perhaps related to permissions issues
  • An opportunity for new or better content or synonyms
  • Users who are confusing your internal web search with employee directory or a Google search, or valid content that has not been indexed

Over the coming weeks and months I’ll be following up additional posts to address enterprise search content, user expectations, security, and more. Feel free to contact me or comment below with any questions you’d like to see me address.


Miles Kehoe About Miles Kehoe

Miles is the Search Practice Director for Avalon Consulting LLC. He has over 20 years of experience in Enterprise Search. Prior to joining Avalon, he was Director of Professional Services at Lucidworks, where he was responsible for building and managing teams of both pre-sales and post-sales search experts. Prior to joining Lucidworks, Kehoe founded New Idea Engineering, a vendor-neural enterprise search and analytics consultancy. During his tenure at NIE, the company created and spun off, the first hosted “Enterprise-Search-as-a-Service”. He started his career at Hewlett Packard and at Verity, which sparked his interest in search.

In addition to writing for the Avalon Search blog, he participates regularly in industry conferences including Enterprise Search Summit, KMWorld, SharePoint Saturday, and the Data Summit, is an author of Professional Microsoft Search (Wrox/2010), and tweets as @mileskehoe and as @searchdev. He is based in Santa Clara, California.

Leave a Comment