Is True Enterprise Search actually Possible? -

The idea of “Enterprise Search” is an attractive one. It certainly would be its weight in gold to have a single search location where key words can be entered, and within a fraction of a nanocentury [1], results would be displayed that include both structured, and unstructured, content from across the numerous repositories, silos, systems, archives, file shares, cabinets, clouds, etc, etc.

But is true Enterprise Search really possible? I know there are several tools that provide “Enterprise Search” functionality, but these usually allow you to search over a fixed number of different repositories, usually containing similar data. Maybe it’s a set of defined documents, or a database, or similar. You certainly get the opportunity to make available content from disparate sources, but can you consider that “enterprise”.

If you consider what’s involved to search across the “Enterprise”, it should be quite easy, right?

1. First off, you need to be able to identify where your structured, and unstructured, data and content is. Remember, here we are dealing with the complete enterprise, so don’t forget that this includes files shares, hard drives, database system, ERP systems, ECM systems, etc, etc. And what happens if new “sources” are added?

2. Next, you need to know what sort of content you have. Can the Enterprise Search application “read”, or parse, the data/content you have? There certainly are ways to make it possible to do this. You can install an ifilter, for example. But, you’ll need one for every format that you have in your enterprise.

3. You need a way that your Search application can connect to all of the different “sources”. In principle this is, again, possible. (However, I would imagine that this would require a lot of configuration).

4. How frequently is your data, and content, changing? For example, in an ECM system, is the content constantly being changed (as new documents are added). Maybe several major and minor versions are kept of each document. Do you need to index all versions, or only the latest? What about data in your ERP system? How accurate do you want your search results to be? Do you just keep continuously indexing?

5. Security. Do you want users to be able to see results of data, or content, that, if they had used the native application, they do not have rights to? If there are disparate security systems in place, how do you translate ACLs from them into a common format? Do you use “early binding”, or “late-binding”?

Until we have a way to be able to “capture” all information from an undefined number of sources, with an undefined number of data, and file, formats, with disparate sets of ACLs, I return to my opening question: “Is True Enterprise Search actually possible?”

[1] A nanocentury is approximately 3.155 seconds

Comments

A little different approach

Submitted by Vivek Abhichandani January 08, 2012 - 9:36 PM

The default approach to your questions 1,2,3 suggest that its supposed to be a pull mechanism from Search to identify, obtain and index the various disparate sources of information in the organisation.

Since search components already have many factors to consider and do a good search; in our system, we tried a push approach where the search system processes a queue where each message has the necessary information on how the search system should obtain the newly added information. Thus the onus and inturn the flexibility of frequency, type of content, security concerns instead of being a complete search system's headache; shifts the burden in some ways to the source of information.

I have to still agree, that this was only possible since we had more leeway on the architecture for the ECM system since its a completely inhouse system; which may not be true for most cases.
And your last question, ACLs; thats the toughest bit for us and is the primary reason why our search takes 10 nanocenturies than one.

Report

Was this helpful? Yes No

Sounds interesting

Submitted by Mark Owen January 09, 2012 - 4:02 AM

Your solution sounds interesting. I had a quick look at your company's web site. I'd like to know more about this, and will have a more detailed read when I get some time.

Report

Was this helpful? Yes No

Is true enterprise search possible? Of course....

Submitted by Aaron Everingham January 08, 2012 - 10:32 PM

The adage 'anything is possible with time and money' holds true on this topic as it does with nearly everything (perpetual motion machines etc not withstanding).

Where is the content? If you don't know where your content is, you are probably in trouble. The question of where is related to how do you connect to those repositories? A good search engine will have a library of connectors for major systems and a framework for developing new connectors. The increasing support of open standards makes connecting to sources systems less complex than perhaps in previous years.

The sort of content you have is obviously important and again, a good search offering will support a wide range of file formats and content types, including structured content in databases. Reading content in databases is pretty easy as it is often just text but you need to be able to join up this content to make it useful and meaningful in a search context.

Addressing the changes in content to maintain search index freshness can be a challenge but can typically be resolved with a combination of capacity in the indexer, the source and more advanced indexing where the source system advises the search engine on what has changed to eliminate the need for brute force crawling and indexing.

Security is always a challenge but again, this problem seems to be largely solved with early binding and post query filtering technologies and approaches.

At a technical level, I would argue that it is absolutely possible to deploy a true enterprise search capability. The technology exists to do this given time and money.

To deliver an effective outcome also requires the application of grey matter and a deep understanding of how enterprise search fits in the overall information management strategy, how the source applications function, how users will utilise the information held in the index to deliver value to the organisation and what that value is.

Sensible decisions need to be made on performance vs cost (both for end users and indexing activity) and the organisation must be committed to the system.

So yes, a true enterprise search solution can be delivered today. Just don't expect it to be easy, fast and cheap.

Report

Was this helpful? Yes No

Excellent feedback

Submitted by Mark Owen January 09, 2012 - 4:13 AM

Hi Aaron

It's great that you were able to expand on the items I had mentioned. I definitely agree that, in principle, it might be possible to achieve "true enterprise search", but your comment on the "application of gray matter" says a lot. It's one thing having the technology, it's another thing being able to provide meaningful results. I've seen many cases where the "enterprise search" solution returns bucket loads of results, but nothing that the user actually wants.

And your last sentence sums ups this topic nicely.

Report

Was this helpful? Yes No

This post and comment(s) reflect the personal perspectives of community members, and not necessarily those of their employers or of AIIM International

The Global Community of Information Professionals

Community

Is True Enterprise Search actually Possible?

Rate Post

Add a Comment

Comments

Blog Search

Information in Motion

ECM Training

ECM and Collaboration

Latest Comments

Get The Research

Grow Your Expertise

Expand Your Network