Big Is Relative

Community Topic(s):

Keywords: big data, social media, knowledge management

Current Rating:
(0 ratings)

 

Last November, I had the privilege of listening to a keynote presentation byChrister Johnson from IBM about Big Data at the Gilbane Conference. I wrote about my reaction to that keynote shortly after hearing it, but somehow I knew that this concept would haunt me. Before I started writing this, I went back and reread that earlier post. Part of my reason was to make sure that I didn’t bore the three people who might read both blogs, but part was to make sure the two entries work together. That doesn’t mean I can’t change my mind in three months, but if I did, I should tell you. Newsflash, I haven’t changed my mind, but I have refined my opinion on Big Data (read that as if I were using a megaphone please). As the title indicates, big is relative.

Clearly, American Nuclear Insurers, the company I work for, will never have big data woes that involve the word Exabyte, as much as I would love buying a multi-Exabyte storage array before I retire. On the other hand, we insure a big industry, we put big money out there in the form of limits, and we are expected to know what we are doing. More than knowing what we are doing, we are expected to know what everyone in this industry is doing. Historically, we digest all available material, review it among ourselves and then disburse that knowledge in the form of white papers and e-Newsletters to our owners. Most of the aggregating and distribution has been handled with apparent ease by a coworker and friend of mine who will be retiring later this year. How we fill the void his retirement will leave behind is anybody’s guess, but we (IT) have begun to experiment with ways in which we can automate at least the aggregation terms in the equation. That’s going to be hard; IBM has Watson at their disposal – we have SharePoint.

The technical task at hand will be to monitor Twitter feeds, newspapers, magazine articles, blogs and other social media for potential candidates, i.e. the things our owners might be concerned about, the things that reference our customers, the things that reference us, well, you know the drill. This could include the predicted path of an Atlantic hurricane, the damage from a Midwest tornado, a new technology, or the release of nuclear material into the environment. Obviously, we have other sources for some of these bits of content, but our need to know is bigger than what we need to conduct our business. We need to know the facts, and we also need to know what other people are saying about these things – including the things that they say that aren’t true.

If you have read this far in search of a description of our solution, I am sorry to disappoint you. We are working on a solution, but it’s nowhere near complete. We can follow Twitter, we can receive email copies of, or links to articles, and we are pretty sure we can alert a network of experts to review, act or opine on what we collect. We can also distribute SharePoint content to others. In other words, the mechanical aspects are all in place, it’s the human element we need to nail down. I’ll let you know how that works for us, after we have it working.

Report

Rate Post

You need to log in to rate blog posts. Click here to login.

Add a Comment

You need to log in to post messages. Click here to login.

Comments

Laurence Hart

Always Relative

I once did a multi-terabyte system and afterwords, any system that had less than a terabyte of content on the same software seemed quaint.

But the tools help define what is big. The more dynamic the data and the faster you need the results, the "bigger" the data problem. Many of our solutions, data warehouses, work well until you want real-time analytics from a more diverse data set.

Good luck with your challenge. It will be interesting to hear how you meet the challenges in your environment.

-Pie
Report
Was this helpful? Yes No
Reply

Bob Zagami

Historically, we digest all available material . . . . .

Your words Dan, not mine. Which then brings up the question, how do you know you have "found" all availaable material? Knowledge experts like your co-worker tend to know where to find what they want, but how do they find what you need? Search is becoming more important as we see the impact big data is making on large organizations. While you may not be dealing with the same amount of information, you still have the same responsibility to find the right information and act on it accordingly, or compile it for distribution to your customers. What role do you see in improved search technology that can help you find information from multiple locations and repositories outside the company to assure that you do indeed have "all available material?"
Report
Was this helpful? Yes No
Reply
Daniel Antion

OK Bob, I was already stressed over this…

You raise an excellent point Bob, and this is why we are starting the process now, months in advance of my coworker’s departure. I guess the short answer to the big question is that we won’t ever know if we have everything. Regarding search, we have invested in search tools within our shop, and we are exploring the options we have for utilizing a lose network of alerts, and activity indicators from outside sources. I think the value we bring to the equation is based more on filtering the stuff that is widely available through the experience and expertise of the people we have on our staff. Nobody wants to deal with Big Data, they want Big Data reduced to items that are easily consumed or acted upon. I’m not sure we fully understand all the issues yet. Fortunately (from a data gathering point of view), the nature of the industry we insure causes things to bubble to the top pretty quick.
Report
Was this helpful? Yes No
Reply
Sanooj Kutty

Are we talking about Findability here?

Dan, you are right about Big is Relative. Is Data "Big-ness" (sic) a matter of size or is it like in your case a matter of diversity and the talent of a human resource?

What if "Pinterest" and as a result of it "infographics" dominate over "140" characters and Twitter? What if with increasing video making capabilities reaching the average consumer PowerPoints are replaced by Short Videos?

Many such questions and innovations will keep coming and going...

When I first hard-coded by Pascal menu and forms, who knew forms building and menus will be a case of drag and drop?

Just as my wife goes to the boutique and likes most of what she sees in there, it will be akin to her time-consuming trial and error mechanism of finding the right "dresses" irrespective of the size of the store and the amount of dresses in there. And just like her shopping, this will be a perennial ongoing exercise.

There is never a perfect pair of shoes and there is never a perfect solution. Keep checking the fashion pages and make sure you are ready to dress for the occasion (or requirement).
Report
Was this helpful? Yes No
Reply
Daniel Antion

Don't Get Me Started

There's big data, and then there's the quest for the perfect pair of shoes. I would much rather work on the former - Thanks for the comment Sanooj
Report
Was this helpful? Yes No
Reply
Sanooj Kutty

I agree

So would I rather work on the former... :)
Report
Was this helpful? Yes No
Reply

This post and comment(s) reflect the personal perspectives of community members, and not necessarily those of their employers or of AIIM International