Sandeep Sachdeva - Sometimes a person does a wonderful job of hiding behind complexity yet other times it is the simplicity that shines out.
We already know that we as humans can only progress with a combination of both complexity and simplicity. Then why is it that instead of pursuing both there are times when we just get lost in complexity alone? It is during those times that we need people like Einstein to see and grab the simple things and bring them forward. Those simple things make a huge impact sometimes game-changing and sometimes radical. This applies to a company, a business unit, a project, and you as a person. Then it makes sense to welcome both complexity and simplicity, encourage them and know that we need to foster an environment that will provide us benefits from both.
It is important to understand that Big Data is both complex and simple at the same time just like any other technology. Take smartphones for instance. They are more powerful than some of the PCs, yet they are so simple and engaging to use. They hide an immense amount of complexity behind them. They challenge us to do more with them and we in turn challenge the manufacturers by expecting more as a user. So let’s consider both complexity and simplicity in the discussion below, but before that I would like to share a secret with you.
The concept is simple. When a user is handed an instrument the user is so immersed in the experience that the means of delivery is no longer material. That instrument then disappears from in front of your user. I call it visible invisibility. This is the secret that some companies fully understand, have it ingrained in their DNA and exploit it to the maximum. Smartphones, cars, TVs are examples that immediately come to mind. Is your team, your project, your business unit or your company building systems that follow the principle of visible invisibility? If not why not?
Complexity of Big Data
Complexity of Big Data is a question I get asked a lot. Yes, there was a time when we were adding new features at a rapid pace with Big Data. That seems to have slowed down although not by much. New technologies and rapid enhancements continue to add to the perception that Big Data is complex. As this technology is in the ascent phase of the s-curve it will continue to evolve and expand into different domains. Features such as real-time capability, query optimization, machine learning, natural language processing, data-stores (not databases), data-archives, rapid real-time visualization, spatial memory and more will continue to be worked on. They all help in making visible invisibilitypossible. It is also important to understand that you may or may not need all of the complexity as you step into Big Data.
Simplicity of BigData
In one sentence, the power of Big Data is in its ability to enhance, enrich, and simplify the life of the users. That is our starting point. The question we need to answer is: How and what can you bring to your users that will enhance, enrich and simplify their life by making the medium or the interface disappear?
It will mean different things to different industries but the goal is the same. Once you have the goal set you are now ready to take the steps needed to get into Big Data. Why Big Data? Because in most cases the answers you get will require a scalable and flexible platform that integrates multiple internal sources, merges external data, learns from patterns and predicts what may happen in the near future. These are all characteristics of a Big Data platform.