Tag: systems

What is the big deal about Big Data?

Prasad AjinkyaLeave a comment

Imagine a typical day at the office for any business executive. All of us would be checking our emails, getting into meetings, attending to our phone calls and planning for that weekend getaway to unwind from a stressful week. Sounds familiar, doesn’t it? These activities will lead to a source of unstructured data being generated, data which until some years back was really difficult to store, much less analyse.

What is Big Data?

The industry has termed all these disparate sets of data as Big Data. Big Data is a collection of data sets which either is unstructured and poses a challenge in storage and retrieval or is such a large set of data that it cannot be analysed by existing systems. Dan Kusnetzky of ZDNet has put it succinctly when he wrote about Big Data –

In simplest terms, the phrase refers to the tools, processes and procedures allowing an organization to create, manipulate, and manage very large data sets and storage facilities.

Big Data could be a series of images that are used in preparing a report, it could be a video recording of an event, and it could also be a behavioural data of users on your web product.

Is Big Data just Hype?

Do we at the end of this article simply write off this jargon … or are there real life applications of Big Data in organizations? The applications of unstructured data needs one to take a step back and look at the bigger picture –

  1. A customer searches for a particular product on an e-commerce site.
  2. The site has the product, but it is currently out of stock.
  3. He tweets about this to the site’s support on Twitter.
  4. They follow-up using Twitter and finally email him about the availability of stock.
  5. The individual goes ahead and purchases the product, he also manages to buy some other gifts for his family.
  6. The site sends him an SMS and email confirmation.
  7. The order is dispatched and the site sends him a separate SMS about this. The order is sent into two separate dispatches with two separate dispatch IDs.
  8. Both the dispatches reach the happy customer. He goes on and thanks the support on Twitter.

If you see this entire cycle, the interactions and data points are across a series of different systems. Some of these systems are not even owned by the e-commerce site, some of them are vendor systems, yet there is a clear need for a better understanding of these data points.

In order to be more responsive to the potential customer, to reduce the lead to cash times, to invest in the right inventory and to justify Social Media expenditures, the organization (our e-commerce site) will have to analyse this data.

Apart from the obvious business applications, big data is also being used to make society safer. For example, in 2009 Google was able to apply big data to search terms to help identify how the H1N1 virus was spreading through communities. They did such a great job at this that not only was their tracking real time but also it was more accurate than the Centres for Disease Control and Prevention (CDC).

The Next Big Challenge

Any business analyst will tell you the importance to have clean, structured and codified data in order to cull out business insights. Without proper codification on a data set, a business analyst cannot run different statistical tests to check their hypotheses.

With Big Data the challenge is that statistical packages such as SPSS, R or the giant in this field SAS cannot handle really high volumes of data (we are talking data of the size of exabytes here!). Multiple problems will have to be addressed and solved to capture the full potential of big data. Policies related to security of data, privacy and ownership of data and intellectual property rights will also need to be formulated.

Understanding the nature of the problem is the first step to solving it. The next step would be start developing methodologies and systems. In the race to develop the next set of software applications, software giants such as Oracle, IBM and even Infosys are looking at developing their own propriety systems.

Summary

For any new technology to be adopted in main stream implementations, it has to go through a chain of events. It will be widely discussed, it will be touted as the next elixir of life, it will also see some failed implementations and then written off as just another snake oil. In the course of these events, some enlightened organizations would end up doing the right implementations, then success stories will be shared and finally it will be adopted by all organizations as a main stream technology.

Big Data has a long way to go, and it is here to stay.

UK Government and its forced IE6 use

Prasad Ajinkya4 Comments

As a developing country, we often take a look at how are some of the infrastructure problems handled by the developed nations. In fact in most of these cases, we end up emulating them … the western culture, the accents, the products … we are more than happy to borrow from our western brothers … since they are more developed.

Or so, that is what they would have us believe.

Imagine a governance system wherein you could simply go to a website, fill out the relevant form and be done with it … instead of going to the local municipal center, finding the relevant department, waiting in line, trying to fill the form and get the proper official’s blessing so that the problem could be addressed. Seems futuristic and too good to be true, eh?

Well, I was lucky to have participated in one such e-governance implementation drive back in 2002. This was for Navi Mumbai Municipal Corporation (NMMC), and we were somehow involved in the implementation of this system. So yes, these things are in India as well. But according to this article in the Inquirer, the UK has already been using this system.

So what’s the catch?

The system which the UK government has so meticulously built supports on IE6 and Windows XP!

Alright, lets take a minute for that to sink in.

  • Microsoft does not support Windows XP anymore
  • Ergo, Microsoft does not support IE6 anymore as well
  • IE6 used to come bundled with Windows XP
  • These days you get IE8 or IE9 … in fact we should be starting to see IE10 any time soon

That means in the real world … in the developed countries, IE6 is dead. Yes, we have asked for it and celebrated its death as well.

A platform created for the people in the UK which can only currently be used by the people of China (at least according to the map in the above link). I wonder, if they have a mobility middleware in place so that the site renders on mobiles.

1984 to 2010

Prasad Ajinkya6 Comments
1984

We are in the process of formalizing our IT and Electronic Access Policies. In this process, often the team drafting the initial proposal finds it pretty easy to switch to the Orwellian mode of 1984, wherein Big Brother is always watching.

Although the intent of these people often stems from the fact that they desire to protect both the system and it’s users, but that sometimes requires that extra bit of control. However, it becomes extremely difficult to tell the need for control from genuine to paranoia.

Further, due to some unforeseen events happening in the recent past, these rules are doubly strict ensuring that the past mistakes of a few have to be borne by the entire team. Only to ensure that something that happens out of the blue should not happen again (and rightly so!).

The problem at hand is how to ensure that people are enabled to work and co-create with each other, but are still protected from any malicious ill will that might exist outside (and even within the system). How can Big Brother start co-operating instead of watching?

One clear method is to assume that all people are good and need to be enabled. The other is to work closely with them step-by-step and layout a simplified process and get the end-user buy-in on each of the steps. Till that happens, Big Brother will continue to exist … we are watching you!