BLOG

The Hope. That Hadoop is the silver bullet that is going to save us from a relentless explosion of data. Data volumes are increasing 100x over the next 5 years, and Hadoop is THE answer to scalably storing and harnessing that data. Forget about relational databases — Hadoop has it all.

The Reality. That one scratch below the surface and you realize that this whole thing is at a much earlier stage than most people appreciate. We’re still living the first baby steps — vendors battling to provide the best low-level Hadoop plumbing. Once you get the cluster up and running, now what? You start to pump data into it. Then what? That’s when things start to unravel. You can write some basic Hive or Pig queries, and wait minutes or hours for jobs that take seconds on MPP databases. You can get down to the metal and try your hand at MapReduce programming in Java, or maybe adapt some Mahout machine-learning code. However you cut it though, unless you are willing to get your hands very dirty, it is going to be a rocky ride. This technology is a long way from main street.

The Future. That’s where we come in. We’re not ready to talk about what we’re doing yet, but we believe that the Hadoop that the world needs (or is able to consume) is a far cry from what you see today. The big data explosion is real and imminent, and companies worldwide need a way to get ahead of the onslaught and be ready to put that data to work. The Hadoop that we envision is not just evolutionary enhancements of today’s stack (although there are plenty of vendors that have that angle covered). It is tangibly more. That’s all we can say about our plans for now.

At Platfora, we are assembling an extraordinary team of big data and scalable systems experts, rock-star data scientists, and sublime UX/visualization designers. If this describes you, then we’d love to talk. It is going to be quite a ride.