Monthly Archives: August 2013

Are VC’s just stabbing around in the Big Data dark? – Part I

Big Data

Image Source: The Internet (i.e. I don’t remember where I found it)

For the past few years, we’ve seen Big Data after Big Data company getting funded. For a while it was my job to stay on top of all the new technologies, but I still do it because I’m interested in the space and where things are going. Unfortunately, I am almost always underwhelmed by what I am seeing getting funded, as there is significant amount of “me too” out there. It seemed like if you had “Big Data” in your pitch, and your founders are coming from Google, Facebook or a few other companies, you could probably raise a few million. Unfortunately, I believe a lot of these companies lack true enterprise experience, and end up with solutions chasing a problem. So what’s going on? Well, I can’t answer the investment rationale behind what has happened (even though I may have a few theories), so instead, I am going to try and categorize the big data market into a few high level segments, and talk about where I see the over-funded vs under-funded (i.e. opportunity!) areas.

Database / Platform

All aboard the big data bandwagonBetween NoSQL, NewSQL, Hadoop + affiliated technologies, etc, there are a plethora of products in this category (10gen/MongoDB, DataStax/Cassandra, Couchbase, MarkLogic, Redis, Cloudera & Hortonworks for Hadoop, Mapr, Neo4J, Basho/Riak, VoltDB, etc etc). And this is a very small list of a few of the more popular ones – there are hundreds more – and that’s without even including the MPP or In-Memory crowd either. It is easily the most over-funded category of “big data” (IMHO)… although the good news is that the rate of new investment here appears to have slowed finally.

I remember when these technologies first came out, I’d often try them out or even just try to understand under what scenarios each one was better suited for. There were lots of discussions about the CAP theorem then, and how each new product tied back to that. You’d see lots of blog posts/questions/etc on NoSQL A vs. NoSQL B, or “why I moved from NoSQL A to NoSQL B”, etc. But after a while, I would (and I think most people would) ignore a lot of the new products coming out because the amount of differentiation between each new company/product was asymptoting towards zero.

The market is tired with these products. Most people have had their fill with playing with new technologies, and want to get back to the business of actually building their own product. As a result, my personal belief is that the success of these companies will now be driven more by network effects than by the value of their underlying technology. Yes, developers want to use the best tools available to them, but the reality is that if there is no community, no information, no tools, etc behind something newly built, you probably aren’t going to invest your time and skills into it unless it is massively better (or cheaper) than what’s already out there. If it’s incrementally better, most will likely go with the safer “mature” options that already have traction. So I would see the established companies continuing to get bigger and bigger now. Of course, there will always be exceptions and new technologies can still break through, but the market today is not the same one as ~2008/2009.

Overall: I’d personally want to see some extremely compelling reason to fund another product in this space right now.

Querying / Analytics

skeptical hippoGiven the proliferation of database companies, we are beginning to see investment now go into the next layers up the stack. That is, since there are so many databases out there, there was a dearth of standardization in querying these databases, so SQL is getting much love again. Further, since many of these databases were designed for transactional processing and not analytics, we have also seen investment go towards the problem of analyzing “big” data with real-time performance instead of batch analysis. Some of the technologies I put into this category are Impala, Stinger, Spark/Shark, Apache Drill, Platfora (although they span the next category too), Google Big Query, and Hadapt. The query language and analytical processing problem is a hard and necessary one, and a huge market too, but I’m skeptical about it making a good investment for a startup right now. Let me explain why.

The combination of really powerful computing power at lower and lower costs, means a system that can do transactions and analytics, batch analysis and real-time, etc all in one technology stack, is the direction the market is swimming in. And all these vendors who own the underlying platforms are well aware of the need for querying, analytics and real-time performance, so most of them are furiously working towards building solutions. I believe they are the ones who will ultimately dominate, and a startup that tries to do just analytics (as a standalone platform) will have to fight upstream against this market direction.

Anyone who has been around in the analytics world should also know the reality of how hard it is to actually manage data by moving it between systems (integration, quality, transforms, etc), which is why getting 1 platform to do more and more in the same stack is an attractive value prop. At this point in time, Hadoop in particular, has created enough publicity (and value) that you (as NewCo) would have to fight really hard to get mind share with most managers/IT depts (i.e. you’d have to put a lot of marketing dollars to work now). I’d love to be proven wrong on this, as I think this market has huge potential while being under-served by startups (there are tons of existing companies with expensive solutions here, but few startups). But, I don’t know that I personally would want to bet on a company in this space unless they had some seriously interesting technology (which I haven’t seen yet).

You could argue: but wait, a startup could still build the best solution for an existing DB/platform, instead of rolling a completely new product. Sure they could, but then I would (especially if I was an investor) be worried about building a company that was completely dependent on somebody else’s platform (a la Zynga on Facebook). It might work for you for a while, but the platform may release something that changes the game on you. Perhaps in this category, you could argue this might be the case with Platfora building their product exclusively for Hadoop, and then Impala and Stinger coming out. To be fair to Platfora here, this is their message on how they see themselves relative to Impala, and nobody “owns” the Hadoop platform so you have a bit more control than a truly private platform (or at least you have a sense of control anyway).

Overall: I think this market has huge potential, and there aren’t a ton of startups here, but I’m not sure it would be a good area to invest in either. This is because (1) the incumbents are all working towards solutions, (2) it’s not clear which platform to bet on, (3) tying your company to 1 platform is very risky anyway, and (4) people are unlikely to buy standalone if the integrated solution from the DB/platform vendor is already good enough.

In Part II of this blog, I cover Business Intelligence. Part III of this blog will discuss Analytic Applications.


Dealing With Conflicting Advice As A Product Manager

homer getting conflicting advice

One of the most underrated skills for Product Management is the ability to synthesize a lot data, with much of it simultaneously pointing you in conflicting directions. It’s not dissimilar to trying to make a decision by looking at what “internet experts” recommend…

As a Product Manager, you will be getting input from a lot of different sources: customers (of course), analysts, competitors, prospects, advisors, industry experts, colleagues, partners, management and more. If you think you’re likely to basically hear the same thing over and over from all these stakeholders, so you can just focus on what you hear the most frequently (or loudest), you’d be wrong (for more than one reason). In fact, if these type of conflicts give you a headache, then Product Management is probably not the job for you.

Fred Wilson recently wrote about a similar experience that startups in an accelerator program go through, with every person they meet as part of the program (smart, experienced people) offering them solid but contradictory advice. He called that specific phenomenon mentor/investor whiplash. What these new startups & CEO’s are going through is what a Product Manager also has to deal with, day in, day out.

But I believe conflicting information should not pose a problem for a few reasons. Firstly, you should already be absolutely crystal clear on what your vision is for where your product is going… what will it do, what will it not do, who will use, why will they use it, what does it look like, what makes it unique, etc. If the feedback you are getting would mean steering you away from your vision, then you want to apply a very high discount factor to it. I’m not saying ignore it or don’t be open to new ideas, but it’s critical you have total commitment to your vision.

There’s nothing worse than a Product Manager who flip flops around on their product strategy based on who they spoke to last (believe me – I’ve worked for more than one of them). You will lose the confidence of your team very quickly, and you’re going to end up with a crappy product in the not-so-distant future. So as long as you have a clear picture of what your future looks like, it makes it very easy to decide how some feedback/advice/idea falls into that picture (or not). If you’re operating without a vision, you have no “filing cabinet” to put the feedback into, so you can’t distinguish good/bad, helpful/useless, relevant/irrelevant.

One of the character traits I can’t stand are people who like to surround themselves by “yes men”. To hear only one view point before you decide something is the worst case scenario: you miss implications and opportunities. So this leads me to the second reason why I don’t see conflicting information as a bad thing: when you get all differing data points, you can understand everybody’s perspective, get new ideas, and see both sides of any decision. This in turn should give you personally new ideas, things to consider, and stretch your own thinking.

People tell me I tend to ask a lot of hard questions, but I just don’t see it that way. I think if you don’t have answers to all of the questions coming your way, then maybe you haven’t thought through everything in the first place. And if that’s the case, I personally would appreciate the back and forth dialog, not complain that the questions are too hard. Questions are not criticism – it’s a way to make sure you’ve thought everything through. I personally always ask people on my team to push back hard on anything I say that they disagree with – it’s much better to hear it early, than after you’ve finished executing on a bad assumption.

5 whysLastly, getting conflicting information will often provide you with the prompt to really make sure you get to the root of the feedback/suggestion. Some people might explain this as asking “why” 5 times. Let’s say you have a customer request your product to add feature XYZ, it looks logical and something other customers you know will want it, so you put it on your backlog. But then another customer asks you for the exact opposite thing of feature XYZ. Because of the resulting cognitive dissonance, you will ask “why” they want that. Only when you ask why, will you get to the real feedback/insight. This is what you should have done the first time around anyway, but my point is that conflicting information often forces you to have that discipline because your brain will want to resolve these two opposing view points.

So as you can tell by now, i think if you are Product manager, you should be seeking out conflicting information, not shying away from it – it is a good thing, not a bad thing. Whenever my managers used to tell me I did a good job on something, I would say “don’t tell me what I’m doing well – tell me what I could be doing better”. So I was very glad to read the following earlier this week:

“They won’t always be right, but I find the single biggest error people make is to ignore constructive, negative feedback. Don’t tell me what you like, tell me what you don’t like.” – Elon Musk

Note that this does not mean you have to agree with the negative feedback and change your approach, but always be seeking alternative views. And on that bombshell, I’ll open up the comments to any hard questions you may have for me 🙂