What is Big Data and Data Science? If you know, Harvard just called you "sexy"

Jan 14, 2014

What did Harvard Business Review call the “sexiest” job of the 21st century?

Data science is growing at least 10% a year (probably more) and is in such demand that freelance professionals can make hundreds of dollars an hour (again, probably more). You’ve may have heard all the articles proclaiming how it will change business and save the world. Gartner estimates that there will be 4.4 million IT jobs created to support data analysis in just the next two years.

But what IS it? Even if you have no plans of becoming a data scientist, you’re going to run into one of these people and should know who they are. It’s sort of like knowing what social media was in 2004. (Meaning it makes you cooler.)

Imagine you’re a water distribution company, and you send out millions of bottles of water to supermarkets all across the country. You also have terabytes of data on when grocery stores tell you they’re out of stock. But how do you take this mountain of bytes and turn it into actionable, explainable insights that convince business leaders to take certain actions? And can you create a system that predicts when product will run out and sends the water in advance?

To do all that, you need a data scientist. And a data scientist’s ability to take all that information and give business leaders the insight to make smart, fast decisions makes them extremely valuable -- especially in this age of endless digital information gathering. Of course, there are a lot of different kinds of data science, and even data scientists debate about what data science really is, but all seem to agree that it’s about finding and telling a story with data.

You’ll see that a lot of data scientists have serious chops, and it’s not usually the kind of career that you just wake up one day and start. The most successful data scientists often have Ph.D.s in Applied Mathematics or Statistics -- or even, as a result of such high demand, the brand-new Data Science degrees offered by forward-thinking universities like NYU and Berkeley.

That said, like other high technical skill careers in tech, it appears that pure talent (even self-taught) can get you far.

Enter Kaggle, “the world’s largest community of data scientists,” where companies put up challenges, usually with large cash prizes, and data scientists compete against one another based on skill and result. The platform is also a kind of recruiting tool for companies looking for data scientists; it ranks all of its members on a giant leaderboard. Kaggle also provides tutorials and lessons for fledgling data scientists.

The founder of Kaggle even told IEEE Spectrum that data scientist are making a full-time living through the site, working freelance for various companies. In fact, the field is increasingly driven by freelancers: “The tool set of data science is extremely broad, and so it’s precisely the kind of thing that you can imagine somebody doing as a freelance way of life, in which you work on different problems,” Chris Wiggins, professor of Applied Mathematics at Columbia, says.

Any freelance data scientists amongst us? What data would you love to know about your business (marketing data? how people hear about you?) that you could turn into actionable goals?