Most Active Stories
Tue November 29, 2011
The Search For Analysts To Make Sense Of 'Big Data'
Second in a two-part series
Businesses keep vast troves of data about things like online shopping behavior, or millions of changes in weather patterns, or trillions of financial transactions — information that goes by the generic name of big data.
Now, more companies are trying to make sense of what the data can tell them about how to do business better. That, in turn, is fueling demand for people who can make sense of the information — mathematicians — and creating something of a recruiting war.
DJ Patil, with venture capital firm Greylock Partners, is on a perpetual manhunt, looking for a rare breed: someone with a brain for math, finesse with computers, the eyes of an artist and more.
"There's one common element across all these people that stands out above everything, and that's curiosity," Patil says. "It's an intense curiosity to understand what's behind the data."
He compares raw data to clay: shapeless until molded by a gifted mathematician. A good mathematician can write algorithms that can churn through billions or trillions of data points and show where patterns emerge.
For example, patterns indicated early on that mothers were heavy users of social networks, which in turn led to the creation of social circles. Patil says a good mathematician can figure out what matters and what doesn't in a huge trove of data.
"Everybody's looking for these people, because they know these individuals can move the needle by themselves. They're that impactful," he says.
Firms are going beyond offering piles of cash and equity to get these people. The chief executive officer at TellApart, a company that helps online marketers profile its customers, invited one math recruit to dinner with other tech luminaries to sell him on the company. Another California firm, Cataphora, which helps firms monitor employee behavior, opened a satellite office in Ann Arbor, Mich., so it could recruit from the University of Michigan's math department.
Often, executives say it's not just about money; they have to appeal to an ideal.
David Friedberg says his company does that by touting the power of risk management to change agriculture. Friedberg, chief executive officer of The Climate Corp., says crop insurance is basically nonexistent in large parts of Africa and Southeast Asia. He says the ability to better model changing climate patterns means his company can provide insurance to small farmers who otherwise might not take on the risk of farming more land.
"We can actually encourage agricultural development and provide a sustainable living for them," Friedberg says. "And that's one of the long-term missions of our organization."
The type of people drawn to this work aren't necessarily what you might expect. Greylock Partners' Patil says his successful recruits have included an oceanographer and a neurosurgeon, as well as people who barely graduated from high school but were brilliant at math. He approaches math majors the way baseball scouts look for young stars.
"I have a list that I track. There's one student who I think is phenomenal; I've been tracking him since he was 16," Patil says.
That student is Dylan Field, now 19 and a junior at Brown University. He remembers that the summer of first grade, he found an algebra book and dug into it. Now, he says math and statistics are the sexiest skills around, and his early aptitude and interest in math have become, literally, his meal ticket.
"The joke is that so many companies come to recruit these days, that you don't need to be on the meal plan, you can just go to their recruiting events and get food there," he says.
Field says it's a happy coincidence the market wants what he loves to do.
"You can understand something that is much bigger than yourself, and I think that is the most interesting property of creating a big data application," he says.
Field plans to spend much of next year working at a big data startup called Flipboard. His ultimate goal, he says, is to start his own big data company.