Is that study relevant? Ask questions, pay attention to your own experiences

As our society has become more diverse, we’ve discovered a myriad reasons to question what once seemed like rock-solid research. Decades of medical findings, for example, turned out to be based on studies conducted almost exclusively on male subjects, raising legitimate concerns that many medicines and therapies might perform differently on women or children.

These days, there’s another factor that could be skewing results in an even wider range of research, from medicine to the social sciences: crowdsourcing. As academic and corporate researchers are forced to deal with tightening budgets, some are turning to “gig” economy sites like Amazon’s Mechanical Turk, where remote workers can earn anywhere from a few cents to a few dollars for participating in a variety of micro-tasks.

More than 800 published academic studies last year based their findings on the responses of survey subjects recruited through Amazon’s mTurk crowdsourcing platform, the Pew Research Center recently reported.

The people working those gigs, however, are not representative of the population at large, Pew found: U.S. workers are younger, less diverse and more educated than the nation’s workforce in general (while on average earning less than the minimum wage). What’s more, median respondents in Pew’s survey participated in a total of 300 academic studies. In other words, a limited population of subjects could be making an outsized contribution to a lot of research, calling into question how applicable and valuable some of those studies are.

What does this mean for business in general, and for customer-focused digital business initiatives in particular? It suggests that any organization should exercise caution about how much importance it places on third-party research until it can get some more details about who those researchers surveyed, how many and how.

Depending upon your type of business, it could be more effective to conduct more in-house research of your own, and to share those findings among your different business units. A study last year by researchers in Texas, for example, found that retailers that share their customer service experiences (inflowing information) with other stores in the same company (that is, outflowing information) tend to be more innovative.

“We were surprised to find that not only is large inflow important but large outflow is also important — how a store tries to teach other stores about how to serve its customers,” said University of Texas at Dallas researcher Eric Tsang. “If an innovative idea works in one particular location, that store should tell other stores. We found that knowledge outflow has a positive impact on the innovative behavior of employees.”

In other words, always exercise some healthy skepticism when considering outside research… but keep an open dialogue with all relevant parts of your business when it comes to discoveries that come from inside research and experience.