The concept of random sampling in employee surveys has been gaining popularity. The main benefit of random sampling is, conceivably, an organization could survey their workforce more frequently without incurring perceived “survey fatigue.” For example, imagine a situation where one-fourth of the workforce is surveyed each quarter. The organization obtains employee feedback each quarter, but each employee is surveyed only once that year.

Random sampling in surveying can be effective when used responsibly. However, there are particular challenges that we want you to be aware of when using it in an employee survey practice:

The most obvious hurdle to effective random sampling is having a population size wherein, if sampled, would still produce a statistically-representative result within an acceptable margin of error. However, the relationship between the size of the sampled population and response rate needed is not linear. In order to see if your sample size is large enough, use the chart below or this online survey size calculator.

Best practice: Only companies that are 1,000+ employees have a large enough population to be sampled, and only if 30% of the population (or more) responds to the survey. Sampling in this scenario would produce results that are representative of the entire company. However, results at the department level would not, statistically, be representative of each department.

When you are sampling a population, you want to make sure that the sample you have chosen is an accurate reflection of the entire population. For example, if your workforce is 30% sales, 30% engineering, and 40% business operations; you’ll want to make sure your sample has the same mix of functions.

You’ll want to make sure the same mix applies to all of the important demographics you care about, such as location, tenure, gender, etc. This way, your results are representative and no demographic group is disproportionately sampled.

Best practice: If you select your sample based on Employee ID, you have a fair chance at producing a sample that is stratified among important demographics. For example, if you want to survey half of your population, first survey everyone with an odd Employee ID; next, an even Employee ID.

If you’ve cleared the mathematical hurdles of random sampling, you’ll want to turn your attention to the human behavior that often prevents Random Sampling from being effective.

Employees are used to taking the survey as a collective unit. When some employees are randomly excluded, it can make the action planning process much more difficult because employees aren’t necessarily “bought-in” to the methodology.

Even if you clearly articulate how the sampling works and why it is statistically robust, you will likely encounter sentiments of “I wasn’t surveyed this time around, so I don’t believe the findings represent my feedback or sentiment.” Make sure you're prepared to answer questions and concerns you are likely to receive about Statistical Significance in Employee Surveys.

Best practice: Pick a sampling cadence where you can take action quickly and have it be felt by the entire organization. That way, even if they aren’t chosen, they still understand that employee feedback drives results and they will participate when asked. If your participation rate drops below your usual rate, it could be due to miscommunication of your sampling methodology.

Your random sample will produce results that are statistically significant at the organizational level, but likely not at the department level. Hence, your action planning process will be inaccurate and ineffective at any level where the threshold of statistical significance is not met. This can be a pretty major drawback when the purpose of the survey is to collect, understand and act on employee feedback. Most employee feedback initiatives are keen to be able to drill down into their data to:

Best practice: If you decide to go down the random sampling route, be sure to design your survey questions to collect feedback on broad, organization-wide priorities that can be acted on by leadership and the people function. Topics like leadership, company confidence, and learning and development tend to be safe topics.