What can I learn from this page? | An update to how our Impact is calculated in survey reports |
Who is this guide for? | Account Admins, Survey Admins, Survey Creators, Report Viewers |
What’s changed?
This upgrade improves how impact is calculated in survey reports. Impact will now update when filters are applied, showing more relevant drivers of engagement for any group with more than 25 people (or other selected index factors).
Why this is important
We know organizations rely on impact (calculated using a statistical technique called ‘driver analysis’) at an overall level to create alignment in action and limit overanalysis. This enhancement to allow a dynamic impact enables our users to further understand the unique drivers for different groups, and create more tailored action plans to have a greater impact on their experience at work.
With the changing world of work where more people are working remotely, distributed or hybrid, employees are feeling more connected to their immediate / local groups. So, being able to address what matters most to each of those groups is important.
When it comes to taking action, we still recommend organizations take action at the company level, but also advise managers to look at impact for their groups and choose specific actions that stand out for them.
How your company can make the most of this
Get more granular insights - you can now identify what matters most to any groups or combination of groups in your organization (where there are more than 25 responses) and take more targeted action.
What?
Survey Reports now show unique impact for your teams - these results will refresh and recalculate for any group filters that are added, whether it’s by location, department, or any combination of demographics. Remember that focus recommendations take into account impact ratings, so your recommendations will also be updated to be more relevant.
How?
View the “Insight” and “Questions” pages on the Survey Report and add a filter to see these new changes.
Example:
Example: Your company is spread out across multiple locations and departments. You know that one country, United States, is completely different compared to the others so you need an engagement plan tailored to them.
You can now add a filter for “country” and can see the unique impact for the United States. You can quickly see that “I believe there are good career opportunities for me at X” has HIGH impact, or, the highest potential to improve the US team’s engagement. This is different to what you saw as the most impactful in the Australia or New Zealand teams.
It is also a recommended focus area based on a combination of multiple inputs (indicated by the green flag / vertical bar). More detail on focus recommendations here.
Next, you have a hunch that there’s a different experience in the US based on gender and department. You can add in more filters to compare and contrast what drives engagement for these combinations of different groups.
Share more within your company - leaders can now see targeted impact for their areas of responsibility
What?
Leaders viewing shared reports now see more relevant impact when using their reports. This helps them dig into specific questions affecting engagement for their team and create a very targeted plan.
How?
This update makes survey results more relevant and actionable by leaders in your company. Use the “share reports” function to get these results out to your leaders. HR admins can also enable the filtering functionality in shared reports to help their leaders get the most out of their team’s results.
Detail:
Example: It has been a tough year with remote working and low morale. The head of Finance wants to understand what’s happening in her team. Previously their HR admin didn’t share Engagement Survey Results with department heads because it took time to figure out what each department should focus on.
With the updated impact analysis, all reports including shared reports show targeted Impact Analysis results and can be filtered to instantly see more granularity. You can automatically see this extra information without changing the way you’re using the report. This also means that focus recommendations are more tailored to the department and any additional subgroups (where there are more than 25 responses).
The HR team now feels comfortable sharing reports to their department heads knowing how easily and quickly leaders can drill in to find insights that matter to them. They can click through to share reports directly from their survey report (top right).
Keep employees in the loop - share the Survey Summary report with employees to thank them for their participation, acknowledge their hard work and play back a summary of their results
What?
The Survey Summary report is a digestible summary for employees. It shows the survey’s top and bottom results as well as a message from your company. Our research shows that companies sharing results broadly through an organization will see, for their next survey, on average, a 7 percentage point increase in participation and significantly improved scores on questions related to action taking.
How?
HR admins can set up the Survey Summary report to send out to all employees. More detail on Survey Summary here.
Detail:
Example: The engagement survey at your company has just closed. You’re aware how important it is to bring employees along the journey. To do this, you set up a Survey Summary report through the platform to thank them for their participation, acknowledge their hard work and play back a summary of their results.
Each employee will receive a link to their Survey Summary report, along with a message from your company.
Consequently at the following survey, your company sees a +7% increase in survey participation and a significant score increase on questions related to action taking (source).
Summary of improvements
When you view survey results, you’ll now see the highest impact questions based on any groups and filters you’ve selected.
In survey results, impact now automatically recalculates according to any demographic filters you add. This gives unique results for each group, and the flexibility to cut results based on your company’s hierarchies. You can find out more on impact and driver analysis here.
Along with the update to impact, you’ll now see more tailored recommendations surfaced through the Focus Agent (green bar/flag).
The Focus Agent algorithm takes the Impact Analysis into the account. Now that impact is tailored to your groups, the Focus Agent will also provide a more tailored recommendation.
What you see in the Insight Report (the survey results) and the Survey Summary report (for sharing results to employees) are now aligned.
Survey Summary was released recently as a way for employers to share survey results with employees. It was the first report to use this new calculation for driver analysis. Now, this new calculation for driver analysis has been rolled out across survey reports. So both reports now show results tailored to the relevant groups.
FAQs
How do I explain these changes to my company?
With the changing world of work where more people are working remotely, distributed or hybrid, employees are feeling more connected to their immediate / local groups. So, being able to address what matters most to each of those groups is important.
As well as organizational-level impact, the reports now help us identify impact and focus areas for departments, teams and demographic groups (where there are more than 25 responses).
When it comes to taking action, we still recommend organizations take action at the company level, but for organizations where managers are able to take action, advise managers to look at impact for their groups and choose specific actions that stand out for their teams.
Will my past results change? Does this mean past focuses are not relevant?
Past focuses are still relevant - we are using the same underlying algorithm to recommend focuses. Previously, we only showed focus at the organizational-level. This update now gives companies a way to be more granular, get even more out of their results, and see focus by demographic cuts.
We know many companies look at their last survey results. Past results will not change with this update.
How do I know how granular to go when looking at results?
There are no hard and fast rules as each organization is different. Looking at the differences between groups can be useful to understand but some organizations may not have the ability to take action at the group-level and may only be able to choose those that have the biggest differences in scores or are a high-risk group.
What will happen when I select a group with less than 25?
You’ll be able to keep drilling down however the Driver Analysis will revert to Overall Company level. There is a clear notification to let you know once you reach that point. You can get around this by making your group size bigger than 25.
Our company is not ready to give this level of detail to our managers, how would that work?
When sharing reports to your managers, you still have the option to disable demographics, which will leave the Driver Analysis at the report level.