What can I learn from this page?
How to best understand the emerging themes within the comments report
Who is this guide for?
Account Admins, Survey Admins, Survey Creators, Report Viewers
The ability to detect unique themes specific to your company, or a particular survey, is available on engagement and other culture type surveys (e.g. Wellbeing, Values) within the comments report.
What do we use to analyse the comments?
We use machine learning techniques that can automatically detect groups of comments based on shared words or patterns of words they contain. These groups, or clusters, are highlighted as potential Emerging themes for you.
Different to our Topic analysis, where comments are tagged with one or two of our 18 pre-learnt Topics, our Emerging Themes algorithms can find themes that are unique to your business or comment set. For example, if people have commented about an office in a new city, the Emerging Themes algorithms are designed to pick up on this and will likely detect comments mentioning the city (even if the new city isn't well known).
The Emerging Theme algorithm works by finding themes within comments that have higher levels of similar word combinations and patterns. To make this more meaningful we do some pre-filtering to remove very common words and weight fairly common words less strongly. The algorithm itself works by iteratively building up groups of comments, assigning each comment to the best-known theme and repeatedly refining, We also use statistical analysis on the clusters as they are built up to determine the optimal number of Themes.
Why do we show emerging themes in comments?
We know it is common for people to feel the need to read all of the comments. This is done for various reasons;
to feel you have a better sense of what people are saying / feeling / needing
to look for sensitive comments
to identify possible opportunities, ideas or areas of focus
We know this process can be time-consuming and often there is time pressure to produce a report for executives or other stakeholders. We saw an opportunity to make this process a little faster and easier by using machine learning to pre-identify some potential key themes.
How do we see this in action?
You can see the Emerging Themes feature by clicking the Comments tab, selecting Topics and Themes, and scrolling down to Emerging themes.
How to interpret Emerging themes
The bold words listed are the most frequent words in the Emerging Theme
Below that are 3 example comments from the Emerging Theme using these words
To the right you will find the overall sentiment associated with that Theme
On the right you'll see the number of comments in that Theme and what percentage of total comments this Theme represents.
If you click into a specific Theme you can see a little more detail.
A full list of the most common words found within that Theme
A list of all the comments associated with that Theme
When is Emerging Themes available?
Emerging themes will populate after the survey is closed.
How soon is Emerging Themes available after a survey has closed?
About 2 hours after close
Who can view Emerging Themes?
Emerging Themes is available to survey admins within the comments report and anyone who can see a shared report where all comments are present. Emerging Themes is not shown when a filter has been applied.
Why is this feature only available for some questions?
We've tested our algorithms across many surveys and we've seen best results on questions with a minimum comment count of 250 comments or more. For this feature we have only run Emerging Themes on questions that meet this comment count.
Can I turn Emerging Themes off for shared reports?
Yes this is possible. Please contact firstname.lastname@example.org for assistance.
Why do I see the Emerging Themes tab but no themes?
Emerging Themes is triggered upon survey close. It does take some time to run in the background, while this is happening you will see a message where Emerging Themes will appear. It may also be that questions in your survey didn't meet the minimum comment count for this feature to work. The messages shown on the tab should make it clear why you are not (or not yet) seeing Emerging Themes.
Can I filter on Emerging Themes?
Who can edit Emerging Themes labels?
All Survey Admins can edit the labels, these will override any previous edits