AI comment summaries in Engagement Results

Guidance on how to use the AI comment summaries feature for analysing comments in your engagement survey reports

Jessie Walsh avatar
Written by Jessie Walsh
Updated this week

What can I learn from this page?

How the AI comment summaries work, and how to use them in your workflow

Who is this page for?

Account Admins

Note:

  • The feature is currently in the EAP release and is set to roll out to all customers in 2024. If you're interested in being part of the EAP, please contact your coach directly or reach out to our Culture Amp Support team via email at support@cultureamp.com to inquire about being included.

  • This feature is not available in shared reports.

What are AI comment summaries?

AI comment summaries provide a quick and succinct overview of your participant’s comments appearing alongside your report results for any Engagement survey. Use them during your analysis to understand the context behind a score, and to easily pull representative comments for your result sharing.

In one click, you can:

  • See positive, negative and neutral comments summarized by question

    • For both your Ratings and Free Text questions

  • See topic level summaries by question where available

  • See representative comments for each summary

  • Copy and paste AI summaries into your slideshows or documents


How to Find AI comment summaries

While the feature is best used for your Engagement/Snapshot type surveys, Account Admins are able to access this feature across all templates available under your Engagement template section (live or closed). This includes the following:

You can learn more about our template options here.

When no demographic filters are applied, Admins will see the AI icon wherever an AI Summary can be generated. To generate, each question has to have a minimum of 5 "meaningful" comments, with detailed feedback. Our current definition of meaningful comments is:

  • has more than 2 words

  • has at least one topic tag (where the topic tag is NOT “NO_COMMENT”)

  • has a sentiment tag (where relevant)

You can find the icon on the below report pages:

  • Insight report page

  • Questions report page

  • Factor report page

  • Select Question report page

Note that Free Text summaries are accessed via the Question report page.


How to Use AI comment summaries

Imagine your company just finished its yearly survey. As you check the results report, you notice that most survey questions now have a AI comment summary icon beside them.

The results only came out this morning but the senior leadership wants a summary by this afternoon. They are especially interested in the results for the question, “I have confidence in the leaders at [COMPANY]”.

The score is low and hundreds of participants have left comments so you click on the AI comment summary icon to get an overview. Although there were many comments, there were not enough to generate a summary of the comments with positive sentiment. This already gives you an idea of what to expect. There is a summary for comments with a negative sentiment which matches what you’ve heard about the workload and unclear expectations. Clicking on the Representative Comments, you can quickly see verbatim.

You easily copy this summary and comments into a message for your manager and add them to a presentation for today's results meeting. You can rest easy knowing you have delivered insights to your stakeholders and can continue with your morning.


You can also leave feedback on the comment summary. Simply click the thumb icon and submit your rating, with the option to leave any additional comments.


How We Are Using Machine Learning & AI

We have partnered with Google to utilize their LLM foundational models in our AI products. This partnership gives us access to best in class technology that will power Culture Amp products to bring the market leading HR products to our customers.

There are 3 foundational product pillars which we use in our AI product design:

1. Trust - Customers look to us for help with sensitive workplace topics. As such, we consider privacy, security, ethics, bias and regulatory concerns in everything we do. Via thoughtful design we'll disclose where and how AI is applied, attempt to explain results, and accept and respond to feedback.

In the AI comment summaries feature, you can see the below text which tells you that this is an AI generated summary.

Also, our AI summaries will show the Representative Comments that went into making each positive or negative sentiment summary.

Representative comments for AI Comment Summary

2. Control - AI capabilities should never assume control, exclude people, nor remove the agency of a person in the course of their work. Employee experience is inherently human and AI should support but never direct. Even the best available data and technology cannot replace the contextual awareness, specific experience, and collective impact of groups of people working together.

We always encourage you to explore your comments in detail and use the summaries as a guideline, to get the maximum value out of our AI comment summaries feature.

3. Value - Product capabilities should always be clear on customer value, and that's just as true for the applications of AI. We ensure that our applications of AI have a clear benefit and are fit for purpose before unleashing them upon the world. Our AI technology aims to enhance the experience of users (people, in all their roles, from employee to manager to HR professional, to exec) by reducing unnecessary toil, surfacing insights, connections, and meaning, and augmenting people’s skill, ultimately allowing them to focus on high value, high impact activities.

Our AI comment summaries feature has a real power to reduce both time and cognitive load when interpreting qualitative data, providing substantial value to our users.

AI Security

Culture Amp prioritises customer data privacy and security throughout our AI capabilities in the same way we do across the rest of our platform. The privacy and security of your data is of the highest priority to us - and we apply the same strict protection measures with your employee feedback data as we do across our entire product offering.

Note: None of your employee data will be shared with Google Vertex. Our customer’s employee privacy and data security remains a top priority for us. No Culture Amp customer data will either be used or retained by Google to improve their models or services.

Responsible AI

Culture Amp's Responsible AI (RAI) means we continuously evaluate our existing and new practices to responsibly and ethically build technology, use data, and prepare for future regulation and customer needs.


FAQs

Who can use this feature in the engagement survey reports?

While the feature is best used for your Engagement/Snapshot type surveys, Account Admins are able to access this feature across all templates available under your Engagement template section (live or closed). You can learn more about our template options here.

Why am I not seeing an AI icon?

The AI icon will appear when:

  • No demographic filters are applied.

  • A minimum of 5 meaningful comments per question are available.

Why am I not seeing the AI summary?

Wherever an AI Summary can be generated - each question has to have a minimum of 5 "meaningful" comments, with detailed feedback. Our current definition of meaningful comments is:

  • has more than 2 words

  • has at least one topic tag (where the topic tag is NOT “NO_COMMENT”)

  • has a sentiment tag (where relevant)

Does the survey have to be launched by a certain date for this feature to be visible?

No, the feature will automatically be available for any of your past surveys, and for any future surveys you create.

How accurate are the AI generated summaries?

Quite accurate, however we always encourage you to explore your comments in detail and use the summaries as a guideline. This is because the summaries are generated by a large language model powered by Google which uses approximations. While we have done extensive testing to ensure a large percentage of these summaries are accurate; it can from time to time generate inaccurate information. Summaries are not a substitute to exploring comments in detail and hence, we always encourage you to use summaries as a guideline only.

What if the AI summary is not what I expected?

Please leave us feedback if you come across an unexpected result! This will help us keep improving the feature. Right next to each summary, you will find a thumbs up/down button which will provide more options to give feedback (like checkboxes and text box to give detailed feedback).

Will the AI summary remain the same if I come back to it a few days later?

Yes, it will remain the same as long as you haven’t updated any of the underlying comment sentiment or topic data.

If I change the sentiment or topics on comments, will it change the AI summary?

Yes, it can change if you change the sentiment or topics on one of the comments associated with the summary. It will then regenerate a new summary based on the new comments data.

Will any customer data be shared with Google Vertex?

No. Our top priority when evaluating potential partners was ensuring that our customers' data remained secure and protected. No customer data will be used or retained by Google to improve their models or services.

Did this answer your question?