AI Comment Summaries in Engagement Results

Guidance on how to use the AI Comment Summaries feature for analysing comments in your engagement survey reports

Jessie Walsh avatar
Written by Jessie Walsh
Updated over a week ago

What can I learn from this page?

How the AI Comment Summaries work, and how to use them in your workflow

Who is this page for?

Account Admins

Note:

  • The feature is currently in the EAP release and is set to roll out to all customers in 2024. If you're interested in being part of the EAP, please contact your coach directly or reach out to our Culture Amp Support team via email at support@cultureamp.com to inquire about being included.

  • Account Admins can access this feature through their Admin reporting view across all their available Engagement surveys (both live and closed), provided that the following criteria is met:

    • Comments are tagged with a topic and sentiment.

    • No demographic filters are applied to the reporting view.

    • Each question has a minimum of 3 "meaningful" comments per sentiment (positive/negative), with detailed feedback.

  • This feature is not available in shared reports.

What are AI Comment Summaries?

AI Comment Summaries provide a quick and succinct overview of your participant’s comments appearing alongside your report results for any Engagement survey. Use them during your analysis to understand the context behind a score, and to easily pull representative comments for your result sharing.

In one click, you can:

  • See positive comments summarized by question

  • See negative comments summarized by question

  • See representative comments for each summary

  • Copy and paste AI summaries into your slideshows or documents

How to find AI Comment Summaries

While the feature is best used for your Engagement/Snapshot type surveys, Account Admins are able to access this feature across all templates available under your Engagement template section (live or closed). This includes the following:

You can learn more about our template options here.

When no demographic filters are applied, Admins will see the AI icon wherever an AI Summary can be generated. To generate, each question has to have a minimum of 3 "meaningful" comments per sentiment (positive/negative), with detailed feedback.

You can find the icon on the below report pages:

  • Insight report page

  • Questions report page

  • Factor report page

  • Select Question report page

Example shown below:

Screenshot of a survey question showing the AI Summaries icon

Screenshot of AI Comment Summary on an Engagement Report

How to use AI Comment Summaries

Imagine your company just finished its yearly survey. As you check the results report, you notice that most survey questions now have a AI Comment Summary icon beside them.

The results only came out this morning but the senior leadership wants a summary by this afternoon. They are especially interested in the results for the question, “I have confidence in the leaders at [COMPANY]”.

AI Comment Summary for "I have confidence in the leaders at my company"

The score is low and hundreds of participants have left comments so you click on the AI Comment Summary icon to get an overview. Although there were many comments, there were not enough to generate a summary of the comments with positive sentiment. This already gives you an idea of what to expect. There is a summary for comments with a negative sentiment which matches what you’ve heard about the workload and unclear expectations. Clicking on the Representative Comments, you can quickly see verbatim.

Screenshot showing representative comments for an AI Comment Summary

You easily copy this summary and comments into a message for your manager and add them to a presentation for today's results meeting. You can rest easy knowing you have delivered insights to your stakeholders and can continue with your morning.

Copy function for AI Comment Summary

How we are using Machine Learning & AI

We have partnered with Google to utilize their LLM foundational models in our AI products. This partnership gives us access to best in class technology that will power Culture Amp products to bring the market leading HR products to our customers.

There are 3 foundational product pillars which we use in our AI product design:

1. Trust - Customers look to us for help with sensitive workplace topics. As such, we consider privacy, security, ethics, bias and regulatory concerns in everything we do. Via thoughtful design we'll disclose where and how AI is applied, attempt to explain results, and accept and respond to feedback.

In the AI Comment Summaries feature, you can see the below text which tells you that this is an AI generated summary.

Also, our AI summaries will show the Representative Comments that went into making each positive or negative sentiment summary.

Representative comments for AI Comment Summary

2. Control - AI capabilities should never assume control, exclude people, nor remove the agency of a person in the course of their work. Employee experience is inherently human and AI should support but never direct. Even the best available data and technology cannot replace the contextual awareness, specific experience, and collective impact of groups of people working together.

We always encourage you to explore your comments in detail and use the summaries as a guideline, to get the maximum value out of our AI Comment Summaries feature.

3. Value - Product capabilities should always be clear on customer value, and that's just as true for the applications of AI. We ensure that our applications of AI have a clear benefit and are fit for purpose before unleashing them upon the world. Our AI technology aims to enhance the experience of users (people, in all their roles, from employee to manager to HR professional, to exec) by reducing unnecessary toil, surfacing insights, connections, and meaning, and augmenting people’s skill, ultimately allowing them to focus on high value, high impact activities.

Our AI Comment Summaries feature has a real power to reduce both time and cognitive load when interpreting qualitative data, providing substantial value to our users.

Note: None of your employee data will be shared with Google Vertex. Our customer’s employee privacy and data security remains a top priority for us. No Culture Amp customer data will either be used or retained by Google to improve their models or services.


FAQs for Early Access Program

Who can use this feature in the engagement survey reports?

While the feature is best used for your Engagement/Snapshot type surveys, Account Admins are able to access this feature across all templates available under your Engagement template section (live or closed). You can learn more about our template options here.

Why am I not seeing an AI icon or summary?

The AI icon will appear when:

  • No demographic filters are applied

  • Wherever an AI Summary can be generated - each question must have a minimum of 3 "meaningful" comments per sentiment (positive/negative), with detailed feedback.

Does the survey have to be launched by a certain date for this feature to be visible?

No, the feature will automatically be available for any of your past surveys, and for any future surveys you create.

How accurate are the AI generated summaries?

Quite accurate, however we always encourage you to explore your comments in detail and use the summaries as a guideline. This is because the summaries are generated by a large language model powered by Google which uses approximations. While we have done extensive testing to ensure a large percentage of these summaries are accurate; it can from time to time generate inaccurate information. Summaries are not a substitute to exploring comments in detail and hence, we always encourage you to use summaries as a guideline only.

What if the AI summary is not what I expected?

Please leave us feedback if you come across an unexpected result! This will help us keep improving the feature. Right next to each summary, you will find a thumbs up/down button which will provide more options to give feedback (like checkboxes and text box to give detailed feedback).

Will the AI Summary remain the same if I come back to it a few days later?

Yes, it will remain the same as long as you haven’t updated any of the underlying comment sentiment data.

If I change the sentiment on comments, will it change the AI summary?

Yes, it can change if you change the sentiment on one of the comments associated with the summary. It will then regenerate a new summary based on the new comments data.

Will any customer data be shared with Google Vertex?

No. Our top priority when evaluating potential partners was ensuring that our customers' data remained secure and protected. No customer data will be used or retained by Google to improve their models or services.

Did this answer your question?