Understand the removal rule in leader-based reports

If you have opened a full reporting line leader-based report, or are using the leader drop-down in an administrator report and found the numbers are not quite what you expected, a confidentiality rule called the removal rule is the most likely reason.

A full reporting line leader-based report shows you results for everyone below a leader, not only their direct team. That wider view is what makes these reports so useful, but it can also create a risk of small groups being identified.

For example, a manager who has only one person reporting to them could see a score for that team, which would really be reading one person's individual answer. To reduce this and similar risks, Culture Amp removes responses that could introduce a risk before the report reaches you.

This article will explore when the removal rule applies, other settings that can interact with it, and how you might be able to identify and explain these cases on your end.

Everything the removal rule does comes back to a single number: the smallest number of responses a group is allowed to have before it needs to be protected. That number comes from a setting on your survey's Confidentiality page, under indirect identification protections.
​

New surveys start on Basic. So unless someone has changed it before the survey launched, only groups of exactly one response are removed by this rule by default.

We will use the same team for every example in this article. Anna is a senior leader, and eleven people sit below her.

The removal rule works through the same six steps every time a full-reporting line leader-based report with drill-down capability is loaded.

You will only see a message about removals being applied when more than 10% of the responses behind a result were removed. When it does appear, it reads:

Steps 1 to 4 account for most removals on their own, and a single change to Anna's team in the example above provides a good example of them working as expected.

Say Omar does not answer the survey. Carmen's team is left with one response, from Nina. No filter is applied, so Carmen's group stays whole and counts as a single group holding one response. That is below the smallest group the survey allows, as it is using Basic indirect protection settings, so Nina's response is removed. Anna's scores are now built from nine responses rather than ten.

Nothing on the screen mentions it. One response out of ten is exactly 10%, and a message needs more than 10% of responses removed before it will appear, so Anna's report looks completely normal and her totals simply come out slightly lower than she expected. It is easy to assume something has gone wrong here, but this is the rule doing what it was designed to do.
​

Step 5 is the one that catches people out, because taking out a small subgroup is sometimes not enough on its own. Its score could still be worked out from what is left behind.

Imagine Anna breaks her report down by Location. Within Bruce's team, Priya works in Melbourne, while Sam, Tom and Uma work in Sydney. That leaves Melbourne with one response and Sydney with three.

Melbourne has too few responses, so it is removed. But if Sydney's score stayed on the page, Anna could subtract it from the score for Bruce's whole team and work out Melbourne's answer for herself. To close that gap, Sydney is removed as well.

Sydney is what we call a buffer group. It has plenty of responses of its own, but it is still removed so that the smaller subgroup beside it stays private. The result is that all four of Bruce's responses disappear from the Location breakdown.

Small subgroups sometimes protect each other without the need for a buffer, and sometimes they cannot. The difference comes down to whether they sit inside the same manager's group, and can help to explain why results can sometimes seem inconsistent.

Continuing with the example we have been using, let's just change one more detail. Priya is in Melbourne, Sam is in Brisbane, and Tom and Uma are in Sydney.

Two subgroups now hold one response each. Melbourne and Brisbane are both removed, just as before. Together, though, they add up to two responses, which reaches the smallest group Anna's survey allows. Anna cannot work out either one from the other, so no buffer is needed, and Sydney's two responses stay on the page.

That leaves two responses removed instead of four. Having more small subgroups actually meant less data was removed in this situation due to the reduced risk.

Now picture a different scenario, where only Priya answered in Bruce's team and only Nina answered in Carmen's.

Those two responses sit in separate Manager groups, so they are judged separately (As per step 2). Each Manager group holds a single subgroup of one response, and each one is removed. The rule never adds them together, because that only happens inside one manager's group.

Because the removal rule only ever reaches two levels down, the same team can be treated differently depending on who you select. This is worth knowing before you compare two reports side by side.

Say Zoe does not answer the survey, which leaves Will as the only person in Priya's team who does.

Bruce therefore sees slightly less than Anna does about the very same people. It can feel back to front at first, but looking further down the organization protects more, not less.

In some cases, the rule may remove responses more strictly than it needs to. This is a deliberate strategy to reduce the risk of edge cases.

Removed responses change some numbers on a report and deliberately leave others alone. On a single screen, you can be looking at both kinds at once.

So the top of your report can say ten responses while a factor score just below it says nine. Both figures are accurate. They are counting different things, which is why setting them next to each other can be misleading.

You may also see a demographic vanish from a breakdown rather than show a zero, which happens when every response behind it was removed. Smaller demographic groups can be gathered together into a single row called All Others.

The Heatmap is where a lot of people notice the removal rule being applied for the first time, and it can be a difficult report to make sense of the rule in action.

The heatmap breaks your results down by a demographic (same as applying a filter), so its groups are smaller than the report's overall participation. Where removal leaves a subgroup below your Reporting group minimum, that cell has nothing left to show. A whole column can also drop out of the heatmap when every response behind it was removed.

If you are looking at an empty cell on a heatmap, it is worth checking the Insight tab for the same leader before assuming your data is incomplete.

The Leader selector cannot be applied to a few reports, which means the rule never runs on them:

If you are looking at one of these and something seems to be missing, the removal rule is not the cause. Comments in particular have their own confidentiality settings, including a separate Comments group minimum.

While it is not possible to switch the removal rule off for a survey that has already launched, there are still a few things that can help if it is proving difficult.

💬 Need help? Open a Support Conversation to speak with our Product Support team.