In this article, you will learn how to build advanced reporting processes with Smart Groups. We recommend customizing comparative reports to show a variety of statistics, from your users’ phishing failures and training statuses to the overall impact of your security awareness program. This article also explains how to build some other common reporting processes.

For general information about using Smart Groups, see our Smart Groups Quickstart Guide.

Comparative Reporting with Smart Groups

Comparative reports allow you to analyze data from multiple groups in one report. When you create a comparative report with Smart Groups, it will automatically stay up-to-date after you set it up. We recommend scheduling your report to be sent periodically to your organization’s stakeholders so that they regularly receive updated versions of your report.

To learn how to plan and set up a Smart Group, see our Smart Groups Quickstart Guide.

Step One: Create Your Smart Groups

First, create Smart Groups that segment your users by your preferred criteria. We recommend referencing our Smart Groups: Glossary of Criteria Types to find the best criteria types for your Smart Group.

In the examples below, you’ll learn how to build Smart Groups to filter your users by a specific department, division, and location. Click on the tabs below to view our recommended settings for each Smart Group.

Users by Department

Create a Smart Group and name it something identifiable, such as “Reporting Smart Group – [Department Name]”. Add the following User Field criterion to this Smart Group:

	1. User Field: Select Department.
	2. Condition: Select Must.
	3. Comparison: Select Equal.
	4. Values: Enter the name of the department.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

User Field

The department must be equal to [name of department].

Users by Division

Create a Smart Group and name it something identifiable, such as “Reporting Smart Group – [Division Name]”. Add the following User Field criterion to this Smart Group:

	1. User Field: Select Division.
	2. Condition: Select Must.
	3. Comparison: Select Equal.
	4. Values: Enter the name of the division.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

User Field

The division must be equal to [name of division].

Users by Location

Create a Smart Group and name it something identifiable, such as “Reporting Smart Group – [Location Name]”. Add the following User Field criterion to this Smart Group:

	1. User Field: Select Location.
	2. Condition: Select Must.
	3. Comparison: Select Equal.
	4. Values: Enter your desired location.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

User Field

The location must be equal to [desired location].

Step Two: Create Your Report

Now that you’ve created your Smart Groups, add them to a report to review data for those groups. There are a variety of report types available in your KSAT console. For example, a comparative Training Status Overview Report displays the quantity of users in each selected Smart Group who have received, started, or completed their training within a specified period of time.

Once you’ve selected a report type, follow the steps below to set up your comparative report:

1. In your KSAT console, navigate to the Reports tab.
2. Select the type of report you want to create.
3. In the Report Type drop-down menu, select the option ending in “by Group”. For example, the Phish-prone Percentage Report includes the Phish-prone Percentage by Group report type.
4. In the User Groups drop-down menu, select your reporting Smart Groups.
5. Click the View Report button to see the results of your report.
6. Click the Save Report button. Select Generate PDF or Generate CSV to download the information from the report.

Since your report will update over time based on changes to your Smart Groups, we recommend scheduling this report to be sent periodically to stakeholders in your organization. To learn how to schedule and send a report, see our How to Save and Send Reports article.

Risk Reporting with Smart Groups

We recommend using Smart Groups to build a report on the high-risk users in your organization. This report helps you identify areas for improvement in your security awareness program.

Step One: Create Your Risk Reporting Smart Groups

In this section, you’ll learn how to build a variety of risk reporting Smart Groups. Your report can include any combination of these groups. Once you’ve created your Smart Groups, move on to Step Two to learn how to build a report with these groups.

Click on the tabs below to see our recommended settings for each Smart Group:

Untested Users

This Smart Group includes users who have never received a Phishing Security Test (PST). Since these users have not practiced identifying phishing emails, they are more likely to fall for a real phishing attack.

To create this Smart Group, add the following Phish Event criterion:

	1. Condition: Select Must Not.
	2. Phish Event: Select Delivered.
	3. Comparison: Select Greater Than.
	4. Count: Select 0.
	5. Time Frame: Select Any.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

Phish Event

User must not have received an email more than 0 times.

Users Who Frequently Fail PSTs

This Smart Group includes users who frequently fail PSTs.

This criterion works best if you send PSTs to your users at least monthly. If you test your users less frequently, lower the Count of phishing failures and expand the Time Frame of this criterion.

To create this Smart Group, add the following Phish Event criterion:

	1. Condition: Select Must.
	2. Phish Event: Select Failed Phishing Test.
	3. Comparison: Select Greater Than.
	4. Count: Select 3.
	5. Time Frame: Select Duration. Then, select In the last, 6, and Months.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

Phish Event

User must have failed a phishing test greater than 3 times in the last 6 months.

High-Risk Users

This Smart Group includes users who have a high risk of failing a PST. This group is filtered by your users’ Personal Risk Scores and phishing test failures.

To create this Smart Group, you'll need to add criteria for User Date, Phish Event, and User Field. For more information on these criteria, click the tabs below:

After you’ve filled out the fields listed above, click Save. Your finished criteria will state:

User Date	User must not have been created in the last 3 months.
Phish Event	User must have had received an email more than 2 times in the last 6 months.
User Field	The Risk Score must be greater than [current organizational Risk Score percentage].

Untrained Users Involved in a Data Breach

This Smart Group includes untrained users who have been involved in a data breach.

Before you create this Smart Group, use our free Email Exposure Check Pro (EEC Pro) tool to search for breached users. When an EEC Pro search finds that a user has been involved in a data breach, this breach event will be automatically added to the user’s User Timeline in your KSAT console.

To create this Smart Group, you'll need to add Breach Event and Training criteria. For more information on these criteria, click the tabs below:

After you’ve filled out the fields listed above, click Save. Your finished criteria will state:

Breach Event	User must have been in a security breach more than 1 time.
Training	User must not have completed any available courses in the last 12 months.

Untrained Users Who Frequently Fail Phishing Tests

This Smart Group includes untrained users who have failed multiple PSTs. These users may benefit from completing security training.

To create this Smart Group, you'll need to add Training and Phish Event criteria. For more information on these criteria, click the tabs below:

After you’ve filled out the fields listed above, click Save. Your finished criteria will state:

Training	User must not have completed any available courses in the last 12 months.
Phish Event	User must have failed a phishing test greater than 1 times in the last 12 months.

Users with a High Phish-prone Percentage

This Smart Group includes users with a high Phish-prone Percentage. We recommend assigning additional security awareness training to this Smart Group.

To create this Smart Group, add the following criterion:

	1. User Field: Select Phish-prone Percentage.
	2. Condition: Select Must.
	3. Comparison: Select Greater Than.
	4. Value: Enter your organization’s average Phish-prone Percentage.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

User Field

The Phish-prone Percentage must be greater than [current organizational average].

Users who Failed a Phishing Test After Training

This Smart Group includes users who failed a PST after completing security training. These users may benefit from completing remedial training.

To create this Smart Group, add the following criterion:

	1. Condition: Select Must.
	2. Phish Event: Select Failed Phishing Test.
	3. Training Event: Select Completed.
	4. Assignments: Select the training modules from your organization’s annual training campaign.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

After Training

User must have failed a phishing test after completing [annual training assignments].

Untrained Users in High-Risk Roles

This Smart Group includes users who have not completed role-specific security training courses. Before you configure this Smart Group, create a static user group that includes the users in the role you want to target. To learn how to build static groups, see our Users and Groups article.

Once you’ve made your static user group, create a Smart Group with User Field and Training criteria. For more information on these criteria, click the tabs below:

After you’ve filled out the fields listed above, click Save. Your finished criteria will state:

User Field	The group name must be equal to [targeted group].
Training	User must not have completed all of these [number] assignments.

Step Two: Create a Risk Report

Once you’ve created your risk reporting Smart Groups, make a report that includes all of these groups. Your risk report shows you how the user count of each Smart Group has changed over time. This report helps you track your organization’s overall security risk level.

Follow the steps below to build your report:

1. In your KSAT console, navigate to the Reports tab.
2. Select User Count and Login Reports.
3. In the Report Type drop-down menu, select User Count.
4. In the User Groups drop-down menu, select your risk reporting Smart Groups.
5. In the Date Range drop-down menu, select your target date range.
6. Click the View Report button to see the results of your report.
7. Click the Save Report button to save your report. Select Generate PDF or Generate CSV to download the information from the report.

Since your report will update over time based on changes to your Smart Groups, we recommend scheduling this report to be sent periodically to stakeholders in your organization. To learn how to schedule and send a report, see our How to Save and Send Reports article.

Success Reporting with Smart Groups

We recommend using Smart Groups to build a report on your users who have strong security awareness skills. Your success report shows how your security awareness program has impacted your organization over time.

Follow the steps below to create your success report.

Step One: Create Your Success Reporting Smart Groups

This section covers a variety of success reporting Smart Groups. Your report can include any combination of these groups. Once you’ve created your Smart Groups, move on to Step Two to learn how to assign these groups to a report.

Click on the tabs below to see our recommended settings for each Smart Group.

Users Who Frequently Report Phishing Tests

This Smart Group includes users who frequently report PSTs using the Phish Alert Button (PAB).

To create this Smart Group, add the following criterion:

	1. Condition: Select Must.
	2. Phish Event: Select Reported.
	3. Comparison: Select Greater Than.
	4. Count: Enter your desired number of reported phishing emails.
	5. Time Frame: Select Any.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

Phish Event

User must have reported a phishing email more than [desired number] times.

Proficient Reporters With No PST Failures

This Smart Group includes users who have never failed a PST and who have reported at least one PST using the PAB.

To create this Smart Group, you'll need to add two Phish Event criteria. For more information on these criteria, click the tabs below:

After you’ve filled out the criteria listed above, click Save. Your finished criteria will state:

Phish Event	User must have reported a phishing email more than [number] times in the last 12 months.
Phish Event	User must not have failed a phishing test more than [number] times in the last 12 months.

Currently Tested Users

This Smart Group includes users who have regularly received PSTs over the past 12 months. This group will help you verify that your users are receiving the PSTs you send.

To create this Smart Group, add the following criterion:

	1. Condition: Select Must.
	2. Phish Event: Select Delivered.
	3. Comparison: Select Greater Than.
	4. Count: Enter your desired number of delivered PSTs. If you send monthly phishing campaigns to your users, enter 11. If you send phishing campaigns less frequently, adjust this number accordingly.
	5. Time Frame: Select Duration. Then, select In the last, 12, and Months.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

Phish Event

User must have had received an email more than [number] times in the last 12 months.

Users Who Improved After Remedial Training

This Smart Group includes users who failed a PST, completed remedial security training, and did not fail any PSTs after finishing that training. This group tracks the effectiveness of your remedial training campaigns.

To create this Smart Group, add the following criteria:

After you’ve filled out the fields listed above, click Save. Your finished criteria will state:

Phish Event	User must have failed a phishing test more than 0 times on or before [desired end date].
After Training	User must not have failed a phishing test after completing these [number] assignments.

Currently Trained Users

This Smart Group includes users who have completed training in the last 12 months. If you want to filter users by a different time frame, adjust the Time Frame setting of the criterion.

To create this Smart Group, add the following criterion:

	1. Condition: Select Must.
	2. Training Event: Select Completed.
	3. Scope: Select Any Available.
	4. Time Frame: Select Duration. Then, select In the last, 12, and Months.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

Training

User must have completed any available courses in the last 12 months.

Step Two: Create a Success Report

Once you’ve created your success reporting Smart Groups, create a report that includes all of these groups. Your success report shows the change in user count of each group over time. This report tracks the effectiveness of your security awareness program.

Follow the steps below to build your report:

1. In your KSAT console, navigate to the Reports page, then click the Reports tab.
2. Under the Reports tab, select User Count and Login Reports.
3. In the Report Type drop-down menu, select User Count.
4. In the User Groups drop-down menu, select your success reporting Smart Groups.
5. In the Date Range drop-down menu, select your target date range.
6. Click the View Report button to see the results of your report.
7. Click the Save Report button. Select Generate PDF or Generate CSV to download the information from the report.

Since your report will update over time based on changes to your Smart Groups, we recommend scheduling this report to be sent periodically to stakeholders in your organization. To learn how to schedule and send a report, see our How to Save and Send Reports article.

Additional Reporting Processes

Smart Groups help you build a variety of reporting processes. The sections below include examples of common automated reports.

Create a Group Report Card

The Group Report Card displays a group’s Risk Score, training status, and other key metrics. If you assign a Smart Group to the Group Report Card, your report will stay up-to-date over time. We recommend scheduling this report to be sent periodically to stakeholders in your organization.

First, create a Smart Group to filter a specific type of user. This group can be customized to fit your organization’s needs. For example, you could create a Group Report Card for users in your Sales division. To create the Sales Smart Group, add the following criterion:

	1. User Field: Select Division.
	2. Condition: Select Must.
	3. Comparison: Select Equal.
	4. Values: Enter the name of the group you’d like to target.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

User Field

The division must be equal to [name of targeted group].

Then, use the steps below to assign the Smart Group to a Group Report Card:

1. In your KSAT console, navigate to the Reports tab.
2. Select Group and User Report Cards.
3. Under the Report Type drop-down menu, select Group Report Card.
4. Under the Group drop-down menu, select your Sales Smart Group.
5. Select a Date Range that fits your organization’s needs.
6. Click the Save Report button. To download the report, select Generate PDF.

Repeat the process above to create additional Smart Group report cards.

Create Manager Reporting for Specific Divisions

Manager reporting allows you to create reports about a certain manager’s team or a specific division in your organization. Then, you can share these reports with the relevant manager.

Before you create this Smart Group, verify that the Manager Email field is up-to-date in the Users tab of your KSAT console.

Then, add manager usernames to sort your users by their manager. We recommend customizing these usernames to fit your organization’s reporting hierarchy. For example, users who report to a manager named Smith could have the username ManagerSmith. Add the usernames to Custom Field 1 in the Users tab of your KSAT console.

Once you’ve added your manager email addresses and usernames, create a Smart Group for each manager. Add the following criterion to each group:

	1. User Field: Select Custom Field 1.
	2. Condition: Select Must.
	3. Comparison: Select Ends With.
	4. Values: Enter the username of the manager whose team you want to create a report for.

After you’ve filled out the fields listed above, click Save. Your finished criterion will state:

User Field

The Custom Field 1 must end with [desired manager username].

After you create your manager Smart Groups, assign them to a report that fits your organization’s needs. Then, send the report to each manager. To learn how to share a report, see our How to Save and Send Reports article.

Provide Managers Custom Reporting Access for Their Team

If your organization requires access to custom reports, our Security Roles feature allows you to give managers access to view Smart Groups in your KSAT console. With this access, managers can review data on specific groups of users, such as their direct reports. For example, you could create a Smart Group of users under a certain manager who have failed multiple phishing security tests (PSTs).

The manager can view this Smart Group and communicate with their team as needed.

First, add your organization’s managers to a static user group. We recommend using our CSV Import or Quick Import features to quickly group your managers in your KSAT console.

Next, create a Security Role for your static group of managers. To set up this Security Role, follow the steps below:

1. Navigate to the Users page, then click the Security Roles tab.
2. Click the New Security Role button in the top-right of the screen.
3. In the Groups drop-down menu, select your static user group.
4. Under the General tab of the security role, select Read permissions for Users & Groups.
5. In the Targeted Groups drop-down menu, select the Smart Groups that you would like your managers to access.
6. Click the Create Security Role button.