How to Test Your Users' Vulnerability to a CEO Fraud or Business Email Compromise Attack

On a phishing campaign, you have the ability to test your users with simulated phishing attacks, while also tracking if they will reply to these phishing attacks. Within your console, replies will be optionally recorded, and the raw data from the reply (in the form of a .eml file) will be available for you to download if you wish.

This is an important addition to your security awareness training plan that will help you to inoculate your users against Business Email Compromise or CEO Fraud. For example, what happens if your users receive an email from a cybercriminal pretending to be your CEO requesting an urgent wire transfer? Will the employee reply back to question the sender for more details or worse, simply confirm that they completed the transfer? Tracking replies in our console will allow you to see who your most vulnerable employees are.

Read the guide below, or watch our Reply-to Phishing tutorial video.

Jump to:

How it Works
How is the reply-to email address populated?
What if I specified a reply-to email address in my email template already?
Do you save ALL the information from the replies?
What if my user sends in sensitive information? Can I delete it?
What if my user replies multiple times to the same phishing email?
Should I track out of office replies?
What kind of phishing email should I use for a reply-to phishing campaign?
Where is the information for the replies recorded?
What do the reply emails look like in the console?
Reply-To in User Details

How it Works

You can choose to track replies on your phishing campaign. The option to track replies will be by default. After you select the Track Replies to Phishing Emails checkbox, a gray box will display with additional options for tracking replies. See below for more information on each option.

Track Replies to Phishing Emails: Enable this option to record whether or not a user replies to a phishing security test email. When this is enabled, replying to a phishing test will be considered a failure.
Custom Reply-to Address Domain: Use this to customize the email address users will see if they reply to a phishing security test. See the How is the reply-to email address populated? section below for details.
Keep reply content for later review: Enable this option to save a copy of the user's reply in your KnowBe4 console. See the Do you save ALL the information from the replies? section below for details.
Record out of office replies: Enable this option to count automated Out of Office replies as a failure. See the Should I track out of office replies? section below for details.

How is the Reply-to Email Address Populated?

The first part of the email address will be created automatically, but if you've specified a "Reply-To" name in your email template, it will use some of that information to populate the first portion of the sender email address.

For your custom reply-to domain, you can choose a domain from the list on the right to populate the reply-to email address domain, and modify the subdomain as well (by default, the subdomain will be the first part of your primary domain).

Note:

If your mail server settings require the return-path header and the reply-to address to match, then you can enable the Overwrite Return-path Address with Reply-to Address option from your account settings page. For more information, see our Account Settings article.

What if I specified a reply-to email address in my email template already?

If you specified a reply-to email address in your email template, but then use that template in a reply-to phishing campaign, the reply-to email address in your email template will be overwritten by your reply-to phishing campaign. This will ensure we will receive the replies from your users and will be able to show them to you in the console.

Do you save ALL the information from the replies?

You can choose to record all of the information from the replies, including the text of their reply and any attachments. Or you can just record that fact the user replied, without saving the content of the reply. To do this, uncheck Keep reply content for later review when creating the Phishing Campaign.

What if my user sends in sensitive information? Can I delete it?

You have the option to delete individual reply content, without impacting your campaign results. If you wish to delete the content of a reply, follow these steps:

From your KnowBe4 console, navigate to the Phishing tab and click on Campaigns.
Here you will see a list of your phishing campaigns. Click the name of the Campaign where you'd like to delete the reply content.
Click the Users tab beneath that Phishing Campaign.
From the failure options in the header, click on Replied.
From the list of users, find the user whose reply content you wish to delete. Then, click the Preview arrow icon.
In the upper right corner of the preview window, click Delete Reply Content.
A confirmation window will pop-up. Click Confirm to permanently delete the content of this reply.

What if my user replies multiple times to the same phishing email?

We will only track the first reply from your user in the console.

Should I track out of office replies?

You can choose to optionally "Track out of office replies" by clicking the checkbox next to that option. If this setting is enabled, it will cause your user to fail the reply-to phishing test if our system receives an "out of office" message as a reply. By default, this setting is turned off, and "out of office" messages will not be recorded or tracked as failures.

There is a reason why you may want to track these replies in your console. You may want to see what sort of information your users are placing in their out of office message. If a hacker can receive email addresses, phone numbers, or other identifiable information from an out of office message from one of your employees, you may want to be aware of that.

Important note: If you ARE tracking out of office replies, and you're using Microsoft Exchange or Microsoft 365, you'll want to go into your Account Settings to turn on the feature: Overwrite Sender Address with Reply-to-Address for OOO Replies.

The reason for this setting is that some mail servers send out of office replies to the “From” address rather than the “Reply To” or “Return-Path” address, in particular, Exchange and Microsoft 365. In those cases, we need to set the “From” address to the reply tracking address if you wish to capture out of office messages. If you are NOT interested in capturing out of office emails or are using a mail service that sends out of office responses to the “Reply To” address (For example, Google Workspace), then it recommended that you keep this option off.

What kind of phishing email should I use for a reply-to phishing campaign?

You can use any of our templates with a reply-to phishing campaign, but it is best to use one that makes it enticing for the user to reply, or that prompts the user to reply, as shown below. We have a category of templates called Reply-To Only which will help get you started with phishing your users in this way. There are no links or attachments in these templates, so they will ONLY track replies from your users.

Where is the information for the replies recorded?

You can find this data in your campaign reporting area. To navigate there, click PHISHING > Campaigns > [[name of your campaign]] > Users > Replied. Note, this navigation will vary slightly if the campaign is recurring, see here for more information.

Here you can see what users replied, and exactly what they replied with if you chose to keep the reply content in your phishing campaign. Click the arrow to view the reply, and the "letter" icon to view the phishing template originally sent to the user.

What do the reply emails look like in the console?

Below is an example of a reply email from employee Fritz to the "CEO". Click Download Raw Email if you want to view the original email (including any photos/attachments/other data).

User Details

Your user's reply will be added to the Phishing tab of their user profile, as shown below.