In our organization, we regularly distribute cybersecurity awareness emailers to keep users informed about new attacks and threats. However, we are interested in understanding how other organizations measure the impact of these awareness initiatives on their users. Specifically, we are concerned about potential methods for assessment, such as quizzes, where there might be a possibility of cheating. Could you please share the methods you use to evaluate the effectiveness of your cybersecurity awareness programs and how you address issues like cheating?
Sort By:
Oldest
Global Intelligent Automation & GenAI Leader in Healthcare and Biotech2 months ago
Reframing the QuestionInstead of focusing on cheating in assessments, ask:
“How can we ensure that our users, clients, or employees have a comprehensive and practical understanding of cybersecurity threats and best practices? What methods can we use to foster continuous learning and gauge their knowledge effectively?”
By shifting the focus from preventing cheating to fostering a deep understanding of cybersecurity, you can build a more knowledgeable and resilient workforce. This proactive approach not only enhances the overall security posture of the organization but also empowers users to take ownership of their role in maintaining a secure environment.
Using my (ACT) Model, I recommend: Alignment with organizational goals, Clarity in communication and expectations, and Transparency in assessment and feedback can guide this strategy. This method ensures that the emphasis is on long-term knowledge retention and practical application, rather than short-term compliance.
Looking at examples: I prefer to take these approaches
Encouraging a Culture of Security:
Leadership Involvement (lead by example): Have leaders set examples by actively participating in cybersecurity training and emphasizing its importance.
Recognition and Rewards (gamification): Recognize and reward employees who demonstrate strong cybersecurity practices and contribute to creating a secure environment.
Open Communication (brave spaces): Foster an environment where employees feel comfortable reporting potential security issues without fear of retribution.
Director of Engineering in Government2 months ago
In the organizations where I have worked security team periodically send test phishing emails to check if employees are able to identify and mark a phishing attempt by clicking a button in outlook. If they identify the phishing email they get congratulated. If they click on the link they are directed to take a training. Fractional CISO in Telecommunication2 months ago
The objective of any security awareness programme is ultimately instilling a cybersecurity aware culture throughout the organisation led from the very top.In terms of measuring the effect of the programme, there are a couple of methods, neither of which are fool proof.
1. Quizzes
As part of your awareness training (personally I recommend micro training modules throughout the year) include a quiz. To minimise cheating, have a reasonable sized bank of questions that are randomly selected from. In the past I have experienced teams sharing the answers.
2. Phishing Simulations
Sending simulated phishing emails is another method of testing, but they need to be done in a supportive way, its not about catching people out. The whole company should be involved (either in one go or phases) from the CEO down. Try different types of complexities of email to get a broad understanding. And if people click the link, send them through to a training module. Do not make it a name and shame exercise.
Director of Supply Chain2 months ago
Assuming you have a multifaceted cyber education goal and are already employing obvious assessment methods like phishing simulations, your primary concern seems to be over assessment tools that allow for simple answer sharing (e.g., 1:A, 2:B, ...).Posting the question/answer pair clearly does not lead to learning, so addressing this issue with anti-cheating measures embedded in the testing tool is crucial. Some strategies to combat this kind of cheating include:
Removing question and/or answer designations while mixing up the answer choice order
Randomizing question order
Pulling questions from a large question bank
Avoiding patterns such as the longest answer always being the correct one
With a reasonable approach to preventing simple question/answer pairing, we could view the act of someone writing out a question and answer to share as a form of reinforcement. It might even be considered community-led learning, knowing that reading the question paired with the correct answer is a valid learning strategy. However you view it, try to work with human nature rather than combat it where you can.
Regardless of your approach, since we have no practical means to deliver content in person or proctor cyber tests at scale, it is necessary to implement a routine process to seek out sharing mediums and use that data to prioritize which questions should be rotated out. This process could also provide an opportunity to improve on existing questions and replace them with more cognitively meaningful ones while your keep your content fresh and relevant. Bloom's Taxonomy is one reference I use when seeking to ask meaningful questions: Bloom's Taxonomy.
If you can measure people's ability to accurately the phishing campaigns your org performs, or reducing the # of repeat offenders of security offenses, that is a better measure of the program. Measure the critical thinking and actions, not the ability to recall the required training class contents.