Know your enemy: Conversational agents for security, education, training, and awareness at scale.

Social engineering attacks are widespread, costly, and challenging to mitigate through technical means. Companies allocate considerable resources toward security education, training, and awareness. However, the cost of these efforts is high, and the efficacy is questionable, particularly for defending against social engineering attacks. To improve security training for employees against social engineering at scale, we have designed a conversational agent to simulate a person becoming the victim of a social engineering attack. We build on Utility theory and previous social engineering, adversarial thinking, and conversational agent research. We designed a conversational agent to allow humans to learn and practice social engineering techniques in a fun and engaging setting. Users learn how an attacker might use these techniques by performing them. We present the design of a conversational agent for social engineering training and demonstrate a process to measure adversarial thinking in a computer-mediated conversational context. We showed that the conversational agent performs well and accurately measures adversarial thinking through two studies.