PROM05 – Research Methods

Navigating My Research Journey: Choosing the Right Methods for My Human-centred Cybersecurity Study

As I embark on my research project, “EVALUATING THE IMPACT OF ADAPTIVE MULTI-FACTOR AUTHENTICATION ON USABILITY AND PERCEIVED SECURITY IN ENTERPRISE ENVIRONMENTS: A Human-centred Study of Context-Aware Authentication Systems and Their Effect on Employee Experience and Trust,” one of the most critical steps is to meticulously plan my research methodology. This project delves into complex human factors within cybersecurity, meaning my approach to assessing and evaluating findings must be robust, nuanced, and capable of capturing both objective measures and subjective experiences.

Why Methodological Rigor is Paramount for My Study

My study isn’t just about technical systems; it’s about how people interact with them. As such, I need methods that can effectively uncover user perceptions, behaviours, and experiences. Research in usable privacy and security often involves concepts such as subjective experience, attitudes, understanding, behaviour, and behaviour change. A well-chosen methodology ensures that I can gather the necessary evidence to draw valid conclusions about the impact of Adaptive Multi-Factor Authentication on usability, perceived security, employee experience, and trust. Without grounded knowledge of qualitative and quantitative research, research results can easily be erroneous.

A Mixed-Methods Approach: The Best of Both Worlds

Given the multi-faceted nature of my research questions, I anticipate employing a mixed-methods approach. This combines both quantitative and qualitative research methods, allowing me to gain a comprehensive understanding of the phenomenon. I want to measure the “what” (e.g., how often do users find AMFA difficult?) and explore the “why” (e.g., why do users feel a certain way about AMFA?). This approach is beneficial as it provides triangulation and allows research questions to be studied from different perspectives.

Quantitative Methods: Measuring the Impact

Quantitative methods will be crucial for assessing the measurable aspects of usability and perceived security. These methods often involve numerical data and statistical analysis.

Method	Key Application in My Project	Benefits
Surveys and Questionnaires	I will design surveys to gather data on users’ perceived usability (e.g., using scales like the System Usability Scale), perceived security, and trust in AMFA systems. These can include Likert scales and multiple-choice questions.	Surveys are excellent for collecting data from a larger sample size, allowing for systematic collection of structured data. They are a strong quantitative research tool when scaled up to encompass a large number of users. This data is typically gathered to inform the broader security and privacy community about user needs, behaviors, and beliefs.
Usability Metrics	Analyzing operational logs from AMFA systems can provide empirical data on user burden and task efficiency (e.g., time to login), as seen in 2FA systems. Software can also provide continuous data logging of user performance and information about patterns of use.	Quantitative analysis of operational logs can support findings from user studies and surveys, providing at-scale impact assessments of 2FA implementations. These methods help compare and systematically assess the trade-offs between security and usability.

Qualitative Methods: Understanding the Experience

To truly grasp the “human-centred” aspects—employee experience and trust—qualitative methods will be indispensable. These methods delve into the depth of human perspectives, providing rich, descriptive data.

Method	Key Application in My Project	Benefits
Interviews	I plan to conduct interviews with employees who use AMFA. This will allow me to explore their daily experiences, specific pain points, feelings of trust or distrust, and any workarounds they might employ due to authentication fatigue.	Interviews can provide detailed and qualitative information. One-on-one interviews offer more qualitative results that can serve as the basis for understanding users’ expectations, vocabulary, goals, and perceptions. These are often used in combination with surveys to identify privacy panic situations and understand privacy concerns.
Focus Group Discussions	These can complement individual interviews by allowing me to observe group dynamics and shared experiences regarding AMFA. Discussions might uncover common themes and concerns that individual interviews might not fully capture.	Focus-group discussions can be productive because the interviewer can pursue specific issues of concern to help in better understanding the users’ perspectives.
Think-Aloud Protocols	During usability testing sessions, asking participants to “think aloud” as they interact with an AMFA system can reveal cognitive processes, confusion points, and decision-making relevant to usability.	Early user studies can be done with prototypes by asking users to perform tasks while thinking aloud and iterate over that feedback.

Assessing Perceived Security and Trust

User perceptions are critical, as they significantly influence how security technologies are used and adopted. Research shows that user perceptions can sometimes deviate from technical security features, which can negatively affect security. My study will directly address this.

• Questionnaires Tailored to Perceived Security and Trust: Metrics for quantifying the impact of personalized knowledge-based user authentication solutions include perceived security, memorability, trust, and likeability, often measured using questionnaires tailored to the domain. The System Usability Scale is also widely used in security studies for perceived usability.
• Context-Specific Evaluations: I recognize that user preferences and perceptions of usability, security, privacy, and trust can be related to the context of authentication. Therefore, I will ensure my evaluations consider various enterprise scenarios and contexts of use.

Data Analysis Strategies

Once I’ve collected my data, the analysis phase will bring it all together.

• Quantitative Data Analysis: I will use statistical methods to analyse survey responses and system logs. This might include descriptive statistics, inferential statistics (e.g., t-tests, ANOVAs) to compare groups or conditions, and correlation analyses to identify relationships between usability, perceived security, and system usage.
• Qualitative Data Analysis: For interview and focus group data, I’ll employ techniques like thematic analysis or grounded theory. This involves systematically coding transcripts to identify recurring themes, patterns, and categories related to employee experience and trust. Dedicated Computer-Assisted Qualitative Data Analysis Software such as NVivo, Dedoose, F4analyse, or Delve can be invaluable for facilitating distinct data analysis actions like coding and clustering data.
• Mixed-Methods Integration: The real power of a mixed-methods design lies in integrating the findings. I plan to use computational notebooks to facilitate this, as they offer a unified environment for mixed-methods analysis, combining qualitative and quantitative data exploration and allowing research questions to be studied from different perspectives.

Ethical Considerations

Given that my research involves human participants, ethical considerations are paramount. I will ensure all data collection adheres to ethical guidelines, including obtaining informed consent from research participants, as is a common ethical practice in human subjects research. Researchers must often secure approval from ethical review committees (e.g., institutional review boards) before conducting human subjects research. It’s crucial to consider the ethical implications of data collection, especially when dealing with sensitive private data or when addressing topics where what is morally right or acceptable is not clear-cut. Informed consent, along with transparency of designer intentions, can assist with trust-building with users.

Moving Forward with My Research

By carefully selecting and applying these research methods and analysis techniques, I am confident that I can conduct a rigorous and impactful human-cantered study. This approach will allow me to not only evaluate the technical aspects of AMFA but also, crucially, to understand its true impact on the employees who use it daily. My goal is to provide insights that can lead to the design and implementation of cybersecurity solutions that are both secure and genuinely enhance the employee experience and trust within enterprise environments.