Ethical Interviews In Software Engineering -- Per Erik Strandberg

During our work on the paper on Information Flow In Software Testing we had many ethical concerns on research interviews with industrial practitioners. We wanted to minimize risks and conduct ethically sound research. Based on these experiences I wrote a paper with guidelines on how to conduct ethical interviews.

Unfortunately the paper is not Open Access (ESEM did not offer it 2019 despite doing so in 2018), so a publicly available PDF can be found at arxiv: [1]

Slides from presentation: [2]

Short video presentation at Youtube: [3]

Abstract

Background: Despite a long history, numerous laws and regulations, ethics remains an unnatural topic for many software engineering researchers. Poor research ethics may lead to mistrust of research results, lost funding and retraction of publications. A core principle for research ethics is confidentiality, and anonymization is a standard approach to guarantee it. Many guidelines for qualitative software engineering research, and for qualitative research in general, exist, but these do not penetrate how and why to anonymize interview data.

Aims: In this paper we aim to identify ethical guidelines for software engineering interview studies involving industrial practitioners.

Method: By learning from previous experiences and listening to the authority of existing guidelines in the more mature field of medicine as well as in software engineering, a comprehensive set of checklists for interview studies was distilled.

Results: The elements of an interview study were identified and ethical considerations and recommendations for each step were produced, in particular with respect to anonymization. Important ethical principles are: consent, beneficence, confidentiality, scientific value, researcher skill, justice, respect for law, and ethical reviews.

Conclusions: The most important contribution of this study is the set of checklists for ethical interview studies. Future work is needed to refine these guidelines with respect to legal aspects and ethical boards.

Informal Summary

Figure: Ten main elements of an interview study.

Important ethical principles relevant for research are: consent, beneficence, confidentiality, scientific value, researcher skill, justice, respect law, and ethics review.

In the paper I argue that there are ten main elements of an interview study: planning for ethical research, pre-interview discussions, room, interview, audio files, transcription, interviewee correspondence, data analysis and thematic data, writing and the publication, as well as archive.

Based on our experiences (and many others) I wrote check-lists for the ten interview elements to improve an interview study in terms of ethics.

The Checklist

Planning for Ethical Research

Are stakeholders identified?
Are ethical challenges considered?
How will the challenges be addressed? Do sponsors and supervisors agree?
How will the instrument be validated?
Has an IRB been consulted?

Pre-Interview Discussions

How will informed consent be obtained?
How will any participant withdrawals be handled?
Are the interviewees informed about purpose, possible positive outcomes, possible harm, expected duration, sampling, sponsor, confidentiality, contact details, project leaders, etc.?
What promises, with respect to third party access to interview data, will be made? Is there a plan for a potential research quality audit?

Room

How will internal anonymity be addressed?
Are managers informed that their participation might have a negative impact on the research?
Are interview artifacts removed after interviews?

Interview

Do the researchers have adequate skills?
How will data minimization be addressed?

Audio Files

What is the data storage plan?
Has the number of people with access to data been limited?

Transcription

Who will transcribe the audio?
How will meta information (such as separation of speakers, timestamps, etc.) be added to the transcripts?
How will consistent transcription over interviews, and over researchers, be achieved?
What will be anonymized?

Interviewee Correspondence

Will interviewees review transcripts?
If yes, how is correspondence to be conducted?
If yes, will they be given the possibility to delete, correct, clarify and/or expand on the transcripts?

Data Analysis and Thematic Data

Will data analysis (and the potential use of third party tools) be done on anonymized data only?
Has the end user license agreements for tools been read?
Is there an inventory of the data (with locations of audio files, transcripts, and processed data)?

Writing and the Publication

How will details on the organizations, and other context data, be reported?
Will reports in different forms, for different audiences, be prepared?
How will feedback for the participating interviewees and organizations be made? (Candidates for anonymization are: names of people, places, companies, organizations, tools, and products; domain-specific details such as programming languages, domain-specific terminology, company specific jargon and technical details that are not of relevance to the topic of the interview; numbers and points in time such as birthdays, graduation years, number of years in a work place, number of colleagues, number of subsystems in a product, or number of lines of code in a product; personal details such as religion, cultural background, military rank, hobbies, nationality, occupation, family relationships, gender, zodiac sign, dietary restrictions, periods of illness, etc.; and also off-topic details, such as the pre-interview discussion, or a rant from the interviewee.)

Cite

@inproceedings{strandberg2019ethical,
  author = {Per Erik Strandberg},
  title = {Ethical Interviews in Software Engineering},
  month = {September},
  year = {2019},
  booktitle = {Empirical Software Engineering and Measurement, 13th International Symposium on}
}

Tillhör Kategori Publikationer