Researching your own ‘backyard’: on bias and ethical dilemmas

This is a post particularly for those in the social sciences and humanities who may be doing a form of ethnographic research within the context in which they work or study – in other words, doing ‘insider research’ to use Paul Trowler’s term. Researching a context with which one is intimately familiar and in which one has a vested interest can create possible bias and ethical dilemmas which need to be considered by researchers in these situations. The last thing you want, in presenting your completed research, is for your findings to be called into question or invalidated because you have not accounted clearly enough for issues of insider bias, and your own vested interests.

Insider bias and vested interests

In the article cited in this post, Trowler considers issues of bias in data generation. Bias in research can be defined as having only part of the ‘truth’ in your data but treating that part as a whole, ignoring other possibilities or answers because you are prejudiced towards the ones that best represent your interests or investment. If you are working in a context with which you are familiar, especially your own department or faculty, or an organisation in which you have worked or do work, you will have a vested interest in that context. Either you want everyone and everything to look amazing, or perhaps you are unhappy about certain aspects of the ways in which they work and you want your research to show problems and struggles so you have a basis for your unhappiness. Either way, you have to acknowledge going in that you cannot be anything but biased about this research.

bias blindspot

However, acknowledging that you are biased, and detailing what that bias might entail for readers and examiners, does not undermine your position as researcher. By making yourself aware of potential blindspots in your research design – for example the participants you have chosen, or the cases you are including and excluding from your dataset (and why) – you can better head off possible challenges to the validity of your data later on, and you can strengthen your research design choices. Be honest with yourself: there is a balance to strike here between being pragmatic and strategic in choosing research participants, sites, or cases that will be accessible and that will yield the data you need to make your argument, and between choosing too neatly and risking one-sided or myopic data generation. Why these participants, these cases, these sites? Are there others that you know less well that you could include to balance out the familiarity, and increase the validity of your eventual findings? If not, how might you maintain awareness of your ‘insiderness’ and account for this in analysis and discussion later on?

You need to account for these decisions and questions in your methodology, and discuss what it means for your study that you are doing insider research, and that this does imply particular forms of bias. I don’t think you can get away from being biased in these cases, but you can think through how this may affect your data generation processes, and your analysis as well, and share this thinking with your readers frankly and reflexively.

Insider bias and ‘intuitive analysis’

Another point Trowler makes concerns insider ‘intuition’ when analysing the data you have generated and selected for your study. You may be analysing a policy process you were part of, or meetings you sat in on, or projects you were involved in. You have insider knowledge of what was said, the tone of the conversations, background knowledge (and perhaps even gossip) about participants – in other words, you have a kind of cultivated ‘intuition’ about your data set that you reader will not be privy too. Accounting for bias here is crucial, because if you cannot see it, you may rely too much on this insider intuition in analysing your data, and too much of the language of description you are using to convey your theorised findings will be tacit and hidden from the reader. They will then struggle to understand fully on what basis you are claiming that X is an example of poor management, or that Y means that the department is doing well in these particular areas.

ideas

It is thus vital that you get feedback here on whether it is clear to your reader why you are making particular claims, and whether they can see and understand the basis on which you are making such claims. Do they understand your ‘external language of description’ or ‘translation device’ to use Bernstein’s and Maton’s terms respectively? If they do not, you may be relying too much on your insider view of your case or participants, and may need to find a way to step back, and try to see the data you are looking at as more strange and less familiar. Getting help from a supervisor or critical friend who can ask you questions, and expose and critique possible points of bias is a useful way to re-interrogate your data with fresher eyes.

Ethical dilemmas

An ethical dilemma is defined as ‘a choice between two options, both of which will bring a negative result based on society and personal guidelines’. In research, this definition could be nuanced to suggest that an ethical dilemma presents itself when you have to make a decision to protect the interests of your research or the interests of your participants or study site. For example, in an interview with a senior manager you learn information that may be better off staying private and confidential, yet would also add an important and insightful dimension to your findings. What do you do? A participant in your study asks you for help, but to help might be to prejudice that participant’s responses in a later survey or interview, possibly skewing your data. Yet it is your job to help them. Study first, or job first? These are the kinds of dilemmas that can arise when you do research in the same spaces in which you work, and with people you work with and have other responsibilities to outside of your research.

Cheating-clients,-ethical-dilemmas

As researchers we have a duty to be as truthful and ethical in our research as possible. We are working to create and add to knowledge, not to simply maintain the status quo. In your study this may mean being carefully but resolutely critical, reflective and challenging, rather than only saying the palatable or easy things to say. This work is always going to present difficulties and dilemmas, but accounting as far as possible for your own bias and vested interests, and for your own relevant insider knowledge, can create space in your study for the development of your own reflexivity as a researcher, and can bolster rather than undermine the validity and veracity of your findings.

Trowler, P. (2011) Researching your own institution: Higher Education, British Educational Research Association online resource. Available online at [http://www.bera.ac.uk/files/2011/06/researching_your_own_institution_higher_education.pdf]

‘Retrofitting’ your PhD: when you get your data before your theory

I gave a workshop recently to two different groups of students at the same university on building a theoretical framework for a PhD. The two groups of students comprised scholars at very different points in their PhDs, some just starting to think about theory, some sitting with data and trying to get the theory to talk to the data, and others trying to rethink the theory after having analysed their data. One interesting question emerged: what if you have your data before you really have a theoretical framework in place? How do you build a theoretical framework in that case?

I started my PhD with theory, and spent a year working out what my ‘gaze’ was. I believed, and was told, that this was the best way to go about it: to get my gaze and then get my data. In my field, and with my study, this really seemed like the only way to progress. All I had starting out was my own anecdotal issues, problems and questions I wanted answers to, and I needed to try and understand not just what the rest of my field had already done to try and find answers, but what I could do to find my own answers. I needed to have a sense of what kinds of research were possible and what these might entail. I had no idea what data to generate or what to do with it, and could not have started there with my PhD. So I moved from reading the field, to reading the theory, to building an internal language of description, to generating data, to organising and analysing it using the theory to guide me, to reaching conclusions that spoke back to the theory and the field – a closed circle if you will. This seems, to me certainly, the most logical way to do a PhD.

But, I have colleagues and friends who haven’t necessarily followed this path. In their line of work, they have had opportunities to amass small mountains of data: interview transcripts, documents, observation field notes, student essays, exam transcripts and so forth. They have gathered and collected all of these data, and have then tried to find a PhD in the midst of all of it. They are, in other words, trying to ‘retrofit’ a PhD by looking to the data to suggest a question or questions and through these, a path towards a theoryology.

Many people start their doctoral study in my field – education studies – to find answers to very practical or practice-based questions. Like: ‘What kinds of teaching practice would better enable students to learn cumulatively?’ (a version of my own research question) Or: ‘What kinds of feedback practices better enable students to grow as writers in the Sciences?’ And so on. If you are working as a lecturer, facilitator, tutor, writing-respondent, staff advisor or similar, you may have many opportunities to generate or gather data: workshop inputs, feedback questionnaires, your own field notes and reports, student essays and exam submissions, and so on. After a while, you may look at this mountain of data and wonder: ‘Could there be a thesis in all of this? Maybe I need to start thinking about making some order and sense out of all of this’. You may then register for a PhD, searching for and finding a research question in your data, and then begin the process of retrofitting your PhD with substantive theory and a theoryology to help you work back again towards the data so as to tell its story in a coherent way that adds something to your field’s understanding or knowledge of the issues you are concerned with.

The question that emerged in these workshops was: ‘Can you create a theoretical framework if you have worked so far like this, and if so, how?’ I think the answer must be ‘yes’, but the how is the challenging thing. How do you ask your data the right kinds of questions? A good starting point might be to map out your data in some kind of order. Create mind-maps or visual pictures of what data you have and what interests you in that data. Do a basic thematic analysis – what keeps coming up or emerging for you that is a ‘conceptual itch’ or something you really feel you want or need to answer or explore further? Follow this ‘itch’ – can you formulate a question that could be honed into a research question? Once you have a basic research question, you can then move towards reading: what research is being or has been done on this one issue that you have pulled from your data? What methodologies and what theory are the authors doing this research using? What tools have they found helpful? Then, much as you would in a more ‘traditional’ way, you can begin to move from more substantive research and theory towards an ontological or more meta-theoretical level that will enable you to build a holding structure and fit lenses to your theory glasses, such that you have a way of looking at your data and questions that will enable you to see possible answers.

Then you can go back to your data, with a fresh pair of eyes using their theory glasses and re-look at your data, finding perhaps things you expect to see, but also hopefully being surprised and seeing new things that you missed or overlooked before you had the additional dimension or gaze offered by your theoretical or conceptual framing. But working in this ‘retrofitted’ way is potentially tricky: if you have been looking and looking at this data without a firm(ish) theoretically-informed or shaped gaze, can you be surprised by it? Can you approach your research with the curious, tentative ‘I don’t know the answers, but let’s explore this issue to find out’ kind of attitude that a PhD requires? I think, if you do decide to do or are doing a PhD in what I would regard as a middle-to-front sort of way, with data at the middle, then you need to be aware of your own already-established ideas of what is or isn’t ‘real’ or ‘true’, and your own biases informed by your own experience and immersion in your field and your data. You may need to work harder at pulling yourself back, so that you can look at your data afresh, and consider things you may be been blind to, or overlooked before; so that you can create a useful and illuminating conversation between your data and your theory that contributes something to your field.

Retrofitting a PhD is not impossible – there is usually more than one path to take in reaching a goal (especially if you are a social scientist!) – but I would posit that this way has challenges that need to be carefully considered, not least in terms of the extra time the PhD may take, and the additional need to create critical distance from data and ‘findings’ you may already be very attached to.

Data: collecting, gathering or generating?

I’m thinking about data again – mostly because I am still in the process of collecting/gathering/generating it for my postdoctoral research. I had a conversation with a colleague at a conference I went to recently who talks about ‘generating’ his data – colleagues of mine in my PhD group use this term too – but the default term I use when I am not thinking about it is still ‘collecting’ data. I’m sure this is true for many PhD scholars and even established researchers. I don’t think this is a simple issue of synonyms. I think the term we use can also indicate a stance towards our research, and how we understand our ethical roles as researchers.

Collect (as other PhD bloggers and methods scholars have said) implies a kind of linear, value-free (or at least value-light) approach to data. The data is out there – you just need to go and find it and collect it up. Then you can analyse it and tell your readers what it all means. Collect doesn’t really capture adequately, for me, the ethical dilemmas that can arise, large and small, when you are working in the ‘field’. And one has to ask: is the data just there to be collected up? Does the data pre-exist the study we have framed, the questions we are asking, and the conceptual and analytical lenses we are peering through? I don’t think it does. Scientists in labs don’t just ‘collect’ pre-existing data – experiments often create data. In the social sciences I think the process looks quite different – we don’t have a lab and test tubes etc – but even if we are observing teaching or reading documents, we are not collecting – we are creating. Gathering seems like a less deterministic type of word than collecting, but it has, for me, the same implications. I used this word in my dissertation, and if I could go back I would change it now, having thought some more about all of this.

Generating seems like a better word to use. It implies ‘making’ and ‘creating’ the data – not out of nothing, though; it can carry within it the notions of agency of the researcher as well as the research participants,  and notions of the kinds of values, gazes, lenses, and interests that the parties to the research bring to bear on the process. When we generate data we do so with a particular sense in mind of what we might want to find or see. We have a question we are asking and need to try and answer as fully as possible, and we have already (most of the time) developed a theoretical or conceptual gaze or framework through we we are looking at the data and the study as a whole. We bring particular interests to bear, too. If, as in my study, you are doing research in your own university, with people who are also your colleagues in other parts of your and their working life, there are very particular interests and concerns involved that impact not just on what data you decide to generate, but also how you look at it and write about it later on. You don’t want to offend these colleagues, or uncover issues that might make them look bad or make them uncomfortable. BUT, you also have a responsibility, ethically, to protect not just yourself but also the research you are doing. Uncomfortable data can also be very important data to talk about – it can push and stretch us in our learning and growth even as it discomforts us. But this is not an easy issue, and it has to be thought about carefully when we decide what to look at, how and why.

These kinds of considerations, as one example, definitely influence a researcher’s approach to generating, reading and analysing their data, and it can help to have a term for this part of the research process that captures at least some of the complexity of doing empirical work. For now, I am going to go with others on this and use ‘generating’. Collecting and gathering are too ‘thin’ and capture very little if any of the values, interests, gazes and so forth that researchers and research participants can bring to bear on a study. Making and creating – well, these are synonyms for generating, but at the moment my thinking is that they make it sound too much like we are pulling the data out of nothing, and this is not the case either. The data is not there to be gathered up, nor is it completely absent prior to us doing the research. In generating data, we look at different sources – people, documents, situations – but we bring to bear our own vested interests, values, aims, questions, frameworks and gazes in order to make of what we see something different and hopefully a bit new. We exercise our agency as researchers, not just alone, but in relation to our data as well. Being aware of this, and making this a conscious rather than mechanical or instrumental ‘collection’ process can have a marked impact, for the better I think, on how ethically and responsibly we generate data, analyse it and write about down the line.

Fieldwork: making and transcribing field notes

This is the second post on fieldwork: this one is specifically about field notes – some thoughts on how to write them and how to transcribe them. I am still working this out, so it’s a thinking process in progress.

For my PhD, I gathered data largely from sitting in on lecturers’ classes and watching them teach, scribbling furiously during each one hour lecture. As you can imagine, over 14 weeks in two courses this ends up being a rather thick pile of notes. In my case it amounted to 5 and a half notebooks full of notes (over 500 A5 pages). These all had to be transcribed and organised so as to make sense of of them, and so that I could put them into NVivo10 to analyse them as part of my larger data set. Also, they needed to be typed up so I could copy and paste relevant pieces into my chapters as needed. I procrastinated a lot about doing the transcription. It’s not my favourite activity as a researcher. But I couldn’t have someone else do it because these notes were something I really needed to read several times, understand and sift through. I think, in hindsight and in agreement with Pat Thomson, that tedious as it is all researchers should try and do as much of their own transcription as possible because of how involved it enables you to become with your data. It makes the analysis process more enjoyable and productive too.

I thought what I would do in this post is list some of the things I did, why and what I learnt along the way in the hopes that you may find it useful if you, too, are gathering some of your data this way.

1. I handwrote my notes for two reasons: the first is that in the one class students were not allowed to use laptops because the lecturers wanted them paying attention as part of their training for the profession they will eventually enter, and the second is because I write faster than I type and writing is quieter. I tend to bash my keyboard a bit, and I did not want to distract other students or stand out too much by using equipment they were not allowed to use. A further plus with handwriting that I learned along the way was that I could easily copy diagrams lecturers put up on slides or drew on the boards, and I could represent what they were saying pictorially or non-linearly as well, which was often quicker and easier and made my notes feel more authentic to me.

IMG00075-20130612-0930

2. Pat Thomson wrote in a blog post a while back that she takes lots of field notes, and that she tries to capture verbatim what is going on as much as she can. I tried to do that too, and often was able to succeed in bits in one of the two courses because both lecturers talked fairly slowly and deliberately and paused for students to ask questions or take notes, so I could keep up. This was harder in the  other course where the lecturers spoke quite fast. I am having this issue now, too, in my post-doc research where one lecturer in particular is a very fast talker. However, I am much more comfortable with my theoretical framework now, so I can note particular kinds of phrases or instructions or comments that he says because I know I will use them in my analysis later on. However, I want to avoid being too selective in my hearing, because I don’t want to pre-empt the data or tell it what to tell me. I want to be surprised by it too, and find things I am not necessarily anticipating. Thus, I try now as I did last year to write as much as I can of what is going on in the moment, and can then sift the notes later and reorganise them during transcription.

3. I developed a shorthand: lecturers’ initials for the lecturers involved, like CA or BM (not their real initials). I also used S for student and tried to keep track of students’ questions or inputs where I could hear them clearly (the classes were large and often noisy). So I would have S1, S2 and so on engaging the lecturer in conversation or debate.

4. I didn’t transcribe everything. By being such a procrastinator about the transcription of the field notes, I ended up transcribing them while I was also beginning to analyse the data, so in a way this worked out well because it meant I had a sense of what I needed and what was just additional information that was unimportant, like comments made about admin issues, or comments I wrote about the lecturer telling the students off for not coming to tutorials the week before. Thus, I did not need to transcribe every word I wrote in my notes, and what I did was to read through my notes first, quickly, to remind myself of what I had written, and noted things I could excise. I learnt that it is okay to cut bits of your notes out and just not transcribe them. You won’t analyse ALL your data.

5. I had to do some reorganising when I transcribed my notes. Field notes are very in the moment – you are just trying to keep up and get it all down as faithfully and fully as possible, and you don’t really have time to think. When you go back to transcribe, though, you do have time and you can see how you can transcribe your sometimes chaotic and messy notes to impose a little more order, often needed in data analysis, and also how you can represent your pictures and scribbles in words so that you know what you mean, and can show readers what you mean if you use those examples in your chapters. I think that you need to be careful with reorganising, though, because you don’t want to rewrite history and make things that were chaotic seem simple, or things that were challenging seem easy. You would be skewing or tampering too much with your data and this would be unethical. It may also rob you of some potentially interesting findings. However, a little reorganisation that makes the notes easier to read and easier to represent to a reader, while staying true to the original scribbles, may sometimes be necessary.

I think the biggest thing I learned, and am still learning, is that it is an ongoing process of learning how to write these notes well, and how to collect rich and interesting data in ways that will be usable and make sense to me later. Stop every few notes and look back – reflect on what is working and what is not, and try to use that reflection to improve your taking of field notes. Capturing them can be tedious but field notes can also give you many-faceted and rich data for later probing and analysis.

Fieldwork: custom, character and questions of ethics

I have been thinking about fieldwork a lot lately, and how to improve on what I did with it during my PhD because I am doing it all again, post-doctorally. I have started a new, connected research project which I will probably write about later on, and I am wondering if the way I am doing my fieldwork is the best way. I am not really doing anything too different yet, and I’m working in the same two departments although with different lecturers. This will probably be one of three posts thinking through different aspects of doing fieldwork, so I’d like to start with considering the question of ethics, and the ethical behaviour of researchers in ‘the field’.

Fieldwork is generally defined as ‘an investigation or search for material, data, etc, made in the field as opposed to the classroom, laboratory, or official headquarters’ (http://www.thefreedictionary.com/field+work).  I gather data in classrooms, and in conversations with lecturers, and from documents. This year I am also adding student voices to the mix. This may not fit this definition but for the sake of using one word instead of two, and because it tends to be a ‘catch-all’ term for this phase of a research project, I am going to use the term fieldwork to talk about the phase of gathering different kinds of data from different sources, even is a classroom or lecture hall.

The lecturers I worked with last year and the ones I am working with this year are colleagues. I know them and they know me, and we respect and like one another professionally and personally as well. So it is very important that we can work together well, and that I am ethical in the way I behave as a researcher because that will affect the way I am received as a colleague. There are several benefits to working like this: access to data is much less complicated – I have been welcomed with open minds and arms, and they are interested in what I am doing; I can ask questions after class and even in class if I want to, so I feel like part of the environment rather than a detached outsider (more about this in another post perhaps); I am really enjoying this work a great deal because these departments are so interested and welcoming.

However, there are also drawbacks. The biggest, for me, stemmed quite specifically from the kind of data gathering I was (and am still) doing – what Paul Trowler calls doing research in your own ‘backyard’ and learning what Kevin Williams has called ‘guilty knowledge’. I work at the same university I gather my data in, and I do other kinds of work from time to time with the lecturers who are talking to me and letting me into their classrooms. So, I am not a detached outsider. I am part of this environment, and it is thus a challenge to try to be more objective about what I am seeing and thinking, and not get too emotionally involved with the courses or the lecturers and students and therefore end up skewing the representations of my data, or omitting important observations because they may not paint the lecturers or students in a good light.  Williams especially talks about this in his paper – he argues that doing research with colleagues in your own university, in a place in which you have invested part of yourself, is difficult because sometimes you learn things you are not sure you should disclose, or dig into deeper. This can leave you and your research in a tricky place, as your professional identity as a staff member can conflict with your identity as a researcher. Choices may have to be made, and this is where custom can clash with character.

Essentially, the literature on research ethics talks about ‘custom’ as being chiefly about the forms you fill in and the ethical protocols you agree to abide by. These are the standard ethical rules to live by in your field. ‘Character’, on the other hand, is to do with how you behave as a researcher when confronted with guilty or difficult knowledge or situations that present you with ethical dilemmas. This is an important distinction. I filled in forms, and got ethical clearance and promised, quite truthfully, to abide by the ethical rules laid down by my university. But when I got into the field, I was confronted by a couple of dilemmas that those forms and rules did not necessarily help me to solve. I had to call more in my character as a researcher, reflect very carefully on the dilemma, and speak to my participants openly about the problem yet without compromising myself or my research in that process. I had to rely on character, rather than on custom, to get me through and to keep the integrity of my research project intact.

This was not easy, but through this part of my fieldwork phase, I realised that while the rules and protocols are there for a reason and need to be observed diligently, there are also things they cannot account for. It is when these unexpected twists and turns arise that you need to call on your own character as a person and as a researcher. You need to cultivate relationships, as far as you can, with your participants that are open, so that when difficulties arise that include or affect them you can share these and reach an understanding, solution or compromise as needed. Share with them, if it’s helpful, pieces of what you are writing and get their feedback. Show them your classroom or interview transcripts, and ask for their input and whether they would like anything omitted. Discuss their requests for omissions or changes with them openly, especially if they may compromise your research. Talk about this in your methodology, so your reader knows what happened too.

It’s important to actually be ethical, and not just to say you will be, and it’s important to realise that things don’t always go according to plan in the field, so having an ethical and upfront character and approach to your research will stand you in good stead in case the unexpected is part of your journey too.