Lecture 11: Qualitative research
“Qualitative work is expressed in natural language, whereas quantitative work is expressed in numbers and in statistical models”
(Gerring 2017: 18)
Strengths and weaknesses of qualitative research
What are the strengths of qualitative research?
Allows for detailed understanding of the process / causal mechanism
Gives room for meaning, interpretation & voice
o With a qualitative case study, you can open the black box, and understand what people believe, how they feel,
what the causal mechanism is exactly.
Allows for surprise / unexpected findings
o ‘Front loading’ in quantitative research (Yanow 2014: 144)
If you do quantitative research, you have to decide and design in advance (e.g. surveys), you have to make
all these choices in advance
o In qualitative research: more room for unexpected findings, you can go in the field and allow for change,
uncertainty, varying choices.
What are the weaknesses of qualitative research?
Small and non-randomized samples (E.g. Documents, people) limited generalizability
No statistical calculations, but interpretation by the researcher limited replicability
Complementarity of qualitative and quantitative research
Qualitative research in political science
Qualitative research in political science
Qualitative research has always played an important role in political science, for example in:
o Public administration, international relations, comparative politics, legal studies
o But less in political behaviour
Research paradigms and qualitative research
Recall distinction between:
o Positivism / objectivism
o Interpretivism / constructivism
Qualitative research is broadly interpretivist, but not in
Ontology and epistemology
Ontology: What is the nature of reality?
Epistemology: What can we know about reality?
Quality criteria for qualitative research
Contrary to statistical analysis, no one ‘cookbook’ for qualitative research. But, three sets of criteria
1) The ‘rigor’ criteria (validity, reliability) for positivist (or realist) research:
Internal validity: is the relationship really causal
External validity: generalizability, applicability in other contexts
Reliability: consistency of the inquiry, stability, replicability
Objectivity: neutrality, free of bias/values
2) The ‘parallel’ criteria for interpretive (or realist) research (Guba & Lincoln 1989)
Rigor criteria of limited use when no ‘real’ world/law-like relationship is assumed. But not anything goes Instead, Guba & Lincoln
o Parallel to internal validity
o ‘isomorphism between constructed realities of respondents and the reconstructions attributed to them’ (Guba &
o Guba & Lincoln offer six ways to check credibility:
1) Prolonged engagement
Substantial involvement at the site of the inquiry, in order to overcome the effects of misinformation,
distortion, or presented “fronts,” to establish the rapport and build the trust necessary to uncover
constructions, and to facilitate immersing oneself in and understanding the context’s culture
2) Persistent observation
Sufficient observation to enable the evaluator to “identify those characteristics and elements in the
situation that are most relevant to the problem or issue being pursued and [to focus] on them in detail
3) Peer debriefing
The process of engaging, with a disinterested peer, in extended and extensive discussions of one’s
findings, conclusions, tentative analyses, and, occasionally, field stresses, the purpose of which is both
“testing out” the findings with someone who has no contractual interest in the situation and also helping
to make propositional that tacit and implicit information that the evaluator might possess.
4) Negative case analysis
The process of revising working hypotheses in the light of hindsight, with an eye toward developing and
refining a given hypothesis (or set of them) until it accounts for all known cases.
5) Progressive subjectivity
The process of monitoring the evaluator’s (or any inquirer’s) own developing construction. The inquirer’s
construction cannot be given privilege over that of anyone else (except insofar as he or she may be able
to introduce a wider range of information and a higher level of sophistication than may any other single
Prior to engaging in any activity at the site or in the context in which the investigation is to proceed, the
inquirer records his or her a priori construction—what he or she expects to find once the study is under
way—and archives that record. At regular intervals throughout the study the inquirer again records his or
her developing construction
6) Member checks
The process of testing hypotheses, data, preliminary categories, and interpretations with members of the
stakeholding groups from whom the original constructions were collected.
This is the single most crucial technique for establishing credibility
This process occurs continuously, both during the data collection and analysis stage, and, again, when
(and if) a narrative case study is prepared.
Member checks can be formal and informal, and with individuals (for instance, after interviews, in order
to verify that what was written down is what was intended to be communicated) or with groups (for
instance, as portions of the case study are written, members of stakeholding groups are asked to react to
what has been presented as representing their construction).
o Parallel to external validity
o Always relative: ‘depends entirely on the degree to which salient conditions overlap or match’ (G&L: 241)
o Empirical process for checking the degree of similarity between sending and receiving contexts. Further, the burden of
proof for claimed generalizability is on the inquirer, while the burden of proof for claimed transferability is on the
o How: thick description of findings within context (time, place, culture)
o Often a relative weakness of qualitative research
o Is linked to case selection / sampling
o Bryman distinguishes:
Theoretical / analytical generalization
Qualitative research can critically test existing theories (most- and least-likely case selection)
Or it can contribute to the formulation of new theories, because it:
Explicates mechanisms that can have wide-ranging application
Brings new ways of seeing and understanding into plain view (see further Wedeen 2010:
Generalisation to broader population
Two steps (reverse of two steps of data selection):
From sample to case -> can we say something about the case, based on the sample of
respondents / documents / etc.?
From case to broader population -> can we say something about the broader population,
based on the case
Generalisations are necessarily bounded and contingent
Example: Cramer (2012)
o Two steps:
From sample to case
From case to broader population
o Parallel to reliability, ‘concerned with the stability of data over time’ (G&L: 242)
o How: ‘process audit’ -> that outsider reviewers ‘can explore the process, judge the decisions that were made, and
understand what salient factors […] led the evaluator to the decisions and interpretations made’ (G&L: 242)
o Parallel to objectivity
o ‘assuring that data, interpretations, and outcomes of inquiries are rooted in contexts and persons apart from the evaluator
and are not simply figments of the evaluator’s imagination’ (G&L: 243)
o How: audit of relationship between data and outcomes by outsider reviewers
o Note: not really tenable in interpretivist research
3) Reflexivity (for all research?) Corlett, Sandra and Mavin, Sharon (2015) “Reflexivity and Researcher Positionality”
o Methodological reflexivity accepts that the researcher makes methodological and method choices (what is
included and what is left out), and acknowledges that research methods, as used by researchers, are not
neutral tools – each have ‘philosophical baggage’
o Reflexive researchers make explicit this baggage to an audience and provide a convincing account of the
knowledge ‘manufacturing conditions’
Data are produced by the researcher, not collected
And their involvement is taken as a source of data in its own right
o Identity: The researcher’s epistemological position and assumptions influence understandings of researcher
field roles, and the related concept of identities.
o Motivations: which interests, values, experiences or political commitments do I bring to the research? (Corlett
& Main 2015: 385)
How should researchers deal with their partiality/ biases?
Positivist researchers: try to minimize (feminist empiricism: “add women”)
Realist / Interpretivist researchers:
o Reflect and/or make it part of your analysis
o Aim for multiple perspectives (feminist standpoint theory)
Critical researchers: Let a normative engagement guide your research
o Positionality: how does my identity affect the research process? ‘how are race/gender/class made meaningful
in this relationship’ (Day cited in ibid: 387)
Mostly relevant for obstrusive data collection
Sometimes called interviewer effect (Lecture 13)
Requires reflection, shapes what people tell you in context dependent ways
o Power: what is the power relationship between the researcher and the researched?; What role do
positionality, identity, and power play in the process of knowledge production?;
o Voice: who speaks for whom? Who has a voice in the case that certain social groups do not have a voice?
Lecture 12: On feminist epistemology and the importance of diversity
“the study of knowledge and justified belief.” Stanford Encyclopedia of Philosophy,
How do we decide what is true, who do we believe, etc.
Epistemology is fundamental to academic practice
Science is a social and political practice shaped by power relations
Feminist perspectives on objectivity
o Situated knowledges
Embrace your bias!
Positivist vs reflexive epistemologies
A metaphor: “Standard Fishbowl Model of Science” (Schliesser 2015)
o He says that the scientist is a cat which stands outside a fish bowl, observing from a distance what is happening in
the close environment of the fish bowl. And scientists should be the objective cat.
Scientist should be:
o Objective, impartial, neutral
o Disinterested truth seeker
o Data, phenomena, object of study
Scientists are part of the world they study; science is a social and political practice
o “The observer and the observed are in the same causal scientific plane” (Harding 1991: 11)
o So, here the cat is in the fish bowl.
o Impossibility of neutral standpoint, because you are always speaking from somewhere in the world.
o Power relations in science
On the possibility of a “neutral point of view”
Roses are red, violets are blue?
How we perceive colors depends on social context
o The words that we use for colors and how we understand the relations between colors depends on the language,
culture, society, etc.
Numbers = objective facts?
o But, how many immigrants live in the Netherlands?
It depends on how you define an immigrant (having a foreign nationality, or just not being born in the
Netherlands, or having a foreign parent)
“One of the core insights of both Women’s Studies and Ethnic Studies is that all knowledge is positional, that it reflects the
perspective and the values of the knower. According to this critique, what has been offered to us by history, science, art,
and literature as universal, neutral and objective truth, was really the way the world looked to the dominant group - the
world seen through their eyes and through the filter of their interests.” (Rothenberg 2003: 23)
Gender and race: Power
The evolution of humankind
o “what has been offered to us by history, science, art, and literature as universal, neutral and objective truth, was
really the way the world looked to the dominant group”
Power: Who gets to “do” science?
Percentage of women among professors at Dutch universities in 2015 18.1%
Percentage of women among professors at Dutch universities in 1990 2.7%
Power: Who benefits from science?
For example, it was asserted that medicine knows very little about women
70-90% of patients with unexplained physical complaints are women
Women are 50-70% more likely to experience side effects from medication than men
Power shapes scientific perspectives
What if WHO focused not on “sexual violence against women” but on “sexual violence by men”? There are different
sources of the problem.
o “Because these alternative formulations implicitly identify different sources of the problem, they will encourage us
to think differently about possible solutions depending upon which we choose.”
On science and power
Science is a social practice
And therefore, it is shaped by power relations just like everything else that people do
The power of science
o “Research shows that….”
Science is a very powerful form of discourse
o A person is given a whole different status as a scientist and is recognized through that research
Power within science
o “Science is politics by other means… Groups with conflicting social agendas have struggled to gain control of the
social resources that the sciences – their “information”, their technologies, and their prestige – can provide.”
(Harding 1991: 10)
“According to one influential tendency in conventional thought, there is only one standard for what
counts as science, and that is provided by the natural sciences. …
Mathematics, physics, chemistry are believed to have way more power than for example, psychology and
social science in general.
o Who do we believe? Who do we trust?
Who gets to say, “research shows that…” and convince us?
Who gets to have a “neutral” point of view? Who gets to be “objective” – and therefore authoritative?
Professor, genius, inventor, philosopher, explorer? How do you perceive them? What do they
“the god trick of seeing everything from nowhere…” (Haraway 1988: 581)
Power of/within science
Feminist epistemologies call for:
1) Science which serves social justice; i.e. which serves the interests of the oppressed rather than the powerful
2) Diversity among scientists
3) Science, which is reflexive, aware of its (power) position
The natural sciences are illuminatingly conceptualized as part of the social sciences.
4) Thinking of the object of science as “an actor and an agent” (Haraway 1988: 592)
It contains both progressive and regressive tendencies.
On objectivity and “alternative facts”
So, science is just another opinion? At least two feminist answers:
1. Feminist empiricism
It says that the fish bowl model is really important, only that actually we’ve been failing at it, as women have been almost
2. Feminist standpoint theory
There is no such thing as a ‘neutral’ point of view
o But science is NOT just another opinion
Donna Haraway and Sandra Harding plead for “feminist objectivity
o Feminist objectivity = Reflexivity
“Feminist objectivity means quite simply situated knowledges.” (Haraway 1988: 581)
No such things as unmediated vision
“So, location is about vulnerability… We seek those ruled by partial sight and limited voice – not
partiality for its own sake but, rather, for the sake of connections and unexpected openings
situated knowledges make possible” (590)
Haraway: “webs of connections” on feminist standpoint
o Hill Collins on feminist standpoint: “Points of connection among multiple epistemologies… Each group speaks from
its own perspective and shares its own partial, situated knowledge. But because each group perceives its own truth
as partial, its knowledge is unfinished” (1990: 270). We must understand other people’s points of view.
On bias and standpoint
Not about eliminating bias
“Feminists… criticize not only ‘bad science’ but also … ‘science – as- usual’.” (Harding 1991: 1)
“We wanted a way to go beyond showing bias in science (that proved too easy anyhow)”(Haraway 1988: 578)
Rather: Acknowledging your standpoint
Harding (1991: 12): “We should think of the social location of our own research – the place in race, gender, and class
relations from which it originates and from which it receives its empirical support – as part of the implicit or explicit
evidence for our best claims as well as our worst ones”.
Embrace your bias:
o Your standpoint is your strength not your weakness
o Your non-academic life is a source of knowledge, understanding and intuition – bring it into dialogue with your
Lecture 13: Interviews and focus groups (Ethnography)
Comparing “obtrusive” methods of data collection
Unobtrusive vs Obtrusive data collection
Obtrusive methods of data collection
o Allow the study of unobservable phenomenon, mental constructions or actual behaviour, but:
o Researcher’s presence (positionality), behaviour and interventions affect the interaction and thereby the data
construction Data is produced.
Unobtrusive methods of data collection = data existed independently of researcher (but still needs to be gathered /
analysed). Not affected by researcher’s presence, but limits research to existing content
Comparing forms of “obtrusive” data collection
Understanding personal interpretations, emotions
Study unobservable phenomena, decisisons made behind closed doors, etc.
Time and cost efficient