Reliability and Validity

Many of the concepts of qualitative research are new and strange to people in libraries (and maybe most places).  That’s not because qualitative research itself is new.  Perhaps the most familiar use of qualitative research is the study of anthropology.  It is, in fact, a very familiar idea that an anthropologist would become immersed in a culture and do a great deal of observing in order to enhance the understanding of how people in that culture behave.  In that context immersing oneself in the subject of one’s studies makes sense, as one is often faced with language barriers, or it may just seem ludicrous to hand out a survey to people to understand how they live.  Why not just live with them?

One of the most common objections to User Experience studies, however, is that the sample size is often not large enough to be able to claim reliability or validity of any conclusions that might be reached.  This can be a tough question to deal with initially as most of us have gone through our education having never been introduced to qualitative research. We were taught about representative samples and value of measuring large numbers of subjects in order to claim reliability and validity.  In fact, when one is taking measurements, reliability and validity do demand large sample sizes relative to the population being measured.

This is where the important distinction lies:  qualitative research does not measure or quantify.  There are no  numbers or measures in qualitative research, essentially because it seeks to illuminate behaviour, or enhance understanding of behaviour. (Hoepfl, 1997) Reliability and validity are, by definition, concepts specific to studies that measure or quantify.

Gradually, libraries are seeing the value of a more qualitative approach to assessing the value of our services, facilities and collections. As Brian Matthews recently put it: “We are witnessing an interesting shift in the library profession toward more anthropological assessment measures—perhaps this will help us inject new thinking beyond the dominant quantitative mindset. When libraries served more as warehouse utilities, data-driven decision-making was crucial. Now as more of our work increasingly revolves around forming complex relationships and ongoing interactions, a more humanistic approach is required for growth and improvement.” (Librarian as Futurist: Changing the Way Libraries Think About the Future, 2014).

For a good review of how reliability and validity play into qualitative research see the article Understanding reliability and validity in qualitative research.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s