Validity and Reliability in Qualitative Research

While writing my peer review assignment on MacMillan's "Soapbox" study I came to the conclusion that qualitative research methods are inherently disadvantaged when it comes to the concepts of reliability and validity. Qualitative research can never be completely reliable because it is ultimately up to the interpretation of the researcher and interpretations can never be standardized. Similarly, when it comes to validity, it seems pretty unlikely that qualitative research will meet minimum standards for both internal and external validity. Qualitative research can never be externally valid because it is not generalizable. Maybe it can be internally valid, but I still feel like that cannot be proven.

While doing this assignment I also read several researcher's takes on evaluating reliability and validity differently for qualitative research; using concepts such as trustworthiness instead. This really resounded with me. Quantitative research is older and more established and these concepts are clearly defined to evaluate quantitative research. It is unfair to have the same standards for a completely different type of research.

Pragmatism

Reality can be a frightening thing sometimes. That is how I felt reading ch.7 of Knight, frightened. But at the same time, realizing the hurdles of conducting academic research, no matter how daunting they may seem, is an important thing for novice researchers to absorb early on. The reason is because these challenges are going to be magnified for first-time researchers and they can be very demotivating. By devoting a whole chapter to the pragmatic challenges of doing academic research, new researchers can brace themselves for the many challenges that lie ahead, which is important for maintaining self-efficacy ( which I believe is half the battle).

Not every topic deals with "benign" issues and Peter helps the researcher understand the importance of these sensitivities. I suppose this is what it means to be a professional in any undertaking, being responsive or empathetic to the will and emotions of others. I wonder though, if this can be learned and internalized from the chapter of a research methods book ? I mean, if one were to apply these tenets literally it might obstruct the flow of whatever information gathering technique is being employed. But then again, I am speaking as an a person who is not shy and quite open (if you weren't able to discern from my in class participation). Not everyone is like this however, and it is important to absorb the professional advice from Ch.7 of Knight.

Studying the Internet

Both Hine's and Orgad's articles on studying the Internet were really interesting. The Internet has created a whole new area of research methods. This topic also reminded me of an article I read for 1001 by Miller and Slater called The Internet: An Ethnographic Approach. This article was revolutionary for how Internet is studied today because it discusses seeing the Internet as integral to society and not as a virtual reality. Researchers are slowly developing a framework for analyzing and conducting online research and making it seem legitimate.

A small-scale example of online and offline worlds coinciding in research was how I conducted my previous research on Afghan-Canadian youth. Facebook actually allowed me to conduct this research. I was able to use it to approach potential participants. Basically, I sent messages to members of an Afghan Association in Hamilton outlining my research and asking if they would be willing to meet with me and be interviewed. Because this is a marginalized group I could not ethically approach them in person or by telephone but through the Internet it was more annonymous and they could simply not respond to me if they did not want to be interviewed. This was very difficult for me to describe in my ethics approval application and in my final research paper. Clearly, the framework for online research is still being created.

Focus Groups Have Value!

While I was doing my peer review assignment (I choose to critique MacMillan's "Soap Box" study) I came across a really great section of a book, Berg's Qualitative Research Methods for the Social Sciences. In this section, Berg discusses focus groups in great detail. According to Knight, focus groups are primarily useful for their cost and time-cutting measures. Knight's section on focus groups is very small and he insinuates that focus groups are for lazy researchers. They apparently cannot generate valid data on their own and are used as a sort of preliminary research. Berg, however, had a completely different perspective on the matter. He believes that focus groups allow the researcher to fade more into the background and said that participants sometimes feel more comfortable sharing their views in a group setting as opposed to one-on-one. He also said that a group has the ability to generate findings that cannot be generated by an individual.

My background is sociology and I also took several psychology courses. The whole principle of sociology is based on the fact that society and groups of people need to be studied as a whole rather than as the sum of individuals. I was definitely happy to see that some methods specialists do no simply brush off the importance of groups and group interviews.

Unobstrusive Methods

The difficulty with studying human behaviour is that you can never be sure that you are getting accurate results. I have a background in qualitative research but it can be frustrating to hear that your interviews are biased or that people are not giving you accurate responses because they are afraid of what you will think of them. Even in quantitative methods there are many criticisms. For survey research some people may check off what they wish applied to them. For example, if you are asking how many hours of television a person watches per day he/she may answer fewer hours than are actually watched because he/she plans to cut down on hours of television watched. Sometimes this really bothers me. My friends who are in science roll their eyes when they speak about social science. I want my research to be taken seriously and to seem credible.

I was, therefore, intrigued to read Knight's section on unobtrusive methods. It is interesting to think of doing a study without leaving a mark. You're simply making observations. The researcher cannot influence subjects under these circumstances. Knight says that most researchers choose not to use unobtrusive methods but I think there might be something very valuable to them. I am not sure if they would make scientists take us more seriously, but they definitely are able to get around some of the criticisms with many social science methods.

Ethics and the anthros

http://www.youtube.com/watch?v=vWaI9UZ-LYw

Our class on ethics reminded me of this song. I don't think I can speak (or sing) on the topic as eloquently as Floyd "Red Crow" Westerman so I'll leave it to him.

Good luck on your proposal

PS I will follow-up with a post that is actually based on the weekly readings ASAP :) 

Ethics

My research study would be a starting point for further investigation in public libraries to
involve more participant in the adult programs by reaching out to the community to meet their needs.
Although my research is not design for individuals to pursue their personal concern or interest, but human is the main subject.

Human in my research is a key person who is the main informant of an organization, therefore related organizational objectives and guidelines should be considered and the ethics policies and regulations must be followed.

In Knight, listed pragmatic responses is a good reminding point to prevent accident.

Nahid