I ask because my research proposal involves a population of texts produced over a long period of time—a much, much longer period of time than this blog represents, but not so different from a methodological point of view. Let's say you have a research question that involves changes over time, meaning that you want to compare the content of posts produced in September, October, November, and December. (You could stratify them some other way, but let's keep this simple.) When I submit this post, the count will look like this:
Sept: 15
Oct: 30
Nov: 21
Dec: 12
(Okay, so there's no need to sample from such a small population, but play along.) Can you simply take a random sample of, say, 10 posts from each month? If you did, your sample would have something like 33% of October's posts and 88% of December's posts. Then again, since the purpose of the study is comparison and not overall generalization, you don't have to worry about your samples being uneven. As long as you can generalize about October from the October samples, you're fine. Still, maybe it would be better to take 50% of each month. I don't know. If you start taking percentages, then do you risk some kind of distortion? Is 6 of 12 as representative as 15 of 30? I need to go back to the textbooks.
For what it's worth, I think that my research design is going to involve nonproportionate stratified sampling. That would be relevant to this hypothetical study if, for example, October had 30 posts and September had 5. There's just no way you could sample from 5 and get a representative picture of the content of the blog posts. I suppose, though, that I have to think about the ways in which nonproportionate sampling constrains and affects the comparison.
Karl
Sunday, December 5, 2010
Hello and goodbye
Before we turn the lights out on this forum, I want to say thank you to all of you who shared thoughts and observations about this process of designing a research project. So much more there than we knew, wasn't there? Thank you Professor Grimes for selecting such a range of readings which I found introduced me to so much new information - games theory, ethnography's intimacies -- and enlightened me in unexpected ways, as light shone into corners i didn't even know were dark . Who knew how difficult it could be to isolate a meaningful research question! Only when I had to make it consistent with all the other parts -- methodology, analysis framework -- did I appreciate how coherent a concept has to be, to have even potential value. As a research investigation anyway.
And thank you also for lectures that were always full of visual as well as intellectual interest.
And thank you also for lectures that were always full of visual as well as intellectual interest.
All tonight's blogs have something in them that resonate; i also feel frightened --at the thought of "going live" with a research project involving people ever again; the ethics talk was excellent, especially as an antidote to grandiose plans; I have used an extra observer to record observations of the "subjects" to capture even more information (that people don't know they're giving even though they have agreed to the process); and my writing process is also bedevilling me right now. Thanks and good luck to all!
I suppose this is my final post for this blog. I would just like to say how helpful I have found this process, both for getting a sense of other people's ideas and projects as well as constructing and making sense of my own.
In reading Chapter 7 of Knight, I began to think a lot about whether or not note taking or using recording devices would be more appropriate in my interviews. The crux of the issue seems to be that in taking notes, there is more of a chance that you can miss something, as well as not being able to be as attentive to the interviewees. These implications can be somewhat avoided if you have two researchers as Knight suggests (168). At the same time, it is still difficult to capture all of what someone says by transcribing it. Using recording devices can assist with this problem, but there is the potential for recording devices to influence behaviour. Ethical considerations would necessitate letting the interviewee know that they are being recorded in any way, so they must be aware of this. It is suggested by Knight that if they are smaller, the interviewees may forget about them and make them more comfortable and thus less influenced by the device in their responses.
I think there is a need to find a balance between these two approaches. I believe that you can allow the interviewee to feel more comfortable about the recording by assuring confidentiality and anonymity. I think that I would definitely use 2 researchers- 1 for conducting the interview and 1 for taking notes not just on what people are saying but their body language, etc. I would also obtain an audio record of the conversation, just because I feel like this is the best way to ensure that I wouldn't miss anything important. I feel that the minor limitations that may come from using a recording device outweigh the potential loss of information from the interview that could happen if there were only transcriptions of the interview.
In reading Chapter 7 of Knight, I began to think a lot about whether or not note taking or using recording devices would be more appropriate in my interviews. The crux of the issue seems to be that in taking notes, there is more of a chance that you can miss something, as well as not being able to be as attentive to the interviewees. These implications can be somewhat avoided if you have two researchers as Knight suggests (168). At the same time, it is still difficult to capture all of what someone says by transcribing it. Using recording devices can assist with this problem, but there is the potential for recording devices to influence behaviour. Ethical considerations would necessitate letting the interviewee know that they are being recorded in any way, so they must be aware of this. It is suggested by Knight that if they are smaller, the interviewees may forget about them and make them more comfortable and thus less influenced by the device in their responses.
I think there is a need to find a balance between these two approaches. I believe that you can allow the interviewee to feel more comfortable about the recording by assuring confidentiality and anonymity. I think that I would definitely use 2 researchers- 1 for conducting the interview and 1 for taking notes not just on what people are saying but their body language, etc. I would also obtain an audio record of the conversation, just because I feel like this is the best way to ensure that I wouldn't miss anything important. I feel that the minor limitations that may come from using a recording device outweigh the potential loss of information from the interview that could happen if there were only transcriptions of the interview.
Saturday, December 4, 2010
The End.
I gather that, for most of us, this blog has come to an end. Endings are, of course, beginnings, and for that reason I will write one more post after this one. So this post is not The End after all.
I have been thinking a lot about writing lately because I need to rethink how I write papers. Not, I mean, how I put sentences together—well, yes, I need to work on that part of it—but the whole business of reading, thinking, making notes, writing, and editing. Curiously, this idea is completely appropriate to this moment because if you think back to the beginning of this course, then you will recall that we began with writing. "Start with Writing" is the first chapter of Knight's book. Here, at the end of the course, I arrive at the beginning.
The main reason that I need to rethink how I write has to do with the fact that I write in too many separate stages. First, I read some text and make notes. Then I read my notes. Then I re-read my notes and re-read portions of the text. Then I re-read my notes and re-read portions of the text and write. Then I re-read the text. Then I re-read my notes. Then I write. And so on. Why don't I just read and write?
So that's where I end, or begin. With a research question.
Karl
I have been thinking a lot about writing lately because I need to rethink how I write papers. Not, I mean, how I put sentences together—well, yes, I need to work on that part of it—but the whole business of reading, thinking, making notes, writing, and editing. Curiously, this idea is completely appropriate to this moment because if you think back to the beginning of this course, then you will recall that we began with writing. "Start with Writing" is the first chapter of Knight's book. Here, at the end of the course, I arrive at the beginning.
The main reason that I need to rethink how I write has to do with the fact that I write in too many separate stages. First, I read some text and make notes. Then I read my notes. Then I re-read my notes and re-read portions of the text. Then I re-read my notes and re-read portions of the text and write. Then I re-read the text. Then I re-read my notes. Then I write. And so on. Why don't I just read and write?
So that's where I end, or begin. With a research question.
Karl
Friday, December 3, 2010
Research ethics in unexpected places
Ever since our class on research ethics, which was fascinating, I have had the funny feeling that, sometime earlier this semester, I stumbled upon an ethical issue in relation to historical research. Typically, the ethical issues that we associate with historical research have to do with academic honesty or, I would add, the fair use and impartial representation of evidence. A few days ago, though, I remembered that I had come across the records of two prison libraries in the Archives of Ontario. The archives hold the library record books that, I presume, indicate which inmate borrowed which book. Interestingly, both fonds come with the following condition, which probably accompanies all prison records in the Archives:
"Access to these records is governed by the Freedom of Information and Protection of Privacy Act. Requests for access to records 100 years old or less must be submitted in writing to the Information and Privacy Unit of the Archives of Ontario."
It occurred to me that writing a request for access to these records would be not unlike writing a research protocol for a study involving human subjects. You should probably make an argument about the value of your research, how you will use the information, how you will secure the information, how you will protect the identities of the inmates, to whom you will disclose your results, what you will do with the information after the study, and so on.
So, as it turns out, our training in research ethics applies just as much to textual records as it does to human subjects. Neat.
Karl
"Access to these records is governed by the Freedom of Information and Protection of Privacy Act. Requests for access to records 100 years old or less must be submitted in writing to the Information and Privacy Unit of the Archives of Ontario."
It occurred to me that writing a request for access to these records would be not unlike writing a research protocol for a study involving human subjects. You should probably make an argument about the value of your research, how you will use the information, how you will secure the information, how you will protect the identities of the inmates, to whom you will disclose your results, what you will do with the information after the study, and so on.
So, as it turns out, our training in research ethics applies just as much to textual records as it does to human subjects. Neat.
Karl
Surveys as a Way of Avoiding Ethical Issues?
My research proposal involves two parts. I want to gain an understanding of how librarians in the TPL view Facebook and whether they use it to reach out to young adults. I also want to see how young adults use Facebook and how they view the library and whether Facebook should be used as an outreach tool. Ideally, I would like to be doing interviews with both sets of subjects. However, interviewing minors poses an ethical risk. Therefore, I plan to do survey research in Toronto highschools. I see survey research as being less invasive and I hope to avoid any ethical dilemmas by using this method as opposed to interviews.
Actually Doing the Research
I really like how Knight's Chapter 7 says that it is okay if things go wrong during the research process. Research isn't always as neat and orderly as the methods books would have us believe. The research that I conducted was messy and emotional and confusing. It did not follow a neat, linear pattern and at times I felt lost along the way. Methods books make research seem so cut and dry. If you're interviewing you follow a set of questions and a procedure. You code a certain way and analyze results a certain way and write your paper a certain way. However, the manuals don't explain how it's going to feel when you're asking a participant really sensitive questions or what to do if they start crying. Even after you get ethics approval there's always a chance that you could emotionally harm a subject. I felt that Knight made the research process seem more real.
Validity and Reliability in Qualitative Research
While writing my peer review assignment on MacMillan's "Soapbox" study I came to the conclusion that qualitative research methods are inherently disadvantaged when it comes to the concepts of reliability and validity. Qualitative research can never be completely reliable because it is ultimately up to the interpretation of the researcher and interpretations can never be standardized. Similarly, when it comes to validity, it seems pretty unlikely that qualitative research will meet minimum standards for both internal and external validity. Qualitative research can never be externally valid because it is not generalizable. Maybe it can be internally valid, but I still feel like that cannot be proven.
While doing this assignment I also read several researcher's takes on evaluating reliability and validity differently for qualitative research; using concepts such as trustworthiness instead. This really resounded with me. Quantitative research is older and more established and these concepts are clearly defined to evaluate quantitative research. It is unfair to have the same standards for a completely different type of research.
While doing this assignment I also read several researcher's takes on evaluating reliability and validity differently for qualitative research; using concepts such as trustworthiness instead. This really resounded with me. Quantitative research is older and more established and these concepts are clearly defined to evaluate quantitative research. It is unfair to have the same standards for a completely different type of research.
Pragmatism
Reality can be a frightening thing sometimes. That is how I felt reading ch.7 of Knight, frightened. But at the same time, realizing the hurdles of conducting academic research, no matter how daunting they may seem, is an important thing for novice researchers to absorb early on. The reason is because these challenges are going to be magnified for first-time researchers and they can be very demotivating. By devoting a whole chapter to the pragmatic challenges of doing academic research, new researchers can brace themselves for the many challenges that lie ahead, which is important for maintaining self-efficacy ( which I believe is half the battle).
Not every topic deals with "benign" issues and Peter helps the researcher understand the importance of these sensitivities. I suppose this is what it means to be a professional in any undertaking, being responsive or empathetic to the will and emotions of others. I wonder though, if this can be learned and internalized from the chapter of a research methods book ? I mean, if one were to apply these tenets literally it might obstruct the flow of whatever information gathering technique is being employed. But then again, I am speaking as an a person who is not shy and quite open (if you weren't able to discern from my in class participation). Not everyone is like this however, and it is important to absorb the professional advice from Ch.7 of Knight.
Not every topic deals with "benign" issues and Peter helps the researcher understand the importance of these sensitivities. I suppose this is what it means to be a professional in any undertaking, being responsive or empathetic to the will and emotions of others. I wonder though, if this can be learned and internalized from the chapter of a research methods book ? I mean, if one were to apply these tenets literally it might obstruct the flow of whatever information gathering technique is being employed. But then again, I am speaking as an a person who is not shy and quite open (if you weren't able to discern from my in class participation). Not everyone is like this however, and it is important to absorb the professional advice from Ch.7 of Knight.
Studying the Internet
Both Hine's and Orgad's articles on studying the Internet were really interesting. The Internet has created a whole new area of research methods. This topic also reminded me of an article I read for 1001 by Miller and Slater called The Internet: An Ethnographic Approach. This article was revolutionary for how Internet is studied today because it discusses seeing the Internet as integral to society and not as a virtual reality. Researchers are slowly developing a framework for analyzing and conducting online research and making it seem legitimate.
A small-scale example of online and offline worlds coinciding in research was how I conducted my previous research on Afghan-Canadian youth. Facebook actually allowed me to conduct this research. I was able to use it to approach potential participants. Basically, I sent messages to members of an Afghan Association in Hamilton outlining my research and asking if they would be willing to meet with me and be interviewed. Because this is a marginalized group I could not ethically approach them in person or by telephone but through the Internet it was more annonymous and they could simply not respond to me if they did not want to be interviewed. This was very difficult for me to describe in my ethics approval application and in my final research paper. Clearly, the framework for online research is still being created.
A small-scale example of online and offline worlds coinciding in research was how I conducted my previous research on Afghan-Canadian youth. Facebook actually allowed me to conduct this research. I was able to use it to approach potential participants. Basically, I sent messages to members of an Afghan Association in Hamilton outlining my research and asking if they would be willing to meet with me and be interviewed. Because this is a marginalized group I could not ethically approach them in person or by telephone but through the Internet it was more annonymous and they could simply not respond to me if they did not want to be interviewed. This was very difficult for me to describe in my ethics approval application and in my final research paper. Clearly, the framework for online research is still being created.
Focus Groups Have Value!
While I was doing my peer review assignment (I choose to critique MacMillan's "Soap Box" study) I came across a really great section of a book, Berg's Qualitative Research Methods for the Social Sciences. In this section, Berg discusses focus groups in great detail. According to Knight, focus groups are primarily useful for their cost and time-cutting measures. Knight's section on focus groups is very small and he insinuates that focus groups are for lazy researchers. They apparently cannot generate valid data on their own and are used as a sort of preliminary research. Berg, however, had a completely different perspective on the matter. He believes that focus groups allow the researcher to fade more into the background and said that participants sometimes feel more comfortable sharing their views in a group setting as opposed to one-on-one. He also said that a group has the ability to generate findings that cannot be generated by an individual.
My background is sociology and I also took several psychology courses. The whole principle of sociology is based on the fact that society and groups of people need to be studied as a whole rather than as the sum of individuals. I was definitely happy to see that some methods specialists do no simply brush off the importance of groups and group interviews.
My background is sociology and I also took several psychology courses. The whole principle of sociology is based on the fact that society and groups of people need to be studied as a whole rather than as the sum of individuals. I was definitely happy to see that some methods specialists do no simply brush off the importance of groups and group interviews.
Unobstrusive Methods
The difficulty with studying human behaviour is that you can never be sure that you are getting accurate results. I have a background in qualitative research but it can be frustrating to hear that your interviews are biased or that people are not giving you accurate responses because they are afraid of what you will think of them. Even in quantitative methods there are many criticisms. For survey research some people may check off what they wish applied to them. For example, if you are asking how many hours of television a person watches per day he/she may answer fewer hours than are actually watched because he/she plans to cut down on hours of television watched. Sometimes this really bothers me. My friends who are in science roll their eyes when they speak about social science. I want my research to be taken seriously and to seem credible.
I was, therefore, intrigued to read Knight's section on unobtrusive methods. It is interesting to think of doing a study without leaving a mark. You're simply making observations. The researcher cannot influence subjects under these circumstances. Knight says that most researchers choose not to use unobtrusive methods but I think there might be something very valuable to them. I am not sure if they would make scientists take us more seriously, but they definitely are able to get around some of the criticisms with many social science methods.
I was, therefore, intrigued to read Knight's section on unobtrusive methods. It is interesting to think of doing a study without leaving a mark. You're simply making observations. The researcher cannot influence subjects under these circumstances. Knight says that most researchers choose not to use unobtrusive methods but I think there might be something very valuable to them. I am not sure if they would make scientists take us more seriously, but they definitely are able to get around some of the criticisms with many social science methods.
Subscribe to:
Posts (Atom)