Winter 2016 (Jan 04 – Apr 04)

Advanced Research Design

EDCI 681 (CRN 21008)


Detailed Course Outline

Resources

PRAAT is a program that allows you to prepare very detailed transcripts and do a variety of analyses, including that of prosody.

Thompson ISI Web of Science [click here]

Textbook: Wolff-Michael Roth, 2015. Rigorous Data Analysis: Beyond Anything Goes. Rotterdam, The Netherlands: Sense Publishers.

The course description can be found at this link: [course description]

 

Detailed Lesson Plans

The outline is a "living" one, continuously growing and changing to meet students' and instructors' needs, though the overall framework [reading, assignments] will stay.

[JAN 04] [JAN 11] [JAN 18] [JAN 25] [FEB 01] [FEB 15] [FEB 22] [FEB 29] [MAR 07] [MAR 14] [MAR 21] [APR 04]

 

 


JAN 04

Lesson topics

  1. Presentations
    1. Students present themselves
    2. A bit about me: How I got here; research methods: Doing Qualitative Research; Doing Teacher-Research; Generalizing from Educational Research; Analyzing Communication; First Person Method
  2. Introduction of the course
    1. Course, course objectives
    2. Readings, structure of the textbook
    3. Assignments
  3. Analyzing some data
    1. Introductory remarks. Bourdieu on methodology.
    2. On the relation between theory, naive conceptions, and phenomena. Epoché (Husserl); Bracketing (Garfinkel)
    3. A mystery transcript: In small groups, figure out as much as you can about what is going on here, who is speaking, where people are, etc. You may state hypotheses, but all of these are to be grounded in the data available. Keep note on our shared online board.
    4. Whole-class discussion
    5. Analysis notes by the whole class
    6. Video
    7. Garfinkel and Sacks: Structure of practical action, Lebenswelt pair (Livingston): "doing [analyzing data]," "doing [taking a graduate class in advance research methods]," "doing [queuing at the checkout]" . . . (Garfinkel, H., & Sacks, H. (1986). On formal structures of practical action. In H. Garfinkel (Ed.), Ethnomethodological studies of work (pp. 160–193). London: Routledge & Kegan Paul.)
    8. An exemplary analysis is provided here

Assignment

  1. Begin to collect the data for your assignment

[top

 

 


JAN 11

Assignments

  1. Reading: Preface–chapter 3 (beginning–p. 63) in textbook
  2. Your Journal/Lab Notebook (due: prior to class, electronically)

Lesson topics

  1. Checking in: Any needs?
    1. Questions concerning assignments
    2. Any other questions?
  2. Supervision, finding suitable advisor
  3. Items 3.5, 3.7, & 3.8 from January 4
  4. Example of a reading of the first three turns of the mystery transcript from January 4
  5. Discussion of readings
    1. The contradiction of learning research method (like children and grammar)
    2. Rigor vs. rigidity
    3. Theory and its role in interpretation, vs. what we are doing with the mystery transcripts. The example of power as property versus power as the effect of relations. (power|knowledge)
    4. The pre-constructed object. Tools of construction. Categorical reconstruction in dialectical materialist science.
    5. Kant and dialectics on concepts (Wittgenstein, family resemblance) (p.12). 5 individuals: (abcde) (abe) (bcd) (acde) (bde)
    6. Invariances; content versus repertoires (discourse analysis, Mulkay & Gilbert); phenomenography versus phenomenology
    7. Ethnographically adequate accounts (p.16) (career interview)
    8. Unit of analysis
    9. Radical doubt (constructions, tools of construction); critique of ideology; epoché
    10. Hypothesis testing
      1. Traditional statistics: p(data|H1)
      2. Bayesian: p(H0) + p(H1) = 1; continuous updating: p(H0|data), p(H1|data) = 1 - p(H0|data)
    11. Student comments on chapter 2
    12. Student comments on chapter 3
  6. Analyzing some data
    1. Use the google doc to record your hypotheses and, most importantly, supporting evidence
    2. Mystery transcript 2
    3. Analysis notes by the whole class
    4. Sample analysis (to be provided after class)

[top

 

 


JAN 18

Assignments

  1. Reading: Chapters 4–6 in textbook
  2. Your Journal/Lab Notebook (due: prior to class, electronically)

Lesson topics

  1. Checking in: Any needs?
    1. On getting the big picture, tasks as puzzle fragments
    2. On reading the textbook, the different format (see also, Derrida, Glas)
    3. On getting the big picture: In all future reflections, include, at the end, a paragraph entitled "This is the big picture of the course for me right now"
    4. Organization of analysis session with student transcriptions/texts + organization of discussion of PhD topic
  2. Left-over topics from January 11
    1. Remembering as the "steam engine" for approaching the Mystery 2 transcript
    2. Transcription as protocol of what people are doing while they are doing it (not talking about the doing. Why did the event unfold in this way?
    3. Theory --> analysis: the example of power: Mystery transcript 1: A in power over B (student) or B in power over A? (wrong question, because the real one is about the relation)
    4. non-teleological reading of the data, not going to the end so that the earlier parts are read through the outcomes
    5. Unit of analysis ~ category
    6. The actor's view, the shop floor of society
    7. Analysis is slow reading, very slow, so slow that it might take 2 hours to do 4 words!A
  3. Discussion of readings (remainder to 1h)
  4. Analyzing a transcription for content and discourse (1h 40min)
    1. Distinction between this task and the two preceding ones (mystery transcript): (Example: Interview about student interest, starting p.3)
      1. Purpose of analysis
      2. Theory
      3. Difference between data source and data
    2. Do 3 forms of analysis
      1. The full transcription of the interview, with line numbers
      2. Analysis 1: What is the content of the talk? (Think of, create representation)
      3. Analysis 2: What is the remembering discourse?
      4. Analysis 3: What is the discourse for talking about topic/content (science)?
      5. Whole class discussion
        1. Presentation of main findings by individual groups
        2. Relation of method and research question
        3. Relation of epistemological assumption to analysis, theory
        4. Class notes from the analyses

[top

 

 


JAN 25

Assignments

  1. Reading: Chapters 7–9 in textbook
  2. Reading the following excerpts exemplifying
    1. Applied conversation analysis (as we did in mystery transcripts of meetings 1 & 2
    2. Content analysis (as per Analysis 1 of Jan 18)
    3. Discourse analysis (as per Analysis 3 of Jan 18)
    4. Mixed forms, as used in Discursive Psychology (appropriate for Analysis 2 of Jan 18)
    5. These analyses should serve you as models for writing your own analyses in Assignment 1
  3. Your Journal/Lab Notebook (due: prior to class, electronically)

Lesson topics

  1. Checking in: Any needs? (10 min)
    1. Assignment 1: Don't hesitate sending transcriptions and first attempts at analysis prior to submitting
    2. How to write the analyses for Assignment 1 and Assignment 4
    3. Class notes from the meeting on January 18
  2. Discussion of readings (1:30 hr)
    1. Analyzing data and writing research: Relation of Introduction—Theory—Data analysis (findings)—discussion—conclusion
      1. PURPOSE: Compare, contrast the following three types of analysis, which we have done so far
      2. Applied conversation analysis (as we did in mystery transcripts of meetings 1 & 2
      3. Content analysis (as per Analysis 1 of Jan 18)
        1. Some materials from the data sources
        2. Grounded theory 1. Example 1;
        3. Grounded theory 2. Example 2;
      4. Discourse analysis (as per Analysis 3 of Jan 18)
        1. TOPICS OF DA: Genres of discourse including Repertoires; (Look for what comes after a claim beginning with "because"; what follows the "because" tends to be unquestioned even if the claim can be questioned)
        2. TOPICS OF DA: relation of text & context;
        3. TOPICS OF DA: relation of discourse & psychological topic (e.g., cognition, memory) (discursive psychology, D. Edwards & J. Potter);
        4. TOPICS OF DA: relation of semantic & syntactic structure of discourse (possibilities of sense-making)
        5. Assumptions: discourse is shared between speaker and recipient; it is a possible way of talking, inherently assumed to be intelligible
      5. Mixed forms, as used in Discursive Psychology (appropriate for Analysis 2 of Jan 18)
        1. What's your QUESTION?
        2. Now analyze
        3. Make your analysis the topic of your reflective interrogation (also, may name it)
        4. In your writing, describe concretely what you actually have done, rather than gesticulating towards it by means of names (authors, methods)
      6. These foregoing analyses should serve you as models for writing your own analyses in Assignment 1 (and Assignment 4)
    2. Chapters 7–9 in textbook
      1. Turn sequences (chapter 7): the insider view, working against teleological reading of the data. What might be some of the losses (dangers) omitting the fact that talk is sequentially organized and recipient designed?
      2. Knowledge-Power & Institutional Relations (chapter 8): Relational analysis vs. imposition of concepts; discourse and power (e.g., Smith, "The Standard North American Family"; struggle: who is in the know? and knowledge-power; ruling relations (Smith)
      3. On the shop floor (chapter 9): ethnographically adequate description (joke; "this is not a joke"; "I am just joking"; "I don't think this is funny!")
  3. Analyzing data (50 min-1 hr)
    1. Interaction analysis--data session (Assumptions) (Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. Journal of the Learning Sciences, 4, 39–103.)
    2. Some background to the history of the analysis
    3. Transcription of the videotape
    4. Class analyzes a video
    5. A published analysis

[top

 

 


FEB 01

Assignments

  1. Reading: Chapters 10–12 in textbook
  2. Your Journal/Lab Notebook (due: prior to class, electronically)
  3. Reading: Roth, W.-M. (2010). An anthropology of reading science texts in online media. Semiotica, 182, 409–442. (access this through your UVic library subscription)
  4. Reading: Roth, W.-M., & Hsu, P-L. (2008). Interest and motivation: A cultural historical and discursive psychological approach. In J. E. Larson (Ed.), Educational psychology: Cognition and learning, individual differences and motivation (pp. 81–105). Hauppauge, NY: Nova Science. [download here].

Lesson topics

  1. Checking in: Any needs?
    1. Doing your dissertation: begin framing your research as early as possible. Work with advisor. Perhaps think of multiple investigations, each covered in a chapter.
    2. Focus on writing your question. The question will drive the method to be used.
    3. If you disagree (with author, participant) don't just hold up your opinion and what you believe to be evidence, but attempt being rigorous and ask questions:
      1. What is the context within which the author's statement(s) make sense?
      2. What is the community within which this makes sense?
      3. Why would the author make this statement (rather than another, or none of that type at all)?
      4. If you want to take a shortcut, look for the authority of the author
    4. Assignment 1: expectations: (a) 1-2 paragraphs of introduction; (b) 3 pages of analysis; (c) full transcription at the end; you use excerpts or full transcription within the text. (excerpts may be better). In your analysis, every claim must be supported in reference to a specific fragment of the text
  2. Discussion of readings
    1. The documentary method (chapter 10)
      1. Mannheim: objective, expressive, documentary sense. ("The steam engine")
      2. Garfinkel: documentary evidence <--> underlying pattern. Evidence points to, stands on behalf of, establishes presupposed pattern; but pattern is used to interpret evidence (part --> whole; whole --> part)
      3. Professional and lay sociologists (psychologists)
      4. RESOURCE: Roth, W.-M. (2015). Analyzing the qualitative data analyst: A naturalistic investigation of data interpretation. FQS: Forum Qualitative Social Research / Sozialforschung, 16(3).
    2. Getting Time Back (chapter 11): From things to process
    3. Socially Responsible Analysis (chapter 12)
      1. Marx/Engels, 11th Thesis on Feuerbach: Philosophers attempted to understand the world; the point is to change it---an ethical project through and through
      2. Methods for real people doing real activities
      3. A key point for doing "rigorous data analysis": not truth in an eternal sense but ethics and the impact our analytic work, lay or professional, has on the lives of others and our own
    4. Discourse psychological approach to interests and motivation (Roth & Hsu, 2008)
  3. Analyzing some data
    1. Textual analysis (Roth, 2010)
    2. Any text is a manifestation of something. It works in a particular way. We cannot say (much, anything) about the author based on the text; and we can say equally little about how the many readers will take it up. The purpose of this analysis in small groups is to get at the "discourse" (not just words) of the alternative health movement. Here today's sample text. Do a textual (discourse) analysis similar to the Roth (2010) analysis that was among the readings for today. (Some scholars use the documentary method of interpretation according to analyze images/text (e.g. Bohnsack, 2010).
    3. For an example of such an analysis of online video, see Roth, W.-M., & Friesen, N. (2014). Nacherzeugung, Nachverstehen: a phenomenological perspective on how public understanding of science changes by engaging with online media. Public Understanding of Science, 23, 850–865. (Preprint here)
    4. Notes are [here]

[top

 

 


FEB 15

Assignments

  1. Your Journal/Lab Notebook (due: prior to class, electronically)
  2. Three students prepare transcript in field of their interest, analysis
    1. Name: Svetlana
    2. Name: Christie
    3. Name: Ining

Lesson topics

  1. Checking in: Any needs?
    1. Adding of second Assignment 4 option [assignment description]
    2. When ready to submit your Assignment 1 (1 week after your data were discussed in class), please use the naming convention
    3. For writing up Assignment 1, use the January 25 sample texts as models
  2. Writing research (and assignments, use the January 25 sample texts as models)
    1. Specify your interest: What is it that you want to find out?
    2. Analyze the data (theoretical commitments may be implicit); use a descriptive heading (look at all posted sample texts)
    3. Organize your analyses according to themes (use subheadings that name the themes)
    4. Write introduction (& background) parts; write HOW you analyzed/approached your data
    5. Write conclusions (use descriptive heading)
  3. We conduct three cycles of analysis using the following structure, and in this order: (1) Ining (2) Christie (3) Svetlana
    1. Data are presented and "owner" talks about her/his interest as a focus
    2. Data are analyzed in small groups (owner is silent observer); groups keep record on Google docs
    3. Small groups report back
    4. Data "owner" makes a brief presentation to class of his/her analysis; use some data, show how interpreted (5 min)
  4. Discussion and additional topics
    1. The written record of the analyses is here

[top

 

 


FEB 22

Assignments

  1. Your Journal/Lab Notebook (due: prior to class, electronically)
  2. Three students prepare transcript in field of their interest, analysis
    1. Name: Jessica
    2. Name: Tatiana
    3. Name: Colin

Lesson topics

  1. Checking in: Any needs?
  2. On writing research, writing your assignment
    1. [The structure] of a recent article (mixed methods); the [full article] as submitted
    2. Most important: Always select your audience first—dissertation, journal, etc.— then write for THAT audience
    3. The process of research is different from writing text, which has its own logic
    4. Research question: you write the one that your research in the end answered rather than the one you started out with
    5. Methods: Describe sources of data ([here an example]), and HOW you analyzed them. (Methods, not methodology, as you are not contributing to the science of methods but are using a method to find out something.)(
    6. Findings: Use descriptive title; do not go linearly through the data. Organize findings into themes, subtitle themes, where appropriate.
    7. Write the Introduction so that it introduces what you actually found.
    8. In the Discussion, you relate your findings to the existing literature, making particularly clear what your study shows over and above what the community already knows.
  3. We conduct three cycles of analysis using the following structure, and in this order: (1) Colin (2) Tatiana (3) Jessica
    1. Data are presented and "owner" talks about her/his interest as a focus;
    2. Data owner formulates the question to be researched by the different groups in the class;
    3. Data are analyzed in small groups (owner is silent observer); groups keep record on Google docs;
    4. Small groups report back;
    5. Data "owner" makes a brief presentation to class of his/her analysis; use some data, show how interpreted (5 min)
  4. Download [Notes from the discussion]

[top

 

 


FEB 29

Assignments

  1. Your Journal/Lab Notebook (due: prior to class, electronically)
  2. Three students prepare transcript in field of their interest, analysis
    1. Name: Gwen
    2. Name: Jing
    3. Name: Meredith

Lesson topics

  1. Checking in: Any needs?
    1. Please send tentative research questions for your PhD, which will be the starting point for the "research design" sessions starting Mar 14.
    2. Why not write reflective notes on the dissertation topic (in preparation for the discussions of your PhD topic)?
    3. Texts for Assignment 4 are posted now.
      1. OPTION 1.1: A conversation between researchers and teachers is [here]
      2. OPTION 1.2: An article (you find it [here] and the associated graphics (2) [graphics]
      3. OPTION 2: A transcription/text of your own. It should reflect the extent and possibilities of OPTIONS 1.1 or 1.2
  2. We conduct three cycles of analysis using the following structure, and in this order: (1) Gwen (2) Meredith (3) Jenny (Jing)
    1. Data are presented and "owner" talks about her/his interest as a focus;
    2. Data owner formulates the question to be researched by the different groups in the class; (Posted in Google Docs)
    3. Data are analyzed in small groups (owner is silent observer); groups keep record on Google docs;
    4. Small groups report back;
    5. Data "owner" makes a brief presentation to class of his/her analysis; use some data, show how interpreted (5 min)
  3. Class Notes are [here]

[top

 

 


MAR 07

Assignments

  1. Your Journal/Lab Notebook (due: prior to class, electronically)
  2. Three students prepare transcript in field of their interest, analysis
    1. Name: Nirmala
    2. Name: Samantha
    3. Name: Lyndzë

Lesson topics

  1. Checking in: Any needs?
    1. On discussing in small groups
      1. Holding each other accountable to the data. Our discussions should not veer off and become unprincipled, but we need to stick to making claims that are grounded in the data.
      2. Holding each other to Bakhtinian (dialogical) principles: Actively listening is the counterpart to contributing by speaking. Not drowning out another voice but taking it up and thereby transforming it.
  2. On doing and writing research
    1. Formatting transcriptions (Sample here)
    2. Dissertation topics, weeks to come (Here a possible structure)
      1. What is the issue? What is the knowledge of the field?
      2. Where is the field lacking something?
      3. Research question
      4. Method: Data, participants, analysis, e.c
      5. Theory only comes afterwards
    3. On writing research—writing for reading to organize itself.
      1. Reading as social practice, that is, as something that is larger than the individual reader; it is an anonymous (social) process that manifests itself in the individual. Writing provides the materials for reading to organize itself, to find what is there to be looked for and to be found, to structure itself, etc.
      2. Example 1 . . . writing the findings;
      3. Example 2 . . . titles & coherence;
  3. We conduct three cycles of analysis using the following structure, and in this order: (1) Nirmala (2) Samantha (3) Lyndzë
    1. Data are presented and "owner" talks about her/his interest as a focus;
    2. Data owner formulates the question to be researched by the different groups in the class; (Posted in Google Docs)
    3. Data are analyzed in small groups (owner is silent observer); groups keep record on Google docs;
    4. Small groups report back;
    5. Data "owner" makes a brief presentation to class of his/her analysis; use some data, show how interpreted (5 min)
  4. Download: Notes from the discussions

[top

 

 


MAR 14

Assignments

  1. Your Journal/Lab Notebook (due: prior to class, electronically)
  2. Four students to have a possible research question in field of interest for research design discussion (40 min/student) (4:30-5:10, 5:10-5:50, 5:50-6:30, 6:30-7:10)
    1. Name: Tatiana
    2. Name: Samantha
    3. Name: Lyndzë
    4. Name: Svetlana

Lesson topics

  1. Four students have possible research question in field of interest discussed (40 min/student). We follow this pattern:
    1. Student presents information on general area/background of the study; brief introduction. Ideally, this includes:
      1. What is the issue? What is the knowledge of the field?
      2. Where is the field lacking something?
    2. Statement of question—to be posted and projected by means of Google Docs
    3. Discussion: Refining research question and all aspects of method: research setting, participants, context, data collection, data analysis
    4. Approximate time brackets: (4:40-5:20, 5:20-6:00, 6:00-6:40, 6:40-7:20)
    5. ORDER: Tatiana; Samantha; Lyndzë; Svetlana
    6. Questions discussed

Final assignment

  1. OPTION 1.1: A conversation between researchers and teachers is [here]
  2. OPTION 1.2: An article (you find it [here] and the associated graphics (2) [graphics]
  3. OPTION 2: A transcription/text of your own. It should reflect the extent and possibilities of OPTIONS 1.1 or 1.2

[top

 

 


MAR 21

Assignments

  1. Four students to have a possible research question in their field of interest for research design discussion (40 min/student) (4:30-5:10, 5:10-5:50, 5:50-6:30, 6:30-7:10)
    1. Name: Ining
    2. Name: Colin
    3. Name: Gwen
    4. Name: Jessica

Lesson topics

  1. Concerns
    1. GOALS OF THE COURSE: (a) exploring research methods to specific problems, questions, and contexts; (b) design of a research question considering the relationship between question and research method
    2. GOALS OF THE INSTRUCTOR: (a) Allow students to go farther than they might have believed they can go; (b) Learn to analyze data rigorously; (c) learn to write reporting research in journals
    3. Academic honesty
  2. Four students have possible research question in field of interest discussed (40 min/student). We follow this pattern:
    1. Student presents information on general area/background of the study; brief introduction. Ideally, this includes:
      1. What is the issue? What is the knowledge of the field?
      2. Where is the field lacking something?
    2. Statement of question—to be posted and projected by means of Google Docs
    3. Discussion: Refining research question and all aspects of method: research setting, participants, context, data collection, data analysis
    4. Approximate time brackets: (4:40-5:20, 5:20-6:00, 6:00-6:40, 6:40-7:20)
    5. ORDER: Ining; Colin; Gwen; Jessica
    6. Questions discussed

[top

 

 


APR 04

Assignments

  1. Final assignment due
  2. Three students to have a possible research question in their field of interest for research design discussion (40 min/student) (4:30-5:10, 5:10-5:50, 5:50-6:30, 6:30-7:10)
    1. Name: Jing
    2. Name: Meredith
    3. Name: Christie

Lesson topics

  1. Three students have possible research question in field of interest discussed (40 min/student). We follow this pattern:
    1. Student presents information on general area/background of the study; brief introduction. Ideally, this includes:
      1. What is the issue? What is the knowledge of the field?
      2. Where is the field lacking something?
    2. Statement of question—to be posted and projected by means of Google Docs
    3. Discussion: Refining research question and all aspects of method: research setting, participants, context, data collection, data analysis
    4. Approximate time brackets: (4:40-5:20, 5:20-6:00, 6:00-6:40)
    5. ORDER: Meredith; Christie; Jenny (Jing)

    [top