Student Self-Evaluations of Open-Ended Projects
in a Grade 9 Science Classroom

 

Student self-evaluations have considerable potential in helping students become better learners. The present study was one teacher's first attempt to understand the role of students' self evaluation of their process and products in a science unit centred on open-ended projects. The research was interpretive and attempted to construct meanings relevant to the participants in the classroom. The data collected for this study included video tapes, transcripts of the videotapes, student self-reflection and self-evaluation writings, student project reports, student interviews, discussions with parents, teacher notes and observations, teaching assistant notes and observations and the assignment outlines. This article provides an interesting perspective on the social structure in student group self evaluation and a better understanding of the important role students' self-evaluation can play as part of an open-ended project learning environment.

Evaluation of students' work is central to schooling. However, grading can be seen as an insidious technology that does more harm than good: it is used stratify students into hierarchies which, because of the middle-class values on which they are built, are inherently biased against those from economically less fortunate classes (Foucault, 1975; Postman, 1992). The end effect is that grading systems shake some students out of the race for a reduced number of coveted spots at university (Roth & McGinn, 1998). Evaluation, and grading as one of its embodiments, has, as all technologies, considerable political component.

There exist a number of serious critiques of evaluation, especially of grading practices which relegate object status to students and, in their wake, have deleterious effects on students' constructions of Self (Roth, 1996; Roth & McGinn, 1998). For example, research in our group has shown that any single test in science is likely to be insufficient to assess students' competencies related to some science topic (McGinn & Roth, in press; Welzel & Roth, 1998). What may be worse for many, different testing formats and social conditions show that competence, rather than being a uniform construct, is likely to vary with changing conditions so that it makes more sense to consider competence as having a heterogeneous topology within individuals and across settings (Roth, in press). On the other hand, involving students in the evaluation process can have some beneficial overall effects on student learning, views of the learning environment, and their levels of engagement (Roth, 1998c). Students become more active learners by setting their own targets, they are more thoughtful and creative due to ownership over all aspects of their project, and they display a greater vested interest in their own completion and their peers' completion of the criteria they helped to establish (Harris, 1997; Stefani, 1998).

Teachers who work within educational systems do not have the option of broadly changing evaluation and grading practices from one day to the next. However, they do have the option to incorporate elements into their evaluation practices that make the process more open and democratic. The present study evolved from one teacher's interest in the positive mediation student self-evaluation may have on process and products of learning. The purpose of the present study was therefore to examine how students' project goals develop and change and how the resulting progressive and culminating self-evaluations mediate their perceived progress.

This study falls into the category of action research, a type of research in which stakeholders engage in cogenerative learning to change their practices and lifeworlds (Elden & Levin, 1991). To ascertain the quality and credibility of the research claims, and therefore our understanding, we enacted criteria that are appropriate in the context of qualitative inquiry (Guba & Lincoln, 1989).

Twenty-three students (11 male, 12 female), from Creekside Secondary School, a public school located in an agricultural region of Western Canada, participated in the study. The class included three students with special needs and three Grade 10 students returning to complete Science 9 on a second or third trial. This was a general science class at Creekside, but students generally were very enthusiastic and participated well in all class activities. Clint taught this class, his seventh time teaching Science 9. His academic background includes a B.Sc. in chemistry and a B.Ed. with a chemistry and general science major and he has been teaching for 5 years.

Student activities emphasised individual and collective development of ideas in project groups. Project groups often contained eight to ten students. Students were given the opportunity to choose their partners for all group assignments and were given the option of an alternate assignment if they chose to work alone or in a much smaller group. All students were encouraged to take individual responsibility for their own progress in projects and to co-ordinate with their group members regarding the organisation of the project, the division of responsibility, the choice of materials to be used and the time for collaborative work.

At the beginning of the course, students were issued a course outline describing the unit learning outcomes prescribed by the Provincial Government.[1] These learning outcomes were discussed when initially distributed and the relevant outcomes again discussed at the beginning of each new unit. Students' prior knowledge was determined in class discussions at the beginning of each unit and ideas developed for a culminating major project. Clint intervened in self-selected groups only when conflicts between students arose. In Science 9 there were two major group projects. In Project I, students assembled a "body," which included the various body systems studied in the biology unit, out of household materials. For Project II, students either built an alarm clock that would wake someone up and completed at least five energy transformations (i.e., devices converting energy from one form to another), or they built a "transporter" to move an egg 10 meters through the air to a 1m x 1m target unbroken without any direct human contact anywhere inside the 10 meters. Students designed these from scratch--no explanatory handouts detailing "methods" were provided--as Clint wanted the students to spend a day or two engaged in framing the parameters of their design and deciding on the materials they would need.

Formal grading requirements were not initially provided. During class time, the Clint guided the student through the relevant material of the units with laboratory assignments and demonstrations and assigning questions from the texts. If students completed the daily assignments early, they were expected to continue work on their major projects. Materials such as cardboard, paper, tape, glue-guns, felt pens, pulleys, string, wooden blocks, and spring scales were provided. Various reference materials were provided, including: a full scale model skeleton, Gray's Anatomy, The Anatomy Coloring Book, biology texts, A.D.A.M. Essentials computer program, Machines Made Simple, The Way Things Work Vol. 1 + 2, and The Incredible Machine Computer Game.

Students were assessed in pairs and individually for their laboratory work and written reports. Each student was also examined individually in the form of written mid-unit and final unit exams. Each student was also assessed by group evaluation, self-evaluation and teacher evaluation on additional small assignments such as a drug or alcohol report and poster.

During these units, students spent approximately 40-50% of their class time working on hands on laboratory assignments. When each of these lab assignments was completed, students would work on an aspect of their major projects. For example, after completing a dissection laboratory assignment on the structure of bone and muscle, the students worked on Project I and continued with the building process of making the skeletal system and parts of the muscular system. This work would be complimented with text readings such as the first two chapters in the student text covered the skeletal and muscular systems of the human body. These sections allowed students the opportunity to explore as a class the intricacies of the human skeleton and relate it to their project. The class would then continue with a new laboratory assignment, then more readings and more work on the projects. As the class moved onto the next chapters, the students had read further ahead and had more questions regarding body systems in order to design their Projects more efficiently.

Each student and group received verbal feedback from the teacher. This feedback included directed questioning of methods and design, suggestions for research materials and building materials, safety procedures when using items such as knives, reflections of body functions in everyday life, discussions of autopsies and operations, interpretation of X-rays, reminders of deadlines, general encouragement, and praise. Marks were assigned (with the opportunity to improve their mark as the project progressed) for work habits and special creativity during the project. These marks accounted for approximately 4% of their final grade. The overall grade of the project was based on self-evaluation, group evaluation and teacher evaluation accounting for approximately 15% of their final grade.

To allow for analysis in this assignment, student presentations of the two major projects were videotaped. The course material during Project I included topics from the biology unit and related to the human body systems including the skeletal system, muscular system, digestive system, circulatory system, respiratory system and excretory system (see Footnote 1). This unit also included a section on factors that affect the human body such as drugs and alcohol. The course material during Project II included physics topics related to Newton's Laws, energy, motion, forces, work, and simple machines. During these units, Clint collected the laboratory reports produced by the students, provided hand-written feedback on these reports, and accepted the reports for re-evaluation. These assignments provided interim analysis on student progress in each unit for the teacher. For analysis of student evaluation, student projects were collected after their videotaped presentations along with their written self-evaluations and group evaluations. Criteria for evaluation was set mid-way through each project so those students had some opportunities to work on the projects before deciding on the basis for analysis. Students were given the opportunity to write their evaluation as they felt, but they were also given a few guiding focus questions if they found themselves encountering difficulty in their self-assessment.

The audio portion of the videotapes was dubbed to audiotape and then two transcripts were independently prepared. The two transcripts were then compared and any differences resolved by re-viewing the appropriate section of the tape, occasionally using the original video tape, until the text was clearly determined or was marked "inaudible". Each videotape was reviewed without the audio component and notes made of body movement, eye movement, and eye contact. During viewing of the video tapes, categories of self-evaluation were constructed such as "weak verbal," "strong verbal," "verbal with project referencing," and "non verbal project referencing." For example, students who contributed a great deal to building a certain portion of their Project often displayed "verbal with project referencing" or "non verbal project referencing" as they touched, gestured over, or pointed to the portions of the body they had conducted the construction work on, particularly in area that were more complex. This pattern showed up again in Project II.

The videotapes and transcripts made up the majority of the data sources for assessing student evaluation processes. The videotapes were analysed following the precepts of interaction analysis (Jordan & Henderson, 1995) and assertions were framed which were then tested according to Guba and Lincoln's (1989) criterion of "credibility." First, assertions were checked against the entire data set, rejected when they did not represent a substantial amount of the data, and subsequently discussed with colleagues (disinterested peers). Clint's intimate knowledge of Creekside, the community, and its students and the extent of the data sources embody prolonged engagement and persistent observation. In addition, several one-on-one interviews were conducted with students about their work. At this stage, the researchers' constructions were checked against students written self-evaluations for confirmation or rejection of assertions.

When we began to examine the question of student self-evaluation, we noticed that there were many social interactions within the groups and within the class that mediated a student's perceived progress and as a result, their self-evaluation. For example, during the development time in class while looking at on-going evaluation, a number of students seemed very certain about how to complete their project and they excitedly showed the teaching assistant their design and indicated that they knew exactly what they were doing. However, they began sharing ideas with another group and their confidence shifted considerably and self-doubt began to appear as to whether or not their idea met the goals of the assignment or whether or not the other group's idea met the goals of the assignment.

This study was designed to examine the results of student's self-evaluation in open-ended project work, which included an examination of the on-going confidence students exhibited when examining their understanding of an open-ended assignment. During the on-going development of their projects, and especially at the beginning, there was a great deal of uncertainty and students' evaluation of their progress showed in many cases a lack of confidence or no idea as to teacher expectation. With a little encouragement from the teacher (there was also a teaching assistant) and the presentation of portions of previous projects, students re-evaluated their work and started to develop solid ideas. This resulted in a dramatic shift to greater confidence and self-evaluations that are more positive. Since the actual grading requirements were not set until well into the projects, students had more opportunities for exploring their ideas, reflecting on difficulties encountered, and formulating the "unspoken" goals of the assignment. Only subsequently did they participate in determining the criteria for evaluation. However, after establishing the evaluation criteria, there was a noticeable period where their confidence appeared to drop as they re-evaluated existing work to see if it met their own criteria. This was followed by a re-negotiation of grading criteria and further adaptations to their existing work before moving ahead with new work. We show here that self-evaluation, in the type of context in which Clint enacted it, is an emergent process and product that arises from students' continued engagement with their project. Students and their projects stand in a Subject-Object (S-O) relation. As projects emerge, student evaluation shifts from evaluation of ideas and their initial con-cretions to an evaluation of the creator (S) of the project (O). That is, as the projects develop, an increasing level of evaluation becomes internalised as students associate projects with themselves: projects become a measure of individual qualities such as ability, commitment, innovativeness and so forth. Thus, when students feel discouraged or frustrated with their work, this is seldom the result of a singular statement or incidence, but emerges from the interactions with Others and with materials in their lifeworlds.

The students' evaluation began as soon as the project was assigned. At this point, the assignment was strictly giving them an idea that they were to think about "before" the actual assignment was given.

Over the next couple of days, I want you to think about the human body and all the parts and systems that make up the body. This is what we are studying in this unit. We will also be starting a major project called Project I where we will build a body and some of its systems using household materials. [Clint]

This idea resulted in immediate questions by the students to clarify process rather than evaluation. For example, some of the questions included, What can we build this body out of?, How big does it have to be?, Do we have to build it?, Can we draw it?, How detailed do we. . . can we. . . should we make it?, How much time do we have?, and Is it all in class?

At this point in the project, the students were simply setting boundaries as the original question posed was very open with regards to what must be included and how they are to complete the project. Initially, Clint did not regard these questions as part of the evaluation process but the answers to these questions appear to be partly the cause of the first confidence block experienced by the students.

In our experience as teachers, we noted that if a teacher presents a problem and associates it with certain scientific processes or concepts, students' solutions will be greatly influenced by using these processes or concepts. That is, students are trained to search for solutions that satisfy the teacher rather than seeking a solution that pleases themselves. (Roth [1998b] reported a case where a teacher vowed never to use glue guns again because it had allowed children to make projects without applying bracing techniques which she wanted to teach them.) When Clint presents his unit-culminating projects, he therefore avoids associating them with specific processes or concepts. As a result, students usually produce a much greater variety of solutions and a greater variety of perceived goals. On the other hand, this openness forces students to set parameters on their work, and associate with it an evaluation that--because they construct ideas and criteria whether the unfolding projects meet their initial ideas--only they can establish. Project I was delimited in that it was to be build with household materials and that the individual students or groups should not spend more $5 to purchase materials.

As students attempt to develop ideas, they begin to assess ideas in terms of their feasibility. At this point, evaluation is clearly concerned with the Other, that is, re-presentations (objects of reflection) and materials. For example, when students returned the next day with some ideas and sought confirmation that their ideas were appropriate and within the confines of the project in which they were to build their "Project." The following questions suggest that they had developed some ideas which now were to be assessed in terms of available materials, and in terms of Clint's yet unstated parameters.
1. "I can build it out of anything? Ree-bar? My dad is a mechanic and we have a welder?"
2. "What about paint cans? Can it include cow skeleton parts?"
3. "I was thinking that I, that we, well our idea is to use marshmallows for the backbo....:, I mean the spine? Is that okay? How can we attach it?
4. "If we are all like doing separate parts, like how do we make them all, well? Fit?"
In the first two quotes, students are still in the process of formulating ideas, but these formulations arise from an interaction with material constraints (availability, feasibility, and teacher consent) on the project. At this stage, evaluation is a self-evaluation projected into the future designed to ascertain that their intervening efforts will not be spent on ideas that are not acceptable or practical. At this point of the project, feedback (another form of evaluation) from teacher (assistant) is crucial. If the teacher's (or assistant's) answers were not positive, or if they posed questions that re-directed a student's work, students' personal evaluation process would suggest focusing on an idea first or re-formulating existing ideas before continuing. In the third example, the student formulated an idea and confirmed that it appropriately fits within the vague boundaries of the project. In the final example, the student did not only indicate a particular solution to a problem, but also moved beyond testing boundaries. He already thought about group processes including dynamics of the collectivity and integration. Such questions help to formulate future progress in the project but are also informal and unofficial (because they do not lead to assessments inscribed as grade or text) self-evaluations of their ideas.

A critical point arises in the project when students--after tinkering with their initial ideas, and after some initial work on "materialising" them into prototypes--they participate in a session where students and teacher collectively decide on the formal evaluation criteria for the project. This establishment of class criteria becomes a milestone, because from this point on, students are afforded to assess their work against a set of external, shared values rather than their own, often-vague ideas about the projects. However, even before this milestone, students begin to assess their ideas and emerging prototypes as they compare them with other emerging projects.

As various groups experienced perceived success with their ideas, the students displayed greater confidence in their understanding of the assignment. Students who lacked this confidence often gravitated to other groups to test their own ideas and re-affirm their progress. At this stage, teacher input was designed to scaffold students' idea formation rather than to eliminate them. However, because of the open nature of the assignment, students may not have seen peers completing tasks in the same way. For some students, this proved to be discouraging; teacher (and assistant) therefore provided encouragement as needed. (It is difficult to provide detailed criteria which lead teachers to help some but not others; knowing when to help or ask a question is one of those teacher competencies that seem to eschew description [Roth, 1998c].) For example, one student worked on the feet of Project and had trouble with co-ordinating the bones. He had been building these "feet" with pebbles and rocks used to represent the bones but after seeing other groups using paper and pipe-cleaners, he discarded his own idea. When Clint approached him, he indicated that he was on the wrong track, as the pebbles were not a household item. Whereas others had flexibly interpreted the introduction to the project (" . . . using household materials") by considering welders and steel bars (see above), this student, after some initial tinkering with pebbles and rocks, and after seeing other projects, rescinded his original idea. Clint therefore told him that these materials were perfectly acceptable; after the student had explained his use of the rocks, Clint further encouraged him in pursuing this idea. The student continued to use the rocks for a period of days--from a teacher perspective, the project turned out very well--however, the student found the difference between his and the other projects simply too great. This resulted in his changing his materials to paper, wood and pipe-cleaners to be like others in his class. In this case, the student's perception of the requirements and the goals did not allow him to continue on his own unique path. In a later interview, Clint discussed this change with the student.

Clint: Why did you no longer continue with the rocks, etc.?
S: It wasn't a household item. You told me it was fine but no one else used it.
Clint: So you felt the need to change what you were doing?
S: Uh. Yeah. I had to make sure I did it right. Mine was not the same. It's Different. We didn't have to make them actually work, so paper and pipe cleaners were better.
Clint: How do you mean they didn't have to make them work?
S: Well, the bones here and here POINTS[feet] are jointed to let them, do, I mean they are for ahm, balance and stuff. If no joints, the foot could not move and you'd fall down. Or maybe not fall down but you couldn't do, some movements?

Initially, the student indicated that he changed not because he felt that the project did not conform to the teacher's expectations, but because in his assessment, the gap with the project in the remainder of the class was too great. At the same time, having worked with pebbles and rocks appeared to have scaffolded this student into working out an understanding of the operation of feet in a three-dimensional world. The level of understanding he demonstrated at this time was far superior to the understanding displayed by the other students with their paper feet. Their understanding appeared to be based on two-dimensional views of the feet and based on aesthetics rather than function whereas the feet made of rocks and resulting discussion indicated an understanding of the function of various parts of the feet.

Here, the student's evaluation of his project led to some frustration with the amount of time he had spent time on an idea that, later, he felt was not acceptable. Although Clint attempted to encourage him to continue his work, he abandoned this line of inquiry. Yet after the project was finished, he returned to the original idea. He indicated that it would have been easier putting the muscles on the bones if he stuck with his rock feet. This illustrates how group and individual interactions can influence a person's on-going evaluation and a form of peer pressure could re-direct efforts on part of the student.

A major point of self-evaluation occurred when the class collectively constructed the grading criteria. Clint chose to establish grading criteria later in the project rather than earlier as he felt it gave the students a better understanding of those aspects of process and product that should be evaluated. This means that they have more productive and constructive input in their evaluation. Clint also allowed for some variance in individual group grading requirements to adjust to special aspects of each group. For example, one group was very particular about accuracy of their skeleton. It was decided that the basic components of the skeleton and their placement must be correct for each group but for the one group, the dimensions and proportions would also be a factor taken into account as they went through great pains to measure portions of their own body to make parts appropriate. Another group paid particular attention to internal details of things like the brain and heart and made some "break-away" models.

The class discussion led the students to evaluate their own work in terms of "what everyone else has done." This is an important regulator of what knowledge, materials, or practices would become the generally accepted shared aspects of a classroom community (Roth, 1998b). Individual students and groups then attempted to decide if they should have included what other people had done in their own work. Clint went to great lengths to encourage them all and suggest that it was perfectly acceptable to emphasise differences. The same problems occurred at individual levels. For example, one group had a two-dimensional spine made of cardboard (just drawn on) and began to feel that their work was not adequate when compared with the marshmallow and baker's dough spine that another group had constructed. Again, Clint encouraged them by pointing out that their spine showed incredible detail by labelling the different types of vertebrae and suggesting their functions. There are many more examples of students adjusting their work as they feel their ideas changing regarding what the expected outcome should be.

On completion of the projects, students presented their work to the classroom community. To scaffold their presentation, Clint had proposed a series of questions that they could address. These questions included, "What part did you work on? What problems did you encounter? How did you solve these problems? What did you learn while doing the project? Would you recommend the project to another student? and Would you like to do a similar project again?" The students were also given the opportunity to speak about anything else that they felt was important about their work in the presentation.

The video tapes of these presentations supports the claim that there was a relation between the depth of the background research and the extent of students' explanations. Those who had spent more effort researching the parts of the body they built also provided longer and more accurate scientific explanations of that part.

CS: Oh, um, I made the hand of straws and pipe cleaners and I put pipe cleaners inside them so you can bend them and there is like the "bendy" parts along the joints and me and JY, we made the heart of paper, the paper on it, and we made the stomach of paper stapled (filled) with toilet paper and the liver and the lower intestine, the (???) intestine, the bladder.
Clint: What did you make the intestine out of?
CS: We took nylons and filled them up with paper, crumpled up paper.
In the first part of this explanation, the student immediately talked about the project parts that she made and about the materials she used. However, CS also provided a rational for her design decisions. The structure of the hand she had constructed also showed incredible detail containing small straws adding up to about 80% of the bones in the hand. In a subsequent interview, she indicated the remaining bones were either too small or too difficult to put in. In comparison, the second part of the explanation was relegated to listing the pieces and does not indicate any extra knowledge. When prompted, CS then added an explanation of what materials were used to complete the project. The answer of another student, JY, who had worked with CS on the intestines, did not indicate any further understanding.
JY: I mean its, we use a funnel for the bladder.
Clint: Excellent, and why did you choose that?
JY: Because, it looked like that in the picture LAUGHS[] in the book. Yeah, and I made the hips also out of cardboard.

The response from CS and JY regarding their work on the internal organs did not reflect the hard work they went through building them. Based on their responses, the students did not seem as sure about their constructions. However, when our videos are analysed in terms of what students not only say, but also in terms their manipulations of the bodies they built, one arrives at a different assessment of their understanding. For example, while discussing her project, CS illustrated an understanding in the way she manipulated the various parts of her project. While talking about the hands, she lifted them up and pointed at or demonstrated movements allowed by the many different "bones" she put into the hands. She not only knew the parts, but also had a thorough understanding of their place in the system and their functioning.

As a result of our analysis, Clint developed a categorisation scheme for student presentation types as possible additional evaluation criteria including "strong verbal, with project referencing," "non-verbal, project referencing," and "weak verbal." The presentation CS enacted in the context of the hand she built fell into the category "strong verbal, with project referencing." This category represents a verbal response that indicates some understanding but also a contact with their work, particularly contact that seems to indicate further understanding or extra efforts made on their part. When CS talked about the intestines, the quality of her understanding changed to "weak verbal," for she only pointed out her contribution and did not show in other, non-verbal ways that she understood this part of the body.

The same assessment was made for JY's follow-up to CS. Their verbal responses did not reveal a good understanding. Yet, while JY was talking, CS made regular contact with the intestine portion of the project, and at one point counted the number of turns in the intestine under her breath. In a subsequent interview (using video for stimulated recall), JY suggested that her count was intended to make sure that the small intestine still had the correct number of bends in it (they determined this from diagrams in the books they used as reference) and that the end leading to the large intestine was still getting larger. She also indicated that she was worried that the paper stuffed inside still had the shredded bits (representing villi) pointing inside the intestine.

"Non-verbal project referencing" is used to represent any referencing to their work in a non-verbal fashion. This category could represent the touching and emphasis on certain parts of the project, the extra evaluation material provided in their written reports and any other form of drawing emphasis to their work in a non-verbal fashion. JY, for example, did not contact the body in any way except to point at the funnel they used to represent the bladder. In a later interview, Clint could ascertain that she did help with the internal organs but it was secondary to her work on the hips and as a result and she therefore did not take much ownership in the intestines. Consequently, she felt it was inappropriate to be evaluated on this aspect of the group work, for she only helped rather than design and take major responsibility in building the part.

A similar case in which students did not take credit for the group work appeared in the following episode.

JB: Well, I made the arms out of, I made the right arm out of paper maché and put plasticene on top of it. And I made the bone out of my lamp and the tendons in the arms are made out of rubber gloves.
C: Very good. Can I actually get someone to move the arm to show how it bends.
JB: It bends at the elbow, like this.
C: Actually, it's quit neat because those muscles actually move when you do, that's very good! Okay.
JB: And the other arm I made with plasticene as well, and those are the veins and arteries, the blue ones are the veins, the other the arteries.
JB was very shy, but the work on his project was exceptional. He had made arms that actually functioned very well including muscle movement. However, in his verbal presentation, he focused strictly, on what the model was built from and required a great deal of prompting for scaffolding his presentation. However, his physical references indicated an excellent understanding of the project. His hands roamed over the ends of the arms drawing attention to the detail put into designing the ball part of a ball and socket joint of an arm. He also traced out details in the plasticene muscles he designed. His presentation did show "verbal with project referencing" but his verbal component was not as strong as his project referencing component. During a subsequent personal interview, he provided what would be described as a "strong verbal" presentation as he discussed why the muscles was shaped the way it was, why he used rubber gloves for tendons, and so on.
There were also situations where new understanding appeared during the presentation, which seemed to increase the confidence of the presenter.
JP: I made the arms, this one's kinda lame. It's made out of cardboard and it's got this outer bone, just like.
EE: The bone's broken.
JP: It's broken there. It really isn't supposed to move but I guest it's good `cause it actually moves now. This one is made out of wood, there's a little cardboard chunk there that it can move around and I made the lung, uh, the liver and it's just two cardboard cut-outs put together.

In this situation, the student begins with a "weak verbal," not happy with his work and one of the members from his group points out a detail he was hoping would be overlooked. However, his confidence seemed to rise as he took advantage of the situation and added a strong project reference at this point to show how the arm moved. In reviewing the tape with the student, Clint also noticed that the student used his fingers during the entire presentation to show the bone structure of the arm, indicating how he had made two bones in the forearm and that his design did allow for rotary movement. The muscle material he designed, however, broke the upper arm portion of the bone when he illustrated how the arm worked and this added the element that he later used to represent the elbow. In the original situation, Clint had categorised the presentation as "weak verbal," but a re-analysis of the video suggested that the presentation should be categorised as "verbal, with project referencing." During an interview one week later, the "weak verbal" category was changed into a "strong verbal."

Another example of a possible misdirection was a student who constructed a rather complex skull: "I made the head. I used it out of file folder cardboard. It's easy to bend and shape." This very short comment constituted what he had to say about his project. This would have to classify as a "weak verbal." However, it also included a strong physical referencing to the project. This referencing continued throughout everyone else's presentation. On closer examination of his work, he had actually built in many of the immovable joint in the skull. In a later interview, he indicated that these joints were all needed to allow the skull to grow. Combining all these material, his category was also changed to a "strong verbal with project referencing."

One week after the presentations, students were given 30 minutes to compose a written self-evaluation of their project-- about which they knew from the beginning--in which they were to address topics such as their individual work, collective work, problems, etc. Surprisingly, students tended to be more negative about their achievement than expected. In most of these self-evaluations, the students seemed to lack a lot of confidence in their work and "put down" most of their own efforts. For example, students suggested such things as "I didn't help as much as I should have. I didn't plan my time. My parts were not that good," "My [PART] did not look very real. . . I don't know how to make [PART]," or "My [PART] was not as good as [PART]. I didn't think of doing it that way but I know better next time." Most of these comments appeared in the evaluations of the people whose work was very good.

Another focus in the written-evaluation was on group dynamics including such comments as "My group worked very well. We all deserve A's," "Not all people in the group worked hard," "We generally agreed on everything, except what to build it out of, how big to make it, what parts should go in, when to work on it. Oh! Maybe we didn't get along. I don't know." It was interesting that the first two comments came from people in the same group, and the praiseful evaluation coming from a student who, in Clint's assessment, did not work very hard on the project. In fact, she had not even completed the project by its due date. There were some defects in her work, and in the presentation, she could not clearly explain what work she had done. The second comment came from someone in the group who had been involved in most of the construction of the body and had learned how things connected and somewhat, how they worked. The last comment came from someone in a group that did get along well but met regularly to discuss how to proceed. That group had almost formed mini-committees to decide how to proceed at each step.

Three stages of self-evaluation were looked at: on-going concept forming evaluation, written and verbal evaluation at the end of the unit, and post-unit interviews. During the on-going evaluation, students tested ideas before proceeding in their work. This type of evaluation may also be classified as an integral part of problem solving. However, because students appeared to seek affirmation by teachers and peers in evaluating their ideas, this stage of the process was included as part of our description of self-evaluation.

The verbal presentations of the project were designed as a terminating activity where the students get to present their final work but it is also a form of evaluation. As the teacher, Clint could hear students explain in their own word what tasks they completed and about the process. This stage is also a form of self-evaluation for the student as they choose what to tell teachers and peers, and about what to talk. However, their verbal responses did not adequately present students' understandings. Closer analysis of their (videotaped) presentations suggested that there is a strong correlation between their body movement and to where they point which also adds to their verbal presentation. Students' gestures and manipulations of their work provided indications as to their understandings. This analysis received subsequent confirmation during the post-unit interviews, which allowed Clint to probe students' understanding in more detail. Consequently, the analysis of students' presentations provided Clint with an additional, often quite revealing means, for evaluating the projects and understandings. In summary, we can state that
* The written self-evaluations tended to emphasise less of the students' own work other than to negatively criticise it. This form of evaluation tended to focus more on group dynamics and the need to be more organised.
* The post-unit interviews tended to be useful for extra clarification when doubt was left as to the product of the student or their understanding of their work. It did allow for one on one discussion that often showed emphasis on materials that were not readily apparent in the videos or their writing.
Overall, this emancipatory project provided some surprising new insights about student (self-)evaluation. It led to a change in which we perceive and think about evaluation in secondary school science. We therefore recommend that secondary science teachers, if they want to be more effective in evaluating students on open-ended projects, employ a variety of evaluation and self-evaluation opportunities. We believe that any single method of evaluation is not going to represent adequately neither what the student understands nor what they have learned. If only one or two methods of evaluation will be used, guiding questions in the evaluation would have to be well chosen. However, the more guiding the questions, the more the teacher will focus the efforts of the student. In order to provide as open an environment as possible, alternative strategies for self-evaluation such as journalising might be used.

[Michael:] Having conducted research as teacher in my own and my peers classrooms, and having taught research courses for teachers over the past half-dozen years, I know that there are many teachers who think that there are big gaps between their own work and research. Yet this does not have to be. I always emphasise that the most important step for teachers is to frame the questions they have about their work environments, their classrooms, the learning of the children in their care. Furthermore, because of their considerable experiences in school settings, their answers to the questions they ask have a lot to contribute to understanding learning, teaching, and the cultures of classroom, school, and districts more generally. This is no different with the work Clint has done in his own classroom. Most importantly, by doing research, teachers also construct themselves as learners, a critical ingredient for the transformation of school culture.

[Clint:] Overall, I found that this project has made me look at the evaluation I do in a different way. I always thought I was relatively open and observant but there were many things that I just did not consider observing before this. This was my first attempt at this form of observation and analysis. I had some initial difficulties spotting patterns to observe and even when I did, I had a very hard time putting the ideas down on paper. Furthermore, the original question I was intending to pursue was not the one I ended up writing about.

If I were to do this project again, I would try to formulate better questions from the start. I realise that this simply comes with experience. I would also focus on smaller groups rather than an entire class. This might have made going through the tapes much easier. In addition, on this line, I would evaluate the tapes the night they were made if possible. I think that I missed many further opportunities by waiting until I had collected all the information that I needed before proceeding. The main flaw in this thinking was the "all the information I needed" especially since my topic question changed, I could have adapted my information collection accordingly.

Elden, M., & Levin, M. (1991). Cogenerative learning: Bringing participation into action research. In W. F. Whyte (ed.), Participative action research (pp. 127-142). Newbury Park, CA: Sage.
Foucault, M. (1975). Surveiller et punir: Naissance de la prison [Discipline and punish: The birth of the prison]. Paris: Gallimard.
Guba, E., & Lincoln, Y. (1989). Fourth generation evaluation. Beverly Hills, CA: Sage.
Harris, M. (1997). Self-Assessment of Language Learning in Formal Settings. ELT Journal, 51, 12-20.
Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Sciences, 4, 39-103.
McGinn, M. K., & Roth, W.-M. (in press). Assessing students' understandings about levers: better test instruments are not enough. International Journal of Science Education.
Postman, N. (1992). Technopoly: The surrender of culture to technology. New York: Knopf.
Roth, W.-M. (1996). Tests, representations, and power. Journal of Research in Science Teaching, 33, 817-819.
Roth, W.-M. (1998a). Against the grade: Student assessment of learning. In D. J. Tippins &T. R. Koballa (Eds.), The promises and dilemmas of teaching middle and secondary science: A classroom case handbook.
Roth, W.-M. (1998b). Designing communities. Dordrecht, Netherlands: Kluwer Academic Publishing.
Roth, W.-M. (1998c). Science teaching as knowledgeability: a case study of knowing and learning during coteaching . Science Education, 82, 357-377.
Roth, W.-M. (in press). Situated cognition and assessment of competence in science. Evaluation and Program Planning.
Roth, W.-M., & McGinn, M. K. (1998). >unDELETE science education: /lives/work/ voices. Journal of Research in Science Teaching, 35, 399-421.
Stefani, L. A. (1998). Assessing in partnership with learners. Paper presented at the 5th European Electronic Conference on Assessment and Evaluation. [http://www.earli.eu.org/member/sig1confpap.html]
Welzel, M., & Roth, W.-M. (1998). Do interviews really assess students' knowledge? International Journal of Science Education, 20, 25-44.

[1] Suggestions for teaching and learning outcomes for all grade levels in British Columbia's schools are assembled in Integrated Resource Packages (IRPs). IRPs provide the basic information that teachers will require in order to implement the curriculum. These can be obtained electronically at [http://www.bced.gov.bc.ca/irp/].