TO TEACH AND ASSESS MODEL-BASED REACTION AND EQUATION KNOWLEDGE

This study investgated the challenges students face when learning chemical reactons in a frst-year chemistry course and the efectveness of a curriculum and sofware implementaton that was used to teach and assess student understanding of chemical reactons and equatons. This study took place over a two year period in a public suburban high-school, in southwestern USA. Two advanced placement (AP) chemistry classes partcipated, referred to here as study group A (year 1), N = 14; and study group B (year 2), N = 21. The curriculum for a frst-year chemistry course (group A) was revised to include instructon on reacton-types. The second year of the study involved the creaton and implementaton of a sofware soluton which promoted mastery learning of reacton-types. Students in both groups benefted from the reacton-type curriculum and achieved profciency in chemical reactons and equatons. The fndings suggest there was an added learning beneft to using the reacton-type sofware soluton. This study also found that reacton knowledge was a moderate to strong predictor of chemistry achievement. Based on regression analysis, reacton knowledge signifcantly predicted chemistry achievement for both groups.


INTRODUCTION
Chemical reactons and the equatons which describe them have long been one of the keystones of chemistry.Our understanding of them has largely been associated with the very laboratory setngs in which they were discovered.Consequently, their signifcance to the laboratory has made it so that treatment of chemical reactons in frst-year chemistry courses has historically been piecemeal (Cassen & DuBois, 1982).In fact, most frst-year texts typically devote litle space to chemical reactons and the equatons which describe them (Hesse & Anderson, 1992).There is a general assumpton, that chemical reactons can be taught throughout the frstyear on an as needed basis, and that reactons are somewhat solitary and unrelated throughout the frst-year (Cassen & DuBois, 1982).The problem with this approach is that the student's terminology of reactons and equatons may be limited to sparse examples, which may hinder the student's ability to conceptualize other chemistry concepts (Ragsdale & Zipp, 1992).For instance, concepts encountered in thermochemistry, electrochemistry and chemical equilibrium all depend on knowledge of chemical reactons and equatons.
The learning of chemical reactons and equatons requires knowledge an understanding of a variety of facts about chemical propertes of substances.It requires chemical knowledge which is knowledge about the resultant diferent substances and propertes typifed in a chemical change (Piaget & Inhelder, 1941).It requires conservaton reasoning, the knowledge of how mass is conserved in a chemical reacton (Hesse & Anderson, 1992).It also requires theoretcal knowledge, like that of atomic molecular theory, and partcle theory (Fazio, Bataglia & Guastella, 2012;Hesse & Anderson, 1992;Jaber & BouJaoude, 2012;Treagust, Chitleborough & Mamiala, 2003).In order to understand chemical change, one must view a substance as an "entty" which: • can change between three states; • can come into and go out of existence; and • can be identfed by its propertes (Johnson, 2002).
And this view must occur at three diferent levels of representaton: macroscopic (experiments and experiences); sub-microscopic (e.g., electrons, molecules, atoms -the partculate nature of mater); and symbolic (e.g., ball & stck models, structural formula, empirical formula, computer models, chemical equatons) (Hesse & Anderson, 1992;Jaber & BouJaoude, 2012;Treagust et al., 2003).All three levels of representaton are integral in developing an understanding of the chemistry concepts under investgaton (Treagust et al., 2003).For example, the experienced chemist will understand chemical change in terms of three levels of representaton, while the beginner will be limited to a single representaton (Hesse & Anderson, 1992;Kozma, Chin, Russell & Marx, 2000;Treagust et al., 2003).The sub-micro level being the most difcult (Wheeldon, Atkinson, Dawes & Levinson, 2012).This is largely a functon of experience, or lack thereof, with chemical change.As the student's experience with chemical change progresses, the student will likely gain capacity to operate between the macro, sub-micro and symbolic representatons (Jaber & BouJaoude, 2012;Treagust et al., 2003).Although, early on, it will likely be in a discrete, compartmentalized and inconsistent fashion -what has been called instrumental understanding (Jaber & BouJaoude, 2012;Treagust et al., 2003).On the other hand, the experienced chemist will be able to form multple representatons easily and in conjuncton with one another.The ability of learners to shif their representatons and reasoning is what has been referred to as emergent process schema (Chi, 2005;Jaber & BouJaoude, 2012), and what has also been termed relatonal understanding (Jaber & BouJaoude, 2012;Treagust et al., 2003), conceptual understanding (Pyat & Sims, 2012), holistc understanding (Wheeldon et al., 2012); and model-based understanding (Treagust et al., 2003).Teachers ofen assume that students can easily transfer from one level to another, when in fact this is not always the case (Robinson, 2003;Treagust et al., 2003).
Just as multple representatons are important to understanding chemical change, multple means of explanaton are also important.The ability of the student to explain chemical change phenomenon, explanatory knowledge (Treagust et al., 2003), is another import area to consider when determining how to efectvely teach chemical reactons.Beginners will typically have ambiguous language and will rely on surface features to classify observatons and subsequent representatons, whereas experts employ an underlying and meaningful basis for their categorizaton (Bond, 1989;Kozma et al., 2000).Because of the emergent schema process, students need an understanding of what consttutes an acceptable explanaton in chemistry (Hesse & Anderson, 1992).For instance, as the student gets command of partcle theory, he/she will be able to explain some of the discrepant events which may have been encountered in studying chemical change.This means that teachers' explanatons must be compatble with students' explanaton knowledge, or student-centered (Treagust et al., 2003).This requires the teacher to communicate and explain abstract and complex chemical concepts and the students' ability to understand the explanatons (Treagust et al., 2003).This can be challenging because, as Stavridou & Solomonidou (1998) showed, the progression the learner makes may be quite diferent from the progression expected in the curriculum, which has historically given litle atenton to the appropriate treatment and sequence of chemical reactons (Hesse & Anderson, 1992).For instance, students may utlize one of several types of explanatons to reconcile their understanding of chemical change: • analogical -a familiar phenomenon or experience is used to explain the unfamiliar; • anthropomorphic -a phenomenon is given human characteristcs to make it more familiar; • relatonal -an explanaton that is relevant to personal experience; • problem-based -an explanaton demonstrated through the solving of a problem; and • model-based -using a scientfc model to explain a phenomenon (Treagust et al., 2003).
The pedagogical implicaton of this is that students will utlize explanaton types with which they are most familiar and support their existng lexicon.Further, the teacher's explanatons of chemical change must take into account the terminology the student possesses to explain chemical change.The learning experiences for the students should encourage development of precise vocabulary from direct experience with demonstratons and lab actvites that involve chemical change and probing questons (Bond, 1989;Pyat, 2013a).The role of the teacher is to create experiences which help students develop the necessity for a well-defned and precise vocabulary (Bond, 1989).
Therefore, regarding the teaching and learning of chemical reactons and equatons, the following theoretcal underpinnings were identfed for this study: • knowledge of chemical reactons and equatons is difcult to acquire and retain; • instructon in such content is largely lacking from most frst-year chemistry classes and texts; and • knowledge of chemical reactons and equatons predicates understanding of other chemistry concepts.
The focus of this study was grounded on these underpinnings.

Objectves
• What challenges do students face when learning chemical reactons?
• What are efectve ways reacton knowledge should be taught and assessed?
• How can students achieve mastery of reacton knowledge?
• In what ways might reacton knowledge be related to chemistry achievement?

METHODOLOGY AND EMPIRICAL APPLICATION
This study took place over a two year period in a public suburban high-school, in southwestern USA.Two advanced placement (AP) chemistry classes partcipated, referred to here as study group A (year 1), N = 14; and study group B (year 2), N = 21.The instructor of record was the same for both years and was an experienced chemistry teacher.The course content of the AP courses was prescribed by the College Board and was equivalent to frst-year college chemistry, in the curriculum taught and the laboratory investgatons (CollegeBoard, 2010).The curriculum for a frst-year chemistry course (group A) was revised to include instructon on reacton-types (Cassen & DuBois, 1982).The second year of the study involved the creaton and implementaton of a sofware soluton which promoted mastery learning of reacton-types.

Procedures (year 1)
As the literature revealed, beginning students may have difculty recognizing chemical reactons and equatons in a categorical manner (Bond, 1989;Kozma et al., 2000).This can lead to an oversimplifed and shallow understanding of chemical change.Therefore, a curriculum was created which focused on reacton-types as a framework to help students categorize the chemical reactons and equatons which describe them.It included the following reacton-types: • combinaton; • decompositon; • single-replacement; • double-replacement; • oxygen reactons; • water reactons; • acid base; • complex ion; and • oxidaton/reducton.The reacton-type curriculum was implemented during year 1 with group A. This was the control group.Each week, one to two reacton-types would be presented to students, in conjuncton with a demonstraton of the representatve reacton(s) (Gray, 2009;Herr & Cunningham, 1999;Shakkashiri, 1983Shakkashiri, , 1985Shakkashiri, , 1989)).The reactontype presentatons gave students opportunites to observe chemical reactons, and helped them refect on the three levels of visualizaton: macro; sub-micro; and symbolic (Jaber & BouJaoude, 2012;Robinson, 2003).This approach was consistent with the recommendatons found in the literature (Cassen & DuBois, 1982;Hesse & Anderson, 1992;Pyat & Sims, 2012;Ragsdale & Zipp, 1992;Stavridou & Solomonidou, 1998).Sample practce problems were also provided students, in similar fashion to what was described by (Bond, 1989).The presentatons took approximately 15 minutes per week and ran for 16-weeks for each two semesters.Students reviewed and practced the reacton-types which were introduced that week in preparaton for a reacton quiz which was given at the end of each week.Students also logged the tme they spent studying reacton-types.

Data Collecton Instruments
Students' symbolic understanding of reactons and equatons was measured in the form of free-response questons, where students predicted the products for a chemical reacton where only the writen form of the reactants was given (CollegeBoard, 1999).For example, students would be given the word equaton for the reactants of a given chemical reacton (i.e., Magnesium metal is heated in air).Students would then write the chemical equaton describing this process: Mg (s) + O 2(g) ---> MgO (s) .This format was congruent with the reacton queston on the AP chemistry exam (CollegeBoard, 1999).Measuring symbolic understanding of chemical reacton and equaton knowledge in this way has been an established approach which has been used for many years previous to this study (CollegeBoard, 1999;Ragsdale & Zipp, 1992).Chemical reacton and equaton knowledge was measured weekly in the form of tmed free-response quizzes.This went as follows.At the end of each week, students were given 10-minutes to complete an eight-item reacton quiz, where they would predict the products for a chemical reacton, given the reactants.Students could retake the quiz on a one-tme basis.These quizzes were considered formatve because they were designed to gauge student profciency of chemical reactons and equatons in a way that allowed on-going revision and reevaluaton.This approach was consistent with the recommendaton that students need to frequently confront their conceptual understanding of chemical change, in a manner that allows for refecton, revision and revaluaton (Stavridou & Solomonidou, 1998;Treagust et al., 2003;Wheeldon et al., 2012).In this case, the focus was on symbolic understanding of equatons and formulas.
Reacton-types instructon took place for 16 weeks during the frst semester of year 1.During second semester, students contnued practcing reacton-types and were assessed on reacton knowledge, weekly.Students logged the amount of tme each week spent studying chemical reactons and equatons.An open-ended survey was given to students at the end of each semester to gauge student's perceptons and attudes towards the reacton-types instructon.Chemical reacton and equaton knowledge was measured, along with chemistry achievement, with an end-of year summatve exam.The exam chosen was the 1999 released AP Chemistry exam (CollegeBoard, 1999).This exam was part of the normal curriculum where the study took place.The exam consisted of two ninety-minute sectons: • multple choice and • free-response.
The chemistry content of the exam was equivalent to a typical frst-year chemistry course (CollegeBoard, 1999).One queston in partcular, free-response queston 4 (FRQ4), referred to in this study as the reacton queston, assessed symbolic understanding of chemical reactons and equatons (CollegeBoard, 1999).Questons were scored based on the scoring guidelines described in the transcripts of the released exam.Scores on FRQ4 were also compared to the overall score on the exam.This was done to see whether chemical reacton knowledge was a predictor of overall chemistry achievement.

Procedures (year 2)
For year 2 of the study, a sofware soluton was created and implemented which provided instructon and assessment on reacton-types for group B. This sofware was designed as a formatve assessment tool for students to practce and assess knowledge of chemical reactons and equatons, which was measured weekly with the reacton-type sofware, for a period of two semesters.A summatve assessment was given at the end of second semester (just as done with group A) to measure chemical reactons and equaton knowledge, and chemistry achievement.Mean scores on the reacton queston were compared to group A. A t Test was carried out to test for performance diferences between groups.This approach was consistent with other reported studies(Rejón-Guardia, Sánchez-Fernández & Muñoz-Leiva, 2013; Salas-Morera, Arauzo-Azofra & García-Hernández, 2012).Regression analysis, much like what was described in (Kallas & Ornat, 2012), was also carried out to see if chemical reacton knowledge could predict overall chemistry achievement.All of which are presented in the results secton.

Design Concept -Reacton-type sofware soluton
Based on the results from year 1, it was determined that further modifcatons to the reacton-type curriculum were necessary.Specifcally, an assessment tool was needed which would measure students' knowledge of chemical reactons and equatons, and allow students to practce, test and retake, if necessary; and provide such opportunites outside of class.This goal was consistent with the fndings in the literature, that students should be encouraged to recognize their existng understandings, while at the same tme, allowing for reorganizaton, extension and abandonment of existng categories (Stavridou & Solomonidou, 1998).A sofware program was therefore desired which taught and assessed chemical reactons and equatons in a manner that emphasized symbolic understanding of chemical reactons and equatons (i.e., where students were given reactants and were asked to predict products for reacton-types).It was postulated that such a program might assist students in the progression of their knowledge of reactons, equatons.While there were applicatons available which taught chemical reactons, none provided instructon on the reacton and equaton content in the context that the students needed (e.g., tutorial, customizability regarding reacton-types, drill/practce, and testng format similar to end-of-year exam).Furthermore, many of the available applicatons were cost prohibitve to students, or were ad-driven with distractng pop-up windows.Therefore, an open sofware soluton was created which had practce, mastery and assessment components for chemical reacton-types.While the focus of this paper was on the use of this sofware, and not on sofware design or instructonal design, it should be noted that a rapid-prototype-design process was followed (Hannafn, Land & Oliver, 1999;Reigeluth & Carr-Chellman, 2009).A summary of the design, development and implementaton for the sofware soluton Reacton Master is described below.

Tutorial
The sofware soluton Reacton Master (Pyat, 2002(Pyat, , 2013b) ) was designed to instruct students on reactons, specifcally, categorical and symbolic representaton of nine reacton-types typically encountered in frst-year chemistry.The sofware was made available online, and could be accessed through a web browser.The opening screen for the sofware is shown here (Fig 1).From this screen students access the tutorial, practce, or test screens.The tutorial feature allowed students to select which of nine reacton types they wanted to study (Fig 2).

Figure 1. Reacton Master Opening Screen
The opening screen shown here is where students will begin their tutorial, practce or test.

Practce-utlity
Students practce their reacton/equaton knowledge with the practce utlity.This utlity uses a random equaton generator (there are over 1500 possible reactons from which the system calls) that displays a word equaton, along with formulas for reactants and products.

Figure 3. Practce Reactons Screen
Shown here is a practce reactons screen.The top feld is where word equatons appear.The boxes below are for equaton inputs for reactants or products.The user dictates which reacton type(s) to practce, and can choose the HIDE opton(s) to hide/show reactants, products, and word products.There is also a HINT buton that displays pertnent informaton about the current reacton.The CHECK functon enables the user to check the whether or not the response is correct.

Test-utlity
Once students are familiar with a given reacton-type, they test their knowledge with the test-utlity.Students select reacton-type(s), select a tme (i.e., 10 min) and begin their test.The random-reacton-generator builds a 5-item assessment based on the reacton-types selected.Students then input their responses ( Fig

Implementaton
The reacton-type sofware soluton, Reacton Master, was implemented at the beginning of year 2, with group B. This was the experimental group.The procedures described in year 1 were followed.The only diference was that students in group B were provided access to the reacton sofware at the beginning of frst semester.Students took weekly reacton quizzes outside of class which were administered and scored by the reacton sofware.Students submited their reacton-types quizzes at the end of each week to their instructor for recording.This routne took place for 16 weeks in the fall semester and again for 16 weeks throughout the second semester.Students logged tme spent each week studying reactons and equatons.An open-ended survey was given to students at the end of each semester to gauge student's perceptons and attudes towards reacton-types instructon.At the end of the second semester, students from group B were given a summatve exam -the same exam given to group A the previous year.

RESULTS
The performance data on chemical reacton knowledge that were gathered over a two year period are shown below.The data were gathered from formatve and summatve assessments, as well as student practce-tme logs.These data are reported in Table 1.Table 1.Performance data for group A and group B

Hypothesis Testng (t Tests)
To investgate whether diferences existed in reacton and equaton knowledge between group A and group B, a series of t Tests were conducted.T Tests were also conducted to determine diferences in chemistry achievement between groups, as well as weekly practce tme.The following assumptons were tested and met: • groups were similar in size; • the variances of the two populatons were equal; • observatons were independent; and • the dependent variable was approximately normally distributed.

Limitatons
Because of the relatvely small sample size of each populaton, there could be validity concerns in terms of variance.However, these concerns should be eliminated so long as the following assumptons are true (Leech, Barret & Morgan, 2008).For t Tests: • groups were similar in size; • the variances of the two populatons were similar; • observatons were independent; and • the dependent variable was approximately normally distributed.
For regression analysis, assumptons of linearity and normal distributons were checked and met.Another limitaton of this study is in the generalizability of the fndings based on the relatvely small sample size.

Is there a diference between group A and group B reacton knowledge?
A t Test was conducted for this sample to determine whether signifcant diferences existed between mean scores on the fnal reacton queston, FRQ4, for group A and group B. There was a statstcally signifcant diference between group A and group B in reacton knowledge, t(33) = 3.02, p = 0.0049, SE = 1.113.Group A (M = 4.36, SD = 2.68) scored lower than group B (M = 7.71, SD = 3.54).The confdence interval for the diference between the means was 5.62 to 1.09.A t Test was also conducted for this sample to determine whether diferences existed between the mean weekly reacton scores for group A and group B. There was no statstcally signifcant diference between group A and group B in weekly reacton quiz scores, t(36) = 1.51, p = 0.1410, SE = 0.427.Group A (M = 13.7,SD = 1.24) scored similarly to group B (M = 14.38,SD= 1.39).The confdence interval for the diference between the means was 1.507 to 0.223.

Is there a diference between group A and group B weekly practce tme?
A t Test was conducted to determine whether signifcant diferences existed between the mean weekly practce tmes for group A and group B. There was no signifcant diference between group A and group B means for weekly practce tme, t(33) = 0.8465, p = 0.4029, SE = 0.311.Group A (M = 1.68,SD = 0.89) had similar tmes to group B (M = 1.95,SD = 1.03).The confdence interval for the diference between the means was -0.89 to 0.37.

Is there a diference between group A and B in overall chemistry achievement?
To investgate whether diferences existed in chemistry achievement between group A and group B, a t Test was computed.There was a statstcally signifcant diference between group A and group B in overall chemistry achievement, t(33) = 3.52, p = 0.0013, SE = 8.065.Group A (M = 37.6, SD = 9.03) scored lower than group B (M = 66.1, SD = 16.6).The confdence interval for the diference between the means was 44.82 to 12.00.

Regression
Simple linear regression was computed to investgate whether reacton knowledge predicted chemistry achievement.This was carried out for group A and group B. Assumptons of linearity and normal distributons were checked and met.Reacton knowledge for group A (I = 4.36, SD = 2.68) signifcantly predicted chemistry achievement (M = 37.6, SD = 9.0), F(1,13) = 0.4798, p< .001,adjusted R2 = 0.42, as shown in Figure 6.According to Cohen (1988) this is a moderate relatonship.
Figure 6.Regression Analysis for group A Reacton Score vs. Exam Score 51%.By comparison, the mean score for group A on reacton knowledge was lower than the natonal average 6.36/15 or 42%, while the mean score for group B was much higher (CollegeBoard, 1999).This patern likely transcended chemistry achievement as well.For instance chemistry achievement for group A was (M = 37.6, SD = 9.0), and for group B (M = 66.1, SD = 16.6).These fndings suggest, even though reacton profciency between groups was similar, the mastery of this knowledge was not.Given that the only diferences between group A and group B was the use of the reacton-type sofware, these fndings suggest an added learning beneft to using this sofware soluton.This study also found that reacton knowledge was a moderate to strong predictor of chemistry achievement.Based on regression analysis, reacton knowledge signifcantly predicted chemistry achievement for both groups.For group A it was a moderate predictor (i.e., R2 = 0.42) and for group B it was a strong predictor (i.e., R2 = 0.56).These data confrm what has been reported in the literature regarding the importance of reacton knowledge as a foundatonal concept in the chemistry classroom (Usak, Ozden & Eilks, 2011).

CONCLUSION
A likely reason for these performance diferences rests in the possibility that the sofware successfully supported learners in forming a model-based understanding of chemical reactons and formulas (Stavridou & Solomonidou, 1998).In this case the conceptual model was reacton-types, and understanding chemical change from this perspectve may have helped students predict products for a given reacton.This may have promoted mastery of chemical reactons and equatons and, consequently, chemistry achievement.This supports the noton that model-based understanding is related to the student's ability and experience with gathering and interpretaton of relevant informaton about chemical phenomena (Pyat & Sims, 2012).Further, without many examples of reacton-types, students will have limited ability to predict products for chemical processes (Cassen & DuBois, 1982).In such instances, students may have litle grasp of reacton knowledge and may have only an instrumental understanding (Jaber & BouJaoude, 2012;Treagust et al., 2003) where they understand how to write chemical formulas they have memorized for word equatons, but will not be able to predict products.While memorizing formulas is an important element of overall reacton knowledge, it is not a model-based understanding.Therefore, based on this analysis, fewer students in group A reached model-based understanding of reactons, while more students in group B formed model-based understanding through support of the sofware soluton.The results of this study have also revealed that there is a link between reacton knowledge and chemistry achievement.As was proposed earlier, chemical reactons are a keystone of chemistry, and therefore represent a foundatonal concept that transcends many other topics encountered throughout a typical frst-year curriculum.For these reasons, a recommendaton is made here that frst-year chemistry courses should include reacton-types as an integral curriculum component that promotes student understanding of reacton and equaton knowledge and consequently promotes overall chemistry achievement.

Figure 2 .
Figure 2. Tutorial Example 4).Once students have completed their entries, they select to have their entries scored.Their entres are scored and students are shown which entries were correct (Fig 5).

Figure 4 .
Figure 4. Test Screen with Example Inputs

Figure 5 .
Figure 5. Test Screen with Completed Evaluaton Note: a N = 14; b N = 21.The data included here were derived from: (1) mean formatve reacton scores; (2) mean weekly practce tmes; (3) mean reacton queston scores; and (4) mean chemistry achievement scores for group A and group B. The maximum score for the weekly reacton quiz and the reacton queston was 15.