AUTOMATIC EVALUATION OF PRACTICES IN MOODLE FOR SELF LEARNING IN ENGINEERING

The frst years in engineering degree courses are usually made of large groups with a low teacher-student rato. Overcrowding in classrooms hinders contnuous assessment much needed to promote independent learning. Therefore, there is a need to apply some kind of automatc evaluaton to facilitate the correcton of exercises outside the classroom. We introduce here a frst experience using surveys in Moodle 2.0 in order to get an automatc evaluaton of practces in our Database course. We report survey valuaton of the autonomous learning tool and preliminary statstcs assessing correlaton to an improvement in the practce exam marks.


INTRODUCTION
Database course is a core compulsory subject in year 2 in Computer Engineering at the Universidad Autónoma de Barcelona (UAB) from 6 ECTS.The course introduces basic contents Database (DBMS) management systems and contents are divided into two blocks: DBMS relatonal design and manipulaton of DBMS relatonal using Structure Query Language, SQL (Date 1987).One of the core skills is the ability of holding consultaton of some complexity using SQL in client-server architecture, therefore practcal exercises totalling 2.52 ECTS (41%) are distributed in 0.48 ECTS practce sessions and 2.02 autonomous individual exercises.
In previous years, students had access to an account on the database server, the DB with which they would work throughout the course, and a set of queries in SQL language to do together with the results.In consultaton class tme was spent in group discussion in order to answer any doubts that may arise.This was considered independent student work and an assessment of it was done during a practcal test.
The emergence of new technologies in evaluaton platorms together with the success of the frst experiences in their applicaton (Arenas, Molleda, Chávez, Domingo & Castañeda, 2003) arouse our interest in using them for SQL language learning in Database courses.Besides these kind of technological tools can provide formatve assessment as established by the European Higher Educaton Area (EHEA).Formatve assessments are evaluaton systems that allow the gathering and interpretaton of data about the student progress in order to provide him/her with a feedback on his progress (Rotger, 1990).Recent studies (Zaragoza, Luis Pascual & Manrique, 2009) have shown these systems assessment possessed considerable advantages for optmizing the learning of students: greater involvement and motvaton of the students, performance improvement, both in relaton to learning as regards the fnal ratng.
Under these consideratons, we decided to use our materials of SQL queries with solutons and code available for several already created databases to design and implement tools for their immediate evaluaton.We also decided to implement these evaluaton tools on Cerbero (Cerbero, 2014) which is the Moodle 2.4 platorm we use in our course to manage on-line interacton with students.
Our experience in the development of self-learning and automatc evaluaton tools started the academic year 2012-13, when we considered the possibility of automatng the correcton of practcal tests with the dual purpose of • ease the load of work of teacher's correctors (Martnez & Crespo, 2007) and • give the students a faster feedback on their results.
To this end, students were asked to send their results using a SQL code in a very concrete form through the Moodle program used in the course; this was then automatcally checked by a very simple home-made php script that evaluated the SQL code on our server.
Encouraged by the positve results of the aforementoned experience, we decided to implement autonomous assessment methodologies in the academic year 2013-14 that would provide students with an immediate evaluaton so they can assess their own degree of maturity in performing SQL queries.The objectves of this online tool for systematc evaluaton of SQL queries are: • Students' self-assessment of the SQL queries that provide and immediate result of the year's correctons.
• An efcient assessment of the practcal problems by teachers to provide students with an approximate grade on test day.
As a side result, this tool also allows efcient and quick collecton of learning's evidence that allow signifcantly improvement in individual contnuous assessment of students in large groups.

WORK DESCRIPTION
Manipulaton (queries) over a Data Base (DB) using SQL consttutes 41% of ECTS of the subject, distributed in 6 tutored practce sessions of 2 hours and 51 hours of individual practce without a teacher.In the year 2013-14 the number of registered students was 260 and they were distributed in 10 groups of 28 students each for practcal sessions.Concerning teaching staf, there was a teachers' team of 2 doing theory/problems and 3 teachers supervising the practcal classes.This teaching load made the monitoring of a periodical assessment delivery unfeasible, as the teachers' feedback cannot be as fast as required.This situaton encouraged the development of alternatve evaluaton methodologies providing an immediate feedback to students.
Since 2011, we have used a Moodle platorm for the publicaton of materials, deliverables and surveys; last year that was upgraded to Moodle 2.0 (Cerbero, 2014).Moodle (Cole & Foster, 2007) is used in a variety of insttutons and agencies and the new upgrade includes tools for group work (group formaton, deliveries, surveys, self-assessment and control) that have been used successfully in other subjects (Mart, Rocarias, Gil, Vivet & Julià, 2008).We have created a new module called SQL Queries (SQLQ) based on the survey module already available in Moddle 2.0.This modifcaton of the Moddle survey module incorporates new optons that are available depending of the user's profle: Teacher user: When creatng a new actvity SQLQ the form allows the teacher to introduce a set of SQL queries with their solutons SQL and the due dates for delivery.The output of the teacher SQL query is compared to the output of the student code to validate his/her soluton.The comparison is performed by checking that the contents of tables obtained from executng the teacher and student SQL codes have the same values for each row and column.
The actvity also has an opton to set the number of tmes that a student can validate his/her SQL code for a given query.In the case of repeated atempts we have a self-assessment system, however, if the students fnd only one possible try the actvity becomes a test of automatc correcton.Furthermore, being an actvity derived from the actvity survey's module, it can be confgured so that the teacher can specify the range of IPs accepted.Figure 1 shows the teacher environment for the defniton of a new actvity SQLQ in case of an examinaton.First, the name of the actvity and its descripton must be introduced.Then, set the permited IPs to use the actvity and the test modality by selectng the number of atempts.Finally, introduce the test solutons.

Figure 1. Teacher environment for the defniton of an SQLQ actvity in case of examinaton
For each actvity SQLQ in use, the Moodle module provides the teacher with several tools to analyze the student's responses to each query and allows downloading the results in an Excel fle.The module ofers several statstcs of the rate of student's success and the possibility to compare the student code with the teacher soluton.The statstcal summaries are computed for each student, each query and also for all students and queries to provide with a global indicator of the rate of student's success.For each statstcal summary, the teacher environment shows the percentage of correct and wrong answers and the percentage of queries that have not been answered in case of an examinaton.Figure 2 shows the environment presentng the statstcal summary in case of a query based analysis.On top of the screen there is a bar menu that allows switching to one of the available summaries and below there is the table reportng statstcs.The module also provides the comparison between student and teacher SQL solutons.This facility is especially useful in case that the student requests a revision of his/her results.Figure 3 shows the environment to compare codes for a given student in a 12-query test actvity.The most lef column shows the mark obtained by direct comparison of the output obtained by the two SQL codes: 2.0 if student and teacher tables are equal, 0.0 otherwise.Student user: When a SQLQ actvity is created the student can access it to send SQL queries that he/she has previously tried on his/her own. Figure 4 shows the student environment to introduce his/her solutons to the test queries.On top, due dates are displayed and the botom dialog box is the space where the student can write his/her SQL soluton to one of the queries (indicated by numbers) from a list that has been previously distributed to students.

Figure 4. Student environment to introduce his/her SQL code solving the queries proposed in the test
In case of self-assessment students can try the queries as many tmes as he/she wants and in any order while the actvity remains open.By submitng the form SQLQ the system returns the number of hits and the students have detailed access to their responses to check which queries have been correctly answered.In the case of an examinaton, the student has only a single atempt, and then he/she receives his/her marks immediately.Figure 5 shows the student environment providing feedback for each of the queries he/she has made.The total grade is displayed on the botom of the screen.

RESULTS
The overall assessment by teachers is positve, given the improvement in the grades that a frst statstcal comparison indicates.Figure 6 (lef) shows a bar chart comparing the practces' grades of the year's 2012-13 students (without self-evaluaton) with the current course.The bar chart shows that in the current year there is a lower percentage of students having marks in the range 0-4, and that the percentage of students in the range of 10 is much higher than in the previous year.In conclusion, more students had passed their exams this year and there is also an increase in the percentage of those with higher grades.The average mark for the year's 2012-13 students was 6.0 and 7.5 for current course students.Finally, a t-test for unpaired data (Montgomery, Runger & Hubele, 2009) at signifcance 0.05 detects a signifcant diference between average marks (p-val = 1.1170e-007) with a confdence interval for the diference in means between the course 2012-13 and the current one equal to [-2.0464, -0.9564].

Figure 6. Results of student assessment
To beter assess the impact of self-assessment module in fnal practce, in Figure 6 (right) we compared the module's exam grade with the exams in a fow chart.We have grouped the self-assessment grade: not submited, -1 suspense, [0,5) pass, [5,7), remarkable, [7,9) and excellent, [9,10].In the case of the practces' exam, the value -1 in the boxes indicates students who did not atend it.The graph shows a qualitatve visual dependency between both grades with an increased tendency in the averaged exam's grades.The number of failed students is the same than those who had failed or got a close pass in the practces' exam.Also, students with good grades in the self-assessment module have obtained the highest grades in the practces' exam, with the excepton of the lower percentle of 25% in the excellent group.We atributed these exceptons to three main factors.Possible cheatng or copying among students in the self-learning module, errors due to exam jiters or inefectve self-learning tool.However, the total number of these exceptonal cases is 20 students, which represents only 7.6% of the total of 262 students enrolled in the course and 7.9% of the 251 submited.Therefore, given the mean and percentles group of excellent notes, we can draw two main conclusions.The frst conclusion is that the tool has benefted the vast majority of students who have used it.This follows from the fact that most of the best marks obtained in the examinaton of practces corresponds to those students who have used the self-assessment tool more intensively.The second is that most students have legitmately used this tool and is an indicaton of the degree of acceptance thereof.
The acceptance and valuaton of the module among students has been assessed by an anonymous opinion survey.This survey is a Moddle module that we have implemented using the available Moodle survey module to allow anonymizaton among other facilites.We report global statstcs for the general valuaton of the selfassessment tool and the practcal classes in the bar plots shown in Figure 7.The results obtained from the survey valuaton show that most students value positvely the autonomous learning tool with a 7 on average.There is, however, a signifcant percentage of students that have a neutral valuaton (5) of the tool.We atribute this neutral percepton to the fact that this is the frst year that the actvity has been proposed and, consequently, during the frst weeks the actvity had to work in prove mode and students were requested to report any incidence.Errors reported by students helped in improving the module and develop a stable version that was the one used to do the self-assessment actvity.Stll the percepton of the practcal actvites was very positve (see bar right plot in Figure 7) with a clear average of 8 points out of 10.In order to make a more precise analysis of this module on the student's results, the relatve weight of the correct number of queries in the self-assessment module in the fnal grade must be taken into account.As mentoned earlier in this work, in previous years we did not have any tools that would allow us to assess in an individual and contnuous way the progress made by the students in mastering the SQL syntax and, therefore, had no way to assess directly the autonomous work done by them.Thanks to this module, while students receive an immediate return on their progress, teachers are able to collect evidence that can be used to mark them.That has allowed us to allocate 10% of the fnal grade for this item, which accounts for 25% of the 2.52 ETs assigned to practcal exercises.We think that this weight has defnitely encouraged the students to use the self-assessment module.

CONCLUSIONS
We present in this paper a frst experience using surveys in Moodle 2.0 in order to get an automatc evaluaton of practces in our Database course.A frst valuaton of this experience allows us to notce, in one hand, a relaton between an improvement in the practce exam marks and the use of the self-learning module, and, in the other hand, a decreasing workload for the teachers.Despite all the positve aspects of this frst experience we have observed some limitatons that could be improved in the future.On the one hand, it has been implemented using the Moodle survey tool, which limits the feedback we can ofer to students whether the query is successful or not.With the current module we have to prepare an actvity for each group of practces, which share the same array of questons.With the module questonnaire we could have generated a single actvity where each student could have had diferent random questons to answer from a set of questons grouped by difculty and type.On the other hand, if we had used the questonnaire module we could have used its full potental to (a) provide beter returns to students according to the failed response and (b) facilitate the generaton of practcal exams.
The current SQLQ module is based on Moddle, which only provide a limited feedback to students.Afer our frst positve experience using this module, our teaching team is developing an actvity version based on Moodle questonnaires.Being based on questonnaires, this new module will manage actvites, groups and feedback given to the student.It will also allow to incorporate generic models of correctons (done in php, for instance) of other types of programming practces, such as C programming or image processing, which verifcaton could be based in the quality of output produced.We will make this new module based on Moodle questonnaires public afer fully evaluatng its performance as autonomous learning tool and its acceptance among students.This autonomous learning module will allow us to add small progress evaluatons at the end of each practce session.This year we have used them to test the module before using it in evaluaton actvites.Once evaluated, it can be used to rate the progress of students in each practce session.
Finally, although the preparaton of the set of solved queries requires some tme and efort in order to get a reliable and complete set, the frst impression among all teaching staf is that such efort is worth the tme saved by professor in comparison to the conventonal system requiring manual correctons.

Figure 2 .
Figure 2. Teacher environment to analyse student queries: general statstcs

Figure 5 .
Figure 5. Student environment to analyze the queries made