PBL ON LINE: A PROPOSAL FOR THE ORGANIZATION, PART-TIME MONITORING AND ASSESSMENT OF PBL GROUP ACTIVITIES

This report presents the organisaton of PBL (Project Based Learning) for a subject included in the IT engineering degree course. It is the result of 10 years of experience of the implantaton and contnuous improvement of the PBL class structure. The latest innovatons include the experience of part-tme monitoring with PBL groups using the Open Meetngs tool in Moodle 2.0, the adopton of actvites that improve learning and interdependence such as the jigsaw classroom, the clear defniton of deliverables that students should present along the semester and the assessment criteria, both on groups and individuals. As a result of this experience, we present PBL student enrolment indexes, student assessment surveys and lecturers’ opinions. We conclude with some topics for discussion about the PBL methodology.


INTRODUCTION
In the last 25 years, informaton technologies, Internet and mobile devices, among others, have revolutonised teaching methodologies in every feld, and especially at universites.The fact that informaton is easily accessible to everyone for free has modifed the role of the university lecturer as the only channel for accessing knowledge.It is no longer essental to atend lectures in order to acquire knowledge.MOOC's (Massive Open On Line Courses) are startng to ofer informaton on the web that can be acquired whenever the students likes, not only during lecture hours, meaning they can decide when and how to learn, thus confguring the 2.0 student.
Meanwhile, in their job ofers, companies are mainly demanding transversal skills and apttudes (initatve, teamwork, leadership, etc.) rather than specifc knowledge (which they also require), given that the later can be provided within the company.
The above is motvatng and compelling universites to reassess their teaching methodologies in order to adapt to these changes, scheduling fewer lectures and more actvites that foster skills and attudes.This is also a new motvaton for lecturers, as they do not have to repeat the same material over and over again each year.areas of knowledge and at diferent universites, some of them using this methodology for all degree courses, as is the case of the University of Aalborg.
Our experience of PBL started 10 years ago with two important circumstances.The frst was the visit to our university by Dr Luis Branda of McMaster University (Branda, 2009), one of the pioneers behind this methodology.The second was how the lecturers started notcing a certain weariness with the lecture methodology in the fnal years of computer engineering at the UAB, which led to repeated absenteeism among students from such classes.But problem-based and practcal sessions, with greater added value from the lecturer, were well atended.
In this report, we present the way we currently structure a PBL subject, the result of the developments and improvements of the last 10 years on the basis of lecturers' experiences and the opinions obtained from student surveys.The methodology used to validate the proposal is based on feedback from students and lecturers' perceptons of the process.
The report is structured as follows: in secton 2 we present the context of the subject (degree, credits, content).In secton 3 we explain how we currently organise PBL, justfying our proposal as much as possible.One of the latest ideas to be used is online or ofine monitoring of group meetngs using the OpenMeetngs (OpenMeetngs, 2014) tool, which is also used for remote tutorials.We highlight the deliverables that we request from students and the assessment criteria.The conclusions in secton 4 present objectve data, such as enrolment fgures and the number of students that choose the PBL itnerary, as well as student assessments of their learning and the work they have done.We end by commentng on the opinions of lecturers and tutors and ofering some topics for discussion in the feld of PBL.

OUR SUBJECT
Computer Graphics 2 is an optonal subject in the fourth year of IT Engineering, with content covering 3D graphics, including transformatons (geometric and display), the modelling of 3D objects, realism (lightng, textures, shadow, colour) and computer animaton.There is an extensive bibliography on the subject and the chosen reference in our case is (Hearn & Baker, 2003) because it covers the subject using the Open GL graphic library, which is used in practcal work.The subject is worth 6 ECTS, and throughout the semester includes 2 classroom hours a week on theory and 1 one on problems, as well as 6 practcal sessions lastng 2.5 hours each.
From the 2004-05 academic year we have implanted a PBL system, which is made compatble with the classic itnerary of theory, problem-based and practcal sessions, and the students could choose one of the two itneraries.The idea was to use the 2 hour theory sessions for PBL tutoring and dedicate the problem-based sessions to explain theory maters and exercises, providing the theory notes and problem exercises at the start of the course and giving to students who chose the classic itnerary more autonomy in their studies.All of the students did the practcal sessions, some to pass (classic itnerary), the others (PBL itnerary)to learn techniques to use in their projects.Success and satsfacton with the PBL itnerary led the number of students choosing this itnerary to increase each year.
In PBL, the project forms the basis for learning.Each project involves certain learning objectves that the lecturer defnes and hopes that the students will discover and work on.
The instructons for the projects are short and aim for the students to assume a professional role (they are asked or 'commissioned' to develop an applicaton) and they have considerable freedom of decision regarding the objectves, through the use of such expressions as "the most realistc display possible" or "the most realistc movements possible" so that it is they who set the limits.
The experience of applying PBL (Mart et al., 2009) has been positve, and some excellent work has been produced, but we have thought about ways of improving the method.These deliberatons have led us to the following conclusions: • To give them more freedom and initatve in their learning we dedicate all monitoring sessions to getng the students to progress with their projects, without organising actvites that are more aimed at fostering learning, which we also believe to be important.
• The work groups atend a face-to-face session every 2 weeks, so in intermediate weeks we did not monitor their work.
In this report, we wish to detail the progress made in recent academic years (2010-11 and 2013-14) regarding each of the three aforementoned conclusions: defniton of project deliverables, use of OpenMeetngs to hold online PBL group meetngs and beter defniton of the assessment criteria.

OUR PBL APPROACH
The students form groups of 4 or 5 people.Teachers don't decide the group members, they form the groups, assuming some responsibility in case there was any problem in the in the operaton of the group.Each group chooses a weekly tmetable (G1 or G2) for atending class and they are tutored every 15 days.
In the frst session, the lecturer proposes 3 projects, and the group chooses one.The projects ask the groups to produce a graphic applicaton that would be useful in a specifc area for displaying graphic informaton and, if possible, analysing it numerically.Examples we could cite are graphic simulaton applicatons (zebra crossings and cars, arrival and departure of aeroplanes at airports, driving or fight simulators, Formula 1 races, representaton of the planets and satellites of the Solar System),graphic representatons of mathematcal functons (2D and 3D fractals, mesh deformatons), games (pool, 3D Tetris, chess) or representatons of artculated objects (cranes, fairground rides, virtual characters).
Figure 1 shows images of the work on two projects done by the students.The picture on the lef shows a 3D chess game, where the learning objectves are modelling the scene (board and pieces), a realistc 3D display and the defniton of the movement and capture of the pieces (animaton).The project on the right shows a simulator of motor racing at Montmeló, the track near to Barcelona, where the most important learning objectves are the modelling of the track and cars, views of the race through diferent cameras and realism, including the movement of the cars on the track (animaton).

Figure 1. Examples of PBL work on this course: Chess and motor racing simulator
In this frst meetng, the students decide what project they would like to do and analyse what they know, what they don't know, and what they need to know in order to progress with the project.In recent years, students have taken the initatve and proposed projects (usually 3D video games) and the lecturer analyses whether they are adequate for the course's learning objectves.As long as it is possible, their proposals are accepted in order to encourage motvaton and implicaton in the project.A report of the frst meetng (and all later meetngs) is made detailing the discussion, agreements and tasks assigned to each member of the group, which are reported on in the following session.
Every ffeen days the group holds a two-hour meetng with the tutor.Each tutor usually atends to some fve groups in each two-hour session, in which the group proceeds with its discussions and work.The tutor monitors the group's behaviour, both individually and as a group, and deals with any doubts that they might have, trying to guide them without conditoning either their work or the objectves of the project, as long as they are in line with the proposed learning objectves, as much as possible encouraging new ideas and learning (Sotjcevski & Du, 2009).
In one of the tutored sessions, the jigsaw classroom dynamic is employed, i.e. one member of each group joins an 'expert group' and these are given an artcle on computer graphics to read, understand and discuss.Each student then makes a summary of what they have understood about the artcle and comments on whether what they have learned might be useful for their own project.In a 'jigsaw' session, 5 or 6 expert groups are formed and once the actvity is over they pool the knowledge learned with that of their project colleagues.This actvity provides new ideas for the project and is highly valued by the students.
Throughout the semester, in two specifc periods, the students are asked to specify their objectves, what we call 'controls'.In these, the group defnes clear objectves about what they want to do (type of graphic applicaton, functonalites, etc.), defne the tasks to be done and which member of the group will be responsible for them and also the tme schedules for these tasks, using a Gant chart or other method.The frst control is done 3 or4 weeks afer the project has started, and the second control is done 4 weeks before the fnal presentaton.At the frst, ambitous objectves are required and appraised, the second asks about completed and outstanding tasks and suggests realistcally defned objectves in order for them to be ready in tme for the fnal presentaton.These documents are delivered using the Moodle Cerbero platorm (Cerbero, 2014).
The oral presentaton and defence of the project take place in the fnal session of the semester, at a session that is open to all students at the School.
We can classify the three improvements made in the last few years in the three areas: deliverables, online tutorials and assessment.

Deliverables
Once the groups have been formed and the projects chosen, the students must present the following deliverables (for which we provide electronic templates), which are sent via Cerbero in digital format: • Meetng report: At the end of each group meetng (in the presence or not of the lecturer) the students should draf a report showing how the discussion developed and any project decisions made.
• Jigsaw: In the third week, in order to provide the groups with tools and ideas, a two-hour jigsaw session is held on subjects related with the course.At the end of the session, each student delivers a report.
• Control: In two specifc weeks (the third and tenth out of a total of 13 weeks) the students deliver a control as a group.
• Co and self-assessment: Afer delivering the control, the students answer a co-assessment survey on their fellow group members and a self-assessment survey based on a template of questons.
• Monitoring report: Following delivery of each control, the lecturer sends the groups a project monitoring report, which presents his/her assessment of the work at this stage.
• Oral presentaton and delivery of the project: In the fnal class, a student from each group has 15 minutes to present their project to three lecturers, two of which are from outside of the course, in a similar manner to the presentaton of a degree project.Each group hands in the following documentaton: writen report, slides from the oral presentaton and the computer applicaton.
The meetng report is intended for the students to learn to summarise the progress made with their project and to reach agreements on the organisaton of their work.In 2012-13, we decided to include a jigsaw session, which provides them with new ideas for their project and also enables beter intercommunicaton between the groups.The control deliverable enables the students to specify ideas, objectves and tasks, and the tutor to assess the level of organizaton, ambiton and innovaton in the group's proposals for the project, while weightng this against the number of students, given that the objectves of a group of 4 students cannot be viewed in the same way as those of a group of 6.
To monitor how the group operates on an internal level, outside of tutorial sessions, they are asked for three co and self-assessments over three separate periods of the semester in order to fnd out how the group is working.
In most cases, these surveys may not be reliable, but in confictve cases that have arisen, these co-assessments have raised warnings of the problems occurring in these groups.These assessments also foster a critcal sense and raise demands among the students.
The monitoring reports made by the lecturer are designed to provide feedback to the students about their learning and the quality of the group's work, trying to have as litle infuence as possible on their initatve with regard to the development of the project and their learning.This survey involves textual answers with test optons, without evaluable numeric assessments.

Online tutorials and PBL Meetng Rooms
The organizaton of the course and the number of students means that each group atends tutorial sessions every 15 days, which we did not feel gave us enough tme to properly monitor the work they were doings.However, the heavy teaching requirements on this course meant that we were unable to provide any more face-to-face classes, so we recurred to new technologies.
OpenMeetngs is a free sofware tool that enables web-based online communicaton.Using a camera and microphone, online student-lecturer or student-student communicatons can be established in meetngs between two or more people at the same tme.It also allows documents to be shared on a virtual blackboard (Figure 2), where we can add handwriten notes and record communicatons or meetngs.We have installed the sofware in our new Moodle server, which means we can defne communicaton actvites as part of a Moodle course.OpenMeetngs works in two modes: meetng with fxed moderator (who controls the board) or meetng in which everyone is a moderator, regardless of when they connect.

Figure 2. Screenshot from on-line tutorial on OpenMeetngs
In the 2012-13 academic year, we used OpenMeetngs for the following actvites: • Virtual tutorials with lecturer: During the lecturer's tutorial periods with the students, he/she opens the OpenMeetngs tutorial actvity, where any student can connect in order to resolve doubts.
• PBL Meetng Rooms: For each PBL group, an OpenMeetngs session was opened for the holding of online work sessions if the students could not meet up, or in the week with no atendance of class.A video recording of the meetng is generated, which the lecturer is able to watch and assess (Figure3).
The results of the experience were positve but not spectacular.The students did not use the group tool as much as was hoped, mainly because they already had their own methods of communicaton (Skype) and had certain difcultes learning how this one worked.On a teaching level, it did not provide diferent indicators to those from face-to-face sessions, merely confrming the group dynamics perceived in face-to-face sessions.

Student assessment
PBL assessment is an important mater, for which we have references, and especially (O'Shea, Verzat, Raucent, Ducarne, Bouvry & Herman, 2013).On the basis of all of these, we have defned student assessment as two elements: group assessment, assessing the fnal work produced by the group, with an equal grade for all members, and an individual assessment, obtained from observatons in tutorial sessions, peer assessments, etc., from which each student's individual grade is drawn.In the fnal grade, we weight both parts (group and individual) equally, without having found arguments for one to be valued higher than the other.
In the group assessment, we sought indicators that foster interdependence among students when working in a group, and in the group criteria we opted to evaluate the aspects that we consider to be the most important or that are most valued in a company environment in the feld of engineering.
We sought to foster interdependence rather than competton between groups, so in both group and individual assessments we valued collaboratons between groups, the exchange of informaton and knowledge between them, and individual initatves within the group.All of these assessments added up to the fnal grade, and a student's grade could surpass the maximum of 10 of the university system.
The assessment is based on diferent indicators: • Group assessment (7 points): The assessment indicators are equal for all members of the group.They are as follows: • Group work (4 points).The members of the tribunal atending the oral presentaton assess the complexity and innovaton of the work, functonalites, user interface and quality of the applicaton in accordance with sofware standards.They use an assessment template.
• Deliverables at the presentaton (2 points): Assessment of the organizaton, clarity and presentaton of the project report and the slides used in the oral presentaton.
• Reports (1 point): Clarity, presentaton and coherence of the delivered reports and controls.
• Merits (0.5 points, maximum of 1 point): Assessment of whether the work has been referenced by other groups, or whether it has been scored in the top three by classmates at the oral presentaton.
• Individual assessment (3 points): Partcular to each member of the group, obtained during face-to-face sessions (a total of 5-6) and PBL Meetng Rooms sessions.Assessed by the course lecturer.They are as follows: • Atendance and punctuality (1 point): Punctual arrival and departure from sessions.
• Attude (1 point): Whether the student takes part in discussions, and whether they behave actvely or passively.
• Initatve and leadership (1 point): Whether the student's opinion is appreciated by peers, whether peers turn to him/her when they have doubts.
• Merits (0.5 points, maximum of 1 point): Whether they made the oral presentaton, scores highly in peer assessments.In the individual secton, we chose three of the criteria proposed in (Doucet, 2004) that we believe to be among the most important in engineering studies: Initatve, perseverance, punctuality.We proposed the assessment of few competences but the use of many indicators to assess them.We are working on rubrics for assessment in the future.
One of the controversial issues was that of publicly announcing the assessment criteria to the students at the start of the course.Our positon has always been not to disclose the details of the assessment method beforehand, other than that there is a group and an individual assessment, in order not to 'contaminate' spontaneity and the way the group organises its work.However, at the end of the course the criteria were explained to justfy the fnal grade to the students.We understand that some were in favour of this and others against, and this is not an issue that we believe to be closed in terms of PBL.

RESULTS AND DISCUSSION
In this report we present the latest developments in a 9-year experience of applying PBL methodology to the Computer Graphics course on the IT Engineering degree, which we have developed and improved by implantng the methodology, improving assessment and fnally using online remote communicaton applicatons such as OpenMeetngs as a means to assess group meetngs outside of the classroom, which students hold in order to work on their projects.In this secton, we present some quanttatve results and some maters for discussion, initated both by lecturers and by students.
The results of our work are shown in two tables.In Table 1 we show developments in student enrolments, and those that opted for a PBL itnerary.In 2006-07, the course had fewer students as it went from being third year optonal to fourth or ffh year optonal.From 2009-10, the fve-year engineering degree was replaced by the four-year engineering degree, so the number of enrolled students fell due to the gradual disappearance of the degree up untl 2012-13, the fnal year with a signifcant number of students.However, it can be seen that the percentage of student optng for PBL increased from 2007-08, which is especially remarkable considering that this is an optonal subject, which supports the idea that the PBL methodology atracts student interest.At the end of the course, the students were surveyed with regard to their assessment of the course and what they had learned.The results are shown in Table 2.Out of a score of 10, we perceive a medium-high assessment of the use of this methodology.In the fnal year of the course (2012-13) we put the decrease in the assessment of the methodology down to the desire among students to fnish their studies, the fact that many of them were already working and how the degree was disappearing year by year.Students confessed in interviews that it was an exhaustng experience, as they had to dedicate a lot of hours to the project, but they had a very high opinion of it in terms of the results obtained and what they had learned about computer graphics techniques.

CONCLUSIONS
This degree of satsfacton is more subjectvely noted as the demand increases among these students for the course lecturers to direct their end of degree projects, and in some cases they do so as part of a group.We therefore believe that PBL fosters positve learning apttudes in students.
In terms of teaching, the level of satsfacton is also high, for the tutoring work is less monotonous than it is in lectures and the students' motvaton regarding their projects also motvates lecturers to help them by providing sources of informaton and infuencing their decision-making as litle as possible.The tutors comment that it is not always easy to maintain this balance between helping with and infuencing the students' work.They all describe how the students perceiving their tutors as facilitators or advisors is much more gratfying and stmulatng than serving the role of evaluator and corrector.
The use of online tools allowed us to maintain much more direct contact with the students when they were not in class, and made it easier for students to meet and work on their projects even though they were in diferent places.In the frst year of implantaton, the expected results were not achieved, so we worked on improving the way the method is used.
Directed actvites, such as the jigsaw, deliverables and the assumpton of roles represent additonal work, but help students to organise the project and encourage greater interdependence and socializaton among groups.
One of the maters for discussion among lecturers is how the current assessment system provides a numeric grade, and hence the student report does not properly refect the skills and competences worked on in PBL, which on the other hand are specifed in the course's teaching guide.We believe that course assessment systems should be adapted to changes in teaching methodologies, to incorporate assessments of skills and competences in the student report.
Our experience has been and is highly positve, and we feel encouraged to contnue improving it and adaptng it to the changing profle of students entering university.We are ofering our templates to the community and are open to discussion, the aim being to promote, share and improve our experience with PBL.

Figure 3 .
Figure 3. Screens hot of a PBL Meetng Rooms session held by a PBL group

Table 2 .
Results of student surveys: Student assessment of the lecturer's tutoring, the methodology used and overall assessment of the course (all scored out of 10), and in the fnal column the number of students that answered the survey