EVALUATION OF PROJECT BASED LEARNING IN THE AREA OF MANUFACTURING AND STATISTICS IN THE DEGREE OF INDUSTRIAL TECHNOLOGY

In the subject Project I in the second year of the Degree in Industrial Technology Engineering taught at the School of Industrial Engineering of Barcelona (ETSEIB), subgroups of 3-4 students within groups of 20 students develop a project along a semester. Results of 2 projects are presented related to manufacturing, measurement of parts and the statstcal treatment of data, placing emphasis on cross-curricular issues, recording of oral presentatons and how this helped improving its quality, as well as evaluaton of the subject by the students by means of questonnaires and open-ended questons.

The main objectves of this paper are: • Presentng an innovatve course in the feld of Industrial Engineering.
• Explaining an evaluaton method for improving oral presentatons based on video recordings.
• Discussing the use of student surveys to assess the development of the subject.

DESCRIPTION
Although project-based learning was originally employed in medical subjects (Barrows, 1992), in recent years it is becoming more important in other areas (Moursund, 2002). It has also been applied to the feld of engineering, for example in drawing and design (Berrio-Otxoa, Arias & Ochoa, 2012) or in design of machines and mechanisms (Mata, 2007). This fts in a social context where teachings tend to approach real life (Gustavsson, 2008). Gordon (1998) distnguished between academic challenges, scenario challenges and reallife challenges, where the later should be addressed in project-based learning. In short, the idea is to develop learning curricula that ft well the needs of students when, afer graduaton, their professional career starts (McLellan, 1996). In partcular, the format of the subject Project I is especially suitable to convey some transversal competences described in the curriculum, such as independent learning, efectve oral and writen communicaton and teamwork.

EVALUATION OF THE SUBJECT
All projects share a common structure in which students of each subgroup must carry out three oral presentatons. For each presentaton, they have to deliver corresponding writen report. Presentatons, but especially reports, become more complete as the course progresses. In order to evaluate both oral presentatons and writen reports, teachers use a rubric. In the rubric, evaluaton of each oral presentaton takes into account content, order and organizaton of the presentaton, verbal skills, nonverbal skills, graphic resources and answers to questons. Regarding writen reports, elements considered are writen presentaton, schemes, drawings, other fgures, tables, objectves, conclusions and bibliography. A numerical mark between 0 and 10 is assigned by the teachers to each item in the rubric for each subgroup. An average numerical mark is obtained for each subgroup.
Second oral presentatons are videotaped and, in the next class, a discussion session takes place about them. However, results of the discussion session are not being evaluated at the moment. In the wake of experience, in the coming years a rubric will be implemented. It will be used by each student in order to evaluate their own performance in the presentaton.
In additon, along the course students have to solve exercises individually, both in the area of Mechanical Engineering and in the area of Statstcs. Results of exercises are included in the fnal mark.
Final mark of the subject for each student is as follows: Where • N1: partal mark 1 for each subgroup, taking into account both the frst oral presentaton and the frst writen report.
• N2: partal mark 2 for each subgroup, taking into account both the second oral presentaton and the second writen report.
• N3: fnal mark for each subgroup, taking into account both the third oral presentaton and the third writen report.
• Nind: individual mark for each student, taking into account results of exercises.
Vol. 5(2), 2015, pp 131 For both the two projects described here (capability analysis and R&R study), there are some characteristcs that make them partly diferent from other projects ofered in the same subject, while being partcularly innovatve:

Projects led by teachers of two diferent departments
Multdisciplinarity is, without no doubt, important in today's world. Many people agree on the fact that collaboraton among experts in diferent disciplines is essental in any current project. However, curricula in many degrees usually consist on a series of courses taught by professors from diferent departments, with no connecton among them.
This subject is partcularly suited to convey the importance of a multdisciplinary approach: we decided to do it by way of example, preparing the projects between two diferent departments. Of course, this does not come without eforts: a high degree of coordinaton is required, we needed to understand and share diferent ways of conductng a subject, and, obviously, learning technical content "from the other part" was necessary.
Some sessions of the course are responsibility of one department; other sessions are responsibility of the other department. But besides the presentaton sessions, where teachers from both departments are present, other sessions are also shared by both departments. In this way, students perceive that teachers from both departments are doing teamwork. The course has been taught for 3 years already. The inital trend when sharing a course between two diferent departments is to divide the content, so that each teacher has "his part." Perhaps this is inevitable in the beginning (knowing the other takes some tme!), and the rapport between teachers of diferent areas of knowledge takes some tme. But in our case, it is curious and pleasant to notce how, as the years have passed, the number of sessions shared between both departments has increased.
Having sessions shared by diferent teachers from both departments also means that, at least to some extent, teachers stay in the classroom more hours than those marked in their teaching assignment. In the same way that some universites give more weight in the calculaton of the teaching assignment when the subject is done in English (of course, in countries not having English as ofcial language), this could be also done when a subject is shared by teaching staf from diferent departments.

Classes taught in the laboratory
All classes are, at the same tme, theory and practce. Therefore, classes are conducted where "the acton takes place", at the Manufacturing Technology Laboratory, so students have easy access to machine tools, manufactured parts, measuring instruments, etc. (Figure 1). For the analysis of collected data using statstcal sofware students need a computer, so they are asked to bring a laptop to the class. Being at the lab having the appropriate tools promotes working deep in the projects. Furthermore, being physically in the right environment makes avoiding lecture classes almost automatc. Imagine a subject of botany: surely it can be done in the form of lectures (the teacher showing slides of trees and fowers and verbally giving informaton that could be transmited more efectvely by other means!). But if the whole class with the teacher walks out to the mountain, surely the environment itself makes the lesson less lecture-kind and more experiental.

Documents and data transferred between groups
Both developed projects (capability study and R&R study) involve diferent students and are in fact independent. However, they have in common the use of the same machine tools and the same parts. Therefore, in the fnal phase of the course, data collected by the two groups are exchanged, so that each group enriches their work with the work of students from the other group.
Knowing that work performed by each group will be used by other students creates a very interestng spirit of cooperatve work (Prince, 2004). The quality of their work not only personally afects their outcome in the subject, but also infuences the result of other colleagues.
Although competton among students may possibly encourage efort, probably creatng conditons for cooperatve work is more efectve (apart from nicer). Based on our experience, the emotonal implicaton of students in a subject is much more powerful than any threat of difcult exams, few people passing the course, etc. A really excitng statement received by one of the authors of this paper by a student read: "I've had a bad quarter for many reasons and I could not bring the subject to date. But now I will study a lot for the fnal test, because you worked so hard for us that I feel I cannot disappoint you". Certainly a motvated person is unstoppable.

Video recording of a presentaton
Clearly a skill can only be improved if practced. It is naive to think that students will be able to successfully communicate ideas orally when they fnish their studies if they have never done that.
When students make writen reports, receiving comments about the task is quite common; however, this is not so usual in oral presentatons. Therefore, we thought it was necessary in our projects to devote tme to explain how to make reports and presentatons, and give feedback. In the case of oral presentatons, we think an especially powerful feedback is recording the oral presentaton and preparing an exercise where students have to view and analyze those recordings. We have seen that the use of recordings help improve the skills of students (Riba, Codina, Flaquer & Marco, 2006). Some suggestons for improvement are evident without any comments (for example, trying to avoid the excessive use of hesitaton markers, looking at the audience rather than to the screen, maintaining a proper body language, etc.). Therefore it was decided to record the second presentaton of the students, so that they had the opportunity to incorporate improvements in the fnal presentaton.
Recording video is today very easy. Interestngly, if you put a camcorder to record a lecture given by a teacher, those more altered are the students (although the camera is recording the teacher). Afer all, teachers, beter or worse, speak in public very ofen, and "practce makes perfect". Our students do not actually have many occasions to speak in public (as they themselves admited), so recording one of the presentatons of this course is very educatonal. Even if you do not make any further comment or analysis (making comments is something recommended and we do that), just giving the recording to each student is valuable: students will surely watch their presentaton even out of curiosity, and will certainly take interestng lessons.

Evaluaton of the results of learning actvites
It is common to evaluate the result of the implementaton of cooperatve learning techniques such as those presented here (Martn et al., 2012). For Engineering Degrees and Masters, Valderrama et al. defned 6 steps for evaluatng the fnal year project, including defniton of skills, defniton of milestones for evaluaton, assignaton of indicators to each assessment acton, defniton of a rubric for each indicator, defniton of the reports to be completed and defniton of the criteria for assigning the fnal grade to the fnal year project based on evaluaton reports (Valderrama et al., 2009). Based on their methodology, Fermin et al. used three milestones (inital, follow-up and fnal milestone) and three actons of evaluaton for assessing the fnal year project (Fermín et al., 2014). In this paper, evaluaton of the results has been made based on three elements: • The own ratngs of students are, obviously, an indicator of their learning. If students who learn (that is, that fulfl the learning objectves) pass the course (and this is what should happen), the ratngs are a good indicator of the degree of learning of the students. Untl now, all students who have followed the course have passed it (although, of course, some with beter marks and others with worse marks). In the same way that the goal in an industrial process should be producing all parts correctly at frst (if not, quality is worsened, costs are increased, etc), we believe that an ideal subject should have all students passing at frst.
• Informal conversatons with students on their thoughts about the subject. The fact of having few students makes it possible to maintain this close relatonship between student and teacher. The informaton by students spontaneously and in an unstructured way can be invaluable. Obviously, it is also difcult to process, compare, etc. But it can be very rich. Asking students what they like and do not like about the subject also has a lot of sense: we try not to invent what they value positvely, we simply ask them.
• Surveys (questonnaire format and open questons). Questonnaires allow an easy understanding of student opinions and the assessment of possible improvements. The "quanttatve" version of an open survey is to deliver questonnaires to students. The big advantage is that questonnaires can be easily processed and compared (have we improved or not over the previous year?).
• The survey contained 5 questons: • What is the main reason why you chose this project?
• Tell three positve aspects of the project.
• Tell three negatve aspects of the project.
• Did you fnd recording an oral presentaton useful?
• List four aspects where you have improved, either in oral presentatons or writen reports, thanks to the subject.
All three methods of data collecton to evaluate results have been used in the subject. Some results are presented in the next Secton.

RESULTS
In this Secton we ofer some results afer evaluatng last year's course with the subject. Afer a general overview, we will give a bit more detail on the opinions related with the recording of oral presentatons.

Results of the use of surveys to improve the course
The use of student surveys to assess the development of the subject has provided valuable informaton for improvement in subsequent years. The surveys were delivered to class partcipants during the week when the oral presentaton recording was analyzed. That week 16 of the 18 students enrolled in the two projects studied atended the lesson.
Regarding the frst queston, What is the main reason why you chose this project?, results are shown in Figure 2.
A number of 50% of students chose the project because they thought it was an interestng subject, while 29% chose the project because of the schedule and 21% had another reason (for example, the project they really wanted to choose was already full). It is remarkable the fact that half of the students have chosen one of the two projects as frst choice. One student wrote literally: "Colleagues that worked on this project a previous year recommended it" (Figure 3), which indicates that the projects start to be consolidated afer 3 years. Another student said: "I found it interestng that in this project we analyze parts made by machines, simulatng how a real control of producton would work". This idea can be used by teachers to try to explain the students the Vol. 5(2), 2015, pp 134 subject of these projects as, a priori, the names "capability study" and "R&R study" mean litle to a student in the second year of Industrial Engineering. Regarding queston 2, Tell three positve aspects of the subject, there have been many diferent answers. The following are the answers given by 3 or more students: • A lot of learning with the oral presentatons.
• Knowledge of manufacturing processes.
• Students work with real cases.
• Measurement devices have been made available to students during the course.
• Applicaton of theoretcal knowledge in a practcal way.
In queston 3, Tell three negatve aspects of the subject, a greater disparity of answers was found. The most common answers (3 or more students) were as follows: • I would like to see more importance given to the technical mechanical part instead of the statstcal methods.
• Theory stll is too preeminent in the course.
• I would have liked to also measure the roughness of the parts, not only the diameter.
• The parts were measured in the second part of the semester, and there was no enough tme to prepare the studies.
• We could not directly manipulate the manufacturing machines.
Vol. 5(2), 2015, pp 135 It is worth to note that some of the students' requests, such as being able to manipulate the machines, cannot be allowed due to security reasons. However, we can think about asking students to prepare, for example, a numerical control program that the lab technician can later use. With the measurement of roughness, the problem is similar: since the roughness measurement instrument is a precision device that needs some experience to be used, students cannot manipulate it directly. However, we could ask them to prepare instructons on how to use the device.
For the next courses, we do plan to make some changes, such as ofering more technical content, measuring the parts earlier in the quarter, and give a litle more importance to practce.
In queston 4, Did you fnd recording an oral presentaton useful?, all students answered afrmatvely. As an example, two answers are transcribed literally here: "It has been a very positve experience trying diferent ways to teach stuf. I fnd it very educatonal and original. I would like you to contnue with the subject in this approach".
"Yes, despite the fact that being recorded was a litle uncomfortable, it is a good opportunity to learn how to deal with such situatons, plus a good way to observe us from the outside...".
Finally, queston 5 List four aspects where you have improved, either in oral presentatons or writen reports thanks to the subject, similar responses were obtained: • To learn to control your nerves.
• To acquire more fuency when speaking in public.
• To learn how to correctly prepare a writen report • To avoid the use of fllers and hesitaton markers.
• To learn how to properly organize a report (index, fgures and tables, appendices, etc.) • To learn to work together, even with colleagues that are not your friends.
• To have a deeper relatonship with teachers (in most subjects this is very scarce).
• To distnguish which aspects are important to include in a report / presentaton and which not.
• To know how to prepare a presentaton in PowerPoint or Prezi (for example, not having too many words and adding graphs and fgures).
An example of the answer of two diferent students is presented in Figure 4.

Figure 4. Students' answers to the queston on issues where they detected an improvement
In short, surveys have provided valuable informaton that will enhance the subject in future editons.

Results of recording the video presentatons
The evaluaton of the second oral presentaton using video has been key to improve the fnal presentaton, having helped students in several issues such as looking at the audience and not at the screen, maintain a proper body language, avoid using fllers, etc.
All oral presentatons were recorded, each group consistng of 3 people. During the following week, the students were able to review the recordings and were asked to note some positve and negatve points detected in the presentatons. It is well worth asking students to choose some video excerpts (startng minute and second) that can serve as an example to other colleagues and can be used as discussion starters. This also helps focus the discussion.
Regarding areas for improvement in the oral presentatons, in Figure 5 a summary of the issues to be improved is presented (length of the bars is related to the total number of groups that showed the same issue, maximum possible length of the bar is 6 if all groups showed the same issue).

Figure 5. Issues to improve in oral presentatons, according to the students
Almost all groups had at least one of the components wearing a sheet in his hand, which he read occasionally. In one of the groups the use of fllers and hesitaton markers was very obvious. Also, to explain a partcular concept, they mentoned a formula not writen in the slide, so the explanaton was quite difcult to understand. Two groups also used slides with a lot of words and without graphics, and some group members basically looked at the auxiliary screen while explaining. Figure 6 shows positve aspects of the presentatons (length of the bars is related to the total number of groups that showed the same issue, maximum possible length of the bar is 6 if all groups showed the same issue).
All groups used the resource of pointng to the screen and relatng diferent concepts in their presentatons. Four groups made a presentaton of themselves at the beginning of his speech, and three of them explained the objectves and conclusions of the work.

CONCLUSIONS
The subject Project I in the Degree in Industrial Technology Engineering represents a unique opportunity to conduct a course of practcal nature, in which students can learn, more closely and in an applied fashion, some of the areas of engineering.
Furthermore, in the case presented in this work, the course content is enriched by the presence of two teachers from diferent felds of Engineering: mechanics and statstcs.
Recording the oral presentatons has allowed the students to actvely get involved in developing and improving their communicaton skills in oral presentatons. Specifcally, main areas of improvement of oral presentaton were as follows: students stopped reading a paper when presentng, they also reduced use of space fllers, they notced they should include a formula or a graphic in the presentaton if necessary, they understood they have to previously prepare the presentaton and they stopped emphasizing errors. Moreover, teachers have also focused their atenton on specifc aspects that students can improve, especially in regard to oral presentatons, but also in relaton to writen reports. The revision of surveys has verifed that, in general, students are satsfed with the way the course has been developed in the two projects studied. As main positve aspects they include the possibility they had to improve their oral presentatons, gaining knowledge of a manufacturing process and partcipatng in an interdisciplinary subject. As key areas for improvement they ask for more emphasis in all technical aspects and keeping an even more practcal approach.
Surely the survey results will help improve the development of the subject in future semesters.