PROBLEM-BASED LEARNING: EFFECTS ON ACADEMIC PERFORMANCE AND PERCEPTIONS OF ENGINEERING STUDENTS IN COMPUTER SCIENCES

In the present article we analyse the impact of problem-based learning (PBL) on learning and perceptions in first-year students undertaking Engineering in Computer Sciences. The module designed took a number of advanced theories of PBL and its application within the Engineering profession. Mixed methods were used to enable data from qualitative and quantitative instruments to be obtained. A quasi-experimental design was specified, employing non-probabilistic sampling, with a control ( N = 40) and experimental group ( N = 39). In comparing PBL with traditional methods, the results reveal statistically significant differences in aspects such as academic performance. Teamwork, oral communication, written communication and students’ perceptions of the learning experience were also all favoured. Nonetheless, lack of adequate team dynamics in previous learning experiences and reluctance to change traditional teaching approaches, could compromise the viability of that proposed. learning:


Problem-Based Learning. Implications for Learning
PBL is an instructional student-centred model in which knowledge is acquired through the identification of gaps between the level of knowledge possessed by students and that required to resolve a given problem (Barrows, 1986). In this way, non-structured problems are presented. In response to this, self-directed learning is developed, alongside the combination of both individual and collaborative learning activities, under the tutelage of the teacher who acts to facilitate this process (Savery, 2015). This didactic technique, considered to be an active methodology, has been employed as a work strategy throughout entire academic courses or to address specific topics within the different disciplines of a study plan (Fernández & Duarte, 2013). A systematic review performed in health sciences by Yew and Goh (2016) confirmed that PBL was effective for knowledge retainment over the long-term and knowledge application in practice.

PBL Models and their Application in Engineering
PBL was conceived in the Faculty of Medicine at McMaster University in Canada by Barrows (1986) who defined the classic or original model. Following this, Yih-Chyn and Huijser (2017) re-affirmed the characteristics of the model. These results were presented in an unpublished text, written before the death of the author on the 25 th of March 2011. These characteristics provide a theoretical benchmark and diverse models in different study disciplines have taken them into consideration. Barrows discussed the authenticity of PBL in that it is developed around real problems, student-centred, and develops problem-solving skills, self-directed learning, self-evaluation, co-evaluation and collaborative learning in small-group situations. All of this occurs in the presence of expert teachers.
Following this, the classic model has been adopted by similar schools (Maastricht, Newcastle, Roskilde and Linköping, amongst others) and introduced into other branches such as social sciences, law and engineering. The model is introduced through more specific models developed within the different subjects and study disciplines (Schmidt, 1993;Hung, 2006;Kolmos, Fink & Krogh, 2004;Koschmann & Stahl, 1998;Savin-Baden, 2007amongst others). The stages through which the process of PBL is developed depends on the characteristics of each degree course and academic discipline. In engineering, the stages that are normally adopted are those of preparation, problem analysis, issue identification, problem solving, conclusions and report (Graaff & Kolmos, 2007). Within the branch of engineering, models emerge such as that developed in the Polytechnic School of Singapore, known as "One-day-one-problem". This is characterised by its modular arrangement around problems and is philosophically alligned with constructivist learning. It functions within a more structured setting in comparison to conventional PBL, encouraging students to build confidence, alongside their teamwork and self-directed learning skills (Wang & Fong, 2006;Yew & O'Grady, 2012). The model proposed by the Technological University of Malaysia (Mohd-Yusof, Hassim, & Azila, 2004;Said et al., 2005;Mohd-Yusof, Sadikin, Phang, & Abdul-Aziz, 2016) is distinguished by a cooperative PBL process based on the course and is instituted through an academic approach. Iron Range Engineering used in Minnesota, the United States (Iron Range Resources, 2010;Allendoerfer, Bates, Karlin, Ulseth, & Ewert, 2015;Bates, & Ulseth , 2013), considered as an adaptation of the Aalborg model (Kolmos et al., 2004), includes the use of actual industry projects with an explicit focus on the technical, professional and creative development of students. The model applied at the University of Minho, Portugal (Moreira, Mesquita, & Hattum-Janssen, 2011;Alves, Sousa, Fernandes, Cardoso, Carvalho, Figueiredo et al., 2016) proposes an inter-disciplinary project focused on the development of technical and transversal skills. Finally, the model applied within the Engineering School of Mondragon University (Spain) aims to produce graduates with technical and transversal skills who are ready to work in industry (Arana-Arexolaleiba, & Zubizarreta, 2015;Guerra, Ulseth & Kolmos, 2017 (Ioannou et al., 2016)) Further, it can be seen that the use of educational technology (ET) for the improvement of PBL processes has gone hand in hand with the development of traditional proposals (Koschmann, Kelson, Feltovich & Barrows, 1996;Hung, Jonassen & Liu, 2008;Donelly, 2010). In engineering, technology has been incorporated in concert with the characteristics of each institution and its students, following proposals made by Koschmann and Stahl (1998). These authors suggest that problems be presented through scenarios, diagrams, dialogue, video, multimedia and other formats (Barret, 2005). Further, the process is developed through four fundamental stages: Recognition, research, report and reflection.
The model applied to the present research combines, at a theoretical level, two of the variants proposed by Savin-Baden (2014). The first, PBL for knowledge management, seeks to develop a student who is competent in the management and resolution of problems in real contexts. Aiming for students to be capable of interpreting and understanding the knowledge behind problems, in addition to its practical application. The second, PBL through activity, is employed in disciplines such as computing and engineering (Booth & White, 2008). It is used to favour student participation in learning and commitment to teamwork. Next, model implementation was conducted in a similar way to the approach taken by Ioannou, Vasiliou and Zaphiris (2016) in multi-modal settings and followed the four stages proposed by Koschmann and Stahl (1998) (see Table 1).

Perceptions and Influence of PBL within Students of Engineering in Computer Sciences
Students' attitudes and perceptions towards their experiences of PBL reveal a common consensus. They are seen to favour the understanding of learning content and its application in practice. This is due to the way that PBL helps to develop professional skills, critical thinking and self-directed learning, with a consequent impact on the performance and academic success of students (Hande et al., 2015;Dagyar & Demirel, 2015;Dalfaro et al., 2017;Fernández & Duarte, 2013).
In this sense, experiences are found, such as those reported by De Camargo-Ribeiro (2008), which are oriented towards the analysis of PBL. Students perceive this to be an attractive and interesting way to construct their own knowledge and develop the skills of research, communication, teamwork, problem solving, analysis and information synthesis. Yadav et. al (2011) evaluated the learning perceptions and conceptual understanding regarding employment of a PBL and traditional methodology. These authors found that students receiving PBL perceived and demonstrated better preparation when applying content to problem-solving, whilst also being more capable of transferring this to new situations. In the same way, Jaeger and Adair (2014) analysed the perceptions of engineering students following the introduction of a PBL methodology based on individual situations, motivation and perceived ability for success. Results outlined the need for mediation of the process by a facilitator, students' individual and collective responsibility for tasks, and the positive influence of one's individual situation on project quality. Research conducted by McLoone et al. (2016) recorded and analysed student's attitudes towards the use of project-oriented PBL. In this case, students rated the methodology to be positive, motivating and an effective way of managing to learn content. These students also stated that the process had led to improvements in communication, developed teamwork skills and prepared them for later professional practice.
Further, it is also useful to highlight that various research studies have outlined benefits associated with teamwork within organisations and businesses (Torrelles, París, Sabrià & Alsinet, 2015), suggesting that competencies associated with this aspect are amongst the most important and in-demand for employers (Barraycoa & Lasaga, 2009). As stated by Torrelles (2011), it is the: Set of knowledge, skills and attitudes that enables collaboration with other individuals in the performance of activities in order to reach common goals, exchanging information, distributing tasks, taking on responsibilities, resolving difficulties that arise and contributing to collective improvement and development (Torrelles, 2011: page. 209).
A number of studies have made it their aim to evaluate the performance level achieved by students with regards to their development of professional skills through active methodologies (De Miguel, 2006;Schmal, 2015). PBL has been one of the methods to demonstrate the best results in this sense. This is true in relation to examination of its effectiveness for developing teamwork skills, self-directed learning, spoken and written communication, leadership, and other aspects (Fernández & Duarte, 2013;Robledo, Fidalgo, Arias & Álvarez, 2015;Warnock & Mohammadi-Aragh, 2015;Macho-Stadler & Elejalde-García, 2013;McLoone et al., 2016).

Methodology
This proposal is developed under the design-based research paradigm (Rinaudo & Donolo, 2010;Kennedy-Clark, 2013). It responds to problems detected in educational practice in relation to engineering careers with traditional curricula. It starts from the lack of teaching-learning strategies and methods to enable the development of professional competencies. In an attempt to find a solution to this lacking, an active method was administered in order to meet formulated objectives.
The research design was descriptive and correlations. If took a mixed approach, sequentially integrating data produced by qualitative and quantitative instruments (Creswell, 2014). A quasi-experimental design was specified with development, pre-and post-intervention phases. Non-probabilistic convenience sampling was used. A study control group was recruited (CG) and administered a traditional methodology, whilst an experimental group (EG) received PBL methodology according to the stages defined in the design section (see Figure 1). At the end of the intervention, data associated with academic performance in both groups was examined in order to compare the achievements derived. In the EG, measurements were taken, prior to and following intervention, of perceptions regarding the teamwork engaged in and attitudes towards the use of PBL in the Discrete Mathematics 2 (DM2) course.

Study Participants
The research was conducted within the Engineering and Computer Sciences degree course. All processes were conducted during the second semester of the 2017-2018 academic course. Sample characteristics are presented in Table 2.
Relative to males, a lower proportion of females enrol on degree courses in STEAM subjects (Hirshfield & Koretsky, 2018). This aspect is evidenced by the population distribution of the present sample.

Instruments
Academic performance was analysed using Official Records of Evaluation and Control (Registros Oficiales de Evaluación y Control, ROEC) for the degree course for both the EG and CG. Post-intervention, the same objective test was carried out in both groups which evaluated content referring to problem solving in Graph Theory. Figure 2, showed in the following sub-section, presents an example of the type of question asked to evaluate, at an application level, the objective related to the subject under study. This test is targeted towards the evaluation system of the discipline and is established in the analytical program of the degree course. This program is reviewed and updated each semester, and is published on the university's support platform for students.
The analysis is similar for all students. This grants it objectivity and responds to the use of the tool for determining final exam scores. In this way, a comparison could be made between the acquisition of learning content related to the topic being studied, between the EG who followed PBL methodology and the CG who worked with a traditional methodology.
The median test was employed to examine whether study groups were comparable and to compare the average scores of both groups following intervention. This non-parametric technique is adjusted to the sample size and to the type of data used in the study.
A questionnaire was also used as another instrument. This was administered to evaluate the attitudes and perceptions of students towards PBL (Cuestionario sobre las Actitudes y Percepciones del estudiantado sobre el Aprendizaje Basado en Problemas, CAPABP) (Hande et al., 2015) and was translated into Spanish. This instrument comprises a total of 15 items, distributed according to three domains: Knowledge acquisition, generic skills and attitudes towards PBL. In order to determine construct validity, factor analysis was performed using the maximum likelihood methods of extraction with orthogonal varimax rotation. A value of .735 was obtained for the Kaiser-Meyer-Olkin measure of sampling adequacy and a significance value of .000 for the Bartlett sphericity test. These outcomes permitted us to proceed with further analysis. Further, all variables exhibited commonality values higher than .600. These results confirm the three considered domains, explaining 80.861% of total variance and largely conforming to the initial internal questionnaire structure (χ² = 88.007, with a significance value of .000). Instrument reliability was evaluated by calculating Cronbach alpha values, achieving the values presented in Table 3 (p ≤ .05, with 95% confidence levels also shown). Perceptions of teamwork prior to and following intervention was measured using a teamwork self-report rubric (Yarosh et al., 2017). This differentiated three performance levels for each of the six established indicators.
For qualitative analysis of students' perceptions in relation to PBL methodology and work conducted using this method, individually developed Wikis were enabled. Emerging perceptions were analysed through content analysis with the software AQUAD 6. In accordance with the type of research, the frequency of code appearances was counted and frequency percentages were analysed. 12 recording units were coded: Oral and written communication, teamwork, acquisition of learning content, collaboration and information exchange, team dynamics, practical application of content, problem solving, skills, self-learning, applied technologies, problem-based learning and conflict resolution.
In order to facilitate the process of data collection and analysis, the courses learning management system (LMS) was used as an adjunct to the administration of questionnaires, rubrics and Wiki reflections. Quantitative data were analysed through the software SPSS v.22.

Procedure for the Implementation of PBL
PBL was applied during the second semester of the first year of study on the topic of Graph Theory, belonging to the subject of Discrete Mathematics 2. Graph Theory is based on discrete and applied mathematics, and is born out of concepts that come from diverse knowledge areas such as algebra, arithmetic, probabilities and topology, and combined areas (Diestel, 2005). It was conceived in 1736 when Euler (1736) published the article "Solutio problematis ad geometriam situs pertinentes". This was directly linked to topology and is considered to be the first theoretical output of the theory. Its theoretical basis was developed, historically, from contributions made by researchers such as Kirchhoff, Guthrie, Cayley, König, Dijkstra and Kuratowski, amongst others. This has directly influenced computer sciences, computing and telecommunication due to the viability that is offers for processes optimization, routing, flowcharts, search algorithms and network analysis (Rosen, 2012). In this way, the study of graphs has become a standard topic in subjects within this branch of study, particularly within the content of subjects from the discipline of Artificial Intelligence (AI). It is integrated into the Engineering in Computer Sciences curriculum as a section of content which corresponds to the Discrete Mathematics 2 module. This forms part of the computational intelligence discipline which includes discrete and applied mathematics and AI.
In the study plan, this topic is made up of two sub-themes: Notions about graphs, and special routing and planarity. The time distribution attributed to study this topic, in both the control and experimental group, can be seen in table 4. Each activity comprises a total of two hours of classes.

PBL Design for the Experimental Group
When designing the PBL delivered to the EG, a number of guidelines established in the consulted literature were considered. This approach is based on the idea that knowledge construction is carried out in a flexible way, through the collaborative working of students when faced with real-world problems and contexts, in this way strengthening meaningful learning development. In this sense, it is assumed that the nucleus of learning activities lies within the problem itself and that the process is not linear. Further, problem generation should not be a simple task, ensuring its relevance, appropriate complexity and appropriateness to real contexts.
PBL was developed according to the four stages of the model proposed by Koschmann and Stahl (1998) and re-asserted by Ioannou et al. (2016): Recognition, research, report and reflection. Executed actions are presented under the first subtheme in Table 1. Analogously, the sequence used when considering the second subtheme was similar, with the exception of the introductory lecture given at the beginning which did not have to be repeated again later. In summary, the following actions were developed in each stage: 1. Recognition: Students developed the general problem analysis and acknowledged learning problems through the lesson imparted and educational resources provided by the tutor. An example of the type of problems to be resolved in PBL can be seen in figure 2.
2. Research: In relation to this theme. Self-directed and collaborative study was conducted of the learning content required to respond to identified learning problems.
3. Report: Here, teams met to apply the information obtained from problem solving. Groups developed and presented potential solutions to problems, analysing and debating the different perspectives relating to the possible solution.
4. Reflection: Here, the state of acquired knowledge, strategies employed, performance of team members and analysis of new problems which could arise were evaluated. Finally, students completed administered rubrics and questionnaires.
Given that it was the first time that a properly structured teamwork dynamic was applied and that this took place with a group of first-year students, support materials were distributed to ensure the appropriate working of teams. The former positively impacted upon team performance and work with the educational resources available on the course platform for learning self-management.
When distributing teams, individual student characteristics were considered, mainly academic performance in the subject up until the point of intervention. Imbalance relating to the learning level of members was avoided. Teams integrated between 3 and 5 students and counted on their lecturer who acted as a facilitator (Prince & Felder, 2006).
The scant or non-existent experience of students in the design and production of educational resources, required consultation materials to be available which served as a guide when elaborating scripts. Classrooms and laboratories were made available for group work sessions. Laboratories were conceived as a viable alternative given certain access difficulties, as it could not be guaranteed that all students had electronic devices with the necessary capabilities for ubiquitous access to information, the internet and resources. To this effect, instructions given around the presentation of deliverable artefacts as evidence of learning were made in consideration of available technology within the reach of those involved.

Methodology Used in the Control Group
The traditional methodology applied within the CG was based on the delivery of theoretical lectures, one for each aforementioned topic, respectively. Six practical classes (PC) were run. Classes were evenly distributed and integrated a seminar presentation which closed the subject course. In relation to the lectures (L), presentational methods were favoured in which the lecturer discussed the learning content that would, in later practical activities, lead to skill development. PC were developed based on initial lecturer orientation of exercises and problems at the beginning of each session, students then working individually to find solutions and, finally, spending time to review the responses given on the whiteboard. Evaluation of whether learning objectives were met was performed by the lecturer. This was conducted using a work checklist, shared with students on the whiteboard or in notebooks, and through systematic written or oral evaluation results.

Results
Results of the present research are presented with respect to the three proposed objectives.
Firstly, for the analysis of academic performance a test of homogeneity was applied in order to determine whether study groups were comparable. The result revealed that significant differences did not exist following application of the median test, verifying that the analysis was viable according to a 95% confidence level (theoretical χ2 = 3.841; empirical χ2 > 2.930). Considered hypotheses were H 1 : Statistically significant differences exist between the EG and the CG; and the null hypothesis, H 0 : Statistically significant differences do not exist between the EG and CG. The combined median of all elements was Mdn = 3, and it was later verified that the treatment given to the EG through PBL affects student's academic performance (95% confidence level; df = 1; theoretical χ2 = 3.841; empirical χ2 < 4.250). Scores achieved in the objective tests performed following intervention was measured in both groups using a quantitative scale which ranged from 2 -5 points, with 2 representing the lowest score (fail). The percentage of those who passed, taken as an indicator of academic performance, was 60% and 79.49% in the CG and EG, respectively. This evidences a significant difference between both groups. It was highlighted that the number of students to achieve maximum scores in the EG was double that seen in the CG. This result may be partly attributed to the extent of meaningfulness achieved through PBL when faced with practical real-world problems. In this respect, comments left by students in the course Wiki included "…we are better prepared because we are faced with problems of daily life" (X). In another sense, this difference could also be attributed to an increase in problem solving skills, with this objective being evaluated in the objective tests pertaining to the subject program. In this respect, some students stated "the PBL learning methodology is really good because it helps us to understand the way in which to resolve problems, it gives us skills to know how to tackle the difficulties that could be presented to us in the course of problem solving " (C). Further, it is reasonable to acknowledge that the number of students who failed the course in the CG was also double that of the EG. This is due to the fact that problem solving skills are some of the most complex to acquire in this subject. In this sense, the traditional methodology followed in the CG does not favour the development of skills to resolve problems, whilst this occurs natural in the EG due to PBL (see Figure 3). Secondly, following examination of students' attitudes and perceptions regarding the experience of PBL delivery, outcomes from the CAPABP questionnaire were considered. Response rate was 100%, with a total of 39 questionnaires being completed in relation to the EG. There were no missing values. Table 5 shows the mean and standard deviation of questionnaire items, dividing according to domains: Generally speaking, evaluated items received a satisfactory evaluation. The value of 4 (agree) was established as the median, with the mean being higher than this in all cases. Standard deviation values were between zero and one. This outcome suggests that student's experiences of PBL were well perceived, with highly positive evaluations being given to the surveyed domains. Item 3 is rated between 4 (agree) and 5 (strongly agree) by 97.4% of students. This could be due to the high feasibility of carrying out the analysis process from different perspectives, counting on diverse sources of information and educational resources made available by the tutor/lecturer. Further, the process is characterised by sharing spaces, the exchange of ideas and information, and making these ideas common knowledge amongst group members. This contrasts with the traditional methodology currently followed in this subject. In this case, work is not considered in the same collaborative way. This leads to the proliferation of one-dimensional thinking in students, with a limited view of problem-solving alternatives. In this domain, the least highly rated item was item 2 (94.9% of students reported scores of 4 or 5), although values were still high and were greater than the mean. Nonetheless, this result denotes that some students, despite positively rating the analysis and information processing in PBL, do not demonstrate cognitive Independence and, as a consequence, do not consider themselves competent to manage their own learning content.
In the second domain, evaluations of PBL are outlined as an alternative for favouring development of communication skills (items 7 and 10), with 89.7% and 84.6% of students, respectively, reporting scores of 4 and 5. Collaboration and information exchange with the tutor, team members and other classmates via the various established channels, enables development of fluid communication processes which use a number of the senses. The social presence of the tutor is essential for removing the barriers established for various reasons. In this respect, students indicate "…it helps us to a great extent to strengthen oral and written expression as during development of the issue at hand we are obliged to investigate different types of places, with different resources, with the aim of ensuring that work objectives are reached" (S). Determined activities exist through which development of this competence is favoured. Such activities include the elaboration of artefacts as learning evidence for the evaluation of suggested solutions for problems, proposed interview script guides and the design of educational resources in the established way. The lowest score achieved was for the item associated with leadership (item 8). This could be linked to the fact that some students still stick to the false belief that the leader is the person who takes on most of the responsibility for tasks because they are more prepared, responsible and committed to their personal results. As a result, they show themselves to be resistant to taking on the role of leader, pointing to their own supposed inability and preparation. Subsequently, they limit their engagement to presenting the results found by the leader and fail to find a route within the chosen method to overcome their limitations. In the final domain, item 14 stands out (92.3% of students reported scores of 4 or 5). The team dynamic followed and the assumption of diverse roles permit students to self-evaluate their capacity for the performance of various tasks. This enables them to recognise their personal challenges and draw up a plan to address them, with this being done in function of the achievement of both individual and common team goals. On the other hand, the lowest score, whilst also being higher than the mean, was obtained for item 12. This could be associated to the fact that some students consider that their opinions are not respected nor considered during the process of solving problems, thus, they appear to be considered irrelevant in analysis or lacking in depth and rigour.
In a general way, the domains of content and competencies are highly positively correlated (ρ = .816). Nonetheless, the most positive scores are found for the former, whilst the lowest score (although also highly positive) is given for the second domain. This situation could be due to the fact that problem solving is a transversal skill developed from primary education using traditional methodologies. The novelty of teamwork via PBL, following a correctly structured dynamic, favourably impacts upon content acquisition. This makes qualitative changes in the observed indicators perceptible. However, it is difficult to administer this given that the development of competencies in previous teachings has never previously been the object of analysis and it does not have a clear and specific system of performance indicators, nor does it count on the pathways or strategies to evaluate them. Administration is even more challenging when the teaching process is centred on the student and formative evaluations are performed which demand individual and group self-evaluations, processes with which students are not familiar with.
Given the size of the EG (N = 39), the Shapiro-Wilk normality test was applied. This indicated that data did not follow assumptions of normality (statistics between .700 and .800, dr = 39 and p < 0.05). Following this, the non-parametric Spearman Rho test was applied to analyse correlations between grouped questionnaire domains. This obtained values 0.7< (ρ < 0.89) in all cases, denoting high positive associations.
Bivariate correlations (see Table 6) underline the values achieved between items 7 and 8 which are associated with communication (ρ = .715). A highly positive correlation between both items is evidenced. This may be attributed to the importance of the communication and expression of ideas throughout the entire process. This is crucial for collaboration, debate and information exchange regarding the research findings uncovered from self-directed learning and, finally, for the presentation of proposed solutions to learning situations for measuring available technologies.
As another aspect, the existing relationship between items 14 and 15 (ρ = .705) stands out, being equally highly positive as the aforementioned correlation. From this is can be understood that students become aware and acquire skills to make evaluative judgements about their actions and those of their course mates. This is facilitated by the prior establishment of specific performance indicators for each activity. In a similar way, a moderately positive correlation was found between the items of 5 and 7 (ρ = .686). This may be largely attributed to the search for, analysis and discussion of research findings, fit of solutions to problems, and the debate developed between the students and the rest of their work team. The plenary session for presenting results will also play a role in establishing this association. Finally, a moderately positive correlation is distinguished between items 9 and 11 (ρ = .678). In this sense, the team dynamic established and the tasks relating to each role determine the active participation of all team members as a function of proposed objectives. Nonetheless, students who do not occupy leadership roles are generally more involved in the search, analysis and processing of information.
In keeping with the third objective, student's perceptions regarding the influence of PBL on teamwork skills was examined through the rubric used throughout the study. Results are observed in Table 7. Prior to the intervention, the majority of students reported maximum self-evaluations scores ( level 3) for each indicator. Indicators 1 and 2 are highlighted, "I actively participate and collaborate in team tasks" and "I promote trust and cordiality in group relations", respectively. Both received approximately 69% of the response total. Thus, these students considered themselves to be more capable of transmitting, debating and exchanging criteria and comments with the course mates in their group in relation to work. In this way, they maintain a climate of collaboration and support, promoting constructive dialogue. In the same way, indicator 6, "I coordinate the group, guaranteeing the achievement of results and positive performance " obtained around 62% of responses in Level 3. In this sense, a large number of respondents stated that they were capable of encouraging other team members to reach more difficult goals than those established.
Following the intervention, results varied significantly. All indicators in level 3 decreased, apart from the sixth indicator which showed a slight increase of approximately 2%. Indicators 4 and 5 were most affected, "I act in a constructive way in order to resolve conflicts within the team " and "I coordinate the group, ensuring the integration and empowerment of members", respectively. Each of these indicators showed a decrease of close to 20% of students reporting the maximum level. In the case of the former indicator, most of these cases passed to level 2, whereas for the latter indicator, cases were distributed between levels 1 and 2. Nevertheless, it is possible to argue that the decline in positive evaluations following intervention does not necessarily imply a reduction in the initially reported level for teamwork skills. Instead, this could represent greater accuracy and self-evaluation due to greater knowledge of the previously established performance indicators. A similar effect could result from greater knowledge of the work dynamics displayed in the team and personal commitment to the application of a formative evaluation process, aspects that are characteristic of PBL. To this effect, one student indicated that "When working in a team it should not always be the leader who does everything, when we talk about teamwork we address this, a team in which each and every one of the "team" members should participate. Before, we did not know what is was to work in a team, now we do. Whenever it functions in this way, the team will work well, where each of the members collaborates and gives their own opinion…" (D).
In the absence of prior experience with regards to the training and development of the aforementioned skill, over-evaluation of individual skills represents a tangible threat. This risk increases when individuals attempt to respond in the way in which they consider to be desired or considered correct by others. Further, through PBL, students become aware of the actual skills developed, with this influencing the final self-evaluation given.
Content analysis of the Wiki platform guided reflection on the development of competencies through PBL and perceptions relating to this. This was conducted by counting frequencies and percentages. Results are shown in Figure 4.
Results denote that students in the GE recognise the impact of PBL methodology on the development of skills in a general way and, particularly, in relation to teamwork, and oral and written communication. A total of 53 records are indicated in which participants observe standpoints such as "In my opinion I think that teamwork is very important given that through this we exchange knowledge about the topic we are developing, whilst it also allows you to educate yourself and incorporate and update our intellect, it helps to a great extent to strengthen oral and written expression as during the development of the problem at hand we are obliged to investigate in different types of places with the aim of guaranteeing good work".
In another sense, favourable criteria emerge in relation to the use of PBL methodology which are unprecedented in the Curriculum. The impact of these criteria can also be seen on the acquisition of learning content. Other opinions express that "PBL instils us with deeper knowledge at the time of studying given that in this way we are better prepared because we are faced with problems of everyday life" (X). This reveals student perceptions about the practical meaning of content.

Discussion and Conclusions
The present research was developed in an Engineering degree course with a traditional curriculum. It followed a mixed approach which integrated both qualitative and quantitative types of data. A quasi-experimental study was specified, comparing results obtained in a control group (CG) and an experimental group (EG). These groups represented a traditional methodology and PBL, respectively. Results of the present study corroborate findings reported in other empirical research studies which used a similar design and were conducted in engineering. These prior findings also supported the three objectives proposed in the present article and are discussed below.
With respect to the first objective, the results achieved indicate that academic performance in Discrete Mathematics is favoured when PBL is employed, in comparison with when a traditional methodology is used. An increase in the average score obtained in the CG was observed, following application of similar tests which measured learning content post-intervention. Present findings are in line with previously conducted research in which, in addition to better academic performance, students reported better performance when solving problems and applying content to practice (Polanco et al., 2004;Rodríguez & Fernández-Batanero, 2016;Morss & Billiar, 2016). In the same way, results agree with presented in a meta-analysis performed by Dağyar and Demirel (2015). This analysed 98 studies, of which 45 had been conducted within the higher education setting and employed similar designs to that utilised in the present research. This meta-analysis evidenced a large impact of PBL on academic success when compared with traditional teaching. However, this contrasts with findings reported by Dochy et al. (2003) which found evidence in 33 studies of a negative effect of PBL, relative to students taught in a conventional learning setting, on the knowledge base constructed. The difference between these two studies is presumably due to the accumulation of experiences following PBL application and the analysis of risks detected as threats in these research studies, aspects which were taken into consideration in later research.
With regards to the second objective, PBL was demonstrated to be greatly accepted by students. Students perceived meaningful gains in knowledge management and in the skills of teamwork and communication.
All of these skills are highly rated by teachers and employers for the achievement of academic success and effective professional performance. The perceptions reflected in the applied instruments further highlight consensus in relation to the novelty of the methodology and its contribution towards the achievement of more solid learning, evaluations of the teaching process and involved individuals, and the application of learning content in real-world contexts. This is in line with findings reported by Mitchell, Canavan and Smith (2010) who found that students perceived PBL to be highly effective in promoting the development of transferrable skills, skills that are highly valued by employers. As has also been highlighted by these same authors, as process of adaptation to PBL is required. This is because the use of traditional methodologies, where poor practice and inappropriate group dynamics prevail, provoke distrust and uncertainty at the time of tackling group tasks.
In the same way, this positively impacts upon the opportunities offered by PBL for approaching problem solving from different perspectives. This issue is conditioned by the availability of multiple educational resources for self-directed learning and the social presence of the tutor to provide the accompaniment and support pertinent to the process. This characteristic is unusual in traditional methodologies which conditions one-directional and convergent thinking, in which interaction between actors is reduced.
In accordance with other research studies, workload was observed to increase considerably with PBL for both teachers and students (Macho-Stadler & Elejalde-García, 2013). Nonetheless, both of these expressed conformity with the process with regards to feeling motivated by the methodology and having addressed subject sub-themes with, at most, two learning situations. The former (feeling motivated), prevents symptoms of fatigue and tiredness from emerging, regardless of the extent of social presence and help shown by the tutor (Pyle & Hung, 2019). It also demands the design and presentation of educational resources in diverse formats for each problem considered. In this sense, analysis of the impact of the workload imposed by PBL on academic performance in the other curricular subjects would be insightful. This could be examined in future research studies. It is understood that generalisation of PBL to other program topics will not guarantee satisfactory results if it only counts on the motivation of implicated staff. In this case, successful application to other ambits will depend on the various factors that are fundamentally associated with the curricular training model of the degree course and the objectives established, alongside their respective forms of evaluations. These aspects will have to be modified in order to focus on the outcomes of contemplating relevant processes.
Further, the communication and expression of ideas throughout the entire process was highly rated and considered essential for information exchange and the presentation of results. Nonetheless, the term collaboration was unknown in this context, despite recognising that PBL enables individuals to earn trust, leave shyness to one side and, in this way, be able to develop a fluid debate and assess the best potential solutions together. This drives the need to create spaces, prior to PBL intervention, to familiarise individuals with associated terms as indicated by Rodríguez & Fernández-Batanero (2017).
In relation to the third objective, positive student perceptions were observed regarding the improvement of their teamwork skills following analysis of opinions expressed in relation to the communication spaces created to develop this competence. In this case, the relevant space pertained to the personal Wikis available in the course platform. Results point to full engagement with the teamwork conducted in PBL. An effective work dynamic was established, with roles and tasks being defined by each member. This is entirely unprecedented, being absent from earlier educational stages. Nonetheless, a degree of rejection still prevails for assuming leadership roles. This is due to poor practice in that the leaders tends to be associated as the person in charge of executing all tasks during problem solving. Typically, traditional methodologies do not encourage collaboration and information seeking equally between all team members. This leads to a lack of knowledge around the function associated with each role within a team and a lack of personal commitment to the execution of tasks. In making available different spaces and resources for collaboration and permitting further evaluation of both individual and group contributions to performing common tasks, favourable perceptions of the process emerged.
It is necessary to highlight that fluctuating self-perceptions of teamwork were reported by students following administration of the post-intervention rubric. In this sense, results recorded a decrease in scores due to the initial lack of knowledge about skill performance indicators, desired levels and forms of measurement. A consequence of this is that students may overrate their actual abilities. Following intervention, evaluations were more objective and accurate. As students were in their first academic year and so could not call on references with regards to working on this skill in previous learning experiences, their scarce or non-existent of an effective team dynamic negatively impacted upon outcomes.
The absence of actions oriented towards the training and development of skills at all levels of learning in the Cuban educational system, in addition to the use of active methodologies, proved to be a limitation of the present study as it sought to examine these approaches in this context. Lack of knowledge of the theoretical support associated with PBL, the teamwork dynamic, and roles and functions of implicated individuals, affects the execution of the process. This is because additional time is required for the training of teaching staff charged with exercising the role of facilitator, the explanation of concepts, PBL processes and evaluation approaches, and the design of educational resources to cover these topics.
In accordance with Robledo et al. (2015), we consider that PBL demands commitment from students and their engagement in the learning process. This requires students to put into action their higher order thinking skills. It also demands management of human resources, and methodological and technological resources, making implicated individuals aware of the greater practical application of this method and its link with future professional performance.
Despite the validity of the achieved results, questions remain regarding the factors that might have led to them given that an uncommon dynamic was introduced into the traditional curriculum. It is worthwhile to question to what extent the breakdown of problems into sub-problems and the availability of a greater number of educational resources on the virtual platform impacted results. These questions are in line with those posed in the study reported by Latasa et al. (2012), focusing towards cost-benefit analysis of PBL application within the traditional curricula of first-year students.
Finally, given the determining role and influence that learning technologies can exercise over the academic spoils of students, alongside the benefits reported in research studies when their incorporation is favoured (Ioannou et al., 2016;Okyere, Tawiah, Lamptey, Oduro & Thompson, 2017;Nordin & Subramaniam, 2013), we propose, as a future projection, that settings should improve their development of mediation methodologies such as PBL. This should take place through the gradual introduction of relevant technologies. In the present context, increasing costs of Internet access and the economic difficulties that students can face in obtaining any type of device or mobile technology are problematic as this represents the minimum requirement to support this learning process. Despite this, the creation of group conditions in teaching laboratories is viable, as is the delegation of sufficient time for completing tasks relating to the search and exchange of information, an aspect highly rated by students.