Email this article 
Hide
Show all
PROBLEM-BASED LEARNING: EFFECTS ON ACADEMIC PERFORMANCE AND PERCEPTIONS OF ENGINEERING
STUDENTS IN COMPUTER SCIENCES
, Vanesa-María Gámiz-Sánchez2, Antonio Cañas Vargas21Universidad de las Ciencias Informáticas (Cuba)
2University of Granada (Spain)
Received March 2020
Accepted July 2020
Abstract
In the present article we analyse the impact of problem-based learning (PBL) on learning and perceptions in first-year students undertaking Engineering in Computer Sciences. The module designed took a number of advanced theories of PBL and its application within the Engineering profession. Mixed methods were used to enable data from qualitative and quantitative instruments to be obtained. A quasi-experimental design was specified, employing non-probabilistic sampling, with a control (N = 40) and experimental group (N = 39). In comparing PBL with traditional methods, the results reveal statistically significant differences in aspects such as academic performance. Teamwork, oral communication, written communication and students’ perceptions of the learning experience were also all favoured. Nonetheless, lack of adequate team dynamics in previous learning experiences and reluctance to change traditional teaching approaches, could compromise the viability of that proposed.
Keywords – Problem-based learning, Teamwork, Education in engineering, Higher education, Academic performance.
To cite this article:
Amaya Chávez, D., Gámiz-Sánchez, V.M., & Cañas Vargas, A. (2020). Problem-based learning: Effects on academic performance and perceptions of engineering students in computer sciences. Journal of Technology and Science Education, 10(2), 306-328. https://doi.org/10.3926/jotse.969 |
----------
1. Introduction
Study programs pertaining to the subjects of Sciences, Technology, Engineering, Arts and Mathematics (STEAM subjects) are crucial due to their close link with economic, scientific and technological development. Indeed, such development comes from the introduction and assimilation of advances in these disciplines within a given country or region (Kumar, 2017). To this effect, teacher training in higher education institutions should be seen as vital. On the one hand, it is relevant for the development of specific skills which provide a solid theoretical and conceptual basis from which advances can emerge. On the other hand, it develops generic skills which are considered to be the most important for educators, graduates and employers. Within the latter set of skills, teamwork, oral and written communication, problem solving and self-directed learning particularly stand out (Passow, 2012; Warnock & Mohammadi‑Aragh, 2015). One option for successfully installing these skills is to bring students closer to the real world. Engineers design and propose solutions for professional problems based on the similarity or analogy of these problems with already resolved cases from their practical experience. Thus, the more students are faced with real-life situations, the better prepared they will be when their job position requires them to put the professional skills they have acquired during their degree studies into practice (Kaplan & Vinck, 2013).
Adoption of these active methods by universities demands the active participation of students. This generates meaningful and enduring learning, and enables content to be applied in real heterogenous contexts. One of these methods is PBL (Problem-Based Learning). Through PBL, various authors reveal that learners improve their understanding of content, strengthen self-directed learning, develop active learning, and broaden their inter-disciplinary perspective and general knowledge (Gijbels, Dochy, Van den Bossche & Segers, 2005; Leary, Walker, Shelton & Fitt, 2013; Kim, 2017). In the same way, it is proposed that PBL contributes to the development of project management, collaboration, teamwork, conflict resolution and communication skills (McLoone, Lawlor & Meehan, 2016; Macho-Stadler & Elejalde‑García, 2013). In engineering, the majority of research related with the use of this method has sought to analyse its influence on the development of student’s critical thinking, training and development of professional skills, conceptual understanding of learning content and the obtainment of meaningful gains in the learning of content. All of this is manifested as improved academic performance (Fernández & Duarte, 2013; Yadav, Sumedi, Lundeberg & Bunting, 2011; Rodríguez & Fernández-Batanero, 2017; amongst others).
In this sense, relevant learning experiences of the aspects described above were developed. It is highlighted that the training model for the Cuban system of higher education is based on objectives rather than competencies. Given this situation, the need to promote student-centred curricular content through which students can develop professional competencies for their later professional practice has been outlined in the Documento Base para el diseño de los Planes de Estudio E, the basic paper for the design of study plans (Ministry of Higher Education [MES], 2016). Recently, many degree courses have conducted a process of curricular re-design to address this, despite the fact that its assimilation is still considered to be in the adaptation and development stage.
Within the degree of Engineering in Computer Sciences, a number of general and short-term goals have been established which address a skillset that must be acquired by students as they progress through the course. In contrast, the methodologies that govern teaching and learning processes in different disciplines and subjects are traditional in nature, with lecturing and independent work approaches predominating. No projection towards the use of active methods is seen, despite the fact that their potential for skills training and development has been proven (Fernández-March, 2006).
In agreement with that presented above, the ability of active methodologies to assist the development of required skills moves authors to consider that the implementation of PBL, as an active teaching-learning method that aligns with degree aims. It should, therefore, provide greater solidity to learning content, with this being reflected in the improved academic performance of students. At the same time, this will positively influence perceptions of teamwork and attitudes regarding the employment of this type of method. The present article pursues three main objectives which are presented next:
1.Analyse the effects provoked by the application of a PBL approach on the academic performance of first-year students undertaking an Engineering in Computer Sciences degree.
2.Examine the attitudes and perceptions of students regarding their experiences of PBL.
3.Analyse student perceptions with respect to the influence of PBL on their ability to work in a team.
2. Theoretical Framework
2.1 Problem-Based Learning. Implications for Learning
PBL is an instructional student-centred model in which knowledge is acquired through the identification of gaps between the level of knowledge possessed by students and that required to resolve a given problem (Barrows, 1986). In this way, non-structured problems are presented. In response to this, self‑directed learning is developed, alongside the combination of both individual and collaborative learning activities, under the tutelage of the teacher who acts to facilitate this process (Savery, 2015). This didactic technique, considered to be an active methodology, has been employed as a work strategy throughout entire academic courses or to address specific topics within the different disciplines of a study plan (Fernández & Duarte, 2013). A systematic review performed in health sciences by Yew and Goh (2016) confirmed that PBL was effective for knowledge retainment over the long-term and knowledge application in practice.
PBL use on engineering courses has demonstrated notable advantages for motivating and involving students in authentic situations of real work, improving meta-cognition, favouring problem solving, and developing critical thinking and professional skills (Othman, Mat-Daud, Ewon, Mohd-Salleh, Omar, Abd‑Baser et al., 2017; Schmidt, 1993). Rodríguez and Fernández-Batanero (2016) carried out a review of the PBL methods applied in engineering courses. They noted that “for engineering disciplines, it is necessary to present real-world problems or those as close as possible to real situations, associating this with applications in the professional context in which the student will practice in the future” (p.17). In this vein, a number of empirical studies report that PBL has been used to promote the acquisition of learning content (Lachiver, Dalle, Boutin, Clavet & Dirand, 2002; Polanco, Calderón & Delgado, 2004; Dochy, Segers, Van den Bossche & Gijbels, 2003; Said, Mahamd-Adikan, Mekhilef & Abd-Rahim 2005; Morss & Billiar, 2016), improve conceptual understanding and perceptions of learning (Yadav et.al., 2011; Hande, Mohammed & Komattil, 2015), and improve academic performance (Dagyar &Demirel, 2015; Dalfaro, Del Valle & Aguilar, 2017, Rodríguez & Fernández-Batanero, 2016, 2017).
2.2. PBL Models and their Application in Engineering
PBL was conceived in the Faculty of Medicine at McMaster University in Canada by Barrows (1986) who defined the classic or original model. Following this, Yih-Chyn and Huijser (2017) re-affirmed the characteristics of the model. These results were presented in an unpublished text, written before the death of the author on the 25th of March 2011. These characteristics provide a theoretical benchmark and diverse models in different study disciplines have taken them into consideration. Barrows discussed the authenticity of PBL in that it is developed around real problems, student-centred, and develops problem‑solving skills, self-directed learning, self-evaluation, co-evaluation and collaborative learning in small-group situations. All of this occurs in the presence of expert teachers.
Following this, the classic model has been adopted by similar schools (Maastricht, Newcastle, Roskilde and Linköping, amongst others) and introduced into other branches such as social sciences, law and engineering. The model is introduced through more specific models developed within the different subjects and study disciplines (Schmidt, 1993; Hung, 2006; Kolmos, Fink & Krogh, 2004; Koschmann & Stahl, 1998; Savin-Baden, 2007, 2014; amongst others). The stages through which the process of PBL is developed depends on the characteristics of each degree course and academic discipline. In engineering, the stages that are normally adopted are those of preparation, problem analysis, issue identification, problem solving, conclusions and report (Graaff & Kolmos, 2007). Within the branch of engineering, models emerge such as that developed in the Polytechnic School of Singapore, known as “One‑day‑one‑problem”. This is characterised by its modular arrangement around problems and is philosophically alligned with constructivist learning. It functions within a more structured setting in comparison to conventional PBL, encouraging students to build confidence, alongside their teamwork and self-directed learning skills (Wang & Fong, 2006; Yew & O’Grady, 2012). The model proposed by the Technological University of Malaysia (Mohd-Yusof, Hassim, & Azila, 2004; Said et al., 2005; Mohd-Yusof, Sadikin, Phang, & Abdul-Aziz, 2016) is distinguished by a cooperative PBL process based on the course and is instituted through an academic approach. Iron Range Engineering used in Minnesota, the United States (Iron Range Resources, 2010; Allendoerfer, Bates, Karlin, Ulseth, & Ewert, 2015; Bates, & Ulseth ,2013), considered as an adaptation of the Aalborg model (Kolmos et al., 2004), includes the use of actual industry projects with an explicit focus on the technical, professional and creative development of students. The model applied at the University of Minho, Portugal (Moreira, Mesquita, & Hattum-Janssen, 2011; Alves, Sousa, Fernandes, Cardoso, Carvalho, Figueiredo et al., 2016) proposes an inter-disciplinary project focused on the development of technical and transversal skills. Finally, the model applied within the Engineering School of Mondragon University (Spain) aims to produce graduates with technical and transversal skills who are ready to work in industry (Arana-Arexolaleiba, & Zubizarreta, 2015; Guerra, Ulseth & Kolmos, 2017).
Table 1. Distribution of the consideration of learning content in each sub-theme
according to the PBL model employed (Adapted from “Problem-Based Learning in Multimodal Learning Environments: Learners’ Technology Adoption Experiences (Ioannou et al., 2016))
Further, it can be seen that the use of educational technology (ET) for the improvement of PBL processes has gone hand in hand with the development of traditional proposals (Koschmann, Kelson, Feltovich & Barrows, 1996; Hung, Jonassen & Liu, 2008; Donelly, 2010). In engineering, technology has been incorporated in concert with the characteristics of each institution and its students, following proposals made by Koschmann and Stahl (1998). These authors suggest that problems be presented through scenarios, diagrams, dialogue, video, multimedia and other formats (Barret, 2005). Further, the process is developed through four fundamental stages: Recognition, research, report and reflection.
The model applied to the present research combines, at a theoretical level, two of the variants proposed by Savin-Baden (2014). The first, PBL for knowledge management, seeks to develop a student who is competent in the management and resolution of problems in real contexts. Aiming for students to be capable of interpreting and understanding the knowledge behind problems, in addition to its practical application. The second, PBL through activity, is employed in disciplines such as computing and engineering (Booth & White, 2008). It is used to favour student participation in learning and commitment to teamwork. Next, model implementation was conducted in a similar way to the approach taken by Ioannou, Vasiliou and Zaphiris (2016) in multi-modal settings and followed the four stages proposed by Koschmann and Stahl (1998) (see Table 1).
2.3. Perceptions and Influence of PBL within Students of Engineering in Computer Sciences
Students’ attitudes and perceptions towards their experiences of PBL reveal a common consensus. They are seen to favour the understanding of learning content and its application in practice. This is due to the way that PBL helps to develop professional skills, critical thinking and self-directed learning, with a consequent impact on the performance and academic success of students (Hande et al., 2015; Dagyar & Demirel, 2015; Dalfaro et al., 2017; Fernández & Duarte, 2013).
In this sense, experiences are found, such as those reported by De Camargo-Ribeiro (2008), which are oriented towards the analysis of PBL. Students perceive this to be an attractive and interesting way to construct their own knowledge and develop the skills of research, communication, teamwork, problem solving, analysis and information synthesis. Yadav et. al (2011) evaluated the learning perceptions and conceptual understanding regarding employment of a PBL and traditional methodology. These authors found that students receiving PBL perceived and demonstrated better preparation when applying content to problem-solving, whilst also being more capable of transferring this to new situations. In the same way, Jaeger and Adair (2014) analysed the perceptions of engineering students following the introduction of a PBL methodology based on individual situations, motivation and perceived ability for success. Results outlined the need for mediation of the process by a facilitator, students’ individual and collective responsibility for tasks, and the positive influence of one’s individual situation on project quality. Research conducted by McLoone et al. (2016) recorded and analysed student’s attitudes towards the use of project‑oriented PBL. In this case, students rated the methodology to be positive, motivating and an effective way of managing to learn content. These students also stated that the process had led to improvements in communication, developed teamwork skills and prepared them for later professional practice.
Further, it is also useful to highlight that various research studies have outlined benefits associated with teamwork within organisations and businesses (Torrelles, París, Sabrià & Alsinet, 2015), suggesting that competencies associated with this aspect are amongst the most important and in-demand for employers (Barraycoa & Lasaga, 2009). As stated by Torrelles (2011), it is the:
Set of knowledge, skills and attitudes that enables collaboration with other individuals in the performance of activities in order to reach common goals, exchanging information, distributing tasks, taking on responsibilities, resolving difficulties that arise and contributing to collective improvement and development (Torrelles, 2011: page. 209).
A number of studies have made it their aim to evaluate the performance level achieved by students with regards to their development of professional skills through active methodologies (De Miguel, 2006; Schmal, 2015). PBL has been one of the methods to demonstrate the best results in this sense. This is true in relation to examination of its effectiveness for developing teamwork skills, self-directed learning, spoken and written communication, leadership, and other aspects (Fernández & Duarte, 2013; Robledo, Fidalgo, Arias & Álvarez, 2015; Warnock & Mohammadi-Aragh, 2015; Macho-Stadler & Elejalde-García, 2013; McLoone et al., 2016).
Evaluation of teamwork skills in different academic disciplines has largely been conducted through rubrics or scoring guides (Badia & Vila, 2013; París-Mañas, Mas-Torelló & Torrelles-Nadal, 2016; Yarosh, Serbati & Seery, 2017). In engineering, students’ perceptions and attitudes regarding the development of this skill has generally been analysed via questionnaires (Robles-Obando, 2013; Evangelia, Lakiotaki & Matsatsinis, 2014; Warnock & Mohammadi-Aragh, 2015; Macho-Stadler & Elejalde-García, 2013). In all cases, results demonstrate the effectiveness of PBL in this regard.
3. Methodology
This proposal is developed under the design-based research paradigm (Rinaudo & Donolo, 2010; Kennedy-Clark, 2013). It responds to problems detected in educational practice in relation to engineering careers with traditional curricula. It starts from the lack of teaching-learning strategies and methods to enable the development of professional competencies. In an attempt to find a solution to this lacking, an active method was administered in order to meet formulated objectives.
The research design was descriptive and correlations. If took a mixed approach, sequentially integrating data produced by qualitative and quantitative instruments (Creswell, 2014). A quasi-experimental design was specified with development, pre- and post- intervention phases. Non-probabilistic convenience sampling was used. A study control group was recruited (CG) and administered a traditional methodology, whilst an experimental group (EG) received PBL methodology according to the stages defined in the design section (see Figure 1). At the end of the intervention, data associated with academic performance in both groups was examined in order to compare the achievements derived. In the EG, measurements were taken, prior to and following intervention, of perceptions regarding the teamwork engaged in and attitudes towards the use of PBL in the Discrete Mathematics 2 (DM2) course.
Figure 1. Research design stages
The research was conducted within the Engineering and Computer Sciences degree course. All processes were conducted during the second semester of the 2017-2018 academic course. Sample characteristics are presented in Table 2.
Relative to males, a lower proportion of females enrol on degree courses in STEAM subjects (Hirshfield & Koretsky, 2018). This aspect is evidenced by the population distribution of the present sample.
Variable | CG | EG | |
Year of study | 1st | 1st | |
Age range | 18-21 | 18-21 | |
Sex | Female | 13/32.5% | 15/38.5% |
Male | 27/67.5% | 24/61.5% | |
N. total=79 | N = 40 | N = 39 | |
N- number of elements | |||
Table 2. Distribution and characteristics of the present student sample
Academic performance was analysed using Official Records of Evaluation and Control (Registros Oficiales de Evaluación y Control, ROEC) for the degree course for both the EG and CG. Post-intervention, the same objective test was carried out in both groups which evaluated content referring to problem solving in Graph Theory. Figure 2, showed in the following sub-section, presents an example of the type of question asked to evaluate, at an application level, the objective related to the subject under study. This test is targeted towards the evaluation system of the discipline and is established in the analytical program of the degree course. This program is reviewed and updated each semester, and is published on the university’s support platform for students.
The analysis is similar for all students. This grants it objectivity and responds to the use of the tool for determining final exam scores. In this way, a comparison could be made between the acquisition of learning content related to the topic being studied, between the EG who followed PBL methodology and the CG who worked with a traditional methodology.
The median test was employed to examine whether study groups were comparable and to compare the average scores of both groups following intervention. This non-parametric technique is adjusted to the sample size and to the type of data used in the study.
A questionnaire was also used as another instrument. This was administered to evaluate the attitudes and perceptions of students towards PBL (Cuestionario sobre las Actitudes y Percepciones del estudiantado sobre el Aprendizaje Basado en Problemas, CAPABP) (Hande et al., 2015) and was translated into Spanish. This instrument comprises a total of 15 items, distributed according to three domains: Knowledge acquisition, generic skills and attitudes towards PBL. In order to determine construct validity, factor analysis was performed using the maximum likelihood methods of extraction with orthogonal varimax rotation. A value of .735 was obtained for the Kaiser-Meyer-Olkin measure of sampling adequacy and a significance value of .000 for the Bartlett sphericity test. These outcomes permitted us to proceed with further analysis. Further, all variables exhibited commonality values higher than .600. These results confirm the three considered domains, explaining 80.861% of total variance and largely conforming to the initial internal questionnaire structure (χ² = 88.007, with a significance value of .000). Instrument reliability was evaluated by calculating Cronbach alpha values, achieving the values presented in Table 3 (p ≤ .05, with 95% confidence levels also shown).
Domain | No. items. | alfa | M | SD |
Content | 8 | .805 | 4.3750 | .28092 |
Competencies | 5 | .800 | 4.2205 | .59435 |
Attitudes | 3 | .708 | 4.2821 | .55429 |
General | 15* | .909 |
|
|
* Item 12 appears in two domains. | ||||
Table 3. Overall reliability analysis and stratified according to domain of CAPABP (N = 39)
Perceptions of teamwork prior to and following intervention was measured using a teamwork self-report rubric (Yarosh et al., 2017). This differentiated three performance levels for each of the six established indicators.
For qualitative analysis of students’ perceptions in relation to PBL methodology and work conducted using this method, individually developed Wikis were enabled. Emerging perceptions were analysed through content analysis with the software AQUAD 6. In accordance with the type of research, the frequency of code appearances was counted and frequency percentages were analysed. 12 recording units were coded: Oral and written communication, teamwork, acquisition of learning content, collaboration and information exchange, team dynamics, practical application of content, problem solving, skills, self‑learning, applied technologies, problem-based learning and conflict resolution.
In order to facilitate the process of data collection and analysis, the courses learning management system (LMS) was used as an adjunct to the administration of questionnaires, rubrics and Wiki reflections. Quantitative data were analysed through the software SPSS v.22.
3.3. Procedure for the Implementation of PBL
PBL was applied during the second semester of the first year of study on the topic of Graph Theory, belonging to the subject of Discrete Mathematics 2. Graph Theory is based on discrete and applied mathematics, and is born out of concepts that come from diverse knowledge areas such as algebra, arithmetic, probabilities and topology, and combined areas (Diestel, 2005). It was conceived in 1736 when Euler (1736) published the article “Solutio problematis ad geometriam situs pertinentes”. This was directly linked to topology and is considered to be the first theoretical output of the theory. Its theoretical basis was developed, historically, from contributions made by researchers such as Kirchhoff, Guthrie, Cayley, König, Dijkstra and Kuratowski, amongst others. This has directly influenced computer sciences, computing and telecommunication due to the viability that is offers for processes optimization, routing, flowcharts, search algorithms and network analysis (Rosen, 2012). In this way, the study of graphs has become a standard topic in subjects within this branch of study, particularly within the content of subjects from the discipline of Artificial Intelligence (AI). It is integrated into the Engineering in Computer Sciences curriculum as a section of content which corresponds to the Discrete Mathematics 2 module. This forms part of the computational intelligence discipline which includes discrete and applied mathematics and AI.
In the study plan, this topic is made up of two sub-themes: Notions about graphs, and special routing and planarity. The time distribution attributed to study this topic, in both the control and experimental group, can be seen in table 4. Each activity comprises a total of two hours of classes.
| Subtheme 1 | Subtheme 2 |
|
| ||||||
EG | L Rec | PC Res | PC Rep | PC Ref | L Rec | PC Res | PC Rep | PC Ref | Workshop | Objective test |
CG | L | PC | PC | PC | L | PC | PC | PC | Seminar | |
L: Lecture, PC: Practical class, Rec: Recognition, Ref: Reflection, Rep: Report, Res: Research. | ||||||||||
Table 4. Time distribution of the intervention in the experimental (EG) and control group (CG).
3.3.1. PBL Design for the Experimental Group
When designing the PBL delivered to the EG, a number of guidelines established in the consulted literature were considered. This approach is based on the idea that knowledge construction is carried out in a flexible way, through the collaborative working of students when faced with real-world problems and contexts, in this way strengthening meaningful learning development. In this sense, it is assumed that the nucleus of learning activities lies within the problem itself and that the process is not linear. Further, problem generation should not be a simple task, ensuring its relevance, appropriate complexity and appropriateness to real contexts.
PBL was developed according to the four stages of the model proposed by Koschmann and Stahl (1998) and re-asserted by Ioannou et al. (2016): Recognition, research, report and reflection. Executed actions are presented under the first subtheme in Table 1. Analogously, the sequence used when considering the second subtheme was similar, with the exception of the introductory lecture given at the beginning which did not have to be repeated again later. In summary, the following actions were developed in each stage:
1.Recognition: Students developed the general problem analysis and acknowledged learning problems through the lesson imparted and educational resources provided by the tutor. An example of the type of problems to be resolved in PBL can be seen in figure 2.
2.Research: In relation to this theme. Self-directed and collaborative study was conducted of the learning content required to respond to identified learning problems.
3.Report: Here, teams met to apply the information obtained from problem solving. Groups developed and presented potential solutions to problems, analysing and debating the different perspectives relating to the possible solution.
4.Reflection: Here, the state of acquired knowledge, strategies employed, performance of team members and analysis of new problems which could arise were evaluated. Finally, students completed administered rubrics and questionnaires.
Given that it was the first time that a properly structured teamwork dynamic was applied and that this took place with a group of first-year students, support materials were distributed to ensure the appropriate working of teams. The former positively impacted upon team performance and work with the educational resources available on the course platform for learning self-management.
When distributing teams, individual student characteristics were considered, mainly academic performance in the subject up until the point of intervention. Imbalance relating to the learning level of members was avoided. Teams integrated between 3 and 5 students and counted on their lecturer who acted as a facilitator (Prince & Felder, 2006).
The scant or non-existent experience of students in the design and production of educational resources, required consultation materials to be available which served as a guide when elaborating scripts. Classrooms and laboratories were made available for group work sessions. Laboratories were conceived as a viable alternative given certain access difficulties, as it could not be guaranteed that all students had electronic devices with the necessary capabilities for ubiquitous access to information, the internet and resources. To this effect, instructions given around the presentation of deliverable artefacts as evidence of learning were made in consideration of available technology within the reach of those involved.
Figure 2. Problem (learning situation) designed for the PBL in the EG
3.3.2. Methodology Used in the Control Group
The traditional methodology applied within the CG was based on the delivery of theoretical lectures, one for each aforementioned topic, respectively. Six practical classes (PC) were run. Classes were evenly distributed and integrated a seminar presentation which closed the subject course. In relation to the lectures (L), presentational methods were favoured in which the lecturer discussed the learning content that would, in later practical activities, lead to skill development. PC were developed based on initial lecturer orientation of exercises and problems at the beginning of each session, students then working individually to find solutions and, finally, spending time to review the responses given on the whiteboard. Evaluation of whether learning objectives were met was performed by the lecturer. This was conducted using a work checklist, shared with students on the whiteboard or in notebooks, and through systematic written or oral evaluation results.
4. Results
Results of the present research are presented with respect to the three proposed objectives.
Firstly, for the analysis of academic performance a test of homogeneity was applied in order to determine whether study groups were comparable. The result revealed that significant differences did not exist following application of the median test, verifying that the analysis was viable according to a 95% confidence level (theoretical χ2 = 3.841; empirical χ2 > 2.930). Considered hypotheses were H1: Statistically significant differences exist between the EG and the CG; and the null hypothesis, H0: Statistically significant differences do not exist between the EG and CG. The combined median of all elements was Mdn = 3, and it was later verified that the treatment given to the EG through PBL affects student’s academic performance (95% confidence level; df = 1; theoretical χ2 = 3.841; empirical χ2 < 4.250). Table 4 presents the statistical comparison between the EG and CG following administration of the intervention.
| N | M | SD | s2 | |
Statistic | Statistic | Standard error | Statistic | Statistic | |
CG | 40 | 3 | 0.16013 | 1.01274 | 1.026 |
EG | 39 | 3.4615 | 0.16773 | 1.04746 | 1.097 |
valid N (listwise) | 39 | ||||
Table 4. Post-test descriptive statistics
Scores achieved in the objective tests performed following intervention was measured in both groups using a quantitative scale which ranged from 2 – 5 points, with 2 representing the lowest score (fail). The percentage of those who passed, taken as an indicator of academic performance, was 60% and 79.49% in the CG and EG, respectively. This evidences a significant difference between both groups. It was highlighted that the number of students to achieve maximum scores in the EG was double that seen in the CG. This result may be partly attributed to the extent of meaningfulness achieved through PBL when faced with practical real-world problems. In this respect, comments left by students in the course Wiki included “…we are better prepared because we are faced with problems of daily life” (X). In another sense, this difference could also be attributed to an increase in problem solving skills, with this objective being evaluated in the objective tests pertaining to the subject program. In this respect, some students stated “the PBL learning methodology is really good because it helps us to understand the way in which to resolve problems, it gives us skills to know how to tackle the difficulties that could be presented to us in the course of problem solving” (C). Further, it is reasonable to acknowledge that the number of students who failed the course in the CG was also double that of the EG. This is due to the fact that problem solving skills are some of the most complex to acquire in this subject. In this sense, the traditional methodology followed in the CG does not favour the development of skills to resolve problems, whilst this occurs natural in the EG due to PBL (see Figure 3).
Figure 3. Post-test results of the objective tests applied in DM2
Secondly, following examination of students’ attitudes and perceptions regarding the experience of PBL delivery, outcomes from the CAPABP questionnaire were considered. Response rate was 100%, with a total of 39 questionnaires being completed in relation to the EG. There were no missing values. Table 5 shows the mean and standard deviation of questionnaire items, dividing according to domains:
Generally speaking, evaluated items received a satisfactory evaluation. The value of 4 (agree) was established as the median, with the mean being higher than this in all cases. Standard deviation values were between zero and one. This outcome suggests that student’s experiences of PBL were well perceived, with highly positive evaluations being given to the surveyed domains.
Questionnaire items | M (1-5) | SD | |
Content | |||
1 | PBL helps you to better understand the topic being examined. | 4.36 | .668 |
2 | PBL facilitates self-directed learning of students. | 4.31 | .655 |
3 | Various hypotheses can be developed through PBL for a given problem. | 4.46 | .555 |
4 | In PBL it is possible to integrate prior knowledge into the context of the current problem. | 4.36 | .628 |
5 | In PBL it is possible to evaluate information collected in relation to a problem. | 4.36 | .668 |
6 | This form of learning promotes the development of decision-making skills. | 4.33 | .737 |
9 | PBL improves information processing skills | 4.41 | .637 |
15 | You learn to critically analyse the information presented by other group members for discussion. | 4.41 | .595 |
Competencies | |||
7 | PBL teaches you to express ideas to the group in an effective way. | 4.36 | .743 |
8 | PBL provides opportunities for improving leadership skills. | 4.03 | .932 |
10 | It allows you to communicate effectively to the group. | 4.31 | .800 |
11 | PBL helps students to participate without having to always give direction. | 4.18 | .823 |
12 | In PBL you learn to respect the opinions of others within the group. | 4.23 | .667 |
Attitudes | |||
12 | In PBL you learn to respect the opinions of others within the group. | 4.23 | .667 |
13 | Students are able to identify their ethical and moral obligations with other group members. | 4.28 | .793 |
14 | You become aware of personal limitations while operating in an PBL group. | 4.33 | .621 |
Valid N (listwise) = 39 | |||
Table 5. CAPABP results (39 valid responses from the EG)
Item 3 is rated between 4 (agree) and 5 (strongly agree) by 97.4% of students. This could be due to the high feasibility of carrying out the analysis process from different perspectives, counting on diverse sources of information and educational resources made available by the tutor/lecturer. Further, the process is characterised by sharing spaces, the exchange of ideas and information, and making these ideas common knowledge amongst group members. This contrasts with the traditional methodology currently followed in this subject. In this case, work is not considered in the same collaborative way. This leads to the proliferation of one-dimensional thinking in students, with a limited view of problem-solving alternatives. In this domain, the least highly rated item was item 2 (94.9% of students reported scores of 4 or 5), although values were still high and were greater than the mean. Nonetheless, this result denotes that some students, despite positively rating the analysis and information processing in PBL, do not demonstrate cognitive Independence and, as a consequence, do not consider themselves competent to manage their own learning content.
In the second domain, evaluations of PBL are outlined as an alternative for favouring development of communication skills (items 7 and 10), with 89.7% and 84.6% of students, respectively, reporting scores of 4 and 5. Collaboration and information exchange with the tutor, team members and other classmates via the various established channels, enables development of fluid communication processes which use a number of the senses. The social presence of the tutor is essential for removing the barriers established for various reasons. In this respect, students indicate “…it helps us to a great extent to strengthen oral and written expression as during development of the issue at hand we are obliged to investigate different types of places, with different resources, with the aim of ensuring that work objectives are reached” (S). Determined activities exist through which development of this competence is favoured. Such activities include the elaboration of artefacts as learning evidence for the evaluation of suggested solutions for problems, proposed interview script guides and the design of educational resources in the established way. The lowest score achieved was for the item associated with leadership (item 8). This could be linked to the fact that some students still stick to the false belief that the leader is the person who takes on most of the responsibility for tasks because they are more prepared, responsible and committed to their personal results. As a result, they show themselves to be resistant to taking on the role of leader, pointing to their own supposed inability and preparation. Subsequently, they limit their engagement to presenting the results found by the leader and fail to find a route within the chosen method to overcome their limitations. In the final domain, item 14 stands out (92.3% of students reported scores of 4 or 5). The team dynamic followed and the assumption of diverse roles permit students to self-evaluate their capacity for the performance of various tasks. This enables them to recognise their personal challenges and draw up a plan to address them, with this being done in function of the achievement of both individual and common team goals. On the other hand, the lowest score, whilst also being higher than the mean, was obtained for item 12. This could be associated to the fact that some students consider that their opinions are not respected nor considered during the process of solving problems, thus, they appear to be considered irrelevant in analysis or lacking in depth and rigour.
In a general way, the domains of content and competencies are highly positively correlated (ρ = .816). Nonetheless, the most positive scores are found for the former, whilst the lowest score (although also highly positive) is given for the second domain. This situation could be due to the fact that problem solving is a transversal skill developed from primary education using traditional methodologies. The novelty of teamwork via PBL, following a correctly structured dynamic, favourably impacts upon content acquisition. This makes qualitative changes in the observed indicators perceptible. However, it is difficult to administer this given that the development of competencies in previous teachings has never previously been the object of analysis and it does not have a clear and specific system of performance indicators, nor does it count on the pathways or strategies to evaluate them. Administration is even more challenging when the teaching process is centred on the student and formative evaluations are performed which demand individual and group self-evaluations, processes with which students are not familiar with.
Given the size of the EG (N = 39), the Shapiro-Wilk normality test was applied. This indicated that data did not follow assumptions of normality (statistics between .700 and .800, dr = 39 and p < 0.05). Following this, the non-parametric Spearman Rho test was applied to analyse correlations between grouped questionnaire domains. This obtained values 0.7< (ρ < 0.89) in all cases, denoting high positive associations.
Bivariate correlations (see Table 6) underline the values achieved between items 7 and 8 which are associated with communication (ρ = .715). A highly positive correlation between both items is evidenced. This may be attributed to the importance of the communication and expression of ideas throughout the entire process. This is crucial for collaboration, debate and information exchange regarding the research findings uncovered from self-directed learning and, finally, for the presentation of proposed solutions to learning situations for measuring available technologies.
As another aspect, the existing relationship between items 14 and 15 (ρ = .705) stands out, being equally highly positive as the aforementioned correlation. From this is can be understood that students become aware and acquire skills to make evaluative judgements about their actions and those of their course mates. This is facilitated by the prior establishment of specific performance indicators for each activity. In a similar way, a moderately positive correlation was found between the items of 5 and 7 (ρ = .686). This may be largely attributed to the search for, analysis and discussion of research findings, fit of solutions to problems, and the debate developed between the students and the rest of their work team. The plenary session for presenting results will also play a role in establishing this association.
.013* |
|
|
|
|
|
|
|
|
|
|
|
|
| |
.936** |
|
|
|
|
|
|
|
|
|
|
|
|
| |
C03 | .265 | .449 |
|
|
|
|