CRITICAL THINKING SKILLS OF CHEMISTRY EDUCATION STUDENTS IN TEAM PROJECT-BASED STEM-METACOGNITIVE SKILLS LEARNING DURING THE COVID-19 PANDEMIC

Critical thinking skills has to be sharpened particularly during the COVID-19 pandemic. This circumstance leads to the lack of student’s passion to apply their thinking skills in doing something necessary. Therefore, it requires a learning improving critical thinking skills. This study aims to describe the critical thinking skills of chemistry education students taking part in the lecture of Team Project Based STEM-metacognitive skills. The method utilized is a Pre-Experimental Design One-Shot Case Study consists of 130 chemistry education students taking part in 3rd and 5th semester. The 3rd semester students attend lectures on the development of chemistry learning programs (P3K) and the 5th semester students attend lectures on the basics of analytical chemistry (DDKA). The instruments utilized in this research are learning scenarios, worksheets, critical thinking test instruments, critical thinking skill assessment rubrics, and project implementation feasibility observation sheets. There are three critical thinking skills very well demonstrated by students in P3K and DDKA subjects, consist of (1) strategies and tactics, (2) providing simple explanations, and (3) providing further explanations. Two other skills, namely concluding and building basic skills shown by students in the good category. The results of this study contribute to chemistry learning in the future, thus efforts are inevitably required to train critical thinking skills for prospective chemistry teachers in order to produce teachers having good thinking skills.

Team Project-Based STEM-metacognitive skills learning is a project-oriented learning that is performed in groups in which metacognitive skills are used and integrated in STEM. This learning is based on several studies (Guo, Saab, Post & Admiraal, 2020), showing that a project-based learning model can improve the quality of learning in higher education. This learning model offers an opportunity to students to participate in real problem solving and knowledge construction which require students to deal with real problems and questions that are relevant to the learning topic (Milentijevic, Ciric & Vojinovic, 2008). The results of another study (Cifrian, Andrés, Galán & Viguri, 2020) showed that during learning, students who work on a project are able to perform self-monitoring and self-management effectively. In addition, it is possible to independently create active communication between students or between students and lecturers, as well as to create a conducive learning atmosphere (Requies, Agirre, Barrio & Graells, 2018). Involving students in real-life projects could train students to actively learn as well as demonstrate critical thinking, science process skill, problem-solving skills, increase students' interests, experiences, and participation in learning, and boost motivation and problem-solving skills (Apriwanda, Mahanan, Ibrahim, Surif, Osman & Bunyamin, 2021;Chiang & Lee, 2016;Rambocas & Sastry, 2017;Uziak, 2016). STEM was first coined by the National Science Foundation (NSF) in 1990 as an acronym for Science, Technology, Engineering, and Mathematics, but the concept started to gain popularity in 2003 (Yllana-Prieto, Jeong & González-Gómez, 2021). Science is the systematic study of the nature and behavior of material and physical universe, based on observations, experiments, and measurements, and the formulation of laws to describe facts in general (White, 2014). Technology is as human innovations that are utilized to modify the nature in a way that it can meet human needs and desires. Engineering is defined as the knowledge and skills to acquire and apply scientific knowledge. Meanwhile, mathematics is a group of related sciences, including algebra, geometry, calculus, numbers, size, shape, and space as well as their correlation using special notations. Several previous studies have revealed that learning using STEM approaches can improve academic learning processes and outcomes, problem solving skills, critical thinking skills, collaborative thinking skills, and integrity (Perignat & Katz-Buonincontro, 2019). Besides, STEM can also promote students' understanding, creativity, and problem-solving skills, foster creative thinking, creative skills, creative processes, innovation, and developing both hard and soft skills (Bequette & Bequette, 2012;Glass & Wilson, 2016;Herro & Quigley, 2016;Preciado-Babb, Takeuchi, Yáñez, Francis, Gereluk & Friesen, 2016;Sharapan, 2012;Yakman & Lee, 2012).
Metacognitive skills are the skills of a person that control his/her ability to organize, monitor, and re-examine his/her understanding and actions in problem solving (Ijirana & Supriadi, 2018). In relation to metacognitive skills, there are three regulation skills that play an important role in organizing the problem-solving learning process of students, namely planning, monitoring, and evaluation skills (Schraw, 2001). Planning skills are related to selecting appropriate strategies and resource allocation that may affect the performance of a person. Performance management and monitoring skills direct students to an aware-ness of understanding the tasks that they should deal with. Several studies have shown the relationship between metacognitive skills and critical thinking skills, showing that there is a relationship between metacognitive skills with critical thinking skills (Magno, 2010;Malahayati, Corebima & Zubaidah, 2015), and that metacognitive skills can develop students' independent learning and critical thinking (Shabani & Mohammadian, 2014), and that people with good critical thinking skills are engaged in many metacognitive activities, especially high-level planning and high-level evaluation strategies (Ku & Ho, 2010). Based on the abovementioned results of previous studies, there is a relationship between team project-based STEM-metacognitive skills learning with critical thinking skills. Thus, it is crucial to conduct a study that answers questions regarding students' critical thinking skills in team project-based STEM-metacognitive skills learning

Method Research
This was a Pre-Experimental study with One-Shot Case Study design. All the sample groups were given the same treatment and the results were observed. The treatment was in the form of team project-based STEM-metacognitive skills learning as the independent variable and critical thinking skills as the dependent variable (Creswell, 2014). The subjects consisted of 130 chemistry education students at Tadulako University, in which 63 of them were third semester students and the remaining 67 were fifth semester students. Thus, the number of the samples was the same as the number of the population. These students were taught using the same learning model in different courses, namely the basics of analytical chemistry (DDKA) and chemistry learning development (P3K) courses. The research flowchart is shown in Figure 1.
The project was implemented by giving structured assignments in groups and one semester independent learning. The Team Project-Based STEM-Metacognitive Skills Learning was done in four main steps, namely: 1) reflection, 2) research, 3) discovery, 4) application and communication. The detailed learning activities of the Team Project-Based STEM-Metacognitive Skills learning can be seen in Table 1.

Learning Steps/Facilities Lecturer and Student Activities Schedule
Reflection (synchronous/zoom) 1. Conducting a course contract with the students and discussing all courserelated matters/rules. 2. Giving the descriptions of the activities to be done in the learning for 16 weeks. 3. Giving the descriptions of the learning outcomes, the materials, the schedule for synchronous/asynchronous activities. 4. Assigning problems to each group in the Worksheet (LK) and giving motivation for problem solving. 1. Each of the student groups applied the results of their designs (practicum for the basics of analytical chemistry course and teaching for the P3K course), assessed the results of their designs and correlated the designs with other sciences. The students made a video and uploaded it on Youtube. 2. The student groups presented their products in turn. The lecturers provided reinforcement/direction/reward, measured competency achievement, assessed the students' assingments on the worksheets, and measured the results of other groups.
3. The students continued the discussion through the https://lms.fkip.untad.ac.id/ discussion forum to reinforce learning outcomes 4. Evaluating the results and giving reinforcement. 10 th to 16 th meetings Table 1. Team project-based STEM-metacognitive skills learning activities The instruments used in this research consisted of learning scenarios, worksheets, critical thinking skills test instruments, critical thinking skills assessment rubrics, assessment sheets of the feasibility of the project implementation results that were observed through video because learning was done online due to the covid-19 pandemic, and student questionnaires which contain questions and statements on both courses. All the research instruments were validated by several experts from the university and from other universities. Valid instruments were then used for the data collection. The critical thinking skills assessment rubrics for each indicator can be seen in Table 2, while the critical thinking skills were calculated using the following formula: The data obtained were then analyzed descriptively quantitatively using the rubrics and criteria as shown in Table 2 and Table 3. Meanwhile, the data obtained from the student questionnaires were analyzed descriptively qualitatively.

Critical thinking skills indicators Assessment criteria Score
Strategies and tactics/determining actions Identifying problems, deciding on appropriate actions based on the students' conditions and logical solutions to problems, and drawing conclusions 4 Writing down the problems, deciding on appropriate actions based on the students' conditions and logical solutions to problems 3 Writing down the problems, deciding on appropriate actions based on the students' conditions 2 Writing down the problems 1 No response 0 Developing basic skills/observing Writing down the problems in the form of questions, mechanisms of making an authentic assessment, problem-solving strategies, and drawing conclusions 4 Writing down the problems in the form of questions, mechanisms of making an authentic assessment, problem-solving strategies, and drawing conclusions 3 The students have the ability to write down the problems in the form of questions and the mechanisms of making an authentic assessment 2 The students have the ability to write down the problems in the form of questions Estimation-based decision making, selecting appropriate methods 2 Estimation-based decision making 1 No response 0 Table 3. Critical thinking skills assessment rubric in team project-based STEM-metacognitive skills learning (Ennis & Weir, 1985)

Results and Discussion
Team Project-Based STEM-Metacognitive Skills Learning was done in four main steps, namely: 1) reflection, 2) research, 3) discovery, 4) application and communication (Laboy-Rush, 2010). In the first step, the lecturer gave a project theme to each group, for those taking the P3K course, the theme was the design of innovative learning instrument by integrating TPACK and HOTS, according to the basic competencies in each class based on the 2013 high school curriculum. Meanwhile, the theme for the DDKA course was a qualitative analysis of elements, ions, and compounds in a sample. The two courses were taught by different lecturers, and the themes of the project were given in the student worksheet. The projects developed by students in the P3K course are high school learning instruments in the form of documents of lesson plans, worksheets, teaching materials, learning media, and assessment instruments. Each group was given different projects. Meanwhile, the projects developed by the students taking the DDKA course were designing, implementing, and reporting the data based on the results of flame tests, cation tests, anion tests, as well as experiment on preservatives and dyes using materials that could be found in the surrounding environment. In this step, the students started to explore some ideas relevant to the projects. In the second step, each of the students in the groups searched the literature to help them gain more understanding about their projects, determined the learning context in which they started to understand the given tasks, and made observations of the surrounding environment to look for materials and tools that could be used for the completion of the projects. The results obtained were then discussed in the groups to determine the next steps to complete the project. In the third step, students make plans, develop learning tools done by students in P3K courses, make experimental designs done by students in DDKA courses, and conduct asynchronous group discussions. Finally, in the fourth step, the activities consisted of monitoring the design results through synchronous class discussions, giving recommendations on the design results, applying the design results, documenting its implementation in the form of a video uploaded on YouTube, and conducting synchronous evaluation. One of the project results reported by students in the form of a video is on the link https://drive.google.com/file/d/1XfkPR-Q45MrZtyCI0vxVOoG92VhmcmdZ/view?usp=sharing for DDKA courses and https://youtu.be/tFObwCKRbSc for P3K courses and is shown in Figures 2 and 3. There were two reasons behind the implementation of this activity. First, due to the Covid-19 pandemic, the lecturers were unable to directly see the implementation of the project activities. The lecturers were only able to make assessments through the video displayed. Second, due to the need to train students to improve their technology skills. Based on the results of the assessment, another class discussion was held to give feedback, direction, and rewards.

Assessment of Students' Critical Thinking Skills
The critical thinking skills of the chemistry education students who took the P3K course and the DDKA course were assessed using the Team Project-Based STEM-Metacognitive Skills learning. The students' critical thinking skills were assessed based on the assessment rubrics as shown in Based on the data shown in Tables 4 and 5, the following is the results of the statistical tests to determine whether there was a significant difference in the critical thinking skills of the students who took the P3K and DDKA courses, using the following hypotheses.
H 0 : there is no difference in the critical thinking skills of chemistry education students in classes A and B H 1 : there is a difference in the critical thinking skills of chemistry education students in class A and B The results of the statistical tests using SPSS are presented in Table 7 for P3K and Table 9 for DDKA, while the results of the normality tests are shown in Tables 6 and 8 Table 9. T-test result of critical thinking skills of chemistry education students in class A and B of DDKA course Based on the results shown in Tables 6 and 8, the data of the critical thinking skills of the students who took the P3K and DDKA courses were normal, so t-test was conducted with a significance (p) > 0.05. Based on the results of the statistical tests, both class A and class B of the P3K and DDKA courses had the same critical thinking skills in the team project-based stem-metacognitive skills learning. In addition, the data in Table 4 and 5 show that the critical thinking skills of the students in the P3K and DDKA courses based on the critical thinking indicators fell in the excellent and good categories. On average, the students were excellent at formulating strategies and tactics as well as giving elementary and advanced clarifications, and they were good at developing basic skills and inferring. In general, findings in this research indicate that learning with the Team Project-Based STEM-Metacognitive Skills applied asynchronously via https://lms.fkip.untad.ac.id and synchronously via zoom during the covid-19 pandemic can improve students' critical thinking skills. Because students can access subject matter widely and discuss it with lecturers and other students in their groups through LMS. Students complete the assignments in groups but are self-reliant, so it forces them to develop strategies, tactics, and skills to make decisions in problem-solving. These findings are relatively the same as other previous studies (Ariawan, 2020) that revealed blended learning during the pandemic is good in accommodating students' critical thinking. Because students can freely access study materials that are available online, and discuss them with teachers and other students outside of class hours.

Ability to Formulate Strategies and Tactics
For this indicator, the results showed that the students were categorized as good in terms of formulating strategies and tactics for problem solving. In fact, 60% of the students who took the P3K course and 90% of those who took the DDKA course were good at formulating tactics and strategies. This means that most chemistry education students had the ability to identify problems as well as decide on appropriate actions and logical solutions to problems in both class A and class B when they were given the problems, i.e., chemistry learning at schools for the students taking the P3K course and problems related to flame test, cation test, anion test, preservatives, and dyes for those taking the DDKA course. On the other hand, in terms of the ability to make inference, only a small number of students had this ability. Based on the data, it can be said that the students' ability to identify problem is an initial step after an analysis of the problem statement. This ability serves as the basis for students to be able to determine which action should be taken if they find a problem and how to solve it. This finding is supported as stated in (Ku, 2009) that analysis skills are the skills of breaking down a structure into components to help a person know the organization of the structure. This aims to enable this person to understand a global concept by describing or breaking down the global concept into more detailed components.
Several opinions that are in line with this finding were given by, as stated in (Sosu, 2013) critical thinking skills dimension implies the ability to understand problems and develop solutions to the problem, such as analysis, interpretation, and drawing conclusions (Chan, 2019). The dispositional dimension refers to the willingness to apply these skills when encountering a problem and finding a solution to it or when faced with decision making (Álvarez-Huerta, Muela& Larrea, 2022). Ref (Sosu, 2013) also argued that there are two dispositional dimensions of critical thinking, namely critical openness and reflective skepticism. Critical openness is a reflection of the tendency to be open to new ideas, to critically assess them, and to be prepared to change someone's views. This study also found that the students demonstrated a tendency to openness to new ideas by providing an alternative action and the best solution to chemistry learning problems.

Ability to Provide Elementary Clarification and Advanced Clarification
The data in Tables 4 and 5 also show that, in general, the students who learned using the team project-based STEM-metacognitive skills learning demonstrated excellent ability in providing elementary and advanced clarifications in both P3K and DDKA courses. A total of 60% of the chemistry education students who took the P3K course had the ability to provide elementary clarification with an average score of 82.8 and 90% of them had the ability to give advanced clarification with an average score of 80.3. In the DDKA course, 77% of the students had the ability to provide elementary clarification with an average score of 82.5 and 87% of them had the ability to provide advanced clarification with an average score of 76.5. This implies that the students who took the course had the ability for assumption-based decision making and the ability to determine appropriate learning media and methods, but they did not have the ability to assess and give a satisfying explanation of the media or methods used. Similarly, in terms of the ability to formulate tactics and strategies, the students who had the ability to decide on actions and solutions to a problem also had an excellent ability for decision making related to the selection of learning methods. This way, it can be said that a person with critical thinking will train him/herself to be creative and productive to properly direct his/her mind. In other words, a person who has critical thinking skills has an awareness of making the right decision whenever encountering a problem. Critical thinking is a higher order thinking which includes having awareness, having inquiry skills, making judgments, conducting evaluation, being open minded, and having the ability to use oral and written language effectively (İşlek & Hürsen, 2014). Critical thinking also serves as a self-regulatory assessment to produce interpretations, analysis, evaluation, and inference, as well as clarifications of evidential, conceptual, methodological, criteriological, or contextual considerations that helps students have initiative in thinking, make self-evaluation based on the assessments of learned knowledge, accuracy, processes, theories, methods, backgrounds, and arguments, resulting in logical decision-making about what they do and what they believe (Hart, Da Costa, D'Souza, Kimpton & Ljbusic, 2021;Qing, Ni & Hong, 2010). Therefore, it is crucial to develop these skills continuously through this learning.

Ability to Develop Basic Skills and Make Inferences
In fact, 38% of the students who took the P3K course had the ability to develop basic skills and 37% of them had the ability to make inference, while among the chemistry education students who took the DDKA course, only 47% of them had the ability to develop basic skills and 25% had the ability to make inference through the team project-based STEM-metacognitive skills learning. This indicates that only a small number of students had the ability to write down problems in the form of questions, make authentic assessments, formulate problem-solving strategies, and make inference on the problem-solving although they were taught using the team project-based STEM-metacognitive skills learning model. This study supports the findings of (Niu, Behar-Horenstein & Garvan, 2013) that there is an effect of critical thinking learning on students, but the effect is insignificant, i.e., only around 0.20 standard deviation of score improvement using a standard critical thinking test. In other words, a particular treatment given to develop students' critical thinking skills does not always have the same effect for all the critical thinking skills indicators. Similarly, other previous studies (Qing et al., 2010) on the effect of the application of task-based learning on critical thinking skills also showed the same results. Such teaching approaches provide an effective way for teachers to develop students' critical thinking disposition, but students do not achieve higher order thinking, making it not significantly different in terms of the sub-scales of systematicity, open-mindedness, truth-seeking, analyticity, inquisitiveness, and maturity. However, another study (Nugraha, Suyitno & Susilaningsih, 2017) revealed that critical thinking skills are significantly related to science process skills and student learning motivation, and this was done in a study that applied SjBL learning to measure students' critical and creative thinking skills (Rusmini, Suyono & Agustini, 2021). Therefore, it is important to conduct further research to analyze the development of critical and creative thinking skills using a problem-solving-based learning with a project problem-solving approach in advanced chemistry learning.

Conclusion
The regulation of the Indonesia Minister of Education and Culture concerning emphasizing team project-based learning in higher education serves as the basis of conducting this study. This study aims to describe the critical thinking skills of students in the chemistry education program in Team Project-Based STEM-metacognitive skills learning. This study found that, in chemistry learning including the field of pure chemistry and chemistry education, it is beneficial to apply STEM-based project that is done in groups. Because these two fields are one of the graduate learning outcomes of the chemistry education program, especially at Tadulako University, i.e., mastering both pure chemistry and chemistry education to be a qualified prospective chemistry school teacher. Although this learning was carried out during the Covid-19 pandemic and carried out online by combining asynchronous and synchronous, it turned out to help students develop their critical thinking skills in completing projects. In addition, developing critical thinking skills is also a goal of education in Indonesia in general. The critical thinking skills indicators which fell in the excellent category are student's ability to formulate strategies and tactics as well as provide elementary and advanced clarifications, while the indicators with a good category are the ability to develop basic skills and to make inferences. One of the limitations in applying this learning model is that the learning is all done online, which potentially prevents achieving optimal learning, which then affects the results. Therefore, it is necessary to conduct a study using team project-based STEM-metacognitive skills learning by Blended-Flipped Learning. In addition, it is also very compelling to test students' self-regulation using this learning because it is closely related to metacognitive skills and it is also a goal of the Indonesian Law concerning National Education System.

Declaration of Conflicting Interest
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The research was supported by DIPA research funding in 2021, Tadulako University, Indonesia.