Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 9, No 3 (2019) > Suwanroj

CONFIRMATORY FACTOR ANALYSIS OF THE ESSENTIAL DIGITAL COMPETENCIES FOR UNDERGRADUATE STUDENTS IN THAI HIGHER EDUCATION INSTITUTIONS

Department of Industrial Education, Faculty of Industrial Education and Technology King Mongkut’s Institute of Technology Ladkrabang (Thailand)

Received January 2019

Accepted March 2019

Abstract

The purpose of this descriptive study was to apply 2nd order confirmatory factor analysis (CFA) and structural relationship models to identify the digital competency components essential to undergraduate students in Thai higher education institutions. The sample comprised 1,126 specialists in Information Technology, Computer Technology, Computer Education, Computer Science, and Computer Engineering working in public higher education instructions throughout the country. The selection was the result of multi-stage random sampling from 76 public higher education instructions that offer undergraduate education. The instrument was a questionnaire form on essential digital competency components for undergraduate students in higher education institutions. The question items employed a 7-point Likert scale and showed Cronbach’s alpha values for the content validity and reliability at a range of.93-.97 per domain and .87-.99 per component. The data were analyzed using descriptive statistics for general data and 2nd Order CFA analysis. The findings revealed that from 24 observed variables, there were 7 competency components.: 1) Fundamental of digital; 2) Accessing digital information; 3) Using digital information; 4) Creating digital information and media; 5) Communicating digital information; 6) Managing digital information; and 7) Evaluating digital information. The discovery from this study was substantially constructive for Thai higher education institutions as it could be used to design an essential digital competency framework of the 21st century.

 

Keywords – Confirmatory factor analysis: CFA, Digital competency, Undergraduate students, Thai higher education institution.

To cite this article:

Suwanroj, T., Leekitchwatana, P., & Pimdee, P. (2019). Confirmatory factor analysis of the essential digital competencies for undergraduate students in Thai higher education institutions. Journal of Technology and Science Education, 9(3), 340-356. https://doi.org/10.3926/jotse.645

 

----------

1. Introduction

The rapid transformation of the global economy and technology has pressured many countries to accelerate developments aiming to catch up with the dynamics. Since the civic capacity of each nation can directly influence the global economy and technology, many countries have begun issuing policies to enhance it. Countries in Europe and the Americas, for instance, began enhancing their civic capacities and labor competencies, especially in digital technology. Similar movements have occurred in Asian countries such as China, Japan, Singapore, Malaysia, and Thailand as well (Burbules, 2018; Inglehart, 2018; Shannon, 2018). Many governments are developing policies and prioritizing the development of their civic capacities. In line with others, the Thai government has been trying to pursue the very goal. Policies had been issued to drive the nation toward under a scheme known as Thailand 4.0 (Jones & Pimdee, 2017; Skeldon, 2018). Simply put, civic competencies digital technology are being enhanced in response to Thailand 4.0 to ensure that the national capacity is sufficiently improved with adequate competitive edges against other nations. Education was given priority for being the key starter of a long-term effect. The fulfill the goal, education management enhancing digital competencies became the prime focus, and higher education institutions were selected as the main starting point of the development (Akarawang, Kidrakran & Nuangchalerm, 2015; Arif, Fleischer, Bogiatzis, Asada, Colombo & Zougbédé, 2018) based on the grounds that they are the scholar-producing institutions that feed workers into various industries (Inglehart, 2018; Kosaiyapattanapundit & Sangthong, 2018; Shyshkina, 2018). For that, it is justified that these institutions should be the focus and the first step for digital competency enhancement (Kasemsap, 2018; Shyshkina, 2018; Turan & Göktaş, 2018).

The higher-level education management for digital competency enhancement in Thailand has been ongoing since 2010 (Chang, 2009; Office of the Higher Education Commission, 2013; Rodmunkong & Wannapiroon, 2013; Rodmunkong, Wannapiroon & Nilsook, 2015; Tantirattanawongse, 2009) purposely to prepare graduates for the industrial sector and equip them with the useful skills to handle everyday work task. It is believed that once specific digital competencies of higher-education students are extensively developed, they can be valuable to the students themselves both while studying and after graduation when they work in business establishments. Considering the significance of the cause, the courses are, therefore, among the most important influencing factors for quality outcomes. Moreover, the concept is in line with the goal that higher education institutions which is to provide appropriate education management and competency enhancement for students. As a result, many institutions have begun to develop short-term courses for the enhancement of essential competencies for undergraduate students and the implementation started with short training sessions (Freud, 2018; Hedge, 2001; Holmes, 2018; Marian Puscas, Sangineto, Culibrk & Sebe, 2015; Phanchalaem, Sujiva & Tangdhanakanond, 2016) provided to improve digital competencies deemed essential for the students. (Akarawang et al., 2015; Kasemsap, 2018; Nosenko, Shyshkina & Oleksiuk, 2018; Roşeanu & Drugaş, 2011; Shyshkina, 2018; Szabo & Marian, 2012; Trip, Vernon & McMahon, 2007)

To strengthen the effort, many education institutions in Thailand came together to formulate a fundamental framework for digital competencies. Nonetheless, it was not clear, in detail, what essential components there should be for the graduate level as each higher-education institution tends to have its unique conditions and missions of existence. Consequently, the design of such a framework was far from success. No research in Thailand was found tackling on and analyzing essential digital competencies for higher-education students using confirmatory factor analysis. Hence, the researcher, as a Computer-Education lecturer in a higher-education institution, took a keen interest in examining and analyzing essential digital competency components so that the findings would be further utilized as a competency model shaping short-term training curricula on essential digital competencies for undergraduate students in Thai higher education institutions. This study aimed to apply 2nd Order CFA to identify the digital competency components essential to undergraduate students in Thai higher education institutions and mark a starting point for such problem-solving. Furthermore, this study also compared between the studies of T. Marusic and I. Viskovic (Marusic & Viskovic, 2018) and that of Cha, Jun, Kwon, Kim, Kim, Kim, et al. (Cha, Jun, Kwon, Kim, Kim, Kim et al., 2018; Suwanroj, Leekitchwatana & Pimdee, 2017; Suwanroj, Leekitchwatana, Pimdee, Thiyaporn & Thanongsak, 2018) as they are different in the use of 2nd Order CFA for component identification and the observed variables for the essential digital competencies for undergraduates. Moreover, a literature review was conducted to examine domestic documents on digital competences and the findings revealed that only extremely limited number of such studies in Thailand discuss definitions and digital competency components for higher-education students. The examination indicated that the existing studies only tackled on skills, understanding, and utilization of digital technology. For instance, Jongsermtrakoom and Nasongkhla (2015) discusses that digital teaching is the ability to use digital materials, employ skills to gain access, evaluate to manage, and integrate to produce communication. Later, Phuapan, Viriyavejakul, and Pimdee (2016) formulated six factors of digital media literacy for Thai students (i.e., access, management, integration, assessment, creation, and communication) using 19 indicators for its model. Recently, Wei, Piaw, and Kannan (2017) reported the development of digital literacy indicators for Thai undergraduate students using mixed method research with a digital-literacy-oriented emphasis. None of the mentioned studies has employed CFA to develop digital competencies for Thai higher-education students, and hence, this study was designed to do so using 2nd Order CFA so that the findings would further be applied in short-term training courses on digital competencies for Thai students.

2. Literature Review

2.1. Competency

It is fair to describe “competency” as a behavioral characteristic one gains out of knowledge and skill (Bora, 2017; Brata & Pemayun, 2018; David, 1973; McRoy, Oglesby & Grape, 2018; Phanchalaem et al., 2016) and demonstrated through a particular action with a level of mastery (Bora, 2017; David, 1973; Jaradat, Keating & Bradley, 2018). Competency must primarily be constituted by both knowledge and skill. For instance, when one conducts an internet search, it is necessary for one to have the right computer knowledge and skill to conduct the searching. Also, secondary latent factors, e.g., calmness; patience; and diligence may play roles reinforcing the person to reach the goal more effectively. These exemplified secondary factors are known as mental features which may help shape the success and speed of the internet search action in conjunction with the knowledge and skill. Nevertheless, the reason they are called secondary is that it might not always be the case (Bora, 2017; Dubois, 1993; Jaradat et al., 2018).

As discussed, it is safe to conclude that competency is a behavioral characteristic, mainly resulted from knowledge and skill and probably from other supplementary mental features, that increases a person’s capability to achieve a goal or purpose as desired. Furthermore, some studies (Leekitchwatana, 2017; Marusic & Viskovic, 2018; Nosenko et al., 2018; Suwanroj et al., 2017; Suwanroj et al., 2018) confirmed the notion that one having a single-dimension capacity, i.e., having either the knowledge or skill; does not interpret as one having a competency. A competency must consist of both capacities: knowledge and skill whereas supplementary or latent mental features may play supportive roles as necessary to shape this competency (Gil-Flores, Rodríguez-Santero & Torres-Gordillo, 2017; Tondeur, Aesaert, Pynoo, van Braak, Fraeyman & Erstad, 2017; Tondeur, Van Braak & Valcke, 2007; Voogt, Knezek, Christensen, Lai, Pratt, Albion et al., 2017). Developing competencies is, therefore, suitable with education that utilizes a fair amount of time, e.g., an enrollment in a curriculum for a designed duration with simultaneous mental enrichment activities. Short-term courses or courses with limited learning time such as short training or supplementary teaching may fail to address such mental enrichment (Leekitchwatana, 2017; Marusic & Viskovic, 2018; Nosenko et al., 2018; Suwanroj et al., 2017; Suwanroj et al., 2018; Tondeur et al., 2007).

2.2. Digital Competency

“Digital competency” is a behavior constituted by the cognitive (knowledge) and psychomotor (skill) domains of learning latent in individuals. Competencies help such individuals to successfully operate the targeted information communication technology (ICT) (Camelo, Torres, Reche & Costa, 2018) and computer technology tasks. Cognitive domain refers to the knowledge, news, and understanding of general and specific matters an individual acquired provided that it can be further utilized in real-life practices. Psychomotor domain refers to the ability to perform tasks that require fair expertise, training, or practice (Cha et al., 2018; Gil-Flores et al., 2017; Goodman, 1987; Suwanroj et al., 2017; Suwanroj et al., 2018). The definitions are consistent with research frameworks from many studies (Krasnova & Shurygin, 2017; Leekitchwatana, 2017; Suwanroj et al., 2017; Suwanroj et al., 2018; Tondeur et al., 2017; Tondeur et al., 2007; Wei et al., 2017) which aimed to examine and develop essential digital competencies for undergraduate students through emphasizing the cognitive and psychomotor domains. Based on the concepts of Tondeur et al. (2017), Gil-Flores et al. (2017), Voogt et al. (2017), Suwanroj et al. (2017), Suwanroj et al. (2018), and Phuapan et al. (2016), the digital competency components comprise 1) Fundamental of digital ; 2) Accessing digital information; 3) Using digital information; 4) Creating digital information and media; 5) Communicating digital information; 6) Managing digital information; and 7) Evaluating digital information.(Gil-Flores et al., 2017; Leekitchwatana, 2017; Krasnova & Shurygin, 2017; Suwanroj et al., 2017; Suwanroj et al., 2018). Moreover, Table 1 illustrates a summary of the digital competency frameworks in geographic categories.

According to Table 1, digital competency components vary significantly across geographic contexts. Although digital competency is an established term for countries in the Americas and Europe, it remains contrarily and considerably new to Southeast Asia, especially in Thailand. Not using English as a primary language is one of the core reasons for the delayed development of digital competencies as the computer use and data access are limited due to inadequate English proficiency.

Geographical context	Defined framework of digital competence
1. America (American Library Association, 2013)	1) Find 2) Evaluate 3) Create and 4) Communicate
2. Europe (European Commission, 2019)	1) Information and data literacy
	2) Communication and collaboration
	3) Digital content creation
	4) Safety
	5) Problem solving
3. Southeast Asian (ASEAN, 2018)	1) Access 2) Manage 3) Integrate 4) Evaluate and 5) Create

Table 1. Geographic-based formulations of digital competency frameworks

2.3. Digital and ICT Situation in Thai Higher Education Institutions

World Economic Forum ranked Thailand “the 38th out of 144 countries worldwide in the overall score for digital and ICT knowledge and skills of higher-education students” (Pichay, 2018). The ranking employed Networked Readiness Index (NRI) to indicate national opportunity from digital and ICT utilization. In other words, the index refers to the impact level of digital and ICT utilization on the nation’s ability to compete. The measurement is based on three elements: 1) fundamental factors for digital such as governmental policy, governance, and infrastructure; 2) readiness of higher education institutions for digital such as the modernity of tools and communication system; and 3) digital competency level of students measured by digital knowledge and skills. The index score for Thai higher education institutions is mid-ranking when compared to other nations (Aslan & Zhu, 2017). When reviewing documents, papers, as well as research direction on digital for education during the past decade, a phenomenon was found. After Thai Qualifications Framework for Higher Education B.E. 2552 (Tantirattanawongse, 2009) became effective, one of the primary competencies being mandated was the communication and digital utilization skill (Leekitchwatana, 2017; Suwanroj et al., 2017; Suwanroj et al., 2018). Consequently, many institutions began to hold competency measurement examinations and the results suggested that only 39.04% of the undergraduate students passed the examinations. Investigations revealed that the cause for the failure was the difficulties in accessing information due to the lack of computer knowledge and skills. With the obstacles, utilizing information via technology appears ineffective. Some studies revealed that the students demanded short-term training to enhance basic competencies during the courses (Leekitchwatana, 2017; Suwanroj et al., 2017; Suwanroj et al., 2018).

3. Scope

The population scope for the confirmatory factor development was the 2,929 specialists in Information Technology, Computer Technology, Computer Education, Computer Science, and Computer Engineering who serve public higher education instructions throughout Thailand (Office of the Higher Education Commission, as of September 10, 2018). The scope for essential digital competencies was the seven digital components. Note that the researcher synthesized all the digital components and observed variables based on the scholarly concepts as discussed in the literature review section.

4. Methodology

This study employed the descriptive research approach with structural relationship model. The research process involved the development of confirmatory factors based on factor data derived from the review of literature; and the development of essential digital competency components for undergraduate students. The components were developed as a result of the focus group including nine specialists in Information Technology, Computer Technology, Computer Education, Computer Science, and Computer Engineering working in both public and private higher education instructions throughout Thailand with at least ten years of experience in the field of expertise who were selected using purposive sampling. Based on the findings of the preceding study (Suwanroj et al., 2017; Suwanroj et al., 2018), the specialists provided the data and confirmation for the components and 24 observed variables. The findings were then used as the conceptual framework to formulate a questionnaire which was further used to collect the data from the specialist samples. The acquired data were analyzed using 2nd Order CFA.

4.1. Sample

The sample comprised 1,126 specialists in Information Technology, Computer Technology, Computer Education, Computer Science, and Computer Engineering working in public higher education instructions throughout the country (38.44% of the population and 46.91 times larger than the observed variables). The selection was the result of a multi-stage random sampling from 76 public higher education instructions nationwide under the Office of the Higher Education Commission which offer undergraduate education in 2018. The sample size in this study was determined by the minimum ratio of 20 samples per variable based on the basic concepts proposed by Marsh, Balla and McDonald (1988) and Gagne and Hancock (2006); and the minimum number of total sample was as proposed by Myers, Ahn and Jin (2011) and Nordin, Zubairi and Ashraf (2018) which stated that the minimum total sample number should be at least 300 (n>300) for the confirmatory factor analysis to work.

4.2. Instrument

The instrument employed in the data collection was a questionnaire form on essential digital competency components for undergraduate students that the researcher developed. The form utilized 24 digital competency variables (Andritchi, 2012; Phanchalaem et al., 2016; Suwanroj et al., 2017; Suwanroj et al., 2018; Thinnukool, 2018) covering seven digital competency components. The instrument was reviewed and validated by the specialist group as discussed earlier to ensure it all digital competency and variable were covered. The question items employed a 7-point Likert scale and showed Cronbach’s alpha values for the content validity and reliability at a range of .93-.97 per domain and .87-.99 per component. Its index of item objective congruence (IOC) ranged between 0.80-1.00 demonstrating that this research instrument is valid and suitable for the data collection. The questionnaire was given out as an initial trial to 100 non-sampled specialists from nationwide public and private higher education institutions being selected through a similar sampling approach. The initial trial suggested that the developed questionnaire was adequately reliable for the data collection.

4.3. List of Component Definitions and Observed Variables

Table 2 illustrates a list of Thai higher-education students’ digital competency components, observed variables, and representing acronyms.

Components	Acronym	Observed variables	Acronym
1. Fundamental of digital	A	1) Basic knowledge of computer	A1
		2) Basic knowledge of information and technology	A2
		3) Ability to install basic applications in Windows operating system	A3
		4) Ability to setup a computer and peripherals	A4
2. Accessing digital information	B	1) Knowledge of information access	B1
		2) Knowledge of information search strategies and techniques using technology	B2
		3) Ability to employ information search tools using technology	B3
3. Using digital information	C	1) Knowledge of information usage	C1
		2) Ability to utilize information	C2
		3) Ability to use a computer to store information	C3
4. Creating digital information and media	D	1) Basic knowledge of media	D1
		2) Knowledge of the principles media design for learning	D2
		3) Ability to use a computer to design media for learning	D3
5. Communicating digital information	E	1) Basic knowledge of via-internet communication	E1
		2) Knowledge of social media utilization for an information exchange	E2
		3) Ability to use a computer for via-internet communication	E3
		4) Ability to use a computer to communicate through social media for information exchange	E4
6. Managing digital information	F	1) Basic knowledge of information management	F1
		2) Knowledge of computer usage for data processing and information generation	F2
		3) Ability to use a computer to process the data into a piece of information	F3
7. Evaluating digital information	G	1) Basic knowledge of information evaluation	G1
		2) Basic knowledge of internet information resource evaluation	G2
		3) Ability to verify and evaluate the searched information	G3
		4) Ability to verify and evaluate information resources on the internet	G4

Table 2. Component definitions and observed variables

4.4. Data Collection

The researcher employed the postal service to mail the samples the physical request letters on data collection along with a QR code pointing to the online questionnaire. The samples were asked to scan the code to fill out the form on the internet and eventually, 1,126 responses were received. The period of data collection was during October 1-November 30, 2018.

4.5. Data Analysis

The basic descriptive data in this study were analyzed using descriptive statistics on IBM SPSS Statistics 14.0 for Windows whereas the 2nd Order CFA was processed via LISREL 8.72. Since the measurement model was designed by the researcher with predetermined component quantity, name, and variable composition, the confirmatory factor analysis was applied to it to statistically validate the measurement (Awang-Hashim & Murad-Sani, 2008; Bandalos, 2018; Muthén & Muthén, 2015). Confirmatory factor analysis uses conditions that can be met easier than that of exploratory factor analysis, for instance, it tolerates deviations of observed variables as these deviations could still be related (Muthén & Muthén, 2015) and hence, it became presently more popular than exploratory factor analysis. Exploratory factor analysis has some limitations such as the diversity of analytical patterns which often create mismatches in the results. Often, the produced components are difficult to interpret.

5. Results of Research

5.1. Findings

There are four parts of the findings: 1) model diagram of the 2nd Order confirmatory factors of the essential digital competency components for undergraduate students in Thai higher education institutions; 2) statistical goodness of fit of the component model and empirical data; 3) factor loading, covariance, and priority values of the essential components; and 4) construct reliability and average variance extracted values of the essential components.

1. 1.Model diagram of the 2nd Order confirmatory factors of the essential digital competency components for undergraduate students in Thai higher education institutions.

Chi-Square = 60.47, df = 65, P-value = 0.64, RMSEA = 0.00

Figure 1. Model diagram of the 2nd Order confirmatory factors of the essential digital competency
components for undergraduate students in Thai higher education institutions

Figure 1 showing the analytical results of the 2nd Order confirmatory factors revealed that the model was the goodness of fit statistics with the empirical data on essential digital competency components for undergraduate students in Thai higher education institutions. All the goodness of fit statistics passed the criteria. The 2nd Order confirmatory factor analysis was applied to confirm two aspects: 1) whether the 24 observed variables represent the sub-components of the seven main components and 2) whether the said main components could generally be the appropriate composition of digital components for undergraduate students in Thai higher education institutions. The analysis demonstrated that all the observed variables represent the sub-components of the main components and main components together were an appropriate composition of the digital components for undergraduate students in Thai higher education institutions.

1. 2.Goodness of fit statistics between the model components and empirical data.

 

Goodness of fit index	Criteria*	Measurement	Interpretation	Conclusion
χ2-test	p>0.05	0.64	Pass	Good
χ2/df	<2	0.93	Pass	Good
GFI	≥0.95	1.00	Pass	Good
AGFI	≥0.95	0.98	Pass	Good
NFI	≥ 0.95	1.00	Pass	Good
CFI	≥ 0.95	1.00	Pass	Good
RMSEA	≤ 0.05	0.00	Pass	Good
SRMR	≤ 0.05	0.01	Pass	Good
RMR	≤ 0.05	0.01	Pass	Good
* Muthén and Muthén. (Muthén & Muthén, 2015) and Bandalos (Bandalos, 2018)

Table 3. Goodness of fit statistics between the model components and empirical data

Table 3 indicated that all the goodness of fit statistical values between the model components and empirical data passed the criteria. When accounting the χ2-test value which has no statistical significance (p=0.64), χ2/df=0.93, GFI=1.00, AGFI=0.98, NFI=1.00, CFI=1.00, RMSEA= 0.00, SRMR=0.01, and RMR=0.01, it can be concluded that the Figure 1 diagram is positively consistent with the empirical data.

1. 3.Factor Loadings, covariances, and priority values of the essential digital competency components.

 

Components	Factor Loading		t	R2	Priority
	b	S.E
1. Fundamental of digital (A)	0.93	0.03	32.39	0.86*	4
1) Basic knowledge of computer (A1)	0.87	0.03	32.39	0.86*
2) Basic knowledge of information and technology (A2)	0.73	0.03	24.95	0.83*
3) Ability to install basic applications in Windows operating system (A3)	0.72	0.03	24.41	0.82*
4) Ability to setup a computer and peripherals (A4)	0.64	0.03	22.06	0.81*
2. Accessing digital information (B)	1.00	0.04	28.71	0.87	1
1) Knowledge of information access (B1)	0.69	0.03	28.71	0.84*
2) Knowledge of information search strategies and techniques using technology (B2)	0.65	0.03	26.41	0.82*
3) Ability to employ information search tools using technology (B3)	0.72	0.03	31.20	0.82*
3. Using digital information (C)	0.99	0.03	33.68	0.98	2
1) Knowledge of information usage (C1)	0.83	0.02	33.68	0.90*
2) Ability to utilize information (C2)	0.86	0.02	35.24	0.83*
3) Ability to use a computer to store information (C3)	0.82	0.03	32.93	0.88*
4. Creating digital information and media (D)	0.87	0.31	27.74	0.85	7
1) Basic knowledge of media (D1)	0.79	0.03	27.74	0.84*
2) Knowledge of the principles of media design for learning (D2)	0.87	0.03	30.17	0.87*
3) Ability to use a computer to design media for learning (D3)	0.91	0.03	31.57	0.83*
5. Communicating digital information (E)	0.95	0.03	29.01	0.90	3
1) Basic knowledge of via-internet communication (E1)	0.77	0.03	29.01	0.81*
2) Knowledge of social media utilization for an information exchange (E2)	0.94	0.02	38.14	0.89*
3) Ability to use a computer for via- internet communication (E3)	0.91	0.02	35.90*	0.82
4) Ability to use a computer to communicate through social media for information exchange (E4)	0.88	0.02	34.13*	0.87
6. Managing digital information (F)	0.91	0.03	34.09	0.82	6
1) Basic knowledge of information management (F1)	0.92	0.02	34.09*	0.84
2) Knowledge of computer usage for data processing and information generation (F2)	0.89	0.03	32.13*	0.88
3) Ability to use a computer to process the data into a piece of information (F3)	0.98	0.02	37.20*	0.96
7. Evaluating digital information (G)	0.93	0.03	35.42	0.87	4
1) Basic knowledge of information evaluation (G1)	0.91	0.02	35.42*	0.83
2) Basic knowledge of internet information resource evaluation (G2)	0.81	0.03	29.70*	0.85
3) Ability to verify and evaluate the searched information (G3)	0.93	0.02	36.03*	0.86
4) Ability to verify and evaluate information resources on the internet (G4)	0.68	0.03	23.97*	0.86
* p < .01

Table 4. Factor Loadings, covariances, and priority values of the essential digital competency components.

Table 4 exhibits the priority of the components from high to low and Priority 1 was “Accessing digital information (B).” The variables ranked from “Ability to employ information search tools using technology (B3)”, “Knowledge of information access (B1)”, to “Knowledge of information search strategies and techniques using technology (B2)” with 0.72, 0.69, and 0.65 standard factor loadings and 82%, 84%, and 82% covariances with the “Accessing digital information (B)” component, respectively.

Priority 2 component was “Using digital information (C).” The variables ranked from “Ability to utilize information (C2)”, “Knowledge of information usage (C1)”, to “Ability to use a computer to store information (C3)” with 0.86, 0.83, and 0.82 standard factor loadings and 83%, 90%, and 88% covariances with the “Using digital information (C)” component, respectively.

Priority 3 component was “Communicating digital information (E).” The variables ranked from “Knowledge of social media utilization for an information exchange (E2)”, “Ability to use a computer for via-internet communication (E3)”, “Ability to use a computer to communicate through social media for an information exchange (E4)”, to “Basic knowledge of via-internet communication (E1)” with 0.94, 0.91, 0.88, and 0.77 standard factor loadings and 89%, 82%, 87%, and 81% covariances with the “Communicating digital information (E)” component, respectively.

Priority 4 contains two components of equal priority. The first component was “Fundamental of digital (A).” The variables ranked from “Basic knowledge of computer (A1)”, “Basic knowledge of information and technology (A2)”, “Ability to install a basic applications in Windows operating system (A3)”, to “Ability to setup a computer and peripherals (A4)” with 0.87, 0.73, 0.72, and 0.64 standard factor loadings and 86%, 83%, 82%, and 81% covariances with the “Fundamental of digital (A)” component, respectively. The second component was “Evaluating digital information (G)”.The variables ranked from “Ability to verify and evaluate the searched information (G3)”, “Basic knowledge of information evaluation (G1)”, “Basic knowledge of internet information resource evaluation (G2)”, to “Ability to verify and evaluate information resources on the internet (G4)” with 0.93, 0.91, 0.81, and 0.68 standard factor loadings and 86%, 83%, 85%, and 86% covariances with the “Evaluating digital information (G)” component, respectively.

Priority 6 component was “Managing digital information (F).” The variables ranked from “Ability to use a computer to process the data into a piece of information (F3)”, “Basic knowledge of information management (F1)”, to “Knowledge of computer usage for data processing and information generation (F2)” with 0.98, 0.92, and 0.89 standard factor loadings and 96%, 84%, and 88% covariances with the “Managing digital information (F)” component, respectively.

Priority 7 component was “Creating digital information and media (D).” The variables ranked from “Ability to use a computer to design media for learning (D3)”, “Knowledge of the principles of media design for learning (D2)”, to “Basic knowledge of media (D1)” with 0.91, 0.87, and 0.79 standard factor loadings and 83%, 87%, and 84% covariances with the “Creating digital information and media (D)” component, respectively.

The presentation of data and ranking for all seven components and 24 observed variables of digital competencies in Table 4 is very important as this stage of discovery could later be used to order the learning contents and management sequence when organizing the short-term training on digital competency for Thai higher-education students. Moreover, such ranking could also be used to further construct a standardized test to assess digital competencies for the students in the next phase of the study.

1. 4.Construct reliability and average variance extracted values of the essential digital components.

 

Components	Construct Reliability (rc)			Average Variance Extracted (rv)
	Criteria*	Measurement	Interpretation	Criteria*	Measurement	Interpretation
1. Fundamental of digital (A)
	>0.60	0.88	Highly Reliable	>0.50	0.65	Highly Valid
2. Accessing digital information (B)
	>0.60	0.75	Highly Reliable	>0.50	0.51	Highly Valid
3. Using digital information (C)
	>0.60	0.88	Highly Reliable	>0.50	0.70	Highly Valid
4. Creating digital information and media (D)
	>0.60	0.90	Highly Reliable	>0.50	0.74	Highly Valid
5. Communicating digital information (E)
	>0.60	0.93	Highly Reliable	>0.50	0.77	Highly Valid
6. Managing digital information (F)
	>0.60	0.95	Highly Reliable	>0.50	0.86	Highly Valid
7. Evaluating digital information (G)
	>0.60	0.90	Highly Reliable	>0.50	0.70	Highly Valid
* Muthén and Muthén (Muthén & Muthén, 2015) and Bandalos (Bandalos, 2018)

Table 5. Construct reliability and average variance extracted values of the essential digital components.

Table 5 demonstrates that the seven digital components are highly reliable (CR>0.60) and highly valid (AVE>0.50) according to the standard factor loadings of the observed variables measured in the latent variables in each component following the criterion concepts of L. Muthén and B. Muthén (Muthén & Muthén, 2015) and D. L. Bandalos (Bandalos, 2018). This indicates that the seven components are suitable as the essential digital competency components for undertenant students.

6. Discussion

The discussion section covers two of the following issues:

Issue 1: The results confirmed that the 24 observed variables could represent the sub-components of the seven digital competency components. The model was consistent with the empirical data obtained from the specialists in Thai higher-education institutions and all the goodness of fit statistics passed the criteria. This means that the digital competencies were highly beneficial for Thai higher-education institutions. After applying the confirmatory factor analysis to the core components and observed variables, it was found that the model remained consistent with the empirical data because all the observed variables and components used in the analysis were from the review of literature (Bora, 2017; Phanchalaem et al., 2016; Suwanroj et al., 2017; Suwanroj et al., 2018; Thinnukool, 2018; Yager & Szabo, 2014) and confirmed by the focus group of specialists (Suwanroj et al., 2017; Suwanroj et al., 2018). Hence, the components and variables can further be adopted as a guideline for curriculum and training development where extensive adaptation could account Thai contexts to ensure that the students could ultimately be equipped with essential digital competencies as the learners of the 21st century and the workers in the era of Thailand 4.0 (Jones & Pimdee, 2017; Skeldon, 2018). The findings are congruent with the discovery made by Marusic and Viskovic and Eger, Klement, Tomczyk, Pisoňová, and Petrová, (Marusic & Viskovic, 2018; Eger, Klement, Tomczyk, Pisoňová & Petrová, 2018) stating that essential digital competencies for higher-education students should comprise of knowledge and skills in seven foundation and essential components. The observed variables, however, may vary depending on national contexts and necessities. In the Thai context, the seven digital competency components and 24 observed variables are sufficient and practical for national-level applications.

Issue 2: The results confirmed that, in general, the main components represented the appropriate composition of digital components for undergraduate students in Thai higher education institutions. In fact, the components proved to be highly beneficial to higher education institutions and can further be adopted as a guideline to help the students enhance their essential digital competencies as the learners of the 21st century and the workers in the era of Thailand 4.0. This notion is in line with the identified digital situation in Thai higher education institutions which is the lack of computer knowledge and skills ( Suwanroj et al., 2017; Suwanroj et al., 2018); and the findings laid out in Cha et al. (2018). In addition, the digital components can also be used to improve students’ essential digital knowledge and skills following the concept of 21st century students and the National Qualifications Framework for Higher Education in Thailand B.E. 2552 (2009) which requires higher education institutions to develop the students’ essential digital competencies (Krasnova & Shurygin, 2017; Leekitchwatana, 2017; Suwanroj et al., 2017; Suwanroj et al., 2018; Tondeur et al., 2017; Wei et al., 2017).

7. Implications

All of the developed digital competency components and variables proved to be highly constructive for Thai higher education institutions, and they can be adopted as model competencies in for three practices as summarized below:

1. 1.Teaching Management: The developed digital competency components may be used to design short-term training and courses aiming to enhance students’ skills. These digital competencies are part of the essentials in the 21st century (Catapano & Slapac, 2010; Khun Puchthonglang & Puchthonglang, 2016; Kovačević & Opić, 2014; Pannasil, Kenaphoom & Kosolkittiamporn, 2015 Prayitno, Suciati & Titikusumawati, 2018) because they cover many developmental skills including learning and innovation; information, media, and technology; and life and career. It is fair to mention that these newly developed digital competencies are collectively a new and essential digital competency model for higher-education students on 21st century’s information, media, and technology.

2. 2.Living in Thailand 4.0 Era: The developed digital competencies are composed of necessary elements for the change as Thailand is transitioning into the digital age as described in Thailand 4.0’s governmental policies mandating the development of information, media, and technological skills (Chantana, 2017). To comply with the mandate, Office of the Higher Education Commission issued Thai Qualifications Framework for Higher Education B.E. 2552 (2009) (Aslan & Zhu, 2017). Consequently, Thai higher-education institutions began to follow the framework and give digital competencies a priority. As a result, many digital development activities have been designed, e.g., short-term competency development courses for students and teachers.

3. 3.Implementing the digital Competencies: As discussed, the competencies created from this study were used to develop a short-term training course so that the competencies could fit the contexts of higher-education institutions in Thailand. The content of the course covered all seven competency components; the teaching included both academic and practice sessions; and the course ran for 21 training hours.

4. 4.Developing Teachers’ Competencies: Teachers are responsible for guiding and running the process to ensure that the learning meets its objectives. In addition to the development of digital competencies for students, the teachers should also possess the competencies and be a model for their students. To achieve this goal, the teachers should be engaged in a self-development simultaneously with the student development. This concept is consistent with Punnee, Siew and Ambo and Douglas, Christopher and George (Leekitchwatana, 2017; Matijević, Opić & Lapat, 2016, Siew & Ambo, N. (2018), Douglas, Christopher and George (2018) as the study developed some competency factors for a group of lower-secondary information technology teachers and improved their knowledge and skills using a short-term training course incorporating the competencies. The final aims were for these teachers to further teach their lower-secondary students using the knowledge and skills they previously acquired.

8. Conclusion

The confirmatory factor analysis of the essential digital competency components demonstrated that the competency component model was consistent with the empirical data. This can be interpreted that the essential digital competency components should contain seven competency components with 24 observed variables. These seven components were 1) Fundamental of digital; 2) Accessing digital information; 3) Using digital information; 4) Creating digital information and media; 5) Communicating digital information; 6) Managing digital information; and 7) Evaluating digital information. The identified digital components are also highly appropriate with the current contexts for a digital competency development in Thailand because learning these components will allow the students to creatively employ the developed knowledge and competencies in creating innovations. In addition, the students will be able to respond to social needs using the essential digital competencies to search for information. Hence, higher-education institutions should support the development and ensure that higher-education students are well-equipped with the essential competencies as they should not be using the digital pointlessly. Creativity and social interaction such as using digital for group collaboration could be applied to help enhance the learning so that the students would not have to be left working alone.

9. Suggestions for Further Studies

The developed competency components are only suitable with graduate-level contexts of Thai higher-education institutions specifically to general education fields dealing basic level digital knowledge. To use the developed competency components with students of specialized fields such as Computer Education, Information Technology, or any filed that requires advanced digital knowledge, the components should only be used only as a guideline and the observed variables should be enhanced with a higher level of complexity, e.g. knowledge and skill of microcomputer maintenance or installation of network peripherals.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Akarawang, C., Kidrakran, P., & Nuangchalerm, P. (2015). Enhancing ICT Competency for Teachers in the Thailand Basic Education System. International Education Studies, 8(6), 1-8. https://doi.org/10.5539/ies.v8n6p1

American Library Association (2013). Digital Competencies. Libraries, and public policy: Report of the Office for information technology Policy’s digital competencies Task force. Available at: http://www.districtdispatch.org/wp-content/upload/2013/01/2012_OITP_digilitreport_1_22_13

Andritchi, V. (2012). The hierarchy of teachers’professional needs from pre-university education from moldova. Journal of Psychological and Educational Research, 20(2), 7.

Arif, A.Z.M., Fleischer, L., Bogiatzis, A., Asada, H., Colombo, A., & Zougbédé, K. (2018). Enhancing governance in Thailand.

Aslan, A., & Zhu, C. (2017). Investigating variables predicting Turkish pre‐service teachers’ integration of ICT into teaching practices. British Journal of Educational Technology, 48(2), 552-570. https://doi.org/10.1111/bjet.12437

Association of Southeast Asian Nations (2018). ASEAN ICT-Digital Standarts Competance. Available at: https://www.cfr.org/backgrounder/asean-association-southeast-asian-nations

Awang-Hashim, R., & Murad-Sani, A. (2008). A confirmatory factor analysis of a newly integrated multidimensional school engagement scale. Malaysian Journal of Learning & Instruction, 5, 21-40.

Bandalos, D.L. (2018). Measurement theory and applications for the social sciences: Guilford Publications.

Bora, C.H. (2017). Cognitive factors related to teachers emotional distress. International Journal of Education and Psychology in the Community, 7(1/2), 31.

Brata, I.B., & Pemayun, A.G.P. (2018). Human Resource Competency Tourism Bali together with ASEAN Economic Community. International Research Journal of Management, IT and Social Sciences (IRJMIS), 5(2), 186-194. https://doi.org/10.21744/irjmis.v5i2.638

Burbules, N. (2018). Watch IT: The risks and promises of information technologies for education. Routledge. https://doi.org/10.4324/9780429503153

Camelo, G.E.H., Torres, J.M.T., Reche, M.P.C., & Costa, R.S. (2018). Using and integration of ICT in a diverse educational context of Santander (Colombia). Journal of Technology and Science Education, 8(4). https://doi.org/10.3926/jotse.314

Catapano, S., & Slapac, A. (2010). Preservice Teachers’ Understanding of Culture and Diversity: Comparing Two Models of Teacher Education. Teacher Education and Practice, 23(4), 426-443.

Cha, S.E., Jun, S.J., Kwon, D.Y., Kim, H.S., Kim, S.B., Kim, J.M. et al. (2018). Measuring achievement of ICT competency for students in Korea. Computers & Education, 56(4), 990-1002.
https://doi.org/10.1016/j.compedu.2010.11.003

Chang, C.T. (2009). Enhancing graudates’ employ ability skills & competencies for hospitality & tourism program through Thai qualification framework and credit transfer.

Chantana, P. (2017). Model of Educational Management for Developing Mass Communication Graduates in Digital Age.

David, C. (1973). Testing for competence rather than for intelligence. American psychologist, 28(1).

Douglas, K., Christopher C., & George M. (2018). Factors that influence Zambian higher education lecturer’s attitude towards integrating ICTs in teaching and research. Journal of Technology and Science Education, 8(4), Available at: http://www.jotse.org/index.php/jotse/article/view/338/343 (Accessed: January 2019)

Dubois, D.D. (1993). Competency-Based Performance Improvement: A Strategy for Organizational Change. ERIC.

Eger, L., Klement, M., Tomczyk, Ł., Pisoňová, M., & Petrová, G. (2018). Different user groups of university students and their ICT competence: evidence from three countries in central europe. Journal of Baltic Science Education, Journal of Baltic Science Education, 17(5). Available at: http://oaji.net/articles/2017/987-1539418548.pdf (Accessed: December 2018)

European Commission, (2019). The Digital Competence Framework 2.0 Available at: https://ec.europa.eu/jrc/en/digcomp/digital-competence-framework

Freud, A. (2018). Normality and pathology in childhood: Assessments of development. Routledge. https://doi.org/10.4324/9780429477638

Gagne, P., & Hancock, G.R. (2006). Measurement model quality, sample size, and solution propriety in confirmatory factor models. Multivariate Behavioral Research, 41(1), 65-83. https://doi.org/10.1207/s15327906mbr4101_5

Gil-Flores, J., Rodríguez-Santero, J., & Torres-Gordillo, J.-J. (2017). Factors that explain the use of ICT in secondary-education classrooms: The role of teacher characteristics and school infrastructure. Computers in Human Behavior, 68, 441-449. https://doi.org/10.1016/j.chb.2016.11.057

Goodman, C.M. (1987). The Delphi technique: a critique. Journal of advanced nursing, 12(6), 729-734. https://doi.org/10.1111/j.1365-2648.1987.tb01376.x

Hedge, T. (2001). Teaching and learning in the language classroom (Vol. 106).  Oxford, England: Oxford University Press.

Holmes, B. (2018). Comparative education: Some considerations of method. Routledge. https://doi.org/10.4324/9781351005067

Inglehart, R. (2018). Culture shift in advanced industrial society. Princeton University Press. https://doi.org/10.2307/j.ctv346rbz

Jaradat, R.M., Keating, C.B., & Bradley, J.M. (2018). Individual capacity and organizational competency for systems thinking. IEEE Systems Journal, 12(2), 1203-1210. https://doi.org/10.1109/JSYST.2017.2652218

Jones, C., & Pimdee, P. (2017). Innovative ideas: Thailand 4.0 and the fourth industrial revolution. Asian International Journal of Social Sciences, 17(1), 4-35. https://doi.org/10.29139/aijss.20170101

Jongsermtrakoom, S., & Nasongkhla, J. (2015). A Group Investigation Learning System for Open Educational Resources to Enhance Student Teachers’ Digital Literacy and Awareness in Information Ethics. International Journal of Information and Education Technology, 5(10), 783-788. Available at: http://www.ijiet.org/papers/611-T051.pdf (Accessed: February 2019) https://doi.org/10.7763/IJIET.2015.V5.611

Kasemsap, K. (2018). Encouraging digital literacy and digital competency in the information age. Encyclopedia of Information Science and Technology (4th ed.) (2253-2263). IGI Global.

Khun Puchthonglang, P., & Puchthonglang, Y. (2016). Learning Provision Model for Develop Right Consumption Behavior Based on the Philosophy of Sufficiency Economy for Adolescents. Journal of MCU Peace Studies, 4(2), 17-37.

Kosaiyapattanapundit, P., & Sangthong, P.S. (2018). Executives in the New Age with Thailand 4.0. Journal of MCU Buddhapanya Review (JMBR), 2(3), 67-74.

Kovačević, T., & Opić, S. (2014). Contribution of Traditional Games to the Quality of Students’ Relations and Frequency of Students’ Socialization in Primary Education. Croatian Journal of Education, 16(1), 95‑112.

Krasnova, L.A., & Shurygin, V.Y. (2017). Development Of teachers’ Information Competency In Higher Education Institution. Astra Salvensis, 10.

Leekitchwatana, P. (2017). Development of Competency Factors for Information Technology. KKU Research Journal, 15(11), 1101-1113.

Marian Puscas, M., Sangineto, E., Culibrk, D., & Sebe, N. (2015). Unsupervised tube extraction using transductive learning and dense trajectories. Paper presented at the Proceedings of the IEEE International Conference on Computer Vision. https://doi.org/10.1109/ICCV.2015.193

Marsh, H.W., Balla, J.R., & McDonald, R.P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological bulletin, 103(3), 391. https://doi.org/10.1037/0033-2909.103.3.391

Marusic, T., & Viskovic, I. (2018). ICT Competencies of Students. Correction of the Journal Itro, 115(56), 13.

Matijević, M., Opić, S., & Lapat, G. (2016). Teachers’ assessments of certain curricular determinants in primary school. Faculty of Educational Sciences, 162-171.

McRoy, R.G., Oglesby, Z., & Grape, H. (2018). Achieving same-race adoptive placements for African American children: Culturally sensitive practice approaches Serving African American Children (85-104). Routledge.

Muthén, L., & Muthén, B. (2015). Mplus. The comprehensive modelling program for applied researchers: user’s guide, 5.

Myers, N.D., Ahn, S., & Jin, Y. (2011). Sample size and power estimates for a confirmatory factor analytic model in exercise and sport: A Monte Carlo approach. Research Quarterly for Exercise and Sport, 82(3), 412‑423. https://doi.org/10.1080/02701367.2011.10599773

Nordin, M.S., Zubairi, A., & Ashraf, R.M. (2018). Factorial Validity and Invariance of the MUET Writing Rating Scale: Empirical and Theoretical Correspondence. Malaysian Journal of Learning and Instruction, 15(1), 265-285.

Nosenko, Y., Shyshkina, M., & Oleksiuk, V. (2018). Collaboration between research institutions and university sector using cloud-based environment. arXiv preprint arXiv:1807.08741.

Office of the Higher Education Commission (2013). Thai Qualifications Framework for Higher Education. TQF: HEd BE, 2552.

Pannasil, P., Kenaphoom, S., & Kosolkittiamporn, S. (2015). The role of local executive in the 21st century. Journal of MCU Peace Studies, 3(2), 146-161.

Phanchalaem, K., Sujiva, S., & Tangdhanakanond, K. (2016). Development of capacity building model on teachers’assessment information use based on collaborative immersion approach. International Journal of Education & Psychology in the Community, 6.

Phuapan, P., Viriyavejakul, C., & Pimdee P. (2016). An analysis of digital literacy skill among Thai university seniors. International Journal of Emerging Technologies in Learning, 11(3). 24-31. Available at: https://online-journals.org/index.php/i-jet/article/view/5301 (Accessed: February 2019) https://doi.org/10.3991/ijet.v11i03.5301

Pichay, C. (2018). Social status: ICT knowledge and skills in Thailand. (2018, December 9th). Bangkok post pp. A1, A4.

Prayitno, B.A., Suciati, & Titikusumawati, E. (2018). Enhancing students’ higher order thinking skills in science through instad strategy. Journal of Baltic Science Education, Journal of Baltic Science Education, 17(6). Available at: http://oaji.net/articles/2017/987-1544860627.pdf (Accessed: December 2018) https://doi.org/10.33225/jbse/18.17.1046

Rodmunkong, T., & Wannapiroon, P. (2013). The design of cloud computing management information system accordance with Thai qualifications framework for higher education. International Journal of e‑Education, e-Business, e-Management and e-Learning, 3(3), 214. https://doi.org/10.7763/IJEEEE.2013.V3.226

Rodmunkong, T., Wannapiroon, P., & Nilsook, P. (2015). The architecture of Information Management System through cloud computing according to Thai Qualifications Framework for Higher Education. Paper presented at the Teaching, Assessment, and Learning for Engineering (TALE), IEEE International Conference on. https://doi.org/10.1109/TALE.2015.7386041

Roşeanu, G., & Drugaş, M. (2011). The admission criteria to the university as predictors for academic performance: a pilot study. Journal of Psychological & Educational Research, 19(2).

Shannon, T.R. (2018). An introduction to the world-system perspective. Routledge. https://doi.org/10.4324/9780429494154

Shyshkina, M. (2018). Holistic Approach to Training of ICT Skilled Educational Personnel. arXiv preprint arXiv:1807.08717.

Skeldon, R. (2018). Of migration, great cities, and markets: Global systems of development Global history and migrations (183-215). Routledge.

Siew, N.M., & Ambo, N. (2018). Development and evaluation of an integrated project-based and stem teaching and learning module on enhancing scientific creativity among fifth graders. Journal of Baltic Science Education, Journal of Baltic Science Education, 17(6). Available at: http://oaji.net/articles/2017/987-1544860506.pdf (Accessed: December 2018) https://doi.org/10.33225/jbse/18.17.1017

Social Status ICT knowledge and skills in Thailand (2018). Bangkok post, A1-A4.

Suwanroj, T., Leekitchwatana, P., Pimdee, P. (2017). Investigating digital competencies for undergraduate students at Nakhon Si Thammarat Rajabhat University. DRLE 2017 The 15th International Conference Faculty of Industrial Education and Technology King Mongkut’s Institute of Technology Ladkrabang, 27(2), 11-19.

Suwanroj, T., Leekitchwatana, P., Pimdee, P. Thiyaporn, K., & Thanongsak, S. (2018). Development of digital Competency Domains for Undergraduate Students in Thailand. International Journal of the Computer, the Internet and Management, 27(2).

Szabo, Z., & Marian, M. (2012). Stress inoculation training in adolescents: Classroom intervention benefits. Journal of Evidence-Based Psychotherapies, 12(2), 175.

Tantirattanawongse, J. (2009). Thai qualifications framework for higher education 2009 (TQF: HEd). Commission on Higher Education.

Thinnukool, O. (2018). Analysis of the Use of Social Network in the 21st Century Active Learning for Undergraduate Students Based on the Subject of Rapid Application Development. International Journal of Innovation and Technology Management, 15(03), 1850020. https://doi.org/10.1142/S0219877018500207

Tondeur, J., Aesaert, K., Pynoo, B., van Braak, J., Fraeyman, N., & Erstad, O. (2017). Developing a validated instrument to measure preservice teachers’ ICT competencies: Meeting the demands of the 21st century. British Journal of Educational Technology, 48(2), 462-472. https://doi.org/10.1111/bjet.12380

Tondeur, J., Van Braak, J., & Valcke, M. (2007). Curricula and the use of ICT in education: Two worlds apart? British Journal of Educational Technology, 38(6), 962-976. https://doi.org/10.1111/j.1467-8535.2006.00680.x

Trip, S., Vernon, A., & McMahon, J. (2007). Effectiveness of rational-emotive education: A quantitative meta-analytical study. Journal of Cognitive & Behavioral Psychotherapies, 7(1).

Turan, Z., & Göktaş, Y. (2018). Innovative Redesign of Teacher Education ICT Courses: How Flipped Classrooms Impact Motivation. Journal of Education and Future, 13, 133-144.

Voogt, J., Knezek, G., Christensen, R., Lai, K. W., Pratt, K., Albion, P. et al. (2017). The International Handbook of Information Technology in Primary and Secondary Education: Part 2. Paper presented at the Society for Information Technology & Teacher Education International Conference.

Wei, L.M., Piaw, C.Y., & Kannan, S. (2017). Relationship between principal technology leadership practices and teacher ICT competency. MOJEM: Malaysian Online Journal of Educational Management, 4(3), 13-36. https://doi.org/10.22452/mojem.vol4no3.2

Yager, S.E., & Szabo, Z. (2014). Using Electronic Resources to Teach Computer Literact: Students’Perceptiond and Use. International Journal of Education & Psychology in the Community, 4.

---