A FRAMEWORK FOR MEASURING THE LEVEL OF ACHIEVEMENT OF VOCATIONAL STUDENTS COMPETENCY OF ARCHITECTURAL EDUCATION

The determination of work competence according to industrial needs that are expected by employers of new graduates in the department of architectural engineering in vocational high schools will help graduates to transit smoothly from the academic education environment to the real conditions of the construction industry in the workplace. This research identifies which group of competencies attributes that graduates should have in explaining appropriate competencies according to construction industry standards in Indonesia. The survey was conducted in 47 building construction companies consisting of 56 respondents using a questionnaire method. Multivariate analyses using the CB-SEM method used to validate the evaluation model. The research analysis used quantitative descriptive and factor analysis with the Explanatory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) methods. The EFA test shows ten factors so that it consists of 30 items of competency measurement. Overall, the CFA test has met the criteria for the goodness of fit statistics. The results of the study concluded that the instrument developed met good validity and reliability and two model frameworks for measuring the level of achievement of student competence in architectural techniques were feasible to use.

As a formal education that develops educated, skilled, and competent VHS graduates' Nurtanto, Arifin, Sofyan, Warju & Nurhaji, 2020), VHS is deemed necessary to improve their quality. The current competency of VHS graduates is still not able to bring them to excel (Ayadat, Ahmed, Chowdhury & Asiz, 2020) and become winners in the increasingly competitive world of work competition (Daryono, Yolando, Jaedun & Hidayat, 2020). Motivation and the way of thinking of students who tend to look for work after graduation and do not think of looking for other alternatives such as opening their jobs or doing entrepreneurship still dominate the minds of most VHS graduates. This condition has an impact on the increasing number of unemployed VHS graduates' (Karstina et al., 2021;Setyadi, Triyono & Daryono, 2021;Vlasova et al., 2018).
The world of work is constantly changing creating new challenges for employers and employees (Berestova, Lazareva & Leontyev, 2020;Khan, Soundararajan & Shoham, 2020). The progress of industry resulted in the advancement of the employment sector and the number of workers in the construction sector (Špaček, Legény & Gregor, 2020). Indonesian employment data explains that the construction sector is in the top four with 18.98% in August 2019. Employment of the population of each business sector shows the ability in the construction sector in the labor absorption rate (Subdirektorat Statistik Konstruksi, 2019).
The data observations conducted at VHS in the architecture department obtained data regarding the recapitulation of the absorption of the number of graduates in the architecture department within three years, from 2016 to 2019 the school year, namely 24.00%, 24.62%, and 41.79%. Absorption data for graduates who have worked for the past three years is 23.53%, 41.54%, and 40.30%. Data on the absorption of graduates for entrepreneurs during the last three years is 0.00%, 3.08%, and 2.99%. While graduate data not detected is 63%, 30%, and 14.93% (Badan Pusat Statistik, 2020). Based on the data absorption of graduates in the category of graduates who work it can be concluded that the graduates have not been able to work optimally.
This research is based on the number of architectural engineering graduates who work not according to their field of expertise. The low readiness of graduates to work is due to the inadequate development and assessment of students' competence. The material in vocational high schools is not by the needs of industry and developments in construction technology. Therefore, the research focused on the evaluation of competencies implemented in VHS in architectural engineering education, which aligned with the urgent needs currently needed by the construction industry. So that the evaluation model is used to measure the extent, to which student's mastery of current competencies as architectural engineering graduates to be able to work according to industry demand.

Sample Construction
This research uses descriptive research with a quantitative approach using survey methods. The population of this research is construction industry organizations in the field of building construction services as employers for architectural engineering graduates in vocational secondary education level. The sample of respondents was 56 practices from 47 organizations in building construction companies. Data collection was carried out through a questionnaire given to each construction company. The sample profiles in this study are based on the profiles of employers who responded to the questionnaire, the sector-specific civil engineering work performed by the responding organizations, and the categories of organizations. A list of construction company profiles showed in Table 1.

Research Instruments
The majority of employers are high-ranking officials in their organizations. Most of the respondents came from the building, roads, and bridges sector, but the responses from sectors related to workshop/factory work were also relatively significant. The majority of organizations that participated in the questionnaire survey were non-governmental organizations. A questionnaire designed with 40 attributes considered important at this time. The questionnaire has designed for employers for graduates in architectural engineering majors based on Indonesian National Work Competency Standards and the architectural engineering education curriculum in Indonesia. The questionnaire is structured using a four-point Likert scale where 1 indicates "not important"; 2 "quite important", 3 "important", 4 "very important". Table 2 shows a list of 40 competency items in ten competency aspects.

Data Analysis
Descriptive statistical analysis, multicollinearity, normality, and data reduction before applied in factor analysis, using SPSS 25.0 software and Amos 22.0. Multicollinearity analysis can conclude the inter-change matrix with a value of ≤0.90 (Kline, 2005). The reliability test used Cronbach's Alpha coefficients, composite reliability, and average variance extracted values. The CR should be more than 0.60 and AVE should be over 0.50 (Hidayat et al., 2018;Liou, 2021). After that, all items are included in the criteria for the factor analysis test.

Preliminary Analysis
After the first stage in preliminary analysis related to multicollinearity and normality tests, as well as the EFA test in the first stage, 30 out of 40 competency items passed the recommended value requirements for further analysis. The other 10 MSA competency score points and loading factors are below the recommended value, and then the next analysis uses 30 competency items that have met the conformity value standard. The 10 competency points that failed were KU3, GT2, MT4, DKB2, TPT2, APLPIG3, KJJ1, EBK4, KUG2, and PKK1.
Descriptive statistical results of all competency items, the level of urgency of competence in architectural engineering education that is needed by the construction industry, namely the competence of calculating the estimated cost of construction work (EBK3) with an average of 3.679, then competence from evaluating and improving measurement results in the form of working drawings for construction work (APLPIG4), and creating interior designs with elements, materials, models, and accessories in every room (TPT4) with an average of 3.661. Overall, 30 items of competence in the category of urgently with a mean acquisition of 3.508 out of 4.00. Simultaneously, all competency items reached univariate normality (skewness and kurtosis values between -1.783 to 1.95) (Zare & Nastiezaie, 2019). In the case of multicollinearity, the correlation between the ten competency items analyzed in the construction value ranges from 0.248 to 0.737.
From the results of the analysis of the results of the inter-item correlation matrix, all Pearson Correlation values in each competency aspect range from 0.405 to 0.703. This shows that every aspect of competence has an adequate level of validity because the value is more value of the rtable is 0.264 and the inter-correlation matrix value is ≤0.90 (Chai, Jong, Yin & Chen, 2019). Likewise for the Sig. (2-tailed), all aspects of the reviewed competency score are met with a result of 0.000 (<0.005). So that the forty competencies required for the construction industry are sufficient.

Reliability Instruments
Reliability is the stability and suitability of each score found. According to Ekolu and Quainoo (2019), if the CA value ranges from 0.90<x≤1.0 then the item is categorized very high, 0.70 <x≤ 0.90 is categorized high, 0.30<x≤ 0.70 is categorized as moderate, and 0.00 ≤x≤ 0.30 is categorized low (Ekolu & Quainoo, 2019). The CR should be more than 0.60 and AVE should be over 0.50 (Hidayat et al., 2018;Suwanroj et al., 2019). The results of the reliability of the instrument showed in Table 3

Exploratory Factor Analysis (EFA)
The EFA test considers each item of competence that has passed the multicollinearity test and normality and reliability for each item. All 40-competency items stipulated in the past questionnaire were included in the 10 competency aspects. The EFA test criteria are based on the value of the KMO Index, Bartlett's Test, Measure of Sampling Adequacy (MSA), communalities, factor loading, eigenvalues, and plot scree (Chai et al., 2019;Mateus & Hernández-Breña, 2019). The results of the calculation of the KMO obtained a value of 0.801 that has exceeded the set standard value, namely 0.50 so that the scope of each work competency according to employer standards in the building construction industry is satisfactory. Bartlett's Test of Sphericity Approx. Chi-Square scored 999.849; df = 435; Sig. = 0.000. The next output is the total variance explained table. Total variance explained is the percentage of the measured construct variance explained by several factors that are formed. From the initial eigenvalues column in the cumulative sub-column, it can seem that reducing 30 items reducing it to 10 factors can explain 72.039%.
The next step is to identify the MSA values, communalities, and factor loading. MSA is useful for knowing and determining which variables are suitable for use in factor analysis. Requirements that must be met in the factor analysis are above 0.50. This Communalities table shows the value of the variable under study whether it can explain the factor or not. The variable is considered capable of explaining the factor if the Extraction value is >0.50. The next output is the matrix value on the rotating component. This value shows the loading factor for each factor. The principle of exploratory factor analysis is that each item correlated with all factors, but a good item only has a high factor loading on the factor it measures. Table 4 shows the analysis results of MSA values, communalities, factor loading, and total variance explained.
The principle of exploratory factor analysis is that each item can be correlated with all factors, from the results of the analysis, the factor loading value is between 0.458 to 0.828, and this value has met the standard because it is more than 0.40 (Harlan & Van Haneghan, 2020;Hidayat et al., 2018). The results of the analysis state that each competency item correlated with other factors.

Measurement Model for Student Competency Achievement Level 3.4.1. Confirmatory Factor Analysis: First-Order Factor Testing
This study examines two measurement models to measure the level of competency attainment of students in architectural engineering education majors in the Indonesian context. The CFA test is intended to verify the factorial validity of the measurement model. CFA can prove the suitability of the suggested model with the identified factor structure through the EFA test. Therefore, the CFA model presented in Figure 1 is a first-order measurement model that shows the measurement structure for the achievement of student competencies in architectural engineering majors in Indonesia. The model is compared with the criteria of goodness of fit statistics (GOF) (Hair et al., 2010;Hidayat et al., 2018;Huang & Hwang, 2019). The model specifications from the CFA output for factor testing in the first order are shown in Table 5. The results of the analysis show that in the testing, all 30 items of competency are declared valid. The P-value (0.129) greater than 0.05 indicates a match between the measurement model concept and field data. The value of RMSEA = 0.040 (≤0.08), then the value of the Incremental Fit Measures which includes IFI = 0.967, TLI = 0.954, CFI = 0.963 (≥0.90) qualifies as a fit model. The first-order CFA test results for the ten-factor model are hypothesized to be very good based on GoF criteria and the model is acceptable.

Confirmatory Factor Analysis: Second-Order Factor Testing
The factor structure of this study is hypothesized and examined. Based on the results of the second-order CFA in the construct of the measurement model for the achievement of student competencies in the architectural engineering department in Indonesia, a path diagram is obtained which is shown in Figure 2.

Discussion
This study proves the validity and estimates the level of reliability of the instrument to measure the level of competency achievement of students in Indonesia in the architectural engineering education program. The EFA test results show that the respondent data from construction industry participants involves the structure of ten factors, namely: general competencies, technical drawings, construction techniques, the basics of building construction, land measurement techniques, software applications and building interior design, road and bridge construction, construction cost estimation, construction and building utilities, and creativity and entrepreneurship products competencies. These ten factors are hereinafter referred to as aspects of architectural technical competence. The structure is also by the original ten-factor structure of determinants, the measurement of student competency achievement. Although all 40 competency items offered cannot be further analyzed, namely competency items with codes KU3, GT2, MT4, DKB2, TPT2, APLPIG3, KJJ1, EBK4, KUG2, PKK1 because they do not meet the value criteria in the construct validity test. So that 30 items can be accepted for construct validity analysis.
Based on the curriculum currently applied in Indonesia in the department of architectural engineering for vocational high schools, General Competencies consist of faith and piety, nationality and love for the homeland, personal and social character, physical and spiritual health, literacy, and creativity. Furthermore, in the Technical Drawing subject, students learn about how students can present and demonstrate the types and functions of technical drawing equipment, then draw various plane shapes, 2D and 3D projections, and the ability to draw sketches and draw projections with BIM software. In the Statically Structures subject, vocational students are taught material about building structural elements, calculating various forces on building structures, and stresses on a simple beam. Application of SAP 2000 software for engineering mechanics calculations in structural planning.
The subject of Basic Building Construction is a science that learns about the types of building construction, how to carry out a job on a building construction which includes wood, concrete, steel, and soil construction. Introduction and application of Tekla Structure to create and plan building structure models and steel structure models. Furthermore, the Land Measurement Engineering subject provides knowledge so that students understand and can carry out work on surveys and mapping. The use tools for survey measurements are leveling and theodolite, in addition to measuring an area and The subject of Construction Cost Estimation competencies provides material on how students can calculate the cost budget plan and make progress from the planned schedule of a building construction work. Applying the time schedule with Ms. Projects. Make technical analysis as a reference for determining unit prices. In the Building Construction and Utility subject, students are given the material on how to draw floor plans and details on architectural, structural, and electrical mechanical drawings in building construction. Furthermore, the subject of Creativity and Entrepreneurship Products competencies provides students with materials on how to make, calculate, test prototypes and packaging of a product, do product marketing and make financial reports. In addition, conducting product inspections in accordance with the eligibility criteria and compiling a descriptive description of the products that have been developed.
The CFA approach also confirms the suitability of the two models for measuring competency achievement. As a result, the model can be used to measure the achievement of student competencies in architectural engineering education programs in Indonesia. The reliability of achieving the objectives for the Indonesian sample is largely acceptable. The findings of this study are consistent with previous studies (Chai et al., 2019;Hidayat et al., 2018;Mateus & Hernández-Breña, 2019). The results of the analysis of convergent validity and discriminant validity have fulfilled the multivariate analysis requirements.
The urgency level of competence based on the average assessment results of participants from the construction industry obtained a category that is needed for current job competencies. All competencies consisting of 30 statement items obtained an average of 3.508 out of 4.00. Of the ten competency aspects, the largest mean acquisition was in the competency aspect of land measurement techniques (3.649), then estimation construction costs (3.583), and software applications and building interior design (3.565). This shows that the construction industry places more emphasis on these three competency aspects to serve as job competencies in construction services, especially architectural engineering in Indonesia today. Overall, every competency aspect has met the work competency requirements that are currently required. Meanwhile, of the 30 items of competence that are needed by the construction industry based on obtaining data, namely competence in calculating the estimated cost of construction work, evaluating and improving measurement results in the form of working drawings for construction work, and creating interior designs with elements, materials, models, and accessories in every room.

Conclusions
This study aims to investigate the competency aspects needed and must be mastered by students to evaluate the competencies that a graduate has in architectural engineering education in Indonesia. To test this model, an empirical study was conducted. The contribution of this research is multifaceted and provides the following theoretical and practical contributions.

Theoretical Implications
The first contribution of this research is to reveal competency aspects that can measure the level of competency attainment of students in architectural engineering education in Indonesia. Proving the validity and estimating the reliability of the instrument is acceptable. The results of the EFA analysis found that the structure of measuring the level of achievement of student competencies in architectural engineering education in Indonesia has ten factors. These ten factors are competency aspects that must be mastered by a graduate in architectural engineering education in Indonesia. The analysis confirms that these ten aspects can be a useful scale for measuring the achievement of student competencies for graduates of an architectural engineering graduate in vocational education in the Indonesian context.
The contribution of these two studies revolves around developing a multi-dimensional model to reveal how many competency items are used to measure the level of competency attainment of students in architectural engineering education in Indonesia based on 10 competency aspects. This model was developed based on an intensive literature review and analysis using construct validity testing. The test for competency items that passed the construct validity was 30 out of 40 items offered. This model has substantially explained 72.039% of the variation in the competencies that must be mastered by a graduate of architectural engineering education.
Finally, this research presents an important theoretical contribution to the field of vocational engineering education to architectural engineering education in Indonesia. In addition, this study confirms the validity of the two models developed to measure the level of competency attainment of students in architectural engineering in Indonesia. This is a recommendation to evaluate the competencies possessed by architectural engineering graduates in Indonesia.

Practical Implications
Because it is very important for vocational-technical education institutions in Indonesia that provide architectural engineering to plan the competencies that must be applied in educational programs as well as in the learning curriculum. On the other hand, the study results and recommendations must be considered to improve the competence of an architectural engineering graduate according to the needs and demands of the construction industry as well as efforts to reduce unemployment in architectural engineering majors in Indonesia. This study provides practitioners with the following practical contributions.
1. The results of the formulation and classification of competencies according to the needs and demands of the construction service industry can be used as material for information, suggestions, and input related to planning, implementation, and supervision of school programs, especially in implementing school curriculum programs, especially architectural engineering education in Indonesia.
2. As monitoring and evaluation as well as developing cooperation with the construction service industry such as intern ships and practices in industry and compilation of competency curricula in architectural engineering education.
3. Establishment of cooperation with the construction service industry so that the competence of graduates is by their fields and expectations

Declaration of Conflicting Interests
The authors declared no potential conflicts of interest concerning the research, authorship, and/or publication of this article.

Funding
The authors received no financial support for the research, authorship, and/or publication of this article.