EMPIRICAL VERIFICATIONS OF WEB-LEARNING ENVIRONMENT TO ENHANCE LEARNING EFFECTIVENESS AND MOTIVATION IN MECHANICAL DRAWING

In mechanical drawing class, it is a problem that a lecturer has students get a wide range of knowledge and skills in a class whose time and place are limited in a semester. The situation surrounding the classes has led to insufficient understanding of students. Accordingly, it has been desired to reveal an effective learning method in mechanical drawing class. In this study, a web-learning environment was created on the basis of motivation-oriented teaching method using ARCS model and Gagne’s 9 events of instruction. Then, the web-learning environment was introduced into mechanical drawing class, and the effectiveness was investigated with the comparison of results from paper-based and web-based tests. After that, learning motivation was also assessed using ARCS evaluation sheet. From the results, there existed the advantage of using web-learning environment. In addition, it was remarkable that attention-related factors greatly contributed the enhancement of learning motivation.


Introduction
Among various subjects with respect to mechanical engineering, design and drawing allow students to learn important knowledge and skills related to manufacturing. The knowledge and skills are particularly essential to obtain a design solution satisfying required specification for designing a machine in practice. Although the time of education is not necessarily sufficient, the course hours have been gradually decreasing. The fact indicates that it is getting difficult for students to learn a wide range of knowledge and skills in mechanical design and drawing. The tendency will probably continue from now on, so that the students' understanding would inevitably deteriorates more and more. From the backgrounds, there are studies with many kinds of viewpoints regarding design and drawing. Huerta, Kus, Unver, Aslan, Dawood, Kofoglu et al. (2019) developed an AR/VR application to support the learning and teaching of design and engineering students in education and industry. Azodo (2017) studied the effects of attitude related to engineering students' learning in engineering drawing. Halim, Yasin and Ishar (2012) discussed students' and teachers' perception on an innovative teaching method of engineering drawing subject at secondary school level. In addition, they reported the effects of teaching problem-solving strategies in the engineering drawing subject on student achievement, students' knowledge of problem-solving and students' problem-solving skills . Katz (2015) proposed integrated thinking as a new approach and showed how it can be applied to various mechanical engineering courses for undergraduate students. Adanez and Velasco (2002) gave an investigation of a visualization psychometric test that could facilitate an early diagnosis concerning the academic performance of technical drawing students. dos Santos, Gonçalves, de Oliveira and Silva (2018) reported a new structure in the educational methodology related to project-based learning. However, the efficient learning environment has been still desired to provide instructive opportunities for not only overcoming the related problems but also obtaining a lot of knowledge and skills within a reasonable time scale.
These days, various information technologies for education are being widely used to enhance the learning and teaching effectiveness. Especially, e-learning is well known as a useful method with several merits. To name a few, we can learn what we want to know without a lecturer. It can provide ubiquitous learning environment available from mobile devices. The system enables us to use a wide variety of learning contents. Individual-oriented instructions also act as a helpful function in some cases. In creating a web-learning environment, each component such as web design is created situationally with learning theory to enhance both learning motivation and learning effectiveness.
Gagne's theory of instruction is one of the effective theories in education. The related works have been reported so far in a variety of fields. Baba, Sale and Zirra (2017) created student-centered learning environment designed using Gagne's theory of instruction, and the multimedia program was applied to education in secondary school. Mei, Ramli and Alhirtani (2015) studied Gagne's nine approaches in teaching Arabic to non-native speakers. Their work led to the importance in education and teaching.  developed e-learning content for advertising theory applying Gagné's nine events of Instruction. The content worked properly in the advertising classes. Khadjooi, Rostami and Ishaq (2011) investigated the applicability of Gagne's model of instructional design in teaching psychomotor skills. Gagne's strategy was also used to a development of a conceptual understanding in mathematics (Erlinda & Surya, 2017). Kutlu and Menzi (2013) examined the effect of internet-based instruction with Gagne's theory in primary schools. Cheung (2016) also studied a teaching method with Gagne's theory for medical procedure. Al-Shalabi, Andraws, Alrabea and Kumar (2012) proposed a V model of E-learning using the well-known Gagne nine steps for quality education. Meanwhile, ARCS model was proposed by Keller (1987). ARCS stands for Attention, Relevance, Confidence, and Satisfaction. The model is one of the useful tools for instructional design to improve learning motivation directly. The overview was given by Arora and Sharma (2018), and many types of contribution have been made with ARCS model. The relationship between ARCS model and persuasive strategies was investigated to provide empirical insight into the mechanism through which persuasive technologies persuade (Orji, Reilly, Oyibo & Orji, 2018). Zhang (2017) designed a micro-lecture teaching platform based on ARCS model. The results showed the suitable characteristics of platform. Lee and Kim (2012) developed Web-based courseware using ARCS model for lower elementary level, number sense concepts. Effectiveness of ARCS Model of Motivational Design was discussed to overcome the motivational problem of distance learning students (Malik, 2014). Huett, Kalinowski, Moller and Huett (2008) examined the improvement of motivation and retention of online students through the use of ARCS-based e-mails. Kurt and Keçik (2017) studied the effects of ARCS model on students' motivation to learn English. Although these approaches have been demonstrated to be very useful in education, authors are unaware of the applied study of ARCS model in mechanical design and drawing.
The purpose of this study is to demonstrate the availability of two educational approaches in mechanical drawing education. In this study, a web-learning environment with instruction of the third angle projection method was created based on motivation-oriented teaching method using ARCS model and Gagne's 9 events of instruction. Then, it was introduced into a mechanical drawing class for Japanese students in our university. After that, the effectiveness was investigated with the comparison of results from two types of evaluation tests. Moreover, the influence of web-learning environment on learning motivation was clarified by questionnaire investigation with a modified ARCS evaluation sheet.

Design of a Web Learning Environment for Mechanical Drawing
There are a lot of research works associated with ARCS model (e.g., Keller, 2010). As one of the outcomes, a motivation-oriented instruction and design procedure were proposed from a problem-solving perspective of learning motivation (Suzuki, 1995). Gagne's 9 events of instruction have drawn academic attention in pedagogical field since it can enhance students' learning motivation. With the proposed strategy, the related studies have been conducted to reveal learning effectiveness from various viewpoints (e.g., Suzuki, 2000;Wang, Ikeda & Li, 2007). Web-learning environment developed in this study was created based on the two instructional design theories. We created web-learning environment through applying some illustrations extracted partially from a test book for mechanical drawing (Hayashi, 2013). The purpose of creating a website was to enhance learning effectiveness and learning motivation of students attending a class of mechanical drawing in our university.
Top page is firstly displayed as shown in Figure 1 when a user accesses the website. This page expresses the site map used for website's design. The learning contents were classified into several topics in the site map. This website design is useful to learn and search what learner would like to know repeatedly. The next page was designed as shown in Figure 2. This page is placed on Lesson 1. Here, example problem is provided to make learners aware of learning objective. Essential information is subsequently given in Lesson 2. Figure 3 expresses a part of Lesson 2. The third angle projection method is explained with effective illustrations to assist an understanding of projection drawing method. In addition, some video contents were prepared as shown in Figure 4. The deficient information in two-dimensionally displayed contents is complemented by using the contents, and they enable us to learn practical drawing techniques in mechanical design. A learner can intuitively interpret essentials through the contents in each webpage based on problem-solving approach.  Lesson 3 illustrates essential information for 3-view drawing. After that, Lesson 4 provides several problems similar to the example problem in Lesson 1. One of the problems is shown in Figure 5. This page is very useful to give learners experience of various exercises. The practices can be repeatedly implemented for 3-view drawing based on the third angle projection method; moreover, these answers are checkable with reference to video contents as exhibited in Figure 6. Finally, evaluation test is given in Lesson 5 to check learner's understanding for the third angle projection method. Figure 7 shows an example of the tests. The test was also designed for a practicality evaluation of web-learning environment used in mechanical drawing class. The answers are quickly saved to a file on a cloud system.
The other advantages were made for further convenience. In each page, the hyperlink for next page is placed on the bottom of current page. We can accordingly go to the next web page by using the hyperlinks. The page for summary was also designed to give a rise to learners' interest in mechanical design. In addition, the page for inquiry was created for learners who would like to get the advanced information.

Introduction of Web-Learning Environment in Mechanical Drawing Class
One of the mechanical drawing classes in beginner's level is annually held for first-year students in our department. There always exist more than 100 Japanese students attending the class, and almost all of the students do not have any knowledge associated with mechanical design. Web-learning environment created in this study was introduced into the class for validation through a practical application. The validation was performed in the past two years. A paper-based evaluation test was used in the former year, while a web-based evaluation test was given in the latter year. The difficulty levels including the time limit of each test were adjusted to the same degree. The third angle projection method is explained with its practical training after some fundamental knowledge necessary for mechanical drawing is offered in the previous classes. In the conventional class, students had taken a paper-based evaluation test as shown in Figure 8. The test-takers had to answer some questions to check the understanding of knowledge related to the third angle projection method. They also had to depict the 3-view drawings for each projection view. The answers and drawings were quantitatively scored by experienced designers. The depth of understanding for the third angle projection method was inevitably reflected in the 3-view drawings.
On the other hand, students different from the former ones took a web-based evaluation test explained in the previous section after they also learned some fundamental knowledge necessary for mechanical drawing. In the both years, the textbook and reference books were announced at the beginning of mechanical drawing class. The test preparation can accordingly be made with these materials. In addition, web-learning environment was available anytime and everywhere since the test was noticed in the latter year. The answers were automatically collected and scored in the web-learning environment with a cloud system. The scored results can reveal the depth of understanding in the third angle projection method. The effectiveness in introducing the environment was assessed based on the comparison of two evaluation tests.

Investigation of Learning Motivation
The investigation was made for the influence of web-learning environment on learning motivation. The participants were 10 Japanese students belonging to school of engineering in our university. Their ages were from 21 to 22 years old. All the participants had already taken mechanical drawing class when they were first-year student. Accordingly, they had the relevant knowledge in mechanical drawing, while there were individual differences in the amount of knowledge. In the investigation, two types of problems were assigned for each student. The one is multiple choice questions to check the fundamental knowledge regarding the third angle projection method. These questions were associated with not only the relevant knowledge, but also an interpretation of three-dimensional shape. The other is to make 3-view drawing from an isometric projection. A fundamental assignment to create the drawing on a paper was set with reference to those in a paper-based evaluation test. The difficulty levels in each question were adjusted to the same degree.
In the test, the participants were classified into two groups. To review the relevant knowledge in mechanical drawing, the textbook and reference books were available for the participants in each group. In addition, the only one group was allowed to use web-learning environment. Then, the questions were assigned for each participant. The answers for multiple choice questions were verified after the test, and the drawings were quantitatively scored by experienced designers.
Questionnaire investigation were also made for all the test-takers. In the questionnaire, we used ARCS evaluation sheet including 14 questions based on 4 factors of ARCS model (e.g., Murakawa, Sakaguchi & Imanaka, 2014). This sheet had Likert scale of one to five in each question. Mann-Whitney U test was used to detect a significant difference due to the small number of samples. Learning motivation was measured based on subjective evaluation by the questionnaire for the test-takers. Table 1 shows the comparative result of two evaluation tests. The correct answer rate was calculated based on the average value of all scored results. It was found that the correct answer rate in the latter year was really higher than that in the former year. The effectiveness of web-learning environment was numerically obvious from the result, while there were several differences between a paper-based and web-based evaluation test. When we checked the system usage of web-learning environment in the latter year, almost all of the students used the entire contents explained in the previous section before the evaluation test.

Former year
Latter year Correct answer rate 53.8% 87.6% Number of research participants 146 108 Table 1. Correct answer rates in each evaluation test As described above, the web-learning environment was organized with Gagne's 9 events of instruction which could give students the effective learning procedure to obtain the necessary knowledge. Thus, it is considered that the effective learning procedure can improve learning effectiveness even in mechanical drawing class. On the other hand, the correct answer rate in the former year was almost half although students should use textbook and reference books in the test preparation. The fact implies two causes. The one is that it is difficult for beginners to grasp key learning points in mechanical drawing since there are too much relevant knowledge to make a suitable drawing. The other is that the learning order designed by students themselves is doubtful to obtain the relevant knowledge efficiently. From the result with implications, learning contents organized using Gagne's 9 events of instruction are invaluable for learning the essential knowledge in mechanical drawing. However, it must have been relatively easy for students to get tired with learning the contents when there is too much relevant knowledge to be learned. In spite of that, it was notable that high completion rate was achieved in web-learning environment. We attempted to investigate the interesting point through the next test in a different manner.

Influence of Web-Learning Environment on Learning Motivation
Questionnaire with ARCS evaluation sheet was assigned for each test-taker to investigate an influence of web-learning environment on learning motivation. Table 2 shows each average score calculated from the results of subjective evaluation by the questionnaire after the test. In the table, the mean of each average score in all questions was 3.89 for the group using web-learning environment. The standard deviation was 0.51. In contrast, the mean of each average score in all questions was 3.03 for the other group. The standard deviation was 0.59. The results indicated that the mean for the group using web-learning environment was higher than that for the other group. Furthermore, the standard deviation for the group using web-learning environment had a smaller value than that of the other group. From the results, it was obvious that web-learning environment demonstrated an advantageous effect on enhancing learning motivation for the test-takers. The average scores of three attention-related factors also showed noticeable differences. The fact clarified that attention-related factors especially contributed the enhancement of learning motivation. From the results, motivation-oriented teaching method with problem-solving strategies is effective to design a web-learning environment with illustrative and explanatory materials in mechanical drawing class. A proper design with suitable contents can lead to enhancing learning motivation of learners. Moreover, the high completion rate is achievable with a proper design even when there is too much relevant knowledge to be learned. One of the possible factors helpful for achieving the high average scores in questionnaire investigation is also that the web-learning environment enabled us to learn something with keeping their paces.
Mann-Whitney U test was carried out to detect a significant difference in the results. The U value is 36, and the two-sided asymptotic p-value was 0.001. There was a significant difference between the two groups since the p-value was less than 0.05. We also checked Cronbach's coefficient alpha for the validation of evaluation scale. As the result, the value of Cronbach's coefficient alpha was 0.988 in all the scales of 14 questions, so that internal consistency was confirmed by the value exceeding 0.90.

Conclusion and Future Work
In this study, a web-learning environment with instruction of the third angle projection method was created based on motivation-oriented teaching method with problem-solving strategies. The web-learning environment was introduced into mechanical drawing class, and the effectiveness was investigated with the comparison of results from two types of evaluation tests. Then, the influence of web-learning environment on learning motivation was clarified by questionnaire investigation with a modified ARCS evaluation sheet. As a result, learning contents organized using Gagne's 9 events of instruction are invaluable for learning the essential knowledge in mechanical drawing. Moreover, a proper design with suitable contents can lead to enhancing learning motivation of learners. The high completion rate is also achievable with a proper design even when there is too much relevant knowledge to be learned.
Future research could focus on the relationship between usability and web-learning environment. In addition, further detailed investigations using the web-learning environment will be continuously made to accumulate research findings helpful for developing pedagogy and educational technology.