EXPERIENCES USING AN OPEN SOURCE SOFTWARE LIBRARY TO TEACH COMPUTER VISION SUBJECTS

Machine vision is an important subject in computer science and engineering degrees. For laboratory experimentaton, it is desirable to have a complete and easy-to-use tool. In this work we present a Java library, oriented to teaching computer vision. We have designed and built the library from the scratch with enfasis on readability and understanding rather than on efciency. However, the library can also be used for research purposes. JavaVis is an open source Java library, oriented to the teaching of Computer Vision. It consists of a framework with several features that meet its demands. It has been designed to be easy to use: the user does not have to deal with internal structures or graphical interface, and should the student need to add a new algorithm it can be done simply enough. Once we sketch the library, we focus on the experience the student gets using this library in several computer vision courses. Our main goal is to fnd out whether the students understand what they are doing, that is, fnd out how much the library helps the student in grasping the basic concepts of computer vision. In the last four years we have conducted surveys to assess how much the students have improved their skills by using this library.


INTRODUCTION
Computer vision is an important subject in computer science degrees.For laboratory experimentaton (practcal teaching), we need a complete and easy-to-use tool.In this work we present a Java library, which is oriented to teaching (JavaVis, 2015).For this purpose, we have designed and built the library bent on readability and understanding rather than efciency.
We have developed three diferent modules to meet the three diferent needs we have spoted as fundamental in our subjects related to machine vision.The frst need is to develop a basic library to process images.The library allows easy sequence processing by incorporatng geometrical data (edges, segments, points, etc.) and implementng several well-known computer vision algorithms.Furthermore, the design allows new methods and algorithms to be added to the library in a simple way, invitng the students to develop their own algorithms almost efortless.The second need is to apply the same working procedures as previously, but contemplatng 3D data.The third need is to actually see the results.For this purpose, we have developed a visual desktop to visualize how the diferent algorithms work and what the results of combining them are.
The rest of the paper is organized as follows: Secton 2 enumerates the subjects where JavaVis are currently being used.Then, Secton 3 describes the pedagogical issues to be addressed with this library.In Secton 4, we describe the main features of JavaVis.Inside this secton, we detail the three diferent parts of JavaVis: JavaVis2D for processing 2D images, JavaVis3D for 3D data, and JavaVisDesktop a visual desktop tool used to learn and build machine vision algorithms.Secton 5 shows frst the teacher perspectve and then, relates the students' experience using JavaVis together with some analytc data from questonnaires.Finally, we draw our conclusions and propose future work lines.

COURSES DESCRIPTION
JavaVis is currently being used in the teaching of several subjects in Computer Science degree at Alicante University in Spain.It was previously used in the delivery of some subjects in master and predoctoral degrees.Since 2001 we have been using JavaVis as a framework to teach Computer Vision in related courses.We shortly describe all of them: • Artfcial Intelligent Techniques: This subject consists of one hour and half lecture and one hour and half lab session per week for ffeen weeks.Eight weeks of the course are dedicated to computer vision and the remaining to machine learning.In the part dedicated to Computer Vision, we use the tool presented in this paper as the basis for carrying out the lab assignments.Sometmes we are able to use JavaVis in the whole course.
• Computer Vision: This subject consists of two-hour lectures and two-hour lab sessions per week for ffeen weeks.It is dedicated to 3D vision, object recogniton, image segmentaton, feature extracton, etc. JavaVis is used for laboratory experimentaton throughout the course.
• BS Degree course: Students tend to favour to complete projects related to computer vision; some of them are implemented with this tool.In fact, the library began its development as a BS Degree project.Throughout the years JavaVis has been used in more than 20 projects, which in turn have contributed with code and new ideas to the library.
• Postgraduate courses: JavaVis is used as a supportng tool in several Master subjects on Computer Vision and Image Processing.It has been also used in several PhD theses.
When students complete any of these courses, they inevitably acquire sound programming skills.However, the concepts related with Computer Vision can sometmes be so complex that it is necessary to have a tool easy to learn and use.

PEDAGOGICAL ISSUES
Javavis library is a teaching tool that we use as teachers to implement our methodological proposals.There are many diferent teaching tools available, each one oriented to a partcular learning actvity.In our case, our main pedagogical goal is to serve as a supportng tool for the laboratory experimentaton.As a teaching tool, the library must satsfy several fundamental requirements: • It must facilitate the learning/teaching process.
• It must contribute to an enjoyable but rigorous learning.
• It must help students to learn (and successfully pass) the course.
• It must motvate students to do research on its applicatons.
• It must enable the interacton between the teacher and the students.
In our opinion, JavaVis satsfes all of these requirements, becoming a powerful tool to support the learning and teaching process.
• The aim of the learning/teaching process is the student training.So, there are two agents involved in this process (the teacher and the student), dealing with some contents that must be taught (learned) and some resources that must be taught (to learn them).JavaVis eases this process to a great extent.
On the one hand it helps the teachers in their pedagogical task because it is an easy to use tool and it provides a friendly graphical environment.On the other hand, it improves the learning task: the student can clearly understand the algorithms thanks to the code inspecton.For the learning process to be constructve, the student must analyse and construct it.As a consequence it will bring a real assimilaton of the knowledge in the long term instead of a mere weak and short-term learning.
• A lot of computer vision algorithms involve complex mathematcal concepts.But our students are not studying Mathematcs as a degree.So we must try to combine a rigorous learning process with an easy and helping way.The library promotes this goal because the students can clearly see the implemented algorithms, which are explained in the theoretcal session.This way they can beter understand concepts and see how the algorithms work.
• JavaVis includes several practcal applicatons, most of them being the result of past courses projects.This fact motvates the student, because the students realize that other students in the past have developed parts of the library.In fact, several students (one or two per year) have been involved in developing some important parts of JavaVis.
In short, the use of this tool not only serves to strengthen the practcal skills of the subject but also substantally strengthens the theoretcal understanding and, thus, the overall understanding of the subject.

USING JAVAVIS FOR TEACHING COMPUTER VISION
A subject on computer vision usually includes the study of basic concepts together with algorithms to solve related tasks.A brief descripton of this kind of subject could be: • Image formaton.Cameras (calibraton, geometry), image representaton, colour and light.
• Object recogniton.Mapping, bag of features, face detecton JavaVis has been primarily developed to assist in the teaching of computer vision.This kind of concepts are, usually, hard to understand for the students.For example, gradients, which are defned as a derivatve of the image functon, are rendered by just a pixel diference.The student understands the concept beter when he/she sees the implementaton of the theoretcal concepts.
In JavaVis, we have developed three diferent modules, based on three diferent needs we have discovered in our surveys across the subjects related.The frst one is the need of a basic library to process images.Next, a special image format to enable easy sequence processing.For this purpose, geometrical data (edges, segments, points, etc.) are incorporated and several well-known computer vision algorithms are implemented.The students can, therefore, develop their own algorithms with ease --the second module consistently applies the same working schema of the frst one, but 3D data is applied.The fnal need is that of providing a visual desktop to visualize how the diferent algorithms work and the results obtained by combining them.The use of Java is justfed as our students use diferent operatng systems.Java is multplatorm and allows us to give a program, which can run in all of diferent operatng systems.
In the next subsectons, we briefy describe the three modules.Figure 1 shows the GUI for the three modules.

JavaVis2D
This part refers to traditonal computer vision and, in fact, it was our frst step.We could import diferent image formats even though traditonal computer vision has its own.In JavaVis2D each image is composed of one or several frames.A frame represents an image that can be either bitmap or geometric.A bitmap frame is an image represented as a matrix in which each element is a pixel (picture cell).JavaVis features fve types of bitmap frames: Bit (0,1), Byte (0..255), Short (0..65525), Float (foat range of Java) and Colour (three bands of Byte type each).In additon, each bitmap frame can be formed by one or several bands.Note that each band must have the same dimensions.Internally, each type stores a one-dimensional array of data to be more efcient.For example, the Bit type is stored using a Boolean Java type, the Byte type uses the Java type byte.
When accessing a pixel, the data is converted from its original value: for example, the byte type in Java is signed but the Byte type of JavaVis is unsigned.Conversion is done internally.A sequence can be organized in two ways: several frames in a sequence, or several bands per frame, which can also form a part of a sequence.However, to be able to insert several bands in the same frame, every band must have the same size and be of the same type whereas diferent frames can have diferent size, type and number of bands.A geometric type frame, on the other hand, manages informaton diferently.For example, the Segment frame type just stores the coordinates of the inital and end points of a segment.A Segment frame may have several segment objects stored not into a pixel matrix but into a segment object list.So the representaton of geometric frames is not only more compact but its computatons are faster.This kind of frames is useful in several computer vision algorithms.
An important point in JavaVis2D is the organizaton of its functons (implementaton of algorithms).This organizaton becomes crucial for other users to be able to easily implement their algorithms in a standardized way.In order to implement an algorithm, the functon code needs to be developed and its input and output parameters specifed.This is precisely one of the main goals of the library: implement once, use anywhere.
When an algorithm is implemented, then the library is in charge of input and output parameters checking, showing images through the GUI, and so on.

JavaVisDesktop
Our goal in this project is to build a utlity which serves to beter understand the partal results when processing an image.We can use diferent algorithms to get a complex one: it is by means of the Desktop utlity that we are allowed to check partal results, by showing the images obtained afer applying an algorithm.
This utlity allows the building a sequence of functons as a fnite-state machine.Each state in the sequence is a functon (algorithm) from JavaVis.A state shows the result (an image) of applying such algorithm.Thus, the student can easily adjust diferent parameters of an algorithm, observing what the consequence of modifying that parameter in the complete sequence involves.Once the parameters are adjusted, there is an opton to generate a new functon in JavaVis2D directly.This new functon will contain the sequence of functons included in the desktop refectng the selected parameters.

Java3D
Since the Kinect device was introduced, the use of 3D data and its algorithms have been more popular.JavaVis3D module allows the representaton and manipulaton of 3D data sets from any 3D sensor, such as stereo cameras, RGB-D sensors, 3D laser, etc.Most of the basic 3D geometric enttes have been implemented in the core of JavaVis3D.Points, vectors, normals, segments and planes classes are included in a geometric package that allows JavaVis3D to operate with them, applying transformatons, performing input/output operatons, etc.We can load a 3D point set captured from a scene using a stereo camera or Kinect device, compute these points to obtain the normal vector for the scene surfaces at every point and fnally store this set of normal vectors in a fle to be used in the future.
JavaVis3D includes a GUI for showing the 3D data.This GUI is built through the 3D Java API Java3D.Java3D can be found for almost every operatng system.3D Data sets can be loaded/unloaded into the GUI.The user can freely transform the observer's point of view, the virtual camera so that he/she can have an adequate view of the scene.JavaVis3D GUI also provides an easy-to-use method for launching diferent 3D functons over the 3D data.We have also implemented some of the state-of-the-art 3D algorithms in JavaVis3D such as Iteratve Closest Point (ICP) (Pomerleau, Colas, Siegwart & Magnenat, 2013) to compute the transformaton that best aligns any two sets of 3D points.

TEACHING EXPERIENCE
JavaVis library has been used for 10 years as a tool to teach computer vision in the subjects detailed in Secton 2 at the University of Alicante (Spain).JavaVis has let us work at two diferent levels.First of all, it has allowed us to show some of the most useful and popular computer vision algorithms, as many of them are already implemented in JavaVis.At this level, students observe what happens when the algorithm already explained in the lecture is applied.Furthermore, they can take a look at the implementaton of the algorithms, also explained in the lectures, thus obtaining a deeper understanding.However, just by using algorithms, they are not learning the complexity of the implementaton, i.e. how much tme it takes to implement.So, the second level follows.Students can try to develop their own algorithm.At this level we aim at simplicity.Students do not have to know much about the library.What they must know the classes and methods for manipulatng an image, but nothing about graphical classes or fle structure.
During these four years, we have proposed several practcal assignments.We have followed a kind of projectbased learning strategy, but only in the practcal part of the subject.Students are grouped in groups of 3-4 people.A problem to be solved is presented to the groups.For example: we need a system that must be able to count the number of coins in a conveyor belt.This belt has a camera that provides images of the belt.The expected output is the number of coins at a given moment on the belt.Summarizing, each group needs to defne the problem and propose a viable soluton, taking into account all the possible problems (changing illuminaton, presence of others circular objects, occlusions, etc.).Some of the groups propose a similar soluton, based on the use of methods already implemented in JavaVis and some additonal code.First, students have to develop a non-implemented algorithm: Hough transform applied to circumferences.This functon is added to the library.Then, a new functon is created.This new functon calls for several functons: Canny (to obtain contours), Binarize (to binarize the Canny output), HoughCirc (the last functon developed, applied to Binarize output) and, fnally, we have to count the number of pixels inside the circumferences detected.This last step let us eliminate false coins --those which do not have enough pixels-in the domain defned by the coin and, having the same colour as the coin.Figure 2 shows an example of the executon of this assignment.
Figure 2. Two examples of the results of the lab assignment: countng coins.The white circumferences are the coins detected.Note that the CD is not detected as it has a diferent colour, and the inner circle of several coins is not detected either

Student opinion
During the terms from 2007/2008 all through 2012/2013, we conducted a survey from where we gathered informaton from the students who used JavaVis for their laboratory assignments.In 2007/08, 54 students answered the questonnaire, in 2008/09, 122, in 2010/2011, 47, in 2011/2012, 30 and fnally in 2012/13, 27.We wanted to know how JavaVis helped them in their overall lab assignment performance and in grasping theoretcal concepts.We also wanted to know if JavaVis was the tool we purported to be easy to understand and use.
We now show questons asked to the students and detail the results.The results from the student answers are shown in the referenced fgures.For the frst two questons, the students were required to answer them by selectng a value from 0 (beginner) to 5 (expert).For the rest of questons, the answers ranged from 0 (strongly disagree) to 5 (strongly agree).
• Your knowledge of Java programming language (syntax, arrays, lists, etc.).The results showed that almost all the students had an elementary knowledge of the Java programming language with a mean value around value 4.This allowed us to use Java as a programming language.Far from being a trivial queston, it proved not only that the use of Java was not widely spread but also that an introductory session was called for.
Figure 3.Your knowledge of Java programming language (syntax, arrays, lists, etc.) Vol. 5(3), 2015, pp 220 • Your knowledge of advanced Java programming language (threads, graphic user interface, etc.).With a mean in 2.1, the frst two terms and the fourth one and above 2.5 in one intermediate term together with the last one, we learnt that not all the students knew much of GUI API, threads and so on.Then, we desestmated introducing lab assignments in which students had to work with this programming language because it required an additonal efort on their part.This situaton created the need to build JavaVis as a tool to bypass advanced programming.The students, then, could entrely focus on implementng the algorithm.
Figure 4.Your knowledge of advanced Java programming language (threads, graphic user interface, etc.) • Laboratory assignments require a solid understanding of the Java programming language.In this queston, we wanted to perceive the students' general feeling from towards the language.The mean value was around 2.5, indicatng that the students did not need any advanced knowledge of Java language, which has been our goal throughout.• JavaVis Desktop framework is useful for laboratory assignments.As in the previous queston, there are diferences in the answers from term to term.In 2008/2009 and 2012/2013 many students did not apply the Desktop, so their answers refected 0 for almost half of them.Nevertheless, the students generally conceded that the framework was defnitely useful.
Figure 7. JavaVis Desktop framework is useful for lab assignments • JavaVis did help you to get a deeper understanding of the concepts.In this case, all the courses agreed: JavaVis helped them get a deep insight of the course contents.The mean value was 3.34, surprisingly higher than that obtained for 2012-2013.

Figure 8. JavaVis did help you to get a beter grasp of the concepts
• JavaVis is useful for laboratory assignment implementaton.For this queston we obtained the greatest support.Students agreed that this tool was defnitely useful.According to them, computer vision had been simplifed by JavaVis, with a mean value close to 4.5 at each year.

Figure 11. JavaVis is useful for laboratory assignment implementaton
The conclusion we draw is precisely what we expected: there is enough knowledge of basic Java programming, but a low level of advanced concepts.As the advanced skills (graphical interface, threads, etc.) are necessary for some computer vision tasks, it is a good idea to provide a framework to provide abstractons of those skills.
Students used JavaVis as a tool not only to carry out the lab assignments but also to learn theoretcal concepts.

DISCUSSION
Teaching experience is covered from two points of view.On the one hand, we take in account our teachers' experience in computer vision subjects.It allows us to stablish the basis of the proposal computer vision library.In this way, teaching experience tell us that it is worthy to make JavaVis an easy to use library for the development of new computer vision applicatons.The main eforts are then focused on ofer an abstracton layer that frees the students from tasks such as image management, applicaton layouts and events, or multthread programming.Thus, the teachers involved in computer vision subjects in which JavaVis is being used have reported that the use of the library makes easier the teaching of the subject as well as its assimilaton by students.
On the other hand, it is also important to know the opinion of the students who have developed their practcal assignments using JavaVis.Thus we made a questonnaire over fve years to learn how students perceived the provided tool.The questonnaire was focused on understanding diferent issues related to level of the students before and afer taking the course both in relaton to their programming skills and their knowledge of machine vision.In this way we could confrm that the programming skills of most students before taking the course is enough to develop a simple to medium applicaton.However, they ofen lack the advanced skills that would enable them to cope with programming mult-threaded user interface like the one they are provided with.As for the advantages of using the library, students positvely valued the tool as a whole.In additon, most students consider that the use of the library has been useful for assimilatng the concepts studied in the course.They themselves realize the advantage to have available an open source implementaton of the algorithms discussed in the lectures.Finally, for most students the use of JavaVis has made easier the resoluton of the problems they faced.

CONCLUSIONS
The contents of the subjects in computer vision could be tough for computer science students.In fact, the subjects dealing with computer vision are usually placed in the last years of an engineering degree.As the students' surveys indicate, the teaching/learning process of computer vision could be assisted using open source libraries or tool kits.This applies to not only the practcal issues but also the theoretcal ones the subject involves.The students are able to break the code open to beter understand how an algorithm works.Furthermore, the JavaVis Desktop is another useful tool for understanding the workfow of a complex algorithm.
For this reason, we have developed a new framework, JavaVis, which is teaching oriented.It is writen in Java and is Open Source.JavaVis is divided into three parts.First, JavaVis2D, which enables 2D-image processing.It features image format and geometrical data.Then, JavaVis3D, which supports 3D data, -3D points, planes, trajectories, etc.Finally, JavaVis Desktop.It enables the students to understand changes in parameters and develop new algorithms with certain ease.JavaVis has been used for several years in several computer vision-related subjects at University of Alicante (Spain).We are presentng a practcal example of JavaVis in this paper.We also discuss the results from the diferent surveys conducted for a number of terms and we have reached the conclusion that JavaVis has, in fact, proved useful for the students and helped them gain insight of theoretcal concepts.
We can conclude by saying that an Open Source library serves the students as a launchpad to keep on learning by themselves.
As future work we are planning to contnue incorporatng new computer vision algorithms and the features we identfy in our teaching.We want also to use JavaVis in conjuncton with OpenCV and PCL.

Figure 5 .
Figure 5. Laboratory assignments require a solid understanding of the Java programming language

Figure 6 .
Figure 6.You have used the JavaVis Desktop framework to do lab assignments