Fgure 1. Student in VRSemLab
ALLPORT, Christopher, SINES, Paul, SCHREINER, Brandon & DAS, Biswajit
Department of Computer Science & Electrical Engineering, West Virginia University, PO Box 6109, Morgantown, WV 26506-6109, USA, vrlab@csee.wvu.edu
Abstract: Virtual Reality (VR) promises to enhance education beyond the level of modern multimedia computer systems by adding a three dimensional, "hands on" approach. Although computers already feature complex modeling and simulated 3D effects, virtual reality provides an immersive environment allowing complete interaction. Traditional computer interfaces suffer from limited workspace and unrealistic display of 3D objects. Although this is not a limitation for some applications, understanding of principles involving spatial relationships can greatly benefit from virtual reality. VR gives the user control over a 360° work space, as well as providing a true, stereoscopic, depth-sensitive environment. VR empowers users to DO, as well as to SEE. The Virtual Reality Semiconductor Laboratory (VRSemLab) is a pioneer system exploring the application of VR techniques to the teaching of semiconductor physics. This project addresses the extreme difficulties students typically have while trying to learn this subject. Elecrons are too small for many students to visualize. Holes present even greater difficulties due to their abstract nature. VRSemLab draws on the strengths of the computer as a modeling tool without succumbing to the limitations of a two-dimensional work space. VR will provide the ability for the user to see the internal structure of the semiconductor, use a virtual space controller (space ball or glove)to rotate it, inspect it from a different angle, and change the intrinsic physical properties of the device.
Keywords: virtual reality, visualization, 3D, training
Visualization and interactivity play important roles in the learning process, in particular for the understanding of complex and abstract concepts. The development of computer based training materials, such as multimedia courseware, is partly motivated due to the fact that they can provide improved visualization and interactivity abilities. Also, the availability of increased computing power at affordable costs is making such systems accessible to larger and larger populations. There are a number of software packages currently available for the development of computer based instructional materials which provide excellent interactivity and visualizations [1-6]. However, the applications of these software packages are limited to a two dimensional computer screen. While such systems are sufficient for a number of applications, they are inadequate for many applications where visualization and interactivity are needed in three dimensions.
Often, it is impossible to devise physical experiments which significantly aid student understanding. In general, this problem occurs when dealing with extremely large or small quantities, complex physical shapes or arrangements, inconvenient or dangerous locations, or complex sets of nonsequential actions. For example, lab experiments cannot convey an appreciation for atomic or galactic scales and chalkboard sketches cannot adequately depict intricate 3D structures such as electron orbitals. Reduction of 3D systems to 2D models is sometimes applicable, but not always.
In some situations, 3D visualization is not necessarily essential to learning, but can make a significant contribution to the depth and completeness of a student's understanding. One example of particular relevance to an electrical engineering program is semiconductor physics. For years, instructors have watched students struggle with the abstract concepts of "holes", tunneling, breakdown, and bipolar junction effects. Electrons are difficult to visualize since they cannot be seen. Holes pose even greater difficulties for visualization since they are defined as the absence of a particle. Recombination, tunneling, hole-electron generation, and breakdown phenomena are often described in such a convoluted manner that students are mystified by these seemingly "magical" behaviors. Many students would benefit from a system which could help them understand that these effects are more than just theoretical mathematical constructs.
As a demonstration of VR's potential for teaching complex themes, we have developed the Virtual Reality Semiconductor Laboratory (VRSEMLAB). It is designed to aid in teaching semiconductor device physics to undergraduate college students. VRSEMLAB demonstrates an effective method for modeling abstract and complicated 3D systems. It does so by presenting semiconductor devices in an experimental environment free of concerns of electrical safety, equipment limitations, and device limitations.
The remainder of this paper is organized as follows. Section 2 provides a brief description of the system developed and the problems addressed by VRSEMLAB. Section 3 provides details of system implementation, and section 4 discusses some of the results of system use.
The Virtual Reality Semiconductor Laboratory (VRSEMLAB) is a pioneer system exploring the application of VR techniques for the teaching of semiconductor device physics. VRSEMLAB was developed specifically to address the extreme difficulties undergraduates students in electrical engineering face in learning this subject (Figure 1).
Fgure 1. Student in VRSemLab
Electrons are certainly too small to see, and their interactions are so complex that they are difficult for many students to visualize. Even more confusing is the introduction of the abstract concept of "holes," an allegory electrical engineers use to describe a bond where an electron could be accepted, and the movement of that bond. VRSEMLAB provides the ability for the user to see the internal structures of semiconductor devices, use a glove to manipulate the virtual device, inspect it from different angles, and change the intrinsic physical properties of the device. The interface was designed to provide the user with comprehensive, asynchronous control over the lab environment, which allows the user to affect the amount of material covered during a given session resulting in a self-paced learning environment. One of the primary design goals of VRSEMLAB involved performance. Since the target group for this program is undergraduate college students, it was thought that the target system should one that is typical for an academic environment. The initial version of VRSEMLAB was developed on a Pentium Pro 200 MHz PC with 128 Mbytes of RAM. Though this system provided enough speed to develop and test the system, it is not powerful enough to fully exploit all of the amenities of VRSEMLAB. It is predicted, however, that current technology has enough processing power to adequately run the simulation.
VRSEMLAB is organized into several main segments. A set of interactive tutorials presents the users with the opportunity to receive detailed information regarding any of several specific semiconductor phenomena. 3D animated conceptual models accompany audio explanations and furnish users with a multi-sensory approach to learning. The set of individual concepts is drawn together in the dynamic device operation. The effects of an experimentation is displayed instantaneously on a complementary output display. A data bank of 15 programmed preset can be exploited when the student wishes to experiment with the behaviors of real semiconductor materials. Additionally, help is available through the use of an additional VR control button.
The system was designed for quick and easy setup; once the hardware is connected, the VR peripherals (HMD, VR glove, etc.) can be donned in under one minute. The HMD has 3 degrees of rotational freedom activated. The glove has all 6 degrees-of-freedom activated. The glove is represented in the virtual environment by a hand avatar. An avatar being a graphical representation of parts of the user which are actively interacting with the virtual environment. The avatar echoes onto the display at the respective position and orientation of the user's hand in space.
The user interface consists of an immersive 3D environment displayed through an HMD. The user sees a simple room containing an environment control panel, a help selection board, a wall display of numerical characteristics, and a large display of the active device/tutorial. The user receives pre-recorded instructions through the HMD's earphones. The user can interact with the environment by using the VR glove. The glove allows the user to grab and move virtual objects and has a simple gesture recognition system affording the user more interaction.
The device/tutorial selection board of the help selection panel permits the user to select either a device model or a tutorial to run. Activating a tutorial present the user with the opportunity to learn about some of the most common semiconductor phenomena or topics. The phenomena presented depends upon the selection of the user. The supported phenomena are recombination, carrier flow, punch-though breakdown, ionization breakdown, and Zener breakdown. Each tutorial consists of a detailed graphical and auditory description of the designated concept. Activating the device from the selection board causes an enlarged graphical model of the indicated device to appear (Figure 2).
Figure 2. VRSemLab Diode
Also, as shown in the picture, particle polarity is indicated by color and is further explained to the user via the audio. A simple DC biased P-N junction device (diode) is included as an example of a semiconductor device. The display models electron-hole flow in the selected device and features an audio/visual explanation of device operation.
Particular attention is paid toward representing carrier distribution. These particular characteristics are demonstrated using an object animation. The animation shows the two types of materials (p-type and n-type) before forming a junction, through the combining of the junction, and all the way through recombination of electrons and holes. Four key frames from this animation can be seen below. The first part of the sequence shows the two semiconductor materials; one p-type material represented by the silver cube, and one n-type material represented by the green cube (Figure 3a). The next frame shows the instant the two materials are brought into contact with one another. Nothing has started to happen at this point (Figure 3b). In the third frame, the electrons, the red objects, from the n-type material start diffusing into the p-type material whereas the holes, the blue objects, start diffusing from the p-type to the n-type material. The gold ball demonstrate the recombination of an electron with a "hole" (Figure 3c). In the final frame of the sequence, the depletion region of the semiconductor is demonstrated (Figure 3d).
|
|
|
|
Figure 3a. P- and N-Type Materials |
Figure 3b. Ideal P-N Junction |
|
|
|
|
Figure 3c. Carrier Recombination |
Figure 3d. Depletion Layer Formed |
The most fundamental goal of VRSEMLAB is to convey to the users a qualitative sense of how different parameters affect device operation. Thus, VRSEMLAB allows the user to change device parameters such as physical dimension, doping levels, operating temperature, and carrier mobilities. As each of these parameters is changed, the system dynamically recalculates the new device parameters. In order to change the parameters, the user must interact with the parameter control panel. Once the desired parameter is selected, the user can slide the lever up and down to increase and decrease, respectively, the corresponding parameter. The calculations are based upon fundamental equations and assumptions of semiconductor behavior. Device operation is modeled over three distinct ranges: low field operation (<1kV/cm), high-field operation (1kV/cm < 100kV/cm), and very-high-field operation (100kV/cm). The assumptions made in the Drude approach are used to circumvent solution of the Boltzmann transport equation. Depletion region dimensions are calculated using a solution to the Poisson equation developed by applying the depletion approximation. While device parameters are alterable, dynamic parameters are not supported. That is, the user may set a device temperature of 300K, but the device does not experience self-heating effects, and the user does not have the option to perform a temperature sweep. However, device performance information is directly observable to the user by means of the 3D device model. Information is also presented in written and audio formats, depending upon user selections.
Multiple help resources are available to the user. These resources are designed for natural interaction with the goal of further reducing user interface complexity. The available help resources will consist of wall displays or audio messages. Error and warning messages are reserved for system status information, such as an unresponsive glove, and for invalid user processes. The help panel (Figure 4) consists of information devoted to specific topics.
Figure 4. Help Panel Display
One topic is a periodic table. This is an abridged table intended to highlight elemental semiconductors. In addition, a brief overview of compound semiconductors is another topic. Another selection is the ability to reset the device parameters to a common semiconductor material. There is also a brief VRSEMLAB users manual which contains instructions for common operating procedures.
Exiting VRSEMLAB is straightforward from the user's standpoint. When finished with the session, the user simply removes the HMD, thus exiting the VRSEMLAB program. As an alternative escape mechanism, keyboard controls can preempt execution at any time.
Before implementing VRSEMLAB, a considerable amount of effort was expended in refining the details of the user interface. Initially, VR was chosen because it allows excellent visualization of and interaction with spatial relationships. The next concern was how the user would interface to the VR. The two typical models of VR, as mentioned previously, are desktop and immersive. Desktop refers to VR that is done on a single screen or computer monitor whereas immersive VR provides for a complete 360° workspace. VRSEMLAB was designed with the immersive approach in an effort to create a more natural environment for the user. One intent was to limit the amount of training time needed to teach someone how to use the system. Utilizing paradigms existent in typical electrical engineering labs, users would already be familiar with how to use the system.
The VR hardware was selected based on two criteria: cost and performance. For the display, General Reality's Cyber-Eye Monoscopic HMD was selected. In VRSEMLAB, true stereoscopic 3D was sacrificed for performance. To simulate 3D, context clues such as depth cues were used. For the interactive device, the 5DT Glove was chosen. This interface uses optical fibers and an array of optical sensors to determine finger flexion. To track the head and hand positions, Polhemus' InsideTrack magnetic tracking system was used.
With the hardware in place, attention was next focused on VR software implementation. After an exhaustive search of available packages, EAI's (formerly Sense8) World ToolKit (WTK) was identified to be the ideal VR library. WTK is a comprehensive library of C functions which takes care of virtual reality simulation management. It provides a high level interface for many commercially available VR hardware products.
Two of the most significant features of WTK are the simulation manager and the scenegraph manager. The simulation manager maintains all of the different features of the virtual environment until execution is terminated. In the default configuration, the sensor values are read first. This allows the head and hand positions to be in the proper place in the frame currently being rendered. Next, the action function is called. The action function is the primary place for users to develop the simulation physics. After that, object and viewpoint positions are updated, and the scene is finally rendered.
At the heart of WTK's operation is a construct called a scene graph which is gaining increasing popularity in VR software. A scene graph is a way of systematically arranging VR simulations to properly illuminate, transform, and isolate simulation objects. The basic units of this scene graph implementation are nodes which come in several different species. The root node is the base of the scene graph to which the rest of the nodes are attached. Light nodes instruct the renderer on the lighting parameters for the simulation. Group nodes allow amalgamation of sub-groups which all relate together. The next three are the most crucial types of nodes. The simplest to understand is the geometry node. It holds the actual geometrical data of the 3D objects. The transform nodes hold rotation and translation information for the geometries, and the separator node keeps the effects of the transform node from propagating throughout the rest of the scene graph. For the sake of clarity, separator, transform, and geometry nodes have been grouped into one node called a moveable node.
The implementation of the device was done using the model discussed above. Many decisions about the actual programming of the device were made with speed in mind. As with the stereoscopy, some realism was traded-off for speed. In its current form, the diode is hard-coded into VRSEMLAB. The next step in its implementation will be to convert the code to be object oriented. This will allow more general interactions of semiconductor materials, hence the lab will no longer be limited to diodes, it will also be able to simulate transistors and other devices.
Since the inception of this project, much has been learned which would contribute significantly to the maturation of VRSEMLAB. Upon completion of the prototype system, many things were observable which were not predicted at the onset.
Gesture recognition based movement and interaction in a virtual environment is difficult to use and difficult to implement. Although the VR glove used provided adequate and relatively stable feedback, there are additional considerations. Perhaps the most difficult problem to overcome is calibration. The glove is designed to be "one size fits all," and while this is true, different results are obtained for people with considerably different sized hands. Coordination and strain proves to be another difficulty of gesture-based interaction. People frequently have difficulty making certain gestures with their hands, and if they do make the gesture, their muscles tire and it is difficult to maintain the gesture. Another problem with the glove interface is related to the speed of the overall system. Virtual movement did not parallel actual movement closely, so users had a tendency to overcompensate hand movement and miss the various "targets" which caused frustration. Furthermore, the system would occasionally perform an action for a gesture that it detected, but the user did not intend to make. Gesture-based interaction requires one to be especially conscious of their hand movement which results in a less comfortable user environment. Plans to rectify these problem currently include switching to a "laser pointer" type system with an identify button (for help) and an activate button (which would replace touch).
The only other negative result of any concern was the speed of VRSEMLAB. As was discussed above, the system on which this software was developed was not believed to have enough processing power to run the simulation. Once the semiconductor device was activated and reached steady state (that is, after all of the calculations were done), the frame rate of the system was approximately 1.5fps. A factor of 10 is needed to get the system up to the design goal of at least 15fps. Efforts are currently in progress to port VRSEMLAB over to a system with more computing power. Current Pentium III class systems should have sufficient power to attain the desired frame rate.
The greatest success of this project was the development of the virtual diode. The device demonstrated all of the specified phenomena. Aside from some extremely minute details, the virtual diode mirrored actual device operation. Parameters of the device and its operation, such as ambient temperature, bias voltage, carrier density, carrier mobility, etc., could be changed while the device was active. The software dynamically updated the parameters of device operation. Future plans for the device is to port it to an object oriented implementation. Upon completion of that task, a desktop version of the device simulator will be created since not everyone has the resources to run this software in a full immersive mode.
Another success of this project was some of the metaphors for interacting in a virtual environment. The "slider" device control panel (Figure 5) proved to be the most exciting 3D-widget, or "morsel," created in this system. It provided excellent control over selecting the desired value and was user-friendly. The customizable wall display was another useful development since it allowed the users to choose what data they wanted to observe. This display, used in conjunction with the slider, allows the user to see how the values of different parameters change as the current variable being affected by the panel is changed.
Figure 5. Device Control Panel
[1] Davis A. Bobick, S. Intille, F. Baird, L. Campbell, Y. Ivanov, C. Pinhanez, A. Schutte, A. Wilson, and G. Davenport; "HidsRoom : Action recognition in an interactive story environment", MIT Technical Report #398, MIT Media Lab, (1996).
[2] J.J. Cromby, P.J. Standen, J. Newman, and H. Tasker; "Successful transfer to the real world of skills practiced in a virtual environment by students with severe learning difficulties", Proc. ECDVRAT : 1 European Conference on Disability, Virtual Reality and Associated Technologies : pp. 103 - 107 (1996).
[3] C. Dede; "The evolution of constructivist learning environments : Immersion in Distributed Virtual Worlds"; Educational Technology 35(5), pp. 46-52 (1995).
[4] R. V. Kenyon and M.B. Afenya; "Training in virtual and real environments", Starkfest Conference proceedings, 23 (4), pp. 445 - 455 (1995).
[5] W.K. Muller, B.H. Wilson, M. Li and N. Qiu; "Virtual reality in surgical arthroscopic training", Journal of Image Guided Surgery, 1 (5), pp. 228 - 294 (1995).
[6] J. Psotka; "Immersive training systems : virtual reality and education and training", Instructional Science, 23 (5-6), pp. 405-431 (1995).
