SALZMANN, Christophe1, LATCHMAN, Haniph2, GILLET, Denis1 & CRISALLE, Oscar3
1 Institut d'Automatique, Ecole Polytechnique Federale de Lausanne, (Swiss Federal Institute of Technology), CH-1015 Lausanne Switzerland, Tel. (+41) (021) 693.51.68, Fax. (+41) (021) 693.25.74, christophe.salzmann@cise.ufl.edu
2 Electrical and Computer Engineering Department, University of Florida, Gainesville, Florida 32611-6005 USA, Tel. (352) 392.49.50, Fax. (352) 392.00.44, latchman@list.ufl.edu
3 Chemical Engineering Department, University of Florida, Gainesville, Florida 32611-6005 USA
Abstract: In traditional engineering classes, concepts taught through lectures are often reinforced by practical experimentation carried out in laboratory sessions that are attended by the students at the physical site of the facilities. A new paradigm to make the experimental activities available to remotely located students is presented in this paper. The development of remote-experimentation facilities or virtual laboratories is motivated by the fact that presently, as never before, the demand for access to instructional laboratory facilities is growing rapidly in all engineering colleges. The proposed solution allows students to have access to the laboratory facilities via the Internet from any place and at any convenient time.
In addition to simply observing an experiment performed, remote experimentation adds closed-loop feedback to the experience in a virtual or remote laboratory which is essential for an intelligent management of the available bandwidth. This work proposes an intelligent use of the bandwidth in order to achive an acceptable Quality of Service (QoS) for remote experimentation over the Internet.
Remote experimentation is almost trivial when you have a deterministic medium with a given or reserved bandwidth to transmit information. At the present time the Internet does not belong to this category and the proposed approach first investigates the communication requirements for the particular case of remote experimentation with multiple data structures and formats for various types of multimedia and control traffic. We then identify the key characteristics representing the state of the network linking the student clients (users) to the experiment (server). Based on actual measurements, we develop a model of the network connection between Ecole Polytechniqe Federal de Lausanne (EPFL - Switzerland) and the University of Florida (UF - Gainesville Florida). A control strategy which takes into account the proposed connection model, the estimation of the available bandwidth and the user QoS settings is presented.
This work emphasizes optimizing the transfer rate and size of the various prioritized information streams with reference to the available bandwidth, so as to give the client the best possible experience and perception of the real process. We will describe and demonstrate a prototype system which has been built
Keywords: Online Laboratories, Internet, Control, Real Time Control
The advent of the Internet as a major communication channel has triggered a great deal of interest in real-time communication over packet-switched networks. While store-and-forward networks such as the Internet were not originally designed to handle real-time traffic, now that this global communication infrastructure exists and is becoming ubiquitous, computer-based hardware and software subsystems are being designed to transport such real-time services as broadcast audio and video (live or via various streaming media technologies) and even interactive audio and video services. Interactive services rely on real-time adaptation of the transmitted content to the available bandwidth. To ensure a given Quality of Service (QoS), adaptation can be performed at different levels ranging from the user interface level to the router level.
The availability and the performance of these communication facilities combined with the generalization of computer use for data acquisition and control of real processes have led to the implementation of the telepresence paradigm. Telepresence is the interaction with real equipment from a distant location.
In distance learning, this paradigm is applied to enable remote experimentation. Direct contact with the physical process, of which remote users are deprived, is replaced by broadcast images and sound. This operation can be performed by adding a digital camera and a microphone near the equipment, and by sharing this information with the user through the network.
Remote experimentation is facilitated if performed via deterministic media with a given or reserved bandwidth to transmit information. The present work investigates and proposes adaptive solutions to use the Internet as a support for remote experimentation in the domain of automatic control.
Traditional classroom teaching gives the students only few opportunities to experiment with the subject matter by themselves, a process that is in fact the best approach to learning. This is especially true at undergraduate levels in engineering where abstract representations can attractively be complemented by practical experimentation. As a consequence teachers have increasingly integrated real-world experimentation based on laboratory-scale processes and Computer-Aided Instruction (CAI) tools in their courses [GIL91]. In automatic control this experimentation material helps students observe dynamic phenomena that are often difficult to display in a classroom by traditional means. Furthermore interactive demonstrations improve the motivation for learning [GIL93] and also develop an engineering approach to solving realistic problems.
Electrical drives are one of the typical processes used for laboratory experimentation in automatic control. A common setup consists of a motor, equipped with a built-in tachometer, that drives a load (steel disk). Measurement and control of the process are achieved by the mean of an AD/DA card connected to a computer. No other hardware is required beside this board.
The software provided to control the real process, a virtual instrument (VI), allows the students to carry out the different steps of the experiment such as generating excitation signals and observing corresponding responses. The virtual instruments’ Graphical User Interface (GUI) forms a layer that gives a complete view of what is happening in the real process, and allows full control of the operations.
As we will see later, the fully computer-based implementation for the local experimentation is a key point for making it accessible via remote experimentation.
The increasing number of students, the constant or decreasing amount of allocated resources, the demand for more flexible class schedules and the changing life style are forcing universities to find an alternative to traditional on-campus classes. Distance education has been around for some time. Videotapes and written material were sent by postal or courier services to remote students. This very unpratical solution has been greatly improved with the advent of the WWW. Students can take advantage of this media-rich source of information at any time and at any place. Complete courses are now provided online using the Web channel forming an Asynchronous Learning Network (ALN). However, virtual classes should be designed to recreate the close interaction between students and between students and professor to avoid the drawback of a lack of interactivity [LAT98].
Currently, virtual universities mainly rely on electronic documents and multimedia presentations enriched with video and audio broadcasting. Since students still have to come on campus to take lab sessions, remote experimentation can overcome this limitation by allowing students to take lab classes in the same manner as ex-cathedra classes. Allowing students to carry out in-time experimentation whenever they feel the need to compare their knowledge to reality is also an important part of their learning process.
Moreover the existence of facilities for distance learning is becoming an alternative for the people who look for the best possible education. "Customers" who want to find the best educational institution will trigger a competition. Enlarging their current course spectrum with remote experimentation could give the real and/or virtual universities a decisive advantage over their competitors.
Remote experimentation is not limited to education. In research and industry remote access also represents an interesting opportunity to meet the growing need of scientists and engineers who wish to share unique or expensive equipment.
Local experimentation has been conducted successfully for many years and has proved to be very effective for student learning. Distancing the user from the local experiment while keeping the same amount of benefits is challenging. Not only the same degree of interactivity must be kept, but the remote solution must also allow the user to feel the real experiment. During local experimentation the student can use their senses of vision and hearing to perceive the effect of their acts on the control system. In a remote experimentation mode, this specification can be addressed by providing audio and video feedback information in addition to the information given to the remote computer through the Graphical User Interface (GUI). Obviously, such feedback needs to be given in a reasonable amount of time, minimizing the misleading (and most likely also disturbing) effects of signal-transport delays. For example, a remote user will not accept feedback that arrives 30 seconds after an action has been accomplished when the local response is achieved in fractions of a second. Consequently, fast system responsiveness is a key goal in all developments for remote experimentation. As one might expect, ideal instantaneous responses are not possible but response time should be minimized.
Not having the feeling of the real experiment would be similar to having a local simulation of the process under studies. Remote experimentation adds the possibility to "remotely touch" the experimentation. "Touching" the remote process can have different meanings. For example, perturbing the real process by pushing a moving part to observe the reaction of the system. A more important meaning is to be able to "see" phenomena which are hard or impossible to simulate otherwise. A saturation in the system or a loose gearing can be very difficult to simulate and the model could become very complex. Since local simulation is only as good as the model used is, remote experimentation has the advantage of making the user aware of problems that are invisible in simulation and allowing him to define and test a solution to solve them. Due to the computational complexity of a model, remote experimentation on real processes is also easier to implement and more versatile compared to experimentation in virtual reality. We will argue later that simulation capabilities can be of significant added or complementary value in some specific cases.
The interactivity must be adequate, otherwise the gain induced by remote experimentation will be minor and this facility will most likely not be used. Defining a high quality of service is difficult and many contradictory parameters are involved. The interactivity provided is also a function of the equipment the user wants to control. The objectives for a fast application such as motion control where the user should be able to catch the dynamics of the process are different from the objectives of a slow chemical plant where the user mostly needs to deal with the complexity of the system. The proposed solution focuses on fast dynamic systems such as electromechanical processes. Since they have mobile parts, they can be remotely monitored by cameras. In the case of visually static systems such as thermal systems, remote observation can be enabled by the use of sensors that modify their appearance according to the measured state.
The Internet offers many advantages over other technologies, which makes it the medium of choice. The first advantage is its price and its availability. Nowadays almost anyone who has a telephone line is eligible for an Internet connection provided that there is an Internet Service Provider (ISP) close by to avoid paying an excessive communication price. Accessing the Internet through a telephone line is almost always possible and moreover at a very reasonable price. Current technologies such as ISDN or 56k modems give us enough capacity to transmit voice with good quality and video with acceptable quality. Real-time communications are possible as has been demonstrated in audio and video conferencing.
Current implementations of the remote experimentation paradigm lack some of the important capabilities and compensate for it by demanding more resources than needed. Theses experiments all suffer from the same problem: they rely on a fast, high-capacity, un-congested network to achieve acceptable performance. As soon as the network performance decreases, synchronization between data and video becomes problematic, jitter or lack of feedback disturbs the user, and good interaction is no longer guaranteed, making this solution inappropriate for regular use.
With an intelligent use of the available capacity, acceptable Quality of Service (QoS) for remote experimentation over the Internet is possible. While there are clearly some similarities in the QoS requirements of remote experimentation with, for example, live video or audio services, this paradigm introduces some new and unique perspectives such as the separation of the transmitted data into various data streams generated by the remote control process, each stream having its own size, priority, delay acceptance and time accuracy requirements. The architecture is based on a client-server configuration and a master-client mode of service. The user wants to see the effect of his action as soon as possible. Real-time, bi-directional communication, which is non-existent in video streaming, must therefore be considered.
Knowing in advance the physical and dynamic proprieties of the system under study offers the possibility to predict the future states of the system. This prediction, which is not available in video conferencing, permits alternative compression schemes for data and video. A tighter integration of the video image with an alternative representation of the process can also be achieved.
To be widely accepted the solution must be cross-platform and cost effective. Ideally the client application is free. These conditions imply a software-only solution.
The term "real-time" originally implying the process control, is nowadays also used to describe systems that do not control processes but depend on time information. A video sequence is a good example of a system that depends on time to have a good synchronization between frames or a good synchronization between audio and video.
"Real-time control" reinforces the original definition by specifically emphasizing the close-loop and temporal aspects of the problem. The feedback aspect is very important and can be observed at different levels. At the application level, the feedback can be observed between the user action and the user observation of results on the systems. For example the user presses a button on the user interface and waits to see the effect on the video feedback. At the controller level, feedback is provided by the algorithm that computes, at a regular pace, the output to be applied to the system based on the measured inputs. Feedback can be observed at the network level when the packet-loss information is taken into account to adapt the source offered load.
Section 2 covers the different problems and solutions in the literature related to bandwidth adaptation for remote control over the Internet, while Section 3 describes the remote experimentation paradigm and requirements. Section 4 gives a brief overview of actual Internet measurements made between the Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland and the University of Florida (UF) in Florida to develop the generic Internet communications model used in this study. The Section 5 discusses the control strategy and related issues for the implementation of remote experimentation and conclusions and suggestions for extensions to this work are given in Section 6.
There are obviously some similarities between internet-based video conferencing and real-time control with video feedback since both applications require bandwidth adaptation for audio/video transmission and remote control over the Internet. However, as we will see real-time control applications have several unique characteristics which must be considered. In this section we briefly survey some early efforts at dealing with these issues.
The first web-based control implementation only allowed the transmission of text and still image pages. Netscape then introduced the notion of server-push which allowed the web server to send information to the client browser at "regular" intervals [NSP97]. This technique is used to transmit images in real time (mainly JPEG) to form a video sequence. Images captured by a camera and made available using web technology blossomed all around the world [YAH97, NAS97]. Web servers could communicate with the outside world using CGI so that clients could control actuators connected to the web server. The ASEA-Irb-6 robot is one of the first physical experiments (1994) made available on the Internet [TAY97]. Users could control the robot arm gripper to manipulate wooden blocks on a table. The server read the user commands entered in an HTML form, performed the required operation and returned a page containing an updated image (GIF) of the setup. The plain HTML page was then abandoned and the control of the robot is now performed through a Java interface.
Different projects for controlling physical experiments through the web were developed over the years. One famous example is Telegarden [STE96]: people can plant, water and monitor the progress of seedlings via an industrial robot arm in a garden filled with living plants.
Web push technology however suffers from a main drawback: it takes too long to transmit video images and the disturbing jitter introduced by the network cannot be removed on the client side. The video feedback, which varies from 1 to 10 seconds, prevents "real-time" communications. Wolf and Froitzheim proposed a solution [WOL97] to improve pushed images bandwidth called Adaptive-JPEG (A-JPEG). It combines the animated GIF technology with a JPEG compression. A-JPEG technology combined with other web improvements is used by the Interactive Model Railroad [IMR97] to control the destinations of two locomotives.
Web clients use plug-ins to allow "external" applications embedding. The plug-in can use an independent channel of communication and therefore implement any of the teleconferencing or video-broadcast solutions.
The LAMI project called "KepOnTheWeb" focuses on the possibility to control a small robot called "Kephera" within a maze [MIC97]. The user downloads the algorithm on the robot and watches it moving through the maze. Different solutions have been tested and evaluated for the video feedback: Internet, ISDN and ATM. It is also possibile to control the "Kephera" robot directly by using a WEB page. Users who are waiting to interact with the robot can control a virtual robot in a virtual maze.
One of the main advantages of the web-based solution is the integration of different media. By combining class material, digitized lectures and other information, courses can be taught asynchronously in time and space [LAT98]. Remote classes can be extended with web-based exercises and on-line or virtual laboratory experiments.
One of the earliest (1995) applications remotely controlled through the Internet was developed at the Oregon State University. The purpose of this application called "Second Best Being There" (SBBT) [AKT96a, AKT96b] is to control a 3-degree-of-freedom robot arm so that it plays music on a 6-key "piano" [SBB97]. The students are able to develop, download, compile, debug and run controllers in real time while having full control over the laboratory environment, including power control of the devices. They can monitor the laboratory with the audio and video tools (NV) relying on ATM to have good transmission.
Another early experiment in this field was conducted at EPFL. A video camera was mounted on a "Khepera" robot. The images captured by the camera were transmitted to the client on the Internet via CU-SeeMe. The user could control the motion of the robots via a CU-SeeMe plug-in especially designed for the purpose [SAU95]. The project evolved and became "KhepOnTheWeb" [SAU98].
In 1996 the Institut d’automatique (EPFL) made three of its laboratory setups available on the Internet. These setups (inverted pendulum, electrical drive and helicopter model) were controlled locally by students during regular class hours [GIL93]. A network layer was added to the local application that controlled the real processes. This layer augmented with a commercial teleconferencing application allowed students to control these experiments.
Since no mechanisms were used to adapt to the available bandwidth, this application, as well as SBBT and others, suffer from the same problem: they rely on a fast, high capacity, un-congested network to achieve acceptable performance. As soon as the network performance decreases, synchronization between data and video becomes problematic, jitter or lack of feedback disturbs the user and good interaction is no longer guaranteed, making this solution inappropriate for everyday use. Requirements for real-time laboratory experimentation over the Internet are defined in [SAL98] and presented Section 3.
Analytical solutions are proposed when the transmission delay is known or within a certain range. Göktas et al. [GOK97] used robust control theory to handle the worst case delay in mobile robot platform over a campus network (155MB ATM). Kevin J. Brady [BRA97] proposed a state space formulation that takes into account the time-varying, non-deterministic nature of the control and observation delays.
In summary it appears that most of the early remote experimentation applications were built on top of videoconferencing tools and assume an deterministic network connection to achieve pseudo real-time performance. In the next section we discuss an approach specifically tailored to for real-time control over packet switched networks.
Figure 1. A physical system (inverted pendulum) communicates with a local server via AD/DA cards, and a network connection gives access to multiple remote clients.
The configuration of the remote-experimentation system is shown in Figure 1, where the pendulum system is shown along with the DA/DA cards that connect it to a local server. The server in turn displays all relevant signals to a number of remote clients that access the local systems via the Internet. The server also receives commands from selected remote clients, and implements the commands locally on the physical system. The server and the computers platforms for the clients have identical display screens, thus giving the remote users the opportunity to interact with the physical system in a fashion analogous to the way local users do. The network transmits input and output data streams as well as audio and video information.The user can interact in real-time with the experiment through a graphical user interface (GUI) built using the LabVIEW graphical programming language [16]. This interface (Figure 2) has been designed to be the same for a local or a remote experimentation. It is composed of an oscilloscope window where the measurements done on the real process (position, angle, etc.) are displayed. Four sliders represent the parameters provided by the user to the controller. Local users have the benefit of being able to introduce perturbations by making physical contact with the pendulum; for example, by touching the moving arm. The "hand" button shown in Figure 2 is used to introduce a perturbation via a software command, a feature of particular utility when the users perform remote experimentation. In the inverted pendulum case, the server simulates a perturbation by adding an error to the angle measurement. Less important information such as the sampling period or the connection states are accessible through an extra window.
Figure 2. Graphical user interface that allows the user to interact with an inverted-pendulum experiment in real time. The area displaying a hand is used to introduce a perturbation.
An effective remote experimentation setup must satisfy a number of requirements. In particular, the user needs to feel like he/she is physically located next to the real experiment. During local experimentation the user can use his senses of vision and hearing to perceive the effect of his acts on the control system. Under a remote experimentation mode, this specification can be addressed by providing audio and video feedback information in addition to the information given to the remote computer through the GUI. Obviously, such feedback needs to be given in a reasonable amount of time, minimizing the misleading (and most likely also annoying) effects of signal-transport delays. For example, a remote user may not accept as real-time a signal that arrives 30 seconds after an action is taken when in fact the local response is achieved in fractions of a second. Consequently, fast system responsiveness is a key goal in all developments for remote real-time control. As may be expected, ideal instantaneous responses are not possible. In our experience, 2 to 5 seconds of response time have proven to be adequate values for transatlantic experiments.
A second issue of importance is to design the overall system in a highly modular fashion. For example, our prototypes consist of three basic modules (Figure 3), namely (1) a real-time module which is responsible for executing local control actuation on the real system, (2) a GUI module that displays data and that manages the user's communication with the real system, and (3) a network module that manages all the Internet transactions. This modularity permits quick and bug-free reconfigurations of the same software to address different needs. For example, taking the GUI and the real-time modules one can build a local set-up. A network server might not need a GUI, since in this case only the real-time and the network modules may be needed. Finally, a network client only needs the GUI and the network components.
A third issue of relevance is the addition of the capability for carrying out simulation studies, where the response to all user actions are produced by a software representation of the physical process rather than by the process itself. This allows the user to evaluate different operational scenarios before attempting the real experimentation. Note that in some complicated systems the simulation of the physical process might be difficult. Skeptics might argue that the whole idea of a laboratory experience is to work with the real process, and hence there should be no emphasis placed on simulation work. We argue later that in some cases simulation capabilities can be of significant value.
Figure 3. The three basic components of the design: (1) real-time, (2) GUI, and (3) network modules.
The three basic modules shown in Figure 3 have specific functionalities and are responsible for different security processes that ensure the robustness of the set up. These are the Real-time Module, the GUI Interface and the Network Interface. We will now briefly describe each of these subsystems.
The real-time module allows the control of the real process from a computer via AD/DA boards that capture measurements signals and issue command signals. All actuations are handled by control algorithms that run on the computer’s main processor. Real-time control is achieved with the assistance of a Real-Time Kernel developed at the Swiss Federal Institute of Technology. The control algorithms are written in C or as S-functions in the format of the Matlab/Simulink software suite [19], and are executed by the real-time kernel and executed by the real-time kernel. This module must include low-level security procedures that prevent the user or the algorithms from carrying out either inadvertent or deliberate actions that might damage the physical experiment. When such procedures are activated, the real process is reset to a known safe state.
The GUI module (Figure 2) performs a display function, allowing the user to follow the time evolution of all signals of relevance to the experiment, such as the internal states of the controller, for example. In addition, through the GUI the user is allowed to also modify the parameters of the controller, as well as other adjustable characteristics of the experiment, like the sampling period for example. Higher-level of security precautions are handled at the level of the GUI module. For example, the GUI may prevent the user from selecting physically unrealizable parameters (such as a negative sampling time), etc. The graphical user interface is based on LabVIEW. A special Real-Time Framework [16] has been developed on the Macintosh platform to ease the development of integrated real-time controllers.
Finally, the network module allows the program to communicate with other computers distributed in different physical locations. This module also takes care of security issues regarding network management, such as preventing unauthorized access and scheduling access in order to avoid conflicts.
Two different kinds of software entities are involved in the communication process, namely, a server and its clients (Figure 1). The server is the local machine connected to the experiment. The server runs the algorithm used to control the experiment in real time. Although not strictly necessary, it may be convenient to use a server GUI similar to that of the clients; this is particularly useful if the server is sometimes used for local experimentation. A digital camera focused on the physical experiment and a microphone are connected to the server to respectively generate video and audio signals that are transmitted over the network.
The client software runs on the remote machines. Its main components are a network module and a GUI module. Each client can adopt two modes of operation, namely, a standard client or a master client mode, as described in the following section.
The typical operation of the system is through point-to-point sessions that progress according to a hierarchy of client requests. In this approach the server is connected uninterruptedly to the physical experiment 24 hours per day, waiting for clients to request access. If there is no client connected the server might, if appropriate, stop or put the real process in an idle state to save energy.
When a user wants to perform a new experiment (i.e., testing new parameters) from a remote location, he/she launches the client software and requests a connection to the server. The server allows the connection if the user has the appropriate permission and if the maximum number of connected clients is not exceeded. The user first logs in as a standard client, a mode that allows the observation of the experiment but does not allow issuing any commands to the physical system or its set up. In this mode the user receives audio, video, and data streams from the server.
The user can then place a request to the server for a connection as a master client, a mode that permits issuing commands that are accepted by the server as legitimate manipulations on the physical process. If the user has appropriate permission to become a master client, the request will be placed in a queue, and the user will remain in the standard client mode waiting for the request to be approved by the server. The server assigns the status of master client for a pre-defined period of time to the client on top of the master-client request queue. Only one client at the time can have the status of master client.
The master client may introduce manipulations to the local system, such as modifying the parameters of the controllers, and then observe their effects. At any time the user can elect to quit from the master mode and return to the standard mode, surrendering control of the experiment. The server forces the transition to the observer mode whenever the pre-assigned connect time expires.
In addition to the point-to-point mode, it is also possible to operate the system in a multi-clients session where the master client is the instructor for the class who carries out a demonstration that can be simultaneously observed by all remote and local students participating in the class.
Different kinds of information are exchanged between the server and the clients according to four different classes, namely, (i) the parameter stream, (ii) the data stream, (iii) the administrative stream, and (iv) the audio/video stream. These streams must share the available bandwidth (which in turn might be different for each client). A comparative summary of the characteristics of all four streams is given in Table 1.
The parameter stream consists of control packets sent by the master client to the server. This stream holds the highest priority. In our inverted-pendulum prototype the parameters are the four gains of the state controller, the sampling period, and the state of the "hand" button used to introduce perturbations (see Figure 2). These values need to be transmitted with a very high priority in order to assure the best possible responsiveness. This is quite easy to achieve since the amount of data transmitted is normally very small. For example for a PID (proportional-integral-derivative) controller, the data required may consist of only three values, namely the values of the P, I, and D constants. A packet is exchanged every time the user changes one of these three parameters. Since the parameter stream provides information that directly manipulates the physical system, the data transmission must be highly reliable. Consequently, additional computational requirements may be imposed by the need of encrypting and/or error control encoding the values transmitted to avoid deliberate or inadvertent corruption of the data during the transmission phase.
The data stream consists of all signals of the physical process which are measured, such as outputs, set-points, inputs and computed internal values of the controller, for example the error between the set-points and the actual position. This stream flows exclusively from the server to the clients and has a priority just below the parameter-stream priority. All values are sent in records consisting of n measured values for each of the measured signals plus the internal values of the controller. The size of the data to be transmitted depends on the amount of data the process generates, and needs to be adapted to the available bandwidth. Solutions such as compression or decimation can be used if the amount of data produced by the process does not match the available bandwidth. The data stream is broadcast by the server to the clients. In this process, packets can be lost or may arrive out-of-order. A buffer can partially solve this problem and smooth the stream of data. The buffering time must be small in order not to increase the latency of the entire process. Another solution is to reconstruct locally the data using a real-time simulation that makes use of a model of the physical system. The model could feature the use of parameters that are constantly re-identified, updated, and broadcast by the server. In this case it is important to alert the user whenever simulated data is generated. The integrity in time of the data stream has only to be guaranteed when a data history is requested by the user (for playback or off-line analysis purposes). For live interaction, this is not quite as critical.
| Stream | Direction | Priority | Bandwidth | Size | Encryp-tion | Packet Drop |
| Parameter | Client to Server | highest | low | small | yes | not allowed |
| Data | Server to Client | high | high | large | no | allowed |
| audio/video | Server to Client | medium | high | largest | no | allowed |
| Administrative | Client to/from Server | Lowest | low | smallest | yes | not allowed |
Table 1. Characteristics of the four types of information streams used in real-time remote experimentation.
As an example of the size of a record in the data stream, consider a situation where the controller works with a 10 millisecond sampling period and there is a circular buffer on the server side which can hold 500 points (equivalent to 5 seconds of data). There are four values measured, and each measurement is stored in the form of short (4-byte) records. If the server sends new measured data every 2 seconds, then the packet will consists of 1600 bytes (i.e. 200 points x 4 values x 4 bytes).
The administrative stream is mainly used at the beginning of the exchanges between a client and the server. It is also used when a standard client requests to become a master client, and vice-versa. Typical information exchanged in this stream are the username and the password; hence this stream needs to be encrypted. This data exchange goes both ways between the client to the server. The information exchanged is very small in size and has the lowest priority.
Finally, the audio/video (A/V) stream is broadcast from the server to the clients. This stream is analogous to the data stream, but it holds lower priority. The A/V stream demands the most bandwidth for timely and accurate transmission; however, in the configuration proposed it actually uses only residual bandwidth due to its lower priority. This is a key difference between the proposed paradigm for remote real-time control and the usual video conferencing/broadcasting solutions. The A/V information is important to the remote user since it provides a facility for "seeing" and "hearing" the effects of all manipulations. Clearly, better quality of the A/V transmission leads to a more satisfactory experience for the remote user of the virtual laboratory.
Some of the solutions used for the data stream can be applied to lower the bandwidth required for the A/V stream. No real distinctions are made between the audio and video part of the stream even though in our prototypes the need for the video feedback is obvious while the need for the sound feedback is more dependent to the experiment. One particular case is the example of an electrical drive set-up that produces an audible vibration if the parameters are not adequately chosen. In this case the audio signal is very important because the video image would not reveal the problem due to the small displacements of the drive that are realized. For our other prototype systems the sound does not provide useful information and can be switched off.
The four different streams needed for remote experimentation have different transmission priorities. The amount of data for each stream varies from a few bits per second for the parameter stream to the full bandwidth for the A/V stream. Prioritization of the transmission is necessary to guarantee that the most crucial information is transmitted in a timely fashion. The parameter stream has the highest priority since it contains critical information concerning parameter modifications and other adjustments that the clients make on the physical system. A parameter modification in the client interface is sent to the server via the parameter stream and its effect is sent back to the client via the data stream. Hence, the data stream, which acknowledges all modifications and reveals their effects, receives the second-highest priority.
The cumulative round-trip time in this parameter-data exchange is the sum of the transmission times for each two streams. Typically, the time for the server to process the parameter adjustments can be neglected. Naturally the cumulative time in the exchange should be as small as possible. Our experience shows that user acceptance is very good when the cumulative time ranges from 1 to 2 seconds. When the cumulative time exceeds 5 seconds the user needs to adapt to the delay and the interactivity decreases considerably. When more than 10 seconds are needed to see the effects of a parameter modification, the user tends to resends the parameter modification because he/she did not get a visible acknowledgment. Most of the time this leads to an unwanted cycle in the loop user/remote-process, and makes the remote experimentation impracticable. In order to partially overcome this limitation, visual information in the GUI can monitor and display the state of the parameter transmission. When such a delay is present, the whole process should be switched to a batch mode where the user sends the operation to be performed to the server and watches the result in a delayed time. While this can no longer be considered real-time remote experimentation it certainly provides a workable alternative for Internet access to physical facilities. Indeed, there are strong parallels in this regard with packet telephony and packet video services where network latency is sensed periodically and the length of playback buffers is adjusted to facilitate smooth replay.
The A/V stream has the third-highest priority, since the information transmitted by this stream is to some extent redundant to the data stream. The A/V stream serves to give an added sensorial experience to the activity; hence, its role is typically less crucial. The data and the video stream should be synchronous. If the video frame rate drops the user needs to be able to determine the relation between the video frame and the displayed measure.
It is desirable that the remote experimentation system should be available 24 hours a day and should require minimal local maintenance. That means the process should be able to go back to a known safe state– for example return the pendulum to the center of the track– immediately after an undesirable state or a dangerous situation is identified. Such undesirable situations may occur due to a number of reasons, including a user action, the selection of wrong parameters, a network problem or a loss of connection, etc. Precautions should be taken so that the physical system is protected from damage in all contingencies.
On the other hand, the precautionary procedures should allow the user to operate sufficiently close to undesirable states so that he/she can learn from the experience. For example, specifying a large gain in the controller that adjusts the arm position will cause the arm to develop position oscillations. This will progressively move the arm to one end of the track where it will activate a built-in position sensor that disconnects the controller and safely reposition the pendulum away from the end of the track. Of course, the precautionary procedures should be robust enough to guarantee recovery from unwanted states, such as when the user specifies exceedingly large controller gains.
The user in a remote location should also be able to perturb the physical process in order to evaluate the ability of a set of control parameters to reject the effect of the induced perturbation. For example, in the case of a mechanical system, a brake could be used to change the acceleration of the process. A mechanism for performing such perturbations must be integrated in the software and made available to the remote user. In the case of our inverted pendulum prototype, when the user selects a perturbation box featuring the drawing of a hand (see Figure 2), an arbitrary bias is temporarily added to the signal sent to the controller, hence effectively inducing a large step-change type of disturbance on the arm position.
A growing collection of software is now available to transmit or broadcast audio and video through the Internet. One may argue that video broadcasting/conferencing is similar to the remote experimentation paradigm, and indeed, this analogy does hold in the sense that both send audio and video images. Furthermore, the data stream (measurements) can conceivably be interpreted as audio or video data. However, there are significant differences.
The main difference between video conferencing and real-time remote experimentation lies in the fact that in the latter the emphasis is on transmitting and receiving the most recent data, even if this is done at the expense of discarding older or out-of-order data. In contrast, video conferencing software tries to send all the data, recent and older, without any perceivable losses from the user’s point of view. Consequently, if possible, no information is dropped, and buffer sizes are increased in order to smooth the transmission. Note that under this perspective buffers should be avoided as much as possible in real-time remote experimentation.
The original version of our system used a commercial off-the-shelf software [18] to transmit audio and video signals. This solution gave acceptable delays– approximately one to four seconds– for experimentation conducted within the local university LAN. Unfortunately the performance became unacceptable for transatlantic experiments because the delays became significantly greater and were highly unpredictable. Since in this case the stream was not adaptively managed by the server, it was not possible to synchronize the A/V stream with the data stream and manage their relative priorities. This problem is more important than transmission delays because the user may be misled about the actual state of the process and may not be able to determine which information to trust.
Another difference lies in the fact that in video-conferencing/broadcasting preference is given to the audio over the video since the human ear is very sensitive to the continuity of the sound. Interruptions of the sound stream, even for short periods of time, are disturbing for the user. In contrast, the opposite is true for the vision since the eye can easily adapt to "jerky" images. However, with some exceptions, in remote experimentation the video stream is often more important than the audio stream because the visual information plays a larger role in making the user participates in the experiment with sensory information.
Most video conferencing/broadcasting software tend to increase the buffer at the reception end in order to smooth the transmission. Increasing buffers means adding delays which does the opposite of what the remote experimentation tries to do. In the later case older (out of order) information is dropped if newer information is available. However, the removal of buffers has its drawbacks. For example, buffers are needed for compression, especially for video signals where more than one video frame is required by high compression-rate algorithms. Video compression may be in some cases unavoidable, depending on the size and the number of colors of the image to be transmitted. Typical compression schemes do not send any data if the image does not change between two consecutive frames. Naturally, when the image changes rapidly the compression scheme delivers a large amount of data. By lowering the frame rate one can reduce the size of the transmitted data, but this introduces some drawbacks. Lets take the example of a frame rate originally set at 10 frames per second which is to be reduced to 1 frame per second. If the codec (coder-decoder) needs the 3 last images to perform the compression, it will take about 1/3rd of a second in the former case and 3 seconds in the latter case. Hence, the compression time increases proportionally to the compression ratio.
In order to use the available bandwidth efficiently the transmission rate of the different streams needs to be adjusted based on the respective stream priorities. Different techniques can be used to lower the required data rate to make better use of the available bandwidth. The first technique is data compression, but it involves a trade-off because of the additional delay introduced through the compression and decompression operations. This delay should be kept much smaller than the transmission delay.
Since the A/V stream requires the most bandwidth great attention should be taken to manage it carefully. Video conferencing/broadcasting offers a compression rate of up to 50:1; however, such a ratio can only be achieved when the compression is done in both the space and time dimensions. These solutions expect a stream as continuous as possible, which might not be the case in remote experimentation.
Another problem arises when clients are not on the same network. For example, some might be on the local LAN while others may be connected from home using dial-up line. The server needs to adapt to these different bandwidth requirements. One solution for the video stream is for the server to layer the image and send it at different resolutions [19]. When a low bandwidth is available the client would only use a low-resolution image. When more bandwidth is available the client would use the low-resolution image and also the higher layer image.
Data decimation, where only one sample over n samples is transmitted, can also be used to lower the bandwidth required. The intermediate steps can be reconstructed on the client's end where interpolation or a real-time simulation can be used to regenerate the missing measurements. However, if two measures are too distant in time the reconstruction can miss fast phenomenon such as high-frequency oscillations.
A virtual-reality model can also be used as a replacement of the video signal. In this case the client software animates a graphical representation of the physical process using regularly updated coordinates provided by the server via the data stream. Another concept called phantom process [20] uses the local simulation to directly reflect the effect of the user actions (i.e., parameter modification) without waiting for the server to send the information back. At the client's end the user sees two windows: one free of delay but showing the results of the phantom-process simulation, and one with a transmission delay but showing the image of the physical process. This technique is used in robotics applications where delays due to distance cannot be avoided. Phantom-process schemes often require the use of some kind of mechanism, such as artificial intelligence schemes, to deal with unmeasured and/or unknown events that may be missed by the base model.
The specifications discussed in the previous section as well as on empirical measurements made during thus study, the required data communication bandwidths for effective real-time control is 12 to 65 Kbytes/sec if using the video and about 3 Kbytes/sec without the video. We also determined that a frame rate of 5 frames per second is acceptable and that 10 frames per second gives reasonable performance. In this section we study a typical Internet communication link to evaluate the feasibility of providing this quality of service as well to determine mechanisms which may need to be implemented to address Internet specific idiosyncracies such as jitter and congestion.
Since the capacity is limited and most likely shared with other users, a fair adaptation of the transmitted content is essential. The non adaptation of the content leads to a waste of resources and may disturb other users. Bandwidth reservation can partially solve the sharing issue, but the adaptation to the current bandwidth value still needs to be made.
Measuring the Internet bandwidth between two points is a challenging task because routers provide only local information and cannot provide a global view of the system. ATM can provide such information. However since we are not considering an end-to-end ATM connection, we have to rely on another type of measurement. Bolot and others proposed to evaluate the bandwidth by sending packets and counting the number of packets losses at the end point [BOL94]. Bolot and Busse tried to estimate the bandwidth by measuring the jitter in the Round Trip Time (RTT) of the packet but they did not find a correlation between their measures and the network load [BOL94, BUS95]. Busse et al. suggested that his packets did not cross enough hops to be reliable and called for measures across more hops [BUS95]. Bolot and others classified the network state in three categories: unloaded, loaded and congested. During the development of IVS, Bolot and his colleagues [BOL95] observed that the average number of consecutive lost packet remained small (1 or 2 packets) as long as the stream used less than 10 % of the bandwidth.
The first step in the analysis of the collected data was to find the right indicator to represent the load of the network. As stated earlier, a key performance measure is the packet loss percentage and clearly this must be examined first. Figure 4 shows the percentage of packets lost over the day for different packet size (packets were sent at 10 packets per second). The x-axis is the European local time (GMT-1). For example, network load increases in the morning to reach saturation at about 10:00 and stops being saturated at about 19:00. There is a valley at 16:00 (5 Kbytes curve) and this confirms that packet loss is a good indication of the network load.
Further conclusions can be drawn from Figure 4: the percentage of packet loss is proportional to packet size, this means that smaller packets are less likely lost and that over 90% of the large packets (in our example 20 Kbytes) are lost during the saturation period. Figure 4 shows that the percentage of packet lost is less dependent to the packet size when the network is un-congested.
Figure 4. Packet Loss Percentage (sent at 10 Pk/sec)
In the early morning when the network is unloaded no packet of size 1200 Bytes is lost. Larger packets however have a small percentage of packet lost. The measurements reveals repetitive peaks in the number of packet lost. These peaks are spaced by about 60 seconds. These peaks appear when packet size is larger than 1476 bytes. Without these peaks the percentage of packet loss for packet size larger than 1500 Bytes would be very close to 0.
The packet Round Trip Time (RTT) varies according to the network load (Figure 5). This is due to the packet splitting at the router. The bigger the packet is, the longer transmission takes because the probability that the packet will be split is higher.
Figure 5. Packet Round Trip Time (RTT) (sent at 10 Pk/sec)
Figure 5 shows that the RTT increases with the network load but unfortunately the variation among RTT is not correlated with the packet size when the network is congested. These values should be compared with default values but they may be difficult to obtain because they vary over time.
The optimum values for packet size and packet rate can be found by comparing the effective bandwidth with the ideal bandwidth. Figure 4 and Figure 5 show that it is better to send small packets at a faster rate than bigger packets at a slower rate. The difference is most noticeable when the network is congested since packet losses occur mainly at that time. During peak hours, the effective bandwidth drops down to 10% of the ideal bandwidth when sending 20 Kbytes packets at 5 Pk/sec whereas 5 Kbytes packets sent at 20 Pk/sec result in a drop of only 45 % of the ideal bandwidth.
The analysis of the measurements leads to four main observations. The first one focuses on packet loss as an indication of the network load. The second one centers on the relation between packet size and packet loss when the network is congested. However packet rate and packet loss are not related in the domain the tests were performed. This leads us to the third observation: it is more efficient to send small packet at a fast rate than larger packets at a slower rate.
Busse et al. [BUS95] divided the network state in three areas. He defined the network as congested when packet loss was higher than 4%, as loaded when packet loss was between 2 and 4% and as unloaded when packet loss was below 2%. Our measurements show that the network under study is congested when packet loss is larger than 30%, it is loaded between 10 and 30% and it is unloaded when packet loss is below the 10% value. The variation between Busse's values and our values can be explained by the fact that his measures crossed only 6 hops whereas ours crossed more than 20 hops. Our unloaded state would come closer to Busse's definition if the measured periodic spikes were not taken into account.
The ultimate applications (the server and the client) also have intrinsic limitations, such as the maximum frame rate for the video grabber. These constraints restrain the possible choice for packet rate and packet size.
Users may have the possibility to request a specific QoS. This setting can for example balance the image quality (packet size) vs. the interactivity (packet rate). The user might also be aware of outside particularity and decide not to use the complete bandwidth.
On the server side the video and the data streams depend on the underlying hardware performance. The current setup uses the internal video grabber of a Macintosh 8600/200. It can grab and compress a 160 x 120, 32 bits/pixel image at about 10 frames per second in a frame by frame mode. The values measured on the real process are handled by the real-time kernel and buffered internally. They can be fetched by the server at any time and are only limited by the server. The real-time kernel sampling period determines the amount of new data the server can fetch per second. The example in Section 3 is based on a 10 ms sampling period and results in a data stream of 2800 bytes/sec. If the server sent the same data 10 times per second, packets would be of size 280 bytes.
On the receiver side the main limitations are the decoding time for the video image and the time taken by the real-simulation (if used). The time needed to decode a 120 x 160 jpeg image is about 25 ms (= 40 images/sec) which is much more than what the server and the network could handle.
The video acquisition is performed with the help of the on-board frame grabber. The frame grabber can work in two modes: the frame-by-frame mode and the continuous mode. The frame-by-frame mode is chosen for its possibility to easily add user-defined compression schemes (in our case Jpeg). The image grabbing, which is performed asynchronously requires synchronization with the other components of the server. The QuickTime toolbox is used to manipulate the frame grabber at the lower levels and the compression of the image.
The Jpeg standard provides a high compression ratio. The original 160 by 120 pixels image takes about 77 Kbytes uncompressed. Once compressed the same image only needs 2.5 Kbytes of memory with the highest compression factor (0). A visually loss less image takes about 12 Kbytes. The compression factor ranges from 0 for the maximum compression to about 1024 for the loss less image. Higher values can be chosen, but the variation is not visible by human eyes.
Figure 6. Jpeg compression
Since the compression factor is not part of the standard, its meaning depends on the codec specification. Figure 6 graphs the compression factor vs. the final image size for a 160 x 120 pixels color image. It also shows the time needed to grab, convert and compress the image. The grabbing time and the compression time are unrelated to the compression factor. The variations in the grabbing time are due to the synchronization between the frame grabber memory and the main memory.
In this section we integrate the system requirements discussed earlier with model developed by real-time measurements in the previous section to give an overall real-time Internet-based control system.
The optimal scheme for the remote experimentation control structure is defined in Figure 7. This ideal scheme takes into account the different factors related to control. This scheme is divided in four blocks: QoS, Optimization/control, Actuator and Estimation.
The Quality of Service (QoS) block provides the controller with the user requirements and the environmental requirements. The user may specify a desired QoS. He can select the priority for the image vs. the data and the interaction vs. the image quality. The hardware constraints such as the maximal frame rate and the client limitations (i.e., too slow) are also given to the optimization/control block.
The controller/optimizer computes the new packet settings on the basis of the set point, the constraints and the estimated bandwidth. These settings, which include the packet rate and the packet size, are provided to the actuator.
The content of each packet is specified at the actuator level according to the constraints. The compression factor (image) and the decimation/compression (data) are also specified in this block.
The packets leaving the actuator block transit through the network before reaching the receiver. At the receiver side the estimator computes the network load and the available bandwidth according to the packet losses. This information is returned to the controller/optimizer as the measurement.
Figure 7. Optimal control solution.
The algorithm used to adapt to the network load should be fair. In other words it should behave with the same aggressiveness as TCP. The chosen solution is similar to the one suggested by Bolot and Turletti [TUR96]. The adaptation is based on the three network states (unloaded, loaded and congested). The aim is to stay below the congested state but above the unloaded state. The algorithm should behave analogously to TCP: quickly backing off when in the congested state and slowly increasing the flow when unloaded.
If (CurrentState > CongestedStateLowLimit)
Flow = Flow / 2;
If (CurrentState < UnLoadedStateHighLimit)
Flow = Flow + Step;
An extension to this algorithm would increase by dichotomy the flow to reach the congested state asymptotically. In this case the Step value added to the current flow equals the half of the difference between the current flow and the maximum flow (congested). The maximum flow can be derived from the model defined in Section 4.
The minimum and maximum values for the flow are defined by the sender application. The sender application flow is highly flexible and can be adapted to almost any situation. In theory the flow could be as low as 4 Bytes per second, in the worst case. The 4 Bytes representing the reference position (set point) and the actual position, would be sent every second. In this case there is no video feedback; the user only receives the VR feedback. The maximum flow is mainly defined by the speed at which the video grabber can supply compressed images to the sender. If the receiver cannot handle the incoming streams, it can also request a lower limit for the maximum bandwidth or flow. Accordingly the client can set the minimal value for the flow.
The flow is function of two variables: the packet size and packet rate. In Section 4 measurements showed that the packet size should be small and the packet rate should be fast. Good interactivity is achieved with a 10-images-per-second frame rate. The data should also be updated 10 times per second. To certify efficiency the image should not be divided in sub-images. The resulting maximum packet rate is therefore 20 packets per second. This assumes that data and video information are not concatenated in the same packet.
When the flow rate and size are specified, the content of each packet still has to be defined. In Section 3 we defined three main kinds of streams with their associated priority. The parameter stream has the highest priority, the data stream has the next highest priority and the video stream has the lowest priority. The administrative stream has actually the lowest priority but it is not taken into account because it is only used at the beginning and at the end of a session.
Figure 8. Streams priority
The different streams are transmitted through a single channel. The decision for the next packet to transit through the channel is defined by the priority and by the user settings (See Figure 8). Since parameters have the highest priority, they will be sent within the next packet. If no parameter is waiting to be sent, the repartition of the bandwidth is split between the data and the video stream. The data stream is cadenced by the sampling period resulting a fixed flow of information (assuming no compression/decimation). On the other hand the video stream can vary reflecting the variation of the compression factor.
The flow will first adapt to the network state by adjusting the compression factor. When the minimum image size is reached, the available bandwidth will be shared between video and data. The user can give preference to the image quality or image rate. He can also choose the ratio between video and data stream. By default, preference is given to interactivity resulting a highly compressed image and a rapid frame rate. The ratio between image and data is evenly shared to keep the synchronization between the video and the scope window.
During the transmission the packet can be lost or can arrive late. In both cases the missing information has to be recovered locally. Packets will be handled differently if they contain video or if they contain data. Provided that the model of the system is known, data packets can be reconstructed by simulation. This is not the case with video image where only the virtual reality part can be derived from the data. The synchronization between the virtual image and the real image will be lost. (See Figure 10).
In the case of packets arriving late (i.e., out of order) packets containing images will simply be discarded. The decision to discard packets containing data depends on the display progression. If the packet can still be displayed in the scope window (Figure 9), the incoming data will replace the simulated one. If its place is outside the scope window, it will be discarded. As seen in Section 4 packets hardly ever arrive out of order.
Figure 9. Packet recovery.
The user might decide to record the data for later use. In this case no packet will be discarded. To avoid penalizing the performance of the receiver, the packet reordering, based on packet ID and timestamp, can be done offline.
Precise synchronization between the data and the image is difficult to guarantee. Moreover, small variations are difficult to detect. Hence instead of trying to tightly resynchronize the two streams by adding a variable delay to packets, the scope window will display a small sign to represent the time the image packet arrived. The video image will also be synchronized with the data using the virtual reality representation of the real process.
The visual feedback presented to the user by the image of the setup can be enhanced by adding a virtual image of the process (Figure 18). The source of information for the virtual image differs from the source of the video image.
Figure 10. Video and virtual reality image.
The virtual image is derived from the measurements made on the real process whereas the video image comes from the video camera. The possible need of synchronization between these two images is a disadvantage, but it is largely compensated by the possibilities offered by the virtual image.
The virtual image is derives from the measurements. If they are not available at the given time (packet or connection loss), the simulation can be used to generate the needed information for the virtual image. In other words the virtual image is always available provided that the model of the system exists.
The virtual image can also be useful when the available bandwidth is small. Instead of using a large portion of the available bandwidth with the video image, only a few (1 or less) video images per second are sent. The missing dynamic of the video is compensated by the dynamic of the virtual image. When using the highest compression factor, the smallest size for the video image is about 2 Kbytes whereas for the "same" information (angle) only 2 Bytes are needed to define the virtual image. This ratio (1000) gives considerable room for adaptation.
We observe that the video image carried far more information than the virtual image. Through the real image, the user can get the feeling of the real setup. This is essential in remote experimentation. The video image also gives environmental information which is completely unseen by the virtual image. For example an external perturbation in the setup such as a wire running across the moving parts can be seen on the video image, but the virtual image is blind and cannot represent this perturbation.
As a conclusion, the hybridization, i.e. the combination of the virtual image and the real image, provides the user with the best visual feedback.
In this paper we have examined the requirements for practical real-time control over the Internet with video and audio feedback from a central server to multiple clients. We have also shown by studying a typical Internet link that the specified real-time control requirements can indeed be met providing that appropriate mechanisms are in place to handle graceful degradation of performance in the presence of jitter and congestion which characterizes Internet communication.
The widespread use of the real-time control and experimentation system described in this paper has the clear benefit of better sharing of expensive or unique resources. It is planned to develop a large access to a distributed laboratory of networked experiments. User feedback will then be collected and evaluated to improve the current solution before integrating it into ALN.
The current single-client applications must be extended to multi-client applications. The new solution should adapt to the heterogeneity of client connection speeds which range from a low-speed modem when connecting from home to a fast LAN connection when working on campus.
The protocol used in the current solution is a simplified version of the Real Time Protocol (RTP). Full RTP compliance and the integration of new protocols such as IPv6 will lead to a better use of the underlying connection and a better integration with other applications.
The real process model is currently used for the reconstruction of lost information. More advanced recovery techniques, especially meant for video images, can be derived from this model and should result in very low bandwidth requirements.
The authors gratefully acknowledge the support of the US and Swiss National Science Foundations for their generous support of this project.
[AKT96a ] Aktan B., Distant learning applied to control engineering education, M.S. Thesis presented at the Oregon State University, Electrical and Computer Engineering Department, February 1996.
[AKT96b] Aktan B., Bohus C.A. , Crowl L.A., and Shor M.H., Distant learning applied to control engineering, IEEE Transaction on Education, Vol. 39, pp. 320-326, August 1996.
[ALT 95] Altpeter F., Salzmann Ch., Gillet D., and Longchamp R., A general instrument for real-time control and data acquisition, 3rd IFAC/IFIP Workshop on Algorithms and Architectures for Real-Time Control, pp. 323-327, Ostend, Belgium, June 1995.
[BOL94] Bolot J-C. and Turletti-T., A rate control mechanism for packet video in the Internet, Proceedings IEEE INFOCOM '94, Conference on Computer Communications, vol.3, pp. 1216-1223, Toronto, Canada, June 1994.
[BOL95] Bolot J-C., Crépin H. and Garcia A.V, Analysis of audio packet loss in the Internet, Proceedings of the Fifth International Workshop on Network and Operating System Support for Digital Audio and Video NOSSDAV'95, pp. 154-165, Durham, NC, April 1995.
[BOL99] Bolot J-C. and Vega-García A., The case for FEC-based error control for packet audio in the Internet, to appear in ACM Multimedia Systems.
[BRA97] Brady K. J., Timed-delayed control of telerobotic manipulators, Ph.D. Thesis presented at the Washington University, Systems Science and Mathematics Department, August 1997.
[BUS95] Busse I., Deffner B. and Schulzrinne H., Dynamic QoS control of multimedia application based on RTP, First International Workshop on High Speed Networks and Open Distributed Platforms, pp. 831-843, St. Petersburg, Russia, June 1995.
[CAR97] Carle G. and Biersack E.W., Survey of error recovery techniques for IP-based audio-visual multicast applications, IEEE Network, pp. 1124-1136, November/December 1997.
[CAS92] Casner S. and Deering S., First IETF Internet audiocast, SIGCOMM ACM Computer Communication Review, Vol. 22, No 3, pp. 92-97, July 1992.
[CEN95] Cen S., Pu C., Staehli R., Cowan C. and Walpole J., A distributed real-time MPEG video audio player, Proceedings of the Fifth International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV'95), pp. 142-153, Durham, NC, April 1995.
[CHA95] Chaddha N, Wall G.A. and Schmidt B., An end to end software only scalable video delivery system, Proceedings of the Fifth International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV'95), pp. 130-141, Durham, NC, April 1995.
[CHE96] Cheung S.Y., Ammar M.H. and Li X., Using destination set grouping to improve fairness in multicast video distribution, Proceedings of the IEEE INFOCOM Conference, San Francisco, CA, March 1996.
[COW95] Cow C., Cen S., Walpole J. and Pu C, Adaptive methods for distributed video presentation, ACM Computing Surveys, Vol. 27, No. 4, pp. 580-583, December 1997.
[DUF98] Duffield N.G., Ramakrishnan K.K. and Reibman A.R., SAVE : An algorithm for smoothed adaptive video over explicit rate network, Proceedings of the IEEE INFOCOM Conference, pp. 1093-1102, San Francisco, CA, March 1998.
[FEN97] Feng W.and Rexford J., A comparison of bandwidth smoothing techniques for the transmission of prerecorded compressed video, Proceedings of the IEEE INFOCOM Conference, pp. 58-66, Kobe, Japan, April 1997.
[FRE94] Frederick R., Experiences with real-time software video compression, Sixth International Workshop on Packet Video, pp. F1.1-F1.4, Portland, OR, September 1994.
[FU97] Fu W-Y., Latchman H.A. and Lu S-Y., Adaptive flow control scheme for real-time traffic in ATM networks, SPIE Proceeding, Vol. 3231, pp. 271-282, October 1997.
[GIL91] Gillet D., Longchamp R. and Bonvin D., Integrated workbench for laboratory projects in automatic control, International Conference on Computer Aided Learning and Instruction in Science and Engineering, Lausanne, pp. 507 - 508, Switzerland, September 1991.
[GIL93] Gillet D., Salzmann Ch., Longchamp R. and Bonvin D., A methodology for laboratory development of scientific teachware with application to adaptive control, American Control Conference, pp. 2443-2448, San Francisco, CA, June 1993.
[GOK97] Göktas F., Smith J.M. and Bajcsy R., Telerobotics over communication networks, Proceedings of the 36th Conference on Decision & Control, pp. 2399-2403, San Diego, CA, December 1997.
[GRO95] Grossglauser M., Keshav S. and Tse D., RCBR: a simple and efficient service for multiple time-scale traffic, Proceedings of the ACM SIGCOMM'95 Conference, pp. 219-230, Cambridge, MA, September 1995.
[H261] Recommendation H.261: Video codec for audiovisual service at 64 kbits/s, International Telecommunication Union (ITU-T), 1993
[HUB98] Huber W., Accounting ein Speil mit Millionen von Zahlen, Switch Journal 1/1998, pp. 29-31, Zurich, Switzerland, November 1998.
[IMR97] Interactive Model Railroad web page, University of Ulm Germany, http://rr-vs.informatik.uni-ulm.de/rr
[KAN95] Kanakia H. and Mishra P.P., Variable bit rate coding for real-time video transmission in ATM network, International Conference on Image Processing, ICIP, pp. 5-8, Washington, DC, October, 1995.
[KOU97] Kouvelas I., Hodson O., Hardman V. and Crowcroft J., Redundancy control in real-time Internet audio conferencing, Proceedings of the 1997 International Workshop on Audio-Visual Services Over Packet Networks, pp. 342-348, September 1997.
[LAK97] Lakshman T.V., Mishra P.P., and Ramakrishnan K.K., Transporting compressed video over ATM networks with explicit rate feedback control, Proceedings of the IEEE INFOCOM Conference, pp. 38-47, Kobe, Japan, April 1997.
[LAT98] Latchman H.A., Salzmann Ch., Thottapilly S. and Bouzekri H., Hybrid asynchronous and synchronous learning networks in distance education, Proceedings of the ICEE Conference, Rio, Brazil, July 1998.
[LI96] Li X. and Ammar M. H., Bandwidth control for replicated-stream multicast video distribution, 5th International Symposium on High Performance Distributed Computing, HPDC '96, pp. 356-363, Syracuse, NY, August 1996.
[MCC95] McCanne S. and Jacobson, V., Vic: a flexible framework for packet video, ACM Multimedia, pp. 511-522, San Francisco, CA, November 1995.
[MIC97] Michel O., Saucy P. and Mondada F., KhepOnTheWeb: an experimental demonstrator in telerobotics and virtual reality, Proceedings of the International Conference on Virtual Systems and Multimedia, IEEE VSMM'97, pp. 90-98, Geneva, Switzerland, September 1997.
[MPG92] International Organization for Standardization / International Electro-technical Commission Joint Technical Committee 1, Sub-committee 29, Work Group 11, ISO/IEC JTC1 SC29, Coding of moving pictures and associated audio.
[NAS97] NASA robot list, http://ranier.oact.hq.nasa.gov/telerobotics_page/internetrobots.html
[NSP97] Netscape online documentation, An Exploration of Dynamic Documents, http://home.netscape.com/assist/net_sites/pushpull.html
[PER98] Perkins C. S., Hodson O. and Hardman V., A survey of packet-loss recovery techniques for streaming audio, IEEE Network Magazine,
pp. 1-15, September/October 1998.
[SAL97] Salzmann Ch., Gillet D., Longchamp R., and Bonvin D., Framework for fast real-time applications in automatic control education, IFAC Symposium on Advances in Control Education, pp. 345-350, Istanbul, Turkey, July 1997.
[SAL98] Salzmann Ch., Latchman H. A., Gillet D. and Crisalle O. D., Requirements for real-time laboratory experimentation over the Internet, ICEE 1998, Rio, Brazil, August 1998.
[SAU95] Saucy P., Robot guide à distance, LAMI- Swiss Federal Institute of Technology, Internal report, Lausanne, Switzerland, June 1995
[SAU98] Saucy P. and Mondada F., KhepOnTheWeb: one year of access to a mobile robot, Internet Workshop on Robot on the Web IROS, Victoria Canada, September 1998.
[STR97] Stride S., Frequently Asked Questions about the Mars Rover and Lander Jet Propulsion Laboratory, California Institute of Technology and the National Aeronautics and Space Administration. http://mpfwww.jpl.nasa.gov/MPF/rovercom/rovfaq.html.
[SBB95] Second Best Being There project, SBBT project home page, http://www.ece.orst.edu/~aktanb/distance-labs.html.
[STE96] Stedman N. and Ditlea S., Thriving community is seeded by tele-gardening, http://telegarden.aec.at/html/nyt.html.
[TAY97] Taylor K. and Dalton B., Issues in Internet telerobotics, International Conference on Field and Service Robotics, FSR'97, pp. 11-18, Canberra, Australia, December 1997.
[TUR96] Turletti T., Controle de transmission pour logiciel de videoconference sur l'internet, Ph.D. Thesis presented at the Université de Nice Sophia Antpolis, diffusée par l'INRIA, Nice, France, 1995.
[WOL97] Wolf H. and Froitzheim K., WebVideo - a tool for WWW-based teleoperation, Proceedings of the IEEE International Symposium on Industrial Electronics, pp. SS268-SS273, Guimarães, Portugal, July 1997.
[YAH97] Yahoo devices connected to the Internet, http://www.yahoo.com/Computers_and_Internet/ Internet/Entertainment/ Interesting_Devices_Connected_to_the_Net/Robots/
[YAV93] Yavatkar R. and Manoj L., Optimistic strategies for large-scale dissemination of multimedia information, Proceedings of the ACM Multimedia ’93 , pp. 1-8, Anaheim, CA, August 1998.