Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research on the visual system, in psychology, in psycholinguistics, marketing, as an input device for human-computer interaction, and in product design. There are a number of methods for measuring eye movement. The most popular variant uses video images from which the eye position is extracted. Other methods use search coils or are based on the electrooculogram.
Eye tracking data is collected using either a remote or head-mounted ‘eye tracker’ connected to a computer. While there are many different types of non-intrusive eye trackers, they generally include two common components: a light source and a camera. The light source (usually infrared) is directed toward the eye. The camera tracks the reflection of the light source along with visible ocular features such as the pupil. This data is used to extrapolate the rotation of the eye and ultimately the direction of gaze
There are many different methods of exploring eye data. The most common is to analyze the visual path of one or more participants across an interface such as a computer screen.
Beyond the analysis of visual attention, eye data can be examined to measure the cognitive state and workload of a participant. EyeTracking’s patented Index of Cognitive Activity (ICA) is among the most widely used of these metrics. It has been validated in multiple contexts as a reliable indicator of mental effort.
The Index of Cognitive Activity (ICA) is a patented metric developed by Dr. Sandra Marshall and commercially available at http://www.eyetracking.com.
The ICA is based on the transformation of pupil diameter through signal processing algorithms of wavelet analysis. It focuses on the dilation reflex but also takes into account both the rapid constrictions of the pupil that result from increased light and the relatively slow dilations that result from accommodation to decreased light. The impact of the light reflex is reduced so that the dilation reflex can be measured.
In the ICA, wavelet coefficients are converted into a second-by-second index ranging from 0 to 1, with low values corresponding to low mental effort and high values corresponding to extreme mental effort. The Index is roughly the proportion of observations in a second that reflect significant mental effort, with signal smoothing to eliminate statistical noise and hardware anomalies.
One advantage of using wavelet analysis is that it preserves both time and frequency information. Thus, when a large value for the ICA is observed, it is possible to map it directly back to the specific instance in the task in which it occurred.
The figures below give an example of how the impact of light may be managed in the analysis of pupil dilation. The data come from a published study. [see REFERENCES — Marshall, S. P. (2007a)]
The first figure shows the raw pupil diameter of one individual measured for two minutes in each of four different test conditions: sitting in a normally lit room (upper left), sitting in the dark in the same room looking at the same dark blank screen (upper right), sitting in the light in the same room while responding to a series of verbal arithmetic problems (lower left), and sitting in the dark in the same room while responding to another series of verbal arithmetic problems (lower right). In all conditions, the individual was asked to face a dark computer screen which was never turned on.
Four pupil signals from the same individual under four different conditions
Thus, these four pupil signals represent two different situations: cognitive task versus no cognitive task (lower two versus upper two) as well as light (leftmost two) versus dark (rightmost two). Time is on the x axis, and pupil size is on the y axis. Each graph shows all pupil values across the full two minutes of testing. Pupil size ranges from 3 mm to 8 mm. The mean pupil diameter values for the four conditions for this subject were: 5.57 mm (Light No Math), 7.32 (Dark No Math), 5.83 (Light Math), and 6.04 (Dark Math).
The figure below shows the result of applying wavelet analysis to the pupil signals of the above figure. The x axis is time, and the y axis is a coefficient derived from wavelet analysis and used in the calculation of the Index of Cognitive Activity. Importantly, all four panels below are plotted on the same scale for the y axis, from -0.1 to +0.1 so they may be directly compared to each other.
Result of wavelet analysis of the five pupil signals shown above
Consider first the impact of cognitive task (lower two) versus no task (upper two). The coefficients in the top two panels are considerably smaller in values than in the lower two. Now consider the impact of light. There are only slight differences between the top left and top right (no math) as well as between the lower left and lower right (math). Mean absolute values across the four plots are 0.0011, 0.0008, 0.0037, and 0.0033 for the conditions of Light No Math, Dark No Math, Light Math, and Dark Math respectively. The two means obtained in the presence of the cognitive task are larger than the two means obtained in the absence of the task by at least a factor of three, with little impact due to the presence or absence of light.
The wavelet coefficients such as those plotted above are the foundation for the Index of Cognitive Activity (ICA). The coefficients are converted into a second-by-second index ranging from 0 to 1, with low values corresponding to low mental effort and high values corresponding to extreme mental effort. The Index is roughly the proportion of observations in a second that reflect significant mental effort, with signal smoothing to eliminate statistical noise and hardware anomalies. To maintain consistency in the ICA across eye tracking systems that have differing sampling rates, all pupil signals are standardized to 60 Hz before indexing.
To return to the above example, the four ICA values for the data shown in the figures are 0.10, 0.15, 0.47, and 0.38 for the four conditions. As reported in the original publication of the study, a repeated analysis of variance across all 30 subjects in the study revealed a significant difference in the ICA for the task factor (present vs absent) but no significant differences for either the effect of light or the interaction between task and light.
One advantage of using wavelet analysis is that it preserves both time and frequency information. Thus, when a large value for the ICA is observed for a particular second, it is possible to map it directly back to the time in the task that it occurred. And as shown above, we can also average the ICA across an entire task to get an average level of workload for an individual on the entire task.
The ICA relies on a pupil signal delivered by an eye tracker. The ICA has been validated with most eye trackers from several of the leading eye tracker manufacturers, confirming that the ICA is a truly independent metric, and not an artifact of, or reliant on the specific processing algorithm of, any one eye tracker manufacturer.
In summary, the Index of Cognitive Activity works as desired: as individuals engage in progressively difficult tasks, their ICA values go up. The ICA is sufficiently sensitive to be used to identify the periods of intense workload for an individual during a task as well as sufficiently stable to be used to measure task load of multiple tasks. It is not hardware dependent. Moreover, the pupil-based ICA satisfies the four basic requirements of a physiological metric of workload: it reflects brain activity, it is a continuous signal, and it can be measured unobtrusively. The remaining requirement—use in real-world settings as well as the laboratory—is discussed in REAL-WORLD APPLICATIONS OF COGNITIVE WORKLOAD.
THE CLIENT: Northrop Grumman, a leading global security company
The Objective: EyeTracking, Inc. utilized the Index of Cognitive Activity (ICA) to evaluate two versions (current and redesigned) of a mission critical system designed to monitor air, surface and subsurface traffic for the Space and Naval Warfare Command. The goal of this research was assess the cognitive workload of operators of each system as they completed their duties.
The Sample: EyeTracking, Inc. recruited and tested a sample of participants with appropriate military background and field experience.
The Methods: Each operator completed realistic scenarios using the current and redesigned mission critical systems. Eye tracking and behavioral data was collected, as well as self-reported feedback on mental effort. Comparisons were made between the two systems and also between novices and experts.
The Results: The ICA successfully identified points of difficulty and increased mental effort throughout the scenarios. Physiological data was supported by self-reported operator feedback regarding mental effort. Statistical comparisons of cognitive workload suggested that learning the redesigned system did not result in increased mental effort for experts or novices. These findings provided the client with objective, statistically-valid evidence that changing from the current system to the redesigned system will not negatively influence the workload of operators.
“The metrics provided in the final report allowed us to demonstrate the effectiveness of the product to our client. The scientific test results confirmed our heuristic field experience and will ensure widespread product acceptance across our customer base.”
Corey Bickmore, Department Manager, Defense Mission Systems, Northrop Grumman Corporation
Currently there are 12 top Eye tracking Hardware companies. As of May 4, 2018, Tobii pro, SMI, and Pupils Labs offer eye tracking with VR.
TobiiPro VIVE – ($10,000 2018)
Tobii was founded in Stockholm in 2001, and has become a key figure in the eye tracking world, with a huge amount of publications using their tools. Providing eye tracking units for research, assistive technologies, and gaming, it has shown itself to be a versatile and formidable company. Eye tracking units with glasses and VR headsets are available.
A Tobii eye tracker, found on all Tobii assistive technology devices (Tobii C12, C15 and Tobii PCEye), uses invisible Infra-red light to illuminate the eyes. From there, two extremely high quality camera sensors capture the reflection off of the retina and the cornea of the eyes, commonly referred to as “red eye” and the glint, respectively. The eye tracker then uses these two points to build a 3D model of the user’s eyes to determine two things: where the user is looking (gaze point) and where the user’s eyes are in space, relative to the location of the computer (track box).
This information is then paired with Tobii Windows Control to allow the computer to know exactly where the user is looking with an accuracy of 1cm. The computer can then track the user’s gaze point and, ultimately, tell the computer where their eyes are looking at all times. By knowing where the user’s eyes are looking, the eye tracking device then can control the computer, similar to the way a mouse lets you control it with your hand.
SMI (SensoMotoric Instruments)
SMI (SensoMotoric Instruments) have been around for 26 years now, and have been tried and tested as experienced providers of eye tracking equipment. SMI was acquired by Apple and have sold over 6000 units, and have featured in about as many publications. They also offered both eye tracking with glasses, and with VR.