Chrome Browser Extension
After adding the extension, you can enjoy the features and demos pre-built into the extension.
This includes handsfree web browsing using our eye-gaze and head controlled virtual mouse pointer. Instructions can be found in the video to the left.
When using the extension, your computer will perform all head/face eye and gaze tracking locally, meaning that no images or other personal data is being sent anywhere.
Be sure to check out the User Manual.
xLabs revolutionary eye tracking software is now available to end users as a free chrome extension.
To install the extension follow these instructions:
Find the zip file you downloaded, and unzip it. It will create a folder called ‘extension_2.6.10’
In Chrome, enter the URL below in the address bar and press ENTER:
Click ‘Load unpacked’ button. A file dialog will appear.
Find the folder you unzipped named ‘extension_2.6.10’ and select it.
The extension will now install.
How to use the extension
After installation, you can use our software on most web pages, including http://, https://, and file:// addresses. Our software does not work on pages that start with chrome://, such as our Options Page.
The easiest way to control our software is via the Popup Menu. You can access the menu by clicking on the ‘X’ icon in the Chrome toolbar:
The popup menu shows 5 modes, including “Off”. Each mode will be described in detail below. They are:
Note that you cannot select Page-defined Mode: Instead, compatible web pages will activate this mode. However, you can use the Popup Menu to turn our software Off at any time. The Popup Menu also provides quick access to our Options Page and Help on our website.
This is a good first test that our software is able to access your camera. Head Mode tracks the position of your head. In the default configuration, it updates a full 6 Degree Of Freedom (DOF) pose 20 times per second. Therefore, web pages can access X, Y, Z position and Roll, Pitch, Yaw angles. By default we display your head pose as a pair of ‘+’ crosses. The green cross shows position; the relative position of the blue cross depicts the orientation of your head. If the crosses don’t appear, or don’t move, there may be a problem with your camera.
Mouse mode allows you to surf the web without touching your computer! We use head movements to control a red dot on screen, that acts like a mouse pointer. To use Mouse Mode you need to do a simple calibration. When you enable Mouse Mode, a black shadow fills the screen from the edges to the centre. When the shadow reaches the centre, you should look straight at the hole in the middle. The shadow then disappears and you will see a red circle that you can control with your head. If you let the red circle rest on a link for some time, the system will “click” the link and take you to that page. If you push the red circle against the left side of the page, a red rectangle will appear. After a few seconds, you will be taken “Back” to the previous page. Similarly, if you push the red circle against the top or bottom of the page a blue rectangle will appear, and the page will scroll up and down respectively. Be sure to keep your head upright. If you tilt (roll, for the pilots out there!) your head to the right, and red triangle will appear and Mouse Mode will temporarily be disabled. You can resume Mouse Mode and reset the calibration by tilting your head to the left. A black triangle appears, before the black shadow. Look at the centre of the screen while the shadow covers it.
When mouse mode is enabled or reset, you will see a shadow fill the screen from the corners to the centre. Look at the centre of the screen to calibrate the mouse control.
After calibration the red circle is controlled by your head movements. Let the red circle rest on a link for a short time to click and navigate.
Push the red circle against the top or bottom edge of the window to scroll the page. Scrolling down is shown.
Tilt your head to the right (note: don’t turn your head to the side, tilt it instead) to suspend mouse mode temporarily. The red triangle indicates that the command has been understood.
Tilt your head to the left to reset or enable mouse mode again. You can do this at any time, e.g. if you forgot to look at the centre during calibration.
Mouse mode calibration
Push on the bottom edge to scroll down
Tilt your head RIGHT to suspend
Tilt your head left to enable and/or recalibrate
Dwell on a link to select and navigate
Learning Mode is a test and demonstration of our gaze-tracking capability. This mode will show you where we think you are looking on screen, as a red circle. The same data can be used by web pages (it doesn’t have to be displayed on screen). In Learning Mode, the system doesn’t need any explicit calibration. You just browse the internet and click around the page. The software watches while you do this, and learns to map your appearance to the points you’re clicking. As long as you look where you’re clicking, eventually we will be able to predict your gaze without a click. Don’t worry if you forget to look a few times; we can usually ignore these errors and you can always delete your calibration data. So let’s try Learning Mode. Select it from the Popup Menu, and then load a web page, e.g. http://www.reddit.com Try clicking around the page. To save time, for now avoid clicking any links – just click near them instead. When you click a blue circle will appear. This shows we’ve registered the click.
Learning mode clicks are indicated with a blue circle and a red dot. If you press and hold the left mouse button, you can move the red dot with your head. This symbol is a reminder that we are using these clicks to calibrate the system.
After 20-30 seconds a red circle should appear wherever you look. At first, the circle will be large and may not always be in the right place. But click a few more times (especially where you observe it is not accurate) and the system should heal itself, becoming more accurate. The system will re-train itself every 10 seconds if new clicks are provided. So, you may have to wait this long for accuracy to improve. You may notice a thin black circle at times; this indicates calibration is happening.
The system tries to generalize between all the clicks you’ve made, so you don’t need to click everywhere; but it’s a good idea to click in all gross areas of the screen, such as the top, bottom, sides and corners with a few clicks somewhere in the middle. For better calibration, you can also click and hold the left mouse button while moving your head a little bit; this teaches the system very rapidly because the data has more variation. In other websites we also use an interactive calibration process, that reliably achieves a good calibration in a fixed period of time. But the Learning Mode included here works on any website and hence is included with the software. Note that gaze-tracking learns the shape of your face and therefore you need to re-calibrate for other users. You can delete old calibration data via the Options Page.
Gaze indicator. The red circle indicates where we think you are looking. Circle size indicates confidence (smaller circles mean higher confidence).
You can’t select the page-defined mode; after all, it is defined by the, err .. page! But you can turn it off. This mode is used by xLabs-compatible web pages to provide you with gaze or head tracking features.
The Options Page
The Options Page can easily be accessed from the Popup Menu. This page contains information about our software and some controls for using it. The Options Page is organised as a set of Tabs. Feel free to have a look around. Some core features are described below.
You can change the camera resolution from the “Camera” tab. Changes will take effect next time the camera is started. Note that some widescreen format megapixel cameras need to be used at their higher, native resolutions for a good, undistorted image.
When you change users, you’ll need to delete any existing calibration data. This can be done from the “Controls” tab in the Options Page.
The installed version of our software can be found in the “About” tab. This can be useful for support.
We use Chrome’s default camera device. You can change this using Chrome’s settings at:
Unfortunately you can’t change the camera from within our software.
Environment Setup Tips
At night or indoors, find a place with uniformly bright lighting rather than small spotlights.
During the day, it is best to face towards windows to illuminate the face.
Don’t use directional spotlights that cast shadows over the face.
Don’t sit with the light behind you! The face will be a black silhouette against a white background!
Much more information is available on this website, www.xlabsgaze.com. Some common questions are answered below. Make no mistake, the eye/gaze tracking system is very complex. It tries to work with everyone, regardless of appearance, using any camera, on any computer, anywhere in the world. It’s not simply running a program again and again. It’s learning and calibrating itself in every new environment it sees. Not easy! So occasionally, things will go wrong.
The environment, and particularly lighting & illumination, are critical for good results. Cameras react to varying levels of light by automatically adjusting the brightness of the image. In low light, long exposures will produce blurry or noisy images that we can’t use for precision eye/gaze tracking. The camera doesn’t know which bit of the picture we’re interested in – the bit with the face in it. Instead it balances brightness over the whole image. So, if the background is very bright, it will make the face very dark. Eye/gaze tracking needs good and even illumination over the whole face, and especially around the eyes. Shadows can be a problem. Good, bright, diffuse lighting – overhead or daylight – is best. Desk lamps can be a problem, because the light can be quite focused and cause shadows elsewhere. If you are using daylight, it is best for the user to face the window. This means that the face will be well lit, rather than having a bright sky behind the user’s head and shadowed face.
Does it work with glasses?
First, contact lenses are fine. Head & face tracking almost always works perfectly with glasses. However, Gaze tracking requires an unobstructed view of your eyes. Many glasses frames distort or break up the appearance of your eyes, preventing accurate gaze tracking. However, we’ve seen it work occasionally! We are working on a solution for glasses in general. Heavy eye makeup, dark eye-shadow and exotic cosmetic effects can also be difficult for our software. Hair that covers the eyebrows and/or eyes (e.g. a long fringe) is also a problem. The software is not trained to recognise faces of very young people (e.g. under 10 years). Likewise, very bushy beards that obscure most of the face may also cause problems. Essentially, we need to be able to see a face to track it This software is designed to track a single face – it might get confused if too many faces are in view of the camera.
I accidentally denied camera permission!
If you forgot to Allow access to your webcam, we won’t be able to use it. Chrome remembers this setting and it’s a bit tricky to undo it. To re-enable camera access, go to:
Click “Show Advanced Settings”.
Click “Content Settings”.
Scroll down to “Media” and click “Manage Exceptions”.
Click to highlight “Chrome-extension://xxxx ” and click the ‘x’ to forget this setting.
Re-visit our Options Page, and when prompted, click “Allow”.
My camera doesn’t work!
We access the camera through the Chrome browser. If the camera works in Chrome on other websites, it should work for us. Can you use the camera on other websites? Note that your camera should be placed in the centre of the screen, e.g. at the either on the top or bottom bevel. Your face must be clearly visible in the camera image. You need to sit approximately 20-100 cm from the camera in most cases, although you can sit further away if your camera has optical zoom. You can test your camera here.
Currently, our software does not understand multiple screens. We can only work with one display.
Tracking suspended, or intermittent function
In various modes you may see the “Tracking Suspended” error. This means the software can’t see the the corners of your eyes, which we need to track very precisely. This is usually due to poor lighting and/or shadows around the inner eye area.
Poor accuracy in one part of the screen
If you observe that gaze-tracking is not accurate in just one part of the screen, try adding more calibration clicks in that area and waiting a few seconds (in Learning mode). If accuracy remains poor, the problem is most likely lighting – there may be shadows hiding the eyes in certain poses.
It’s too slow!
We’re continually making it faster, but at the moment we need a fast, modern computer (e.g. Intel i5 or i7 processors). Also make sure you don’t have too many browser tabs open, or many other programs running.
Moving too fast
Very rapid movements make the camera give blurred images. When this happens, the system will briefly lose track of the user’s face, and then find it again a fraction of a second later. For best results, try not to move too quickly, especially during calibration clicks. Blurring is also more common in low light, because cameras will choose longer exposures.
Since the system remembers gaze calibration data from earlier sessions, if you change user you need to go to our Options Page and delete the old calibration data. If you forget to do this, you may find poor accuracy during gaze tracking.
The system learns to associate particular poses and positions of the head and face with gaze coordinates on screen. It generalizes beyond the observed training data to try to “fill in the gaps” – to be able to predict gaze for poses it has not seen before. With more training, the system gets better at generalization. Imagine the set of poses the system has seen as a bubble or cloud around your head. The more poses you have calibrated, the larger this cloud becomes. Eventually, all poses you find comfortable will be within the calibrated cloud. Changing the angle of a laptop screen can move your head into an uncalibrated position. This is easily fixed by clicking a times in Learning Mode or interactive calibration interfaces, which adds the new pose to the calibrated “cloud”. Calibration does not transfer between users. You need to re-calibrate for every user. When changing users, you need to enable calibration to clear the old user’s model.