Are mental performance sensors accurate?

I write a lot about mental performance for gaming, and our company, MaddCog, has the sole purpose of measuring mental performance. That is all fine and well, but it begs the question, are the sensors used actually accurate?

MaddCog uses two sensors to measure mental performance:

• Electroencephalography (EEG) — measures electrical activity on the scalp which reflects brain activity
• Heart rate variability (HRV) — measures beat to beat variability to assess autonomic nervous system function

We could get accurate data in a lab with cables attached all over your head, but that wouldn’t be very practical for gaming. So MaddCog uses sensors known as consumer grade. This means they are smaller, cheaper, require less power, and are just easier to use. The tradeoff is that they aren’t as accurate as what you might see in lab research. But how do we know they are good enough to track mental performance for gaming?

EEG Accuracy

MaddCog uses an EEG chip made by NeuroSky as it is known to be the most accurate consumer grade EEG. It is a single channel on the forehead and uses a dry electrode meaning there is no need for electrode gel. You just pick it up and put it on.

This chip is well established and has been extensively researched, so let's have a look at some of the results.

Rieiro (2019) concluded that NeuroSky ‘the results obtained are comparable to those obtained with a medical-grade ambulatory device’ and it ‘is noise-limited and provides stable recordings even through long periods of time’.

In some research on mental performance during driving, Morales (2017) summarized that the NeuroSky chip ‘is able to detect changes in mental state while performing a complex and dynamic everyday task as driving’.

Other published work on the NeuroSky chip show a high level of accuracy, such as 91% accurate from Sadeghi (2016), 98% accuracy by Abdel-Rahman (2015), and 88.9% accuracy from Shin (2013). The variation in accuracy is a result of the testing protocol; Sadeghi and Abdel-Rahman focused their research on driving, while Shin looked at sleep and drowsiness.

Overall, NeuroSky isn’t perfect, but it performs well in most situations. It’s ease of use far outweighs the small loss in accuracy when compared to traditional medical grade EEG systems.

HRV Accuracy

Heart rate chest straps are known to be the most accurate means of capturing heart rate variability in consumer grade equipment. They are cheap, easy to use and comfortable to wear. This is exactly what MaddCog used in a lot of our early research.

But as a gamer you don’t want to be a science experiment. You don’t want to have to put on multiple sensors before you play. Our early testers constantly reported this to us. So, we decided to incorporate the HRV sensor in the sensor band on the head, meaning you just put on the sensor band and get EEG and HRV. To achieve this, we needed an optical sensor.

Optical sensors, or photoplethysmogram (PPG), have become popular in recent years as nearly every wrist worn wearable has them, from the Apple watch to a FitBit fitness tracker. Most of these device's measure both heart rate and HRV, however HRV is far more sensitive to errors than standard heart rate. Specifically, accuracy can quickly deteriorate if the sensor doesn’t have good contact with the skin, or you are moving lots.

For gaming, we needed the optical sensor to be on the head. For comfort and accuracy purposes we chose the temple, although other locations near the forehead would also be suitable. We are in the enviable position that the sensor band holds the optical sensor on the skin, and the gamer isn’t running around like someone might do with an Apple watch. This allowed us to get accurate HRV data but let's look at some results.

Longmore (2019) looked at the use of PPG at various locations and found the temple was more accurate than the wrist for heart rate measurements and was one of the most accurate sites on the body for the use of PPG. Even with more demanding use, Hettiarachchi (2019) showed that an optical sensor on the temple was accurate when measuring heart rate during exercise. This is likely because of the plentiful blood vessels around the temple.

We then undertook dozens of our own trials comparing the temple worn optical sensor to a chest strap, while the gamer played. These results show a high level of accuracy as in the example below.

Conclusion

This article shows that consumer grade sensors are accurate, and therefore a useful tool to track mental performance, especially in a sedentary activity like gaming. Our internal experience has shown that the key to getting good results is the positioning of the sensor. The MaddCog sensor band make this easy, and ensures the sensors have firm skin contact while maintaining comfort.