Platform for the development of neuro tracers and neural interfaces
Design of a platform for the development of neuro tracers and neural interfaces for controlling mobile robotics based on public domain software and hardware.
A platform for developing neuro tracers and neural interfaces for controlling mobile robotics based on public domain software and hardware, including a device for capturing the brain signal (neuro headset), software for signal processing and software for controlling the robot.
Task
The project aims at solving the problem of the lack of simple tools with high reliability of the results obtained for the study of neurocommunications.
It was necessary to develop a comprehensive modular solution that would offer not just a standard neurointerface, but a neuro-educational platform focused on the practical application and development of children; its main features are:
- Control of the direction and speed of the movement of the slave devices (robotic platform) using the neurointerface (a device that reads the signals of the user’s brain activity and transfers them to the processor, which processes these signals and transfers them to the process based on data from the accelerometer / gyroscope
- Control of slaves’ actions (capture, throw, etc.) Based on data on brain activity from a neuro headset
- Operator-used augmented reality elements (view directly from the device based on interaction with an external camera and 3d glasses).
The developed software and hardware should be published under a free license, which will allow schoolchildren, students and developers to understand the device in detail and create their own gadgets based on this educational kit.
Solution
At this stage of the R&D, a neuro headset has been developed that allows to receive brain activity signals (EEG), which can be used to generate control actions on external devices, including a robotic platform.
That is, it was found that a possible option of spreading the capture function for the neuro headset being developed is controlling the level of alpha waves, which increases significantly if the system user is in a relaxed state with the eyes closed.
Also, the developed software-hardware complex allows determining the orientation of the user’s head from the data of the gyroscope and accelerometer located on the microcircuit of the neuro headset: that is, the system determines the rotation and angle of the head of the person on which it is located. Based on such data, a control action for changing the speed or direction of movement can be generated for the robot.
The developed software-hardware complex is intended for organizing robotics control with the help of a neuro-interface, developing its own neurotracers and neurointerfaces on the basis of this complex, as well as for creating neurotracers for increasing the brain’s ability to concentrate, switch attention, etc.
The primary cost of the developed neuro headset is significantly lower than for similar solutions (under 8,000 rubles). Publishing it under a public domain license will allow users to understand this device in detail and create their own based on this educational kit.
Documentation and source code
- The principle electrical circuit. The device diagram and component list files with links to purchasing sources. In the root directory, there is a file with a brief description of the design, materials and a warning for those wishing to use the project files.
- The source code of the project in Keil. The source code for the software part of the neurointerface is microcontroller firmware. You can use it as a template for implementing your logic inside a neuro headset.
- A set of Gerber files for ordering printed circuit boards. A set of files necessary for the production of printed circuit boards of the device (the whole set is in one archive). Using these files, you can order the manufacture of a printed circuit board on rezonit.ru or in China at jlcpcb.com.
- Models for the case 3D printing: stl-files for printing the case of a neuro headset on a 3D printer. See ibid for an explanation of printing as a text file.
- Flutter application allows you to receive data from a neuro-headset Robbo Neuro (using BLE), visualize data in a graph, and also transfer control commands to an external device (a cart with a claw-type manipulator) via Bluetooth SPP.
Details
At stage 1, a prototype of a neuro headset was developed, in which the Neurosky MindWave device was used as the case with silver chloride electrodes and no integrated hardware platform.
In this prototype, the device uses one active frontal electrode, which is the contact. On the board of this electrode, a preliminary low-noise amplifier (with a gain of 24) with active noise suppression circuits is installed. The electrode connections to the amplifier board are made with an unshielded cable. It has been verified that electromagnetic interference is suppressed by the integrated circuits of the active amplifier.
The prototype of the neuro headset had the following composition:
- gyroscope + accelerometer + magnetometer MPU9250
- ADS1299 Texas Instruments analog-to-digital converter
- USB-UART converter
- Nordic debug board nRF52-DK
For data visualization and preprocessing, fast Fourier transforms were used, which were fed to the input every second and presented as a frequency card (spectrogram); a low-pass bandpass filter was also applied, which cuts frequencies outside the range from 0.5 to 40 Hz.
The filtering of the EEG signal was adjusted in the specialized BrainBay software.
The filtering of signals for determining the position of the head was adjusted using the Majdvik filter.
The figures below show examples of EEG spectrograms, when the system user of is in various states along with visual displays of the EEG during the transitions to these states.
During the experiment, the following relationships with human activities were identified:
- When analyzing the brain activity of a system user, the “noise” blinks are clearly distinguished; however, this parameter can be used as one of the control factors when generating a signal for a robotic platform
- Based on the level of alpha waves, it is also possible to control the platform, but this signal can be detected only with the eyes of the user of the neuro headset closed
- The alpha signal level is an individual parameter of a person; that is, in exceptional cases, even when the user of the neuro headset is in a relaxed state with the eyes closed, it may not be high enough when forming the control effect of the robotic platform. It might be possible to change it during repeated trainings, but studying this problem was not in the scope of study.
At this stage of the R&D implementation, the following tasks were performed:
- Development of a prototype neuro headset
- Implementation of full functionality (EEG analysis, API, interaction with an external camera and 3D glasses).
- Designing scenarios for using the platform for the development of neurotracers and neurointerfaces in neuro constructor mode and in neurotracer mode.
Intellectual Property
Project team
- Project manager: E. Pridanova
- Developers: D. Kazakov, D. Grafov
- Engineer: A. Korepanova