Among the more esoteric and “no way, really?” technology prototypes exhibited at this year’s NTT R&D Forum in Tokyo was a new mind captioning technology. Designed to generate text descriptions of the human brain at the mind’s eye level, the technology is capable of detailing a text-based readout of what a person is actually visualizing.
The technology was illustrated at NTT Inc.’s showcase innovation event, which welcomed software development and data science practitioners from customers and partners alike. By decoding brain information through a linguistic AI model, the technology is capable of “verbalising non-verbal information in the brain” at surprisingly accurate levels.
Mind Captioning, Provided By Our Sponsor
NTT researcher Tomoyasu Horikawa and team came up with the term mind-captioning after undertaking an initiative built to drive AI services into and across brain pattern analysis. The goal was to create an application service capable of generating text that mirrors information being “described in the brain” to accurately denote visual details about objects, places, actions, events and the relationships that might exist between them.
This is not the first time projects have been launched in this space; we have been trying to build brain dictation transcriber machines (for want of a better term) for some time. Previous attempts have only managed to identify a few key words and failed to build up any level of coherent context to clarify what a person might actually be thinking about.
Horikawa grasped the breadth of technologies available in the modern age of AI and worked with a deep-language AI model that was set to analyse the text caption output of humans watching thousands of videos. Each video analysis produced a numerical meaning signature and, in aggregate, the team was able to build up a brain decoder from this work. Finally, a separate AI text generator was used to find a human language sentence that represents the closest match to the meaning signature being output from the person’s brain.
NTT says that human informatics laboratories have long been actively researching concepts such as brain interfaces using bio-signals, enabling computer operations based on mental imagery to be created.
Electromyography, Neurotechnology & Cybernetics
“There has recently been significant progress in miniaturizing and improving the precision of devices that acquire vital information such as brainwaves and electromyography and that generate movement by directly electrically stimulating muscles. Therefore, the results of our research are becoming possible to introduce into society. Based on the evolution of such devices, we at NTT Human Informatics Laboratories are researching neurotechnology and cybernetics,” said the company.
This technology sequentially presents images to the user wearing an electroencephalogram (EEG) headset. It then conducts similarity analysis of brain activities on the basis of the EEG data recorded when the user views each image. By using the analysis results, a sensibility map is created, where elements are arranged linearly with similar items placed close together and dissimilar ones farther apart
Slightly more cerebral than work carried out to refine database-level administration and maintenance updates, NTT’s focus on developing brain-representation visualization technology is focused on interpreting various types of brain information, such as emotions and cognitive states, all extracted from brain activities.
Neurotechnology & Cybernetics
As highball as these branches of neurotechnology and cybernetics are, it is not uncommon to find NTT focused on emerging technologies that may even depend upon still-nascent platforms that we can’t even use yet. The company’s haptic (digitised touch robots) are being developed at the NTT Docomo headquarters in Tokyo, and these were also on show this week… but they won’t even work to their full potential until the arrival of 6G… and that should be 2030.
As the company known for forging its logo in steel on every other manhole cover on the streets of Japan, NTT is synonymous with communications and telecommunications. The company says its goal is to create new communication technologies that directly connect minds and minds, bodies and bodies and minds and bodies.
