SONY

What's Your Story? Junior Reporters' Takeaways Seeing with Sound: What is the Intuitive Voice Navigation Facilitated by LinkBuds?

Haruko Miki and Mitsunori Tanaka sitting on benches talking to each other

Sony's LinkBuds truly wireless earbuds employ a ring-shaped driver unit that does not block the ear canal, allowing sounds in the surrounding environment to be heard as well. Sony has teamed up with Eye Navi, a walking support app for people with visual impairments developed by Computer Science Institute Co., Ltd.,(CSI) to create a new kind of voice guidance using LinkBuds.

"By leveraging the features of LinkBuds, Eye Navi could potentially be made even more user-friendly Eye Navi."
"We want to realize a society in which everyone can enjoy going outdoors through the power of the latest technology."

With passionate aspirations, the developers embark on new challenges. Young engineer Haruko Miki goes behind the scenes to interview Mitsunori Tanaka, who served as the project leader for Sony on the LinkBuds × Eye Navi project.

A portrait of Haruko Miki

Asking the questions!

Haruko Miki

Dept.5 Software Technology Div.1 Technology & Engineering Center Sony Corporation

Joined Sony in 2022. In addition to being involved in the development of the Android platform for mobile devices, Haruko is also taking on new challenges such as new music-related ventures. She enjoys chatting to people over a drink and dancing to the music and lyrics at live concerts.

A portrait of Mitsunori Tanaka

Providing the answers!

Mitsunori Tanaka

Professional Solution Business Office Personal Entertainment Business Unit Sony Corporation

Mitsunori oversees product planning for personal audio devices such as Walkman, headphones, and microphones. Recently, he has been focusing on products for creators, high-end audio, and accessibility. On his days off, He spends time with his children at movies, amusement parks, piano practice and more. He is also trying his hand at music/video production.

LinkBuds × Eye Navi, a fateful encounter

A person holding a phone whose display is showing EyeNavi app UI
Eye Navi, a walking assistance application for the visually impaired developed by Computer Science Laboratory, Inc.

Haruko: Mitsunori, it is a pleasure to meet you. I am Haruko Miki, and I joined Sony fresh out of university in 2022. The LinkBuds × Eye Navi is finally released! Can you please tell us how this collaboration started?

Haruko Miki listening to Mitsunori Tanaka

Mitsunori: LinkBuds first went on sale in February 2022. The occasion that led to collaboration was when Sony exhibited at the Japan Lighthouse Exhibition for the first time in October of that year. This is the largest event in Western Japan for exhibiting information and devices to help people with low vision—whether completely blind or visual impairments—and Sony exhibited several products that take accessibility into consideration.

At that time, the booth next to us just happened to be the booth of CSI who developed Eye Navi. Eye Navi’s product manager, Shohei Takata, tried LinkBuds and was of the opinion that they would be the perfect headphones for people with visual impairments to use to listen to voice navigation in the Eye Navi app.

Haruko: The unique open-ring design of LinkBuds suited the needs of people with visual impairments who want to be able to hear the voice guidance and sounds of the outside environment at the same time.

Black LinkBuds and white LinkBuds, both of which lids are open, are placed on a brown surface side by side

Mitsunori: That is right. The major feature of the LinkBuds is the unique design with the ring-shaped driver unit. As this means the part you insert in the ear has a hole in it, the ear is not blocked, and you can hear sounds from the surrounding environment directly.

For people with visual impairments, sounds in the surrounding environment can convey vital information. When going out alone, a person with a visual impairment relies not only on the haptic feedback from their white cane, but they move through the environment while analyzing auditory information one by one, such as the sound of cars, people’s footsteps, and the sounds emanating from stores. Being able to pick up on subtle echoes and reverberation is also essential for pinpointing location, and Shohei thought that the LinkBuds design was extremely well-matched to this usage scenario. He was very keen to work together, and so we embarked on joint research and development.

Haruko: CSI is based in Kitakyushu, Fukuoka Prefecture. That is quite a long way away, so did you have any difficulties in working together?

Mitsunori: Things actually went very smoothly. Development of Eye Navi is carried out by a small, elite team whose members are agile in determining what is beneficial for the user and implementing it. When we give technical feedback in conversation with the team’s engineers, they respond with great speed and flexibility.

LinkBuds have a function to optimize the spatial sound according to how you wear the earbuds, and this is usually set through Sony’s Headphones Connect app. However, we were concerned that people with visual impairments may find it difficult to switch between the two apps (Eye Navi and Headphones Connect) and offered their support. I was incredibly grateful that the Eye Navi development team was able to implement this feature within the Eye Navi app itself.

Video: Click here to see Ena Asano, lead user at Inclusive Design Solutions Co., Ltd., try LinkBuds together with Eye Navi

Intuitive navigation using Sony's spatial sound technology

Haruko: I had the opportunity to try LinkBuds × Eye Navi before the interview. You hang your iPhone around your neck with the rear camera facing forward, enter your destination into the Eye Navi app, then receive real-time voice guidance using your GPS location, such as “In 20 meters, turn left.” If there are obstacles ahead, such as roadside trees, people, etc., you will hear the name called out from the direction of the object. I could hear the sounds around me, and I was able to use the app while talking to other people, resulting in a fun walking experience.

Haruko Miki is wearing neck pouch in which a smartphone is inside
By putting your phone in the special pouch that goes round your neck like shown in the picture, Eye Navi can extract information from the surrounding environment.

Also, when I heard “pedestrian crossing” announced on my left, I deliberately turned and went closer to it. On doing so, I almost immediately heard “pedestrian crossing” directly in front of me, and I was so astonished by the accuracy that I let out a gasp (laughs).

Haruko Miki and Mitsunori Tanaka sitting on benches talking to each other

Mitsunori: This comes from a combination of the LinkBuds’ spatial sound and Eye Navi’s obstacle detection function using the phone’s camera. For the audio announcement of obstacles and other objects, the direction the camera is facing is assumed to be the direction of travel, and the audio announcements come from the relative direction in which target objects are located. LinkBuds ’spatial sound technology has made voice navigation more intuitive and easier to understand.

Mitsunori Tanaka sitting on benches talking to Haruko Miki

Haruko: Certainly, what I felt keenly when trying the LinkBuds was surprised at the fact that we as humans can hear sound in three dimensions. That is because, at first, I tried Eye Navi with the sound coming from the phone’s mono speaker, and voice announcements such as “person in front of you” and “tree on the left” came in such quick succession that I was overwhelmed by the amount of information.

However, the moment that I connected LinkBuds, the announcements were cut down to the minimum information necessary, such as “person” and “tree,” and I could hear them coming from the direction in which the obstacle was located. I was able to immediately grasp the situation in front of me because I was listening to the spatial sound of the LinkBuds. I was once again awed by the incredible abilities of humans and how they contribute to broadening our personal experiences.

Mitsunori: In this project, development proceeded by having people with visual impairments test the prototype many times and by receiving feedback from the many users who visited accessibility-related exhibitions where we exhibited it.

One of the hardest things was how to use the spatial sound technology. When you think of playing spatial sound, you generally think of listening to music or watching a movie. However, in this case, we have employed a somewhat special use of spatial sound technology that allows the listener to accurately discern which direction a person’s voice (i.e., the voice guidance) is coming from in 360 degrees.

During the development, the presence of Junya Suzuki, a Sony engineer with a visual impairment, was of great help. He often told us to avoid causing front-back confusion, in which one misinterprets the positioning of things in front and behind. Making a voice sound from the left or right is relatively simple, as this can be controlled by changing the volume balance between left and right, but we cannot do the same thing for the spatial relationship between front and back.

Haruko Miki and Mitsunori Tanaka sitting on benches talking to each other

That is where Junya was able to help, and together with the Eye Navi development team, we thoroughly adjusted the way the voice guidance sounds based on his feedback. Junya is an engineer specializing in audio technology development, so he was able to give us advice such as, “If you process the sounds from behind in this fashion, then it will give a sense of direction.” As he is also a user, his input allowed us to pursue a more realistic and easier to understand kind of voice guidance.

Products made with particular people in mind open up an inclusive future.

Mitsunori: Sony’s accessibility philosophy states, “Through our commitment to an inclusive future, we aim to create a world where everyone belongs, and strive to enable and empower individuals of all abilities to share Kando (emotion).” We will pursue creativity and technology so that we can provide valuable experiences regardless of whether or not a person has a disability. This is something that we have always firmly believed in when it comes to accessibility.

Previously, we developed the neckband-style personal sound amplifier for the Japanese market (SMR-10), aimed at senior customers and released in 2017, and we still hear from people who bought it as a gift for their parents, telling us their parents had said positive things such as that it had changed their lives, and that they were able to enjoy music concerts again. The SMR-10 has a stylish design that does not look like a sound amplifier device, and this is also well-received, as it lowers the psychological barrier to using it.

Product photo of SMR-10 neckband-style personal sound amplifier
SMR-10 neckband-style personal sound amplifier

This kind of product is not limited to senior customers, however, as it carries the general value of allowing people to hear quiet or distant sounds more clearly in terms of functionality. This also removes the communication barrier from the speaker’s point of view, as they do not need to raise their voice. Taking this recent collaboration with Eye Navi as an example, in the future, it should be possible for people with normal vision to use the navigation function and walk safely with voice guidance without looking at their smartphone screen.

Haruko: I also had a lot of fun when I tried out LinkBuds × Eye Navi. I thought that this technology could also be expanded to the realm of entertainment. It excites me to think how products originally designed to remove inconveniences for a particular set of people can become something for everyone to enjoy. That is something that I want to do in my own work.

Haruko Miki and Mitsunori Tanaka sitting on benches talking to each other

Mitsunori: Exactly, the ideal would be to make it an experience that anyone can enjoy. In fact, I am currently in conversation with The Going Out Walking Support Project, a proposal to support walking using a white cane. This involves using a sensing device that can detect obstacles to give an audio alert about obstacles that a white cane cannot reach. For example, in the future, if it became possible to sense obstacles that are behind you, then you could be notified when an electric vehicle is quietly approaching, thereby realizing safer and more enjoyable walks around the city.

In that sense, the possibilities of accessibility technologies such as voice UI and UX are immeasurable. Because I work in a cutting-edge field that is rapidly advancing, I feel that it is important to incorporate any products that are good instead of sticking to in-house technologies and promote research and development in an open manner.

A person holding a stick
Prototype of The Going Out Walking Support Project demonstrated at CEATEC 2023

A world where anyone can enjoy going out for a walk: the future imagined by LinkBuds × Eye Navi

After Haruko Miki’s interview with Mitsunori Tanaka, we were able to speak online to Shohei Takata from CSI.

A portrait of Shohei Takata

Shohei Takata

General Manager of Sales Planning / Planning Development COMPUTER SCIENCE INSTITUTE CO., LTD.

A Fukuoka native, Shohei has a diverse background, including studying in Ireland, experience as an entrepreneur, and working for an IT company in India. He joined Computer Science Institute Co., Ltd. in 2015. Shohei is responsible for a wide range of duties, such as sales, planning, development, PR, general affairs, and HR. He is the project manager and also oversees national support projects for Eye Navi, a walking support app for people with visual impairments. It is his goal to create a society in which anyone can travel anywhere freely and enjoyably, and he is doing his utmost to realize that vision.

Shohei: We started developing Eye Navi eight years ago. The motivation for the development came from a development project to create a robot to replace guide dogs, which our president had previously envisioned when he was working at a map company. At that time, AI technology was not as advanced as it is today, and GPS devices, etc., were prohibitively expensive, so it was difficult to bring anything to fruition, but with the improvement of IT and spread of the iPhone, the feasibility of the idea in app form became evident. Therefore, in April 2023, we launched Eye Navi, a walking support app for people with visual impairments, using government subsidies and other funding, with the aim of realizing a society in which anyone can travel anywhere freely and enjoyably.

Shohei Takata is holding a smartphone, and another male with a white cane wearing LinkBuds is putting his left hand on Takata's shoulder

I will never forget the words of a blind person who assisted in the development. They noticed the button in the app labeled “Go for a stroll,” and said, “I’ve never gone for a stroll before.” They had always traveled with a particular purpose, and they had never thought of going for a stroll. They told us how delighted they would be if Eye Navi allowed them to do that.

At one exhibition, a mother told us that her son, who is a university student and has a visual impairment, had tended to stay in, but has been venturing out more since he started using Eye Navi. According to her, he had even been to a live concert by himself recently. Afterwards, he said, “It was amazing! Let’s go together next time,” she told us, with tears in her eyes. It made me appreciate what a wonderful thing it is to be able to go out for enjoyment rather than for a specific purpose.

Then, when I saw Sony’s LinkBuds at the Japan Lighthouse Exhibition, I felt certain that they would be a good match for Eye Navi. In the past, our monitors with visual impairments had often asked me what headphones I recommended, but I had only been able to answer, “Whichever ones you like best.” However, the LinkBuds, with their open-ear style, seemed like just the thing to be able to use Eye Navi comfortably.

I was ecstatic when we had agreed to work together, and Mitsunori asked me if he could visit us in Kitakyushu. Not only that, but he turned up with a team of 10 or so engineers, and we spent almost half a day brainstorming (laughs). There was much stimulating conversation, and I remember being excited at the prospect of what we could develop together.

I hope that with the LinkBuds integration, even more people will be able to use Eye Navi in the future. There are many people who experience difficulties getting around: not only people with visual impairments, but also elderly people and people from other countries who feel insecure due to the language barrier. I hope that we can further evolve LinkBuds × Eye Navi, with a view on global expansion, and realize a world in which anyone can enjoy walking around the city.

Shohei Takata is helping a male with visual impairments waring LinkBuds

Junior reporters’ takeaways

・The pursuit of UX and the power of technology can further enhance human abilities.

LinkBuds × Eye Navi uses spatial sound technology to achieve an innovative experience that makes the most of hearing, made with the help of people with visual impairments who perceive the world through hearing. As Mitsunori said, this project’s pursuit of UX and the technology to achieve it has the potential to evolve into an experience for everyone, regardless of whether or not they have a disability. This experience made me also want to become an engineer who constantly explores the potential of human abilities and continues to pursue new experiences.

・Getting closer to people in order to create products that they will love.

Mitsunori has devoted himself to accessibility as part of his life’s work. I felt that there is a constant awareness of getting closer to people behind that dedication. The developers of Eye Navi and the Sony engineers involved in the development consider the issues faced by people with visual impairments and what they need. Sony as a company considers what kind of teams to put together and what they should aim for in order to bring smiles to people’s faces. Creating a product that people will love requires making the effort to gain a clear understanding of the customer profile and to speak directly to customers who are different from oneself in order to get closer and understand them. I would like to uphold this attitude that I learned from Mitsunori, and work on developing products that meet people’s needs.

・Seize opportunities, take on challenges, and expand beyond your field.

Various departments at Sony are investigating accessibility, but it varies by department and by project, and users also exist outside the company. Just like the LinkBuds × Eye Navi project, I think it is people like Mitsunori Tanaka, who seizes opportunities and runs with them, that are capable of transcending boundaries and connecting together the various possibilities. Above all, when he speaks about his experiences his eyes light up, and he seems to enjoy his work wholeheartedly. I would also like to enjoy cross-disciplinary challenges so that I can try my hand at various things, gain experience, and create new value.

Haruko Miki and Mitsunori Tanaka sitting on benches holding LinkBuds

*Click here for information on software updates for LinkBuds and Eye Navi app integration.

SHARE ツイートする facebookでシェアする

関連