News

SINGAPORE – Like many other children and teenagers, Josh Tseng spent some of his adolescent years playing video games such as Pokemon, Minecraft, and Team Fortress 2. However, he was robbed of this joy as his eyesight began to gradually deteriorate due to congenital glaucoma. By the age of 16, he was unable to read or write. Thanks to a new software, however, Mr Tseng, 27, and other visually impaired individuals like him can continue to enjoy the thrill of some first-person shooter (FPS) games, such as Counter-Strike 2. The JBL Quantum Guide Play software uses advanced artificial intelligence to process in-game visuals, and translate them into audio cues, said Ms Lee Yeeling, senior director of Asia consumer marketing in the Asia-Pacific region at Harman International. Harman is the parent company of audio equipment manufacturer JBL. “The software allows visually impaired gamers to rely on sound for navigation and gameplay,” said Ms Lee in an interview with The Straits Times, adding that it utilises spatial audio technology, which gives players a three-dimensional audio experience that reflects the players’ surroundings. Launched on April 4, the software is free to download and was developed by JBL in collaboration with non-profit organisation AbleGamers. The latter, which aims to improve accessibility in the gaming space, contributed two years’ worth of research and facilitated testing with visually impaired gamers. With the software, players will be aurally alerted to the presence of in-game obstacles such as crates and cars, enemies, and teammates, with different sounds, like clicks and beeps, which are customisable. Although the software works only with Counter-Strike 2 currently, it will be made open-source soon, according to the software’s website. This means other companies and developers can modify and implement it in their games. In October 2023, Mr Tseng - whose vision is less than five per cent in both eyes - and Singaporean paralympic swimmer Sophie Soon were approached to be part of a global group of test gamers that helped with the software development. The duo gave very valuable feedback, said Ms Lee. In-game functions such as navigation have been improved greatly because of feedback provided, said Mr Tseng during an interview with ST on April 26, where he also gave a demonstration of the software. For example, initially, players would hear a sound when they were near walls, but the testing team realised that this was not essential as players would automatically no longer be able to move when they are against a wall, and sounds of their footsteps would stop, said Mr Tseng. “So what we actually wanted to look for are landmarks, so that they can be used to orientate the player. For instance, when I’m walking into a room, the fact I that I know there are four walls in a room has a limited purpose,” he added. “But if there’s a chair next to the door in the room, then I would want to know where the chair is instead, so I can leave the room. Obstacles in the game’s map act as landmarks.” There was a learning curve that came with getting used to the software, said Mr Tseng, although he said this is also the case with mastering any other piece of assistive technology. With time and practice, though, the digital accessibility specialist eventually found himself instinctively reacting to audio cues that popped up. Ms Soon, who is also visually impaired, agreed that it takes time and practice to get used to the software. “But it definitely opens a whole new world and quite unchartered territory for many visually impaired people, who now have the option to explore gaming,” said the 26-year-old, who was diagnosed with cone rod dystrophy as a child. If software such as Guide Play becomes more commonplace, gaming as a recreational activity could eventually be among the services provided by social service organisations when coaching visually impaired people, added Mr Tseng. “When we think of rehabilitation, we think of all the boring things like for work, or outdoor navigation,” he said. “But things like recreation and social activities are also very important. People with disabilities deserve to have fun too.” Guide Play was downloaded around 300 times within two weeks of its launch. The software also attracted more than 5000 visitors to its webpage. “We’re optimistic these figures will grow as more gamers discover the software,” said Ms Lee. “We do have plans to work with AbleGamers to expand the software to more games, but right now we are still trying to get more user feedback, and to make sure whatever we’ve rolled out is going to be the best experience.” Although the software is still in its early stages, the possibility of it being adapted to more games in the future is what excites Mr Tseng the most. At the top of his wishlist is Baldur’s Gate 3, a popular role-playing game that has won multiple awards. “This software is making possible something that was previously considered impossible,” said Mr Tseng. “I’ve spoken to many people who have worked on accessible gaming, and it is a very difficult problem set.” “So I’m very glad this has started, because this lays the foundation for more people to continue working on it in the future.”