When Google unveiled Project Gameface, the company was proud to showcase an AI-powered hands-free gaming mouse that, according to its announcement, allows people to control a computer cursor using head movement and facial gestures. While this may not be the first AI-powered gaming tool, it was certainly one of the first to put AI in the hands of players, rather than developers.
The project was inspired by Lancy Carr, a quadriplegic video game streamer who uses a mouse for head tracking as part of his gaming setup. After its existing hardware was lost in a fire, Google stepped in to create a low-cost, highly configurable, open source alternative to expensive replacement hardware, powered by machine learning. While the wider existence of AI is proving divisive, we set out to find out whether AI, when used for good, could be the future of accessibility in games.
It is important to define artificial intelligence and machine learning to clearly understand how they work in Gameface. When we use the terms AI and machine learning, we are referring to both the same and different things.
AI is a concept, Laurence Moroney, head of AI defense at Google and one of the masterminds behind Gameface, tells WIRED. Machine learning is a technique you use to implement that concept.
Machine learning, therefore, falls under the umbrella of artificial intelligence, along with implementations such as large language models. But where familiar applications like OpenAIs ChatGPT and StabilityAIs Stable Diffusion are iterative, machine learning is characterized by instruction-free learning and adaptation, drawing inferences from readable models.
Moroney explains how this is applied to Gameface in a number of machine learning models. The first was being able to detect where a face is in an image, he says. The second was, once we had the image of a face, being able to understand where the obvious points are (eyes, nose, ears, etc.).
Later, another model can map and decipher gestures from those points, assigning them to mouse inputs.
It is an explicitly assistive implementation of AI, as opposed to one often advertised as redundant of human input. In fact, this is how Moroney suggests we best apply AI, to broaden our ability to do things that weren’t previously feasible.
This sentiment goes beyond Gameface’s potential to make games more accessible. AI, suggests Moroney, can have a major impact on accessibility for gamers, but also on how developers build accessibility solutions.
Anything that allows developers to be orders of magnitude more effective at solving classes of problems that were previously unfeasible, he says, can only be beneficial in accessibility or any other space.
This is something developers are already starting to figure out. Artem Koblov, creative director of Perelesoq, tells WIRED that he wants to see more resources directed towards solving routine tasks, rather than creative invention.
This allows AI to assist in time-consuming technical processes. With the right applications, AI could create a leaner, more forgiving development cycle where it both aids in the mechanical implementation of accessibility solutions and gives developers more time to consider them.
As a developer, you want to have all the tools that can help make your job easier, says Conor Bradley, creative director at Soft Leaf Studios. He points to the benefits in current implementations of AI in accessibility, including real-time text-to-speech generation and speech and image recognition. And he sees the potential for future developments. Over time, I see more and more games using these powerful AI tools to make our games more accessible.
Koblov believes he can go even further. He would like to see AI training on specific models to create a basic, adaptable accessibility framework that could be built into games. Such a structure would adapt the visual, audio and interactive aspects of the games, he says. In other words, smaller developers like us shouldn’t have to conduct expensive research, develop unique solutions, and go through numerous iterations of testing on their own.
Bradley urges caution when moving primacy away from human input. Asked whether AI could prove to be an aid or a distraction to existing accessibility efforts, he said he was optimistic about its potential, but stressed that AI is not a shortcut.
Can’t you say, AI, make my game accessible! and presto, you now have the most accessible game of the year, she says. We need gamers, including those from disabled and neurodiverse communities, to test our games. At the end of the day, a human will be playing your game, not a machine.
While Koblov believes AI can be invaluable for implementing and testing accessibility features, he acknowledges that thinking about AI requires an additional mindset, rather than a substitute approach.
But confusing content-driven, generative tools that spark fears of human redundancy with the kind of AI implementations that aid accessibility is, according to Moroney, downright dangerous. He continues, if we are to be the adults in the room when it comes to AI, we need to acknowledge the hype and the bandwagon.
This makes clarity and transparency about AI capabilities even more important, especially in relation to accessibility. It’s not a magic wand. AI and machine learning were fine until recent releases, Moroney says. Now they have fallen back into the hype cycle.
AI can be an excellent tool for developers, but they need to remain dedicated to accessibility throughout the process whether AI is present or not. After all, as Bradley says, at the end of the day, it’s still up to developers to want to make their games accessible by design.
The gradual progress of AI is evident in Gameface. But another project demonstrates how AI-assisted accessibility can be implemented on a larger level. Minecraft Access is a mod that seeks to create Minecraft accessible to blind and visually impaired players. Logic, part of the team behind the mod, tells WIRED how a suite of AI tools, including ChatGPT and Google’s Tensor Flow, are helping with the project.
We hope AI can fill in the visual context for blind and visually impaired gamers by providing information about the world when it’s needed or on demand, says Logic.
Particularly exciting is the potential for AI to not only enforce accessibility, but also actively learn what a player needs. This will prove particularly useful for broader applications in accessibility given the layers of spectrum that make up disability and how personalized each player’s needs are.
However, we have to curb our expectations. However promising these recent implementations have proven to be, and however instructive they may be for the future, significant barriers to entry remain. In its current stage of development, Minecraft Access requires more programs to function, something Logic acknowledges makes it less accessible than it could be.
The average user won’t want to collect a bunch of programs from different parts of the web, says Logic.
Similarly, Ben Green, a disabled gamer, finds the potential of Gamefaces exciting but is concerned about the diversity of data. He may be able to recognize many faces, he says. But some people with facial differences, like a fan in my case, or asymmetrical facial features, might be barely or not at all represented.
When asked about this, Andrs-Clavera says: We set out to build a feature for people to customize the expressions they use to control their mouse. This includes the ability to customize the intensity of gestures for different needs. He continues, That said, we’re always looking for ways to increase the accessibility of our technology for more people. Our hope is that over time Project Gameface will continue to improve and become even more useful.
Even with these caveats, it’s interesting to see how confident people are about the role of AI in accessibility. Once we can distinguish between unethical applications of generative content-driven AI and meaningful AI tools and implementations that can help people solve problems and benefit others, there are plenty of reasons to be optimistic with the understanding that the true value of AI is in our ability to make it work for us.
The future of AI is ambiguous, but it has the potential to benefit individual players and the industry at large. Its use requires caution and we can expect pitfalls, but there is every reason to believe that careful AI implementation can contribute to a gaming landscape that encompasses a broader spectrum of gamers.
This is the world Moroney wants to live in: a world where people like Lance aren’t confined because solutions are technically infeasible, but rather a world where developers have such superpowers that building solutions to connect them to the world is easy.
This story originally appeared on wired. com.
#improve #game #Ars #Technica #players
Image Source : arstechnica.com
