Coding accessibility: How Della found her voice with open source AAC

For Della Calder, none of the Augmentative and Alternative Communication (AAC) apps she tried seemed to resonate. She’d been in speech therapy for a few years and had used a few different apps and devices, but nothing stuck—not to mention that each came with a hefty price tag. That’s why her brother, Archer, decided to build an open source app that could be tailored to Della’s needs and preferences. 

With help from the open source community, Archer created FreeSpeech, an app that allows users to communicate by pushing individual buttons for words and ideas, which the app then translates to speech. When Archer first showed Della the app, he put words for all of her favorite things—snacks, TV shows, and school buses among them—instead of the usual collection of words found in proprietary AAC apps. 

It was an immediate hit, and ever since, Della has been increasingly able to communicate with and participate in the world around her.  

“Communication shouldn’t be restricted just to people who can talk,” says Archer. “People who are nonverbal have the same right to communication, and they shouldn’t have to pay for it.”

Check out the video below—the latest in GitHub’s Coding Accessibility video series—to learn how the open source accessibility software Archer created, with Della at his side, has enabled her to participate in the world where proprietary alternatives failed.

A copy of this video with audio descriptions of visual content is also available on YouTube.

Della’s ability to be a part of FreeSpeech’s development process has been key to its success. “Watching Della use software, then attempting to emulate those usage patterns in FreeSpeech, has been really beneficial to her experience using the application,” Archer says. For example, he says that watching her navigate YouTube by clicking through suggested videos until she found what she was looking for, rather than using navigation elements like search or categories, inspired FreeSpeech’s initial design.

“It’s this mode of collaboration, where she's on a canary branch trying new features as I refine what she uses and what she doesn't, that I think makes FreeSpeech so intuitively useful for so many people,” Archer says.

This mode of collaboration is also known as participatory design, or co-design as it’s commonly called these days, which emphasizes the importance of actively including all stakeholders in the design process to ensure the end product meets their needs. 

Having watched Della use other apps, Archer decided to allow users to disable the ‘sentence builder’ feature—something that comes standard in other AAC apps. With the sentence builder disabled, the app acts as more of a sound board, allowing users to communicate with single words rather than forcing grammatically complete sentences. He points to this decision as one example of this co-design process, where Della contributed to a more accessible and useable product for herself and others by participating in its design.

“That communication style can sometimes be 10x more effective for Della,” Archer says. “I’ve seen a lot of people get hung up and frustrated when they’re made to use full sentences and nothing less. The problem is caused by the AAC software's inability to meet users where they're at, communication-wise.”

Archer also points to another FreeSpeech feature—the ability to add new vocabulary remotely via the cloud, rather than just on her device—as one that meets Della’s individual needs, since taking away her hardware to make changes means taking away her voice. “She doesn’t want me to take her iPad,” he says. “That’s her connection to the world.”

Since FreeSpeech makes it quick and easy to add new buttons, Della has expanded her vocabulary to include a wider range of topics she cares about. Previously, her family might need to scroll through endless options when searching for a movie to watch until they happen to land on the right one. Now, Della has had all her favorites added, so she can request them directly instead. 

Beyond logistics like picking a movie or a snack, she has also added the people in her life, such as classmates and teachers, so that she can talk about what happened at school that day. One day, while they were watching a TV show, she asked Archer to add a new button using a screenshot of a particularly sad moment in the episode. Now, Della uses that button in combination with others, such as people at her school, to better express herself. 

“She would use that button to talk about if she was sad, or if something sad happened at school,” says Archer. “That linking of ideas was something that she was never able to do before with other AAC apps. I think she’s really into FreeSpeech because it’s tailored to her.”