Computer science; computing; Amazon / en AI-assisted computer vision research aims to improve accessibility for deaf, hard of hearing /news/2023-02/ai-assisted-computer-vision-research-aims-improve-accessibility-deaf-hard-hearing <span>AI-assisted computer vision research aims to improve accessibility for deaf, hard of hearing</span> <span><span lang="" about="/user/1441" typeof="schema:Person" property="schema:name" datatype="" xml:lang="">Teresa Donnellan</span></span> <span>Mon, 02/27/2023 - 09:55</span> <div class="layout layout--gmu layout--twocol-section layout--twocol-section--30-70"> <div class="layout__region region-first"> <div data-block-plugin-id="field_block:node:news_release:field_associated_people" class="block block-layout-builder block-field-blocknodenews-releasefield-associated-people"> <h2>In This Story</h2> <div class="field field--name-field-associated-people field--type-entity-reference field--label-visually_hidden"> <div class="field__label visually-hidden">People Mentioned in This Story</div> <div class="field__items"> <div class="field__item"><a href="/profiles/kosecka" hreflang="und">Jana Košecká</a></div> </div> </div> </div> </div> <div class="layout__region region-second"> <div data-block-plugin-id="field_block:node:news_release:body" class="block block-layout-builder block-field-blocknodenews-releasebody"> <div class="field field--name-body field--type-text-with-summary field--label-visually_hidden"> <div class="field__label visually-hidden">Body</div> <div class="field__item"><p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US"><span class="intro-text">Digital assistants like Amazon’s Alexa aren’t currently useful for, say, the hard of hearing and deaf community. AV researchers led by Jana Košecká are making the Internet of Things more inclusive and accessible to those for whom it has not been designed. For the next year, her work to improve "seeing" computer systems to translate continuous American Sign Language into English will be funded by Amazon’s Fairness in AI Research Program. </span></p> <figure role="group" class="align-right"><div> <div class="field field--name-image field--type-image field--label-hidden field__item"> <img src="/sites/g/files/yyqcgq291/files/styles/small_content_image/public/2023-02/kosecka.png?itok=WnlhnQuc" width="350" height="340" alt="portrait of Jana Kosecka" loading="lazy" typeof="foaf:Image" /></div> </div> <figcaption>Jana Košecká. Photo by Ron Aira/Creative Services</figcaption></figure><p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">Having worked at Mason for more than 20 years, Košecká began studying computer vision as it applies to American Sign Language in 2019 with colleagues Huzefa Rangwala and Parth Pathak in collaboration with Gallaudet University. Their work resulted in three academic publications on the topic in 2020. The team’s initial work focused on computer vision recognizing American Sign Language at the word level.  </p> <p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">Košecká describes her current work as a continuation of earlier work, but now, especially with the help of AI, she’s tackling more complex ASL content, such as sentence-level communication, facial expressions, and very specific hand gesticulation.</p> <p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">“The challenge of extending some of these ideas [of computer translation] to American Sign Language translation is the input is video as opposed to text; it's continuous, and you have a lot of challenges, because you have a lot of variations about how people sign,” says Košecká.</p> <p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">The project is accordingly multifaceted. “We are focusing on better hand modeling, focusing on incorporating the facial features and extending to continuous sign language, so you can have short phrases the model can translate to English,” Košecká explains. “We are basically trying to capture continuous sign language and not just individual words." </p> <p><span lang="en-US" xml:lang="en-US" xml:lang="en-US"><span>To accomplish this goal,Košecká is using weakly supervised learning machine learning methods that provide mechanisms to teach the system without excessive human labelling effort.<span lang="en-US" xml:lang="en-US" xml:lang="en-US"><span><span><span><span><span><span><span><span><span><span> </span></span></span></span></span></span></span></span></span></span></span></span></span></p> <figure class="quote"><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span lang="en-US" xml:lang="en-US" xml:lang="en-US"><span><span><span><span><span><span><span><span><span><span><span><span><span>“</span></span></span><span><span><span>Weakly supervised learning</span></span></span><span><span><span> techniques</span></span></span> <span><span><span>don't need</span></span></span> <span><span><span>perfect alignment of video sequences that contain multiple words</span></span></span><span><span><span>,” she says. <span><span><span>“</span></span></span><span><span><span>I</span></span></span><span><span><span>n the word</span></span></span><span><span><span>-</span></span></span><span><span><span>level recognition</span></span></span><span><span><span>, t</span></span></span><span><span><span>he </span></span></span><span><span><span>model is presented with examples of a</span></span></span><span><span><span> video </span></span></span><span><span><span>snippet of a signed word and the word text</span></span></span><span><span><span>,</span></span></span><span><span><span> so it</span></span></span><span><span><span> has</span></span></span><span><span><span> perfect supervision.</span></span></span> <span><span><span>G</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span lang="en-US" xml:lang="en-US" xml:lang="en-US"><span><span><span><span><span><span><span><span><span><span><span><span><span>iven many examples of </span></span></span><span><span><span>the sign</span></span></span> <span><span><span>‘</span></span></span><span><span><span>a</span></span></span><span><span><span>pple</span></span></span><span><span><span>’</span></span></span><span><span><span> as a video snippet</span></span></span><span><span><span>,</span></span></span><span><span><span> the system will learn to recognize the word 'apple.' ” </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></figure><p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">“There are some techniques which can discover patterns without this need of direct supervision. If you just give the model a lot of examples, the model will figure out repeating patterns of certain words occurring in certain contexts,” she says. “So we are applying these machine-learning techniques to the setting of American Sign Language.”  </p> <p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">Relating her work to AI-powered chatbots like chatGPT, Košecká says, “There has been a lot of headway made in this space for written and spoken languages, and we would like to make a little bit of headway for American Sign Language, using some of these insights and ideas.” </p> <p lang="EN-US" xml:lang="EN-US" xml:lang="EN-US">Košecká envisions her research helping improve the interface between hard of hearing people and their environment, whether that be when they’re communicating with Amazon’s Alexa or ordering at a restaurant counter. No doubt her work will help improve inclusivity and accessibility for the deaf and hard of hearing both at Mason and beyond.  </p> </div> </div> </div> <div data-block-plugin-id="field_block:node:news_release:field_content_topics" class="block block-layout-builder block-field-blocknodenews-releasefield-content-topics"> <h2>Topics</h2> <div class="field field--name-field-content-topics field--type-entity-reference field--label-visually_hidden"> <div class="field__label visually-hidden">Topics</div> <div class="field__items"> <div class="field__item"><a href="/taxonomy/term/9191" hreflang="en">American Sign Language</a></div> <div class="field__item"><a href="/taxonomy/term/6921" hreflang="en">Computer science; computing; Amazon</a></div> <div class="field__item"><a href="/taxonomy/term/5606" hreflang="en">Inclusion</a></div> <div class="field__item"><a href="/taxonomy/term/11076" hreflang="en">Artifical Intelligence</a></div> <div class="field__item"><a href="/taxonomy/term/271" hreflang="en">Research</a></div> </div> </div> </div> </div> </div> Mon, 27 Feb 2023 14:55:35 +0000 Teresa Donnellan 104416 at Mason partnering with Amazon to provide computer science education to teachers /news/2021-01/mason-partnering-amazon-provide-computer-science-education-teachers <span>Mason partnering with Amazon to provide computer science education to teachers</span> <span><span lang="" about="/user/231" typeof="schema:Person" property="schema:name" datatype="" xml:lang="">Colleen Rich</span></span> <span>Tue, 01/26/2021 - 15:55</span> <div class="layout layout--gmu layout--twocol-section layout--twocol-section--30-70"> <div class="layout__region region-first"> </div> <div class="layout__region region-second"> <div data-block-plugin-id="field_block:node:news_release:body" class="block block-layout-builder block-field-blocknodenews-releasebody"> <div class="field field--name-body field--type-text-with-summary field--label-visually_hidden"> <div class="field__label visually-hidden">Body</div> <div class="field__item"><p><span><span><a href="https://www2.gmu.edu/">AV</a> is collaborating with Amazon and the Maryland Chamber Foundation to provide local teachers with computer science externships and education in a pilot program to begin this summer. The program will provide current and potential computer science teachers in Virginia, Maryland and the District of Columbia with an opportunity to shadow Amazon technology experts and take a companion three-credit graduate-level class at Mason.</span></span></p> <p><span><span><a href="https://cehd.gmu.edu/people/faculty/ekaya3/">Erdogan Kaya</a>, an assistant professor of computer science education at Mason’s <a href="https://cehd.gmu.edu/">College of Education and Human Development</a> (CEHD), will teach the eight-week, project-based companion class. In addition, participating teachers will also get an opportunity throughout the fall to meet with each other and exchange best practices. Virginia teachers can also participate in a free Praxis workshop in the fall to prepare to take the exam to become licensed to teach computer science. </span></span></p> <p><span><span>“We believe that through this opportunity, teachers can develop meaningful learning experiences that will help them in how they teach computer science to children,” said Kaya, a former K-12 computer science teacher. “Computer science is a dynamic field, and we want to engage with teachers to increase their understanding of the workforce skills and knowledge their students will need.”</span></span></p> <p><span><span>The program is currently accepting applications. Preference will be given to local middle school and high school teachers who are interested in teaching computer science or who want to integrate computer science into their instruction. Teachers who work in underserved communities will be prioritized. </span></span></p> <p><span><span>The Amazon externship will allow teachers to shadow Amazon professionals during the summer, said Shanika Hope, head of Amazon Future Engineer, U.S., giving them exposure to technology professionals in a substantive, practical way. The pilot program will accept 20 participants. </span></span></p> <p><span><span>Mason has been partnering with Amazon on other projects since it established a second headquarters in Northern Virginia. <a href="https://cehd.gmu.edu/people/faculty/ahutchi9/">Amy Hutchison</a>, associate professor and director of CEHD’s Division of Elementary, Literacy and Secondary Education, said that collaborating with Amazon has been seamless.</span></span></p> <p><span><span>“They are very open to our ideas,” said Hutchison. “This has been very exciting for us to bring together industry and education.”</span></span></p> <p><span><span>The Maryland Chamber Foundation is in its third year of offering teaching externships to Maryland educators. The foundation works to bridge the gaps between businesses and educators by pairing teachers with top local businesses. </span></span></p> <p><span><span>If successful, the pilot program could be offered again and expanded. In addition, CEHD researchers will study the effectiveness of the new program in changing how computer science is taught in the classroom.</span></span></p> <p><span><span>Hope said the genesis of the program was a recognition of the gaps between the skills taught in schools and those needed for the job market.</span></span></p> <p><span><span>“Teachers inspire, support and expose students to new opportunities, so their proficiencies and expertise have a direct impact on the paths that students pursue,” said Hope. “We want to invest in teachers in a way that will affect students in the long term.”</span></span></p> </div> </div> </div> <div data-block-plugin-id="field_block:node:news_release:field_content_topics" class="block block-layout-builder block-field-blocknodenews-releasefield-content-topics"> <h2>Topics</h2> <div class="field field--name-field-content-topics field--type-entity-reference field--label-visually_hidden"> <div class="field__label visually-hidden">Topics</div> <div class="field__items"> <div class="field__item"><a href="/taxonomy/term/116" hreflang="en">Campus News</a></div> <div class="field__item"><a href="/taxonomy/term/191" hreflang="en">College of Education and Human Development</a></div> <div class="field__item"><a href="/taxonomy/term/6921" hreflang="en">Computer science; computing; Amazon</a></div> <div class="field__item"><a href="/taxonomy/term/1016" hreflang="en">Amazon</a></div> <div class="field__item"><a href="/taxonomy/term/7096" hreflang="en">Mason Momentum</a></div> </div> </div> </div> </div> </div> Tue, 26 Jan 2021 20:55:53 +0000 Colleen Rich 44451 at