Student researcher urges natural language processing research focus on signed languages

Advancements successful earthy connection processing (NLP) alteration computers to recognize what humans accidental and assistance radical pass done tools similar instrumentality translation, voice-controlled assistants and chatbots.

But NLP probe often lone focuses connected spoken languages, excluding the much than 200 signed languages astir the satellite and the astir 70 cardinal radical who mightiness trust connected them to communicate.

Kayo Yin, a master's pupil successful the Language Technologies Institute, wants that to change. Yin co-authored a insubstantial that called for NLP probe to see signed languages.

"Signed languages, adjacent though they are a important portion of the languages utilized successful the world, aren't included," Yin said. "There is simply a request and an value successful having exertion that tin grip signed languages."

The paper, "Including Signed Languages successful Natural Language Processing," won the Best Theme Paper grant astatine this month's 59th Annual Meeting of the Association for Computational Linguistics. Yin's co-authors included Amit Moryossef of Bar-Ilan University successful Israel; Julie Hochgesang of Gallaudet University, a schoolhouse for the Deaf and hard of proceeding successful Washington, D.C.; Yoav Goldberg of Bar-Ilan University and the Allen Institute for AI; and Malihe Alikhani of the University of Pittsburgh's School of Computing and Information.

The authors wrote that communities relying connected signed language person fought for decades some to larn and usage those languages, and for them to beryllium recognized arsenic legitimate.

"However, successful a predominantly oral society, Deaf radical are perpetually encouraged to usage spoken languages done lipreading oregon text-based communication," the authors wrote. "The exclusion of signed languages from modern connection technologies further suppresses signing successful favour of spoken languages."

Yin archetypal became funny successful sign language portion doing outreach enactment astatine a stateless structure portion she was an undergraduate astatine École Polytechnique successful Paris. There, she met a Deaf pistillate and saw however hard it was for her to found societal connections with others. Yin started learning French motion connection and pursued motion connection translation arsenic portion of her undergraduate research.

Once astatine the LTI, she noticed that astir each NLP probe addressed lone spoken languages. Computer imaginativeness probe sought to recognize signed languages but often mislaid the linguistic properties that signed languages stock with spoken languages.

Signed languages usage hand gestures, facial expressions, and caput and assemblage movements and tin convey aggregate words astatine once. For example, idiosyncratic could motion "I americium happy," but shingle their caput portion doing it to bespeak that they are not happy. Signed languages besides employment shortcuts akin to the use of pronouns successful spoken languages. Natural connection processing tools are amended equipped than machine imaginativeness methods unsocial to grip these types of complexities.

"We request researchers successful some fields to enactment manus successful hand," Yin said. "We can't afloat recognize signed connection if we lone look astatine the visuals."

Hochgesang, a Deaf linguist who focuses connected signed languages, said that erstwhile she was studying for her degree, determination was hardly immoderate notation of signed languages successful the literature, successful her linguistics classes and successful probe similar NLP. Language was speech; different methods of expressing connection were ignored.

"On a idiosyncratic scale, this hurt. It wholly ignored my mode of being," Hochgesang said. "When I was a student, I didn't spot myself successful the information being described and that made it truly hard for maine to connect. That it inactive hasn't improved overmuch these days is unfortunate. The lone mode this benignant of happening volition alteration is if we are included more."

Yin said the insubstantial was good received by some natural language processing researchers and radical studying and utilizing signed languages—the 2 groups she sought to bring together.

"It's truly breathtaking to spot a insubstantial the I wrote motivate people, and I anticipation tin marque a alteration successful these communities," Yin said.

---

---

More information: Including Signed Languages successful Natural Language Processing, arXiv:2105.05222 [cs.CL] arxiv.org/abs/2105.05222