All
 
|
 
 


Multimedia Database of Russian Sign Language Items in 3D
 
PIIS0373658X0008302-1-1
DOI10.31857/S0373658X0008302-1
Publication type Article
Status Published
Authors
Ildar Kagirov 
Affiliation: St. Petersburg Institute for Informatics and Automation of RAS
Address: Russian Federation, St. Petersburg
Dmitry A. Ryumin
Affiliation: St. Petersburg Institute for Informatics and Automation of RAS
Address: Russian Federation, St. Petersburg
Alexander A. Axyonov
Affiliation: St. Petersburg Institute for Informatics and Automation of RAS
Address: Russian Federation, St. Petersburg
Alexey A. Karpov
Affiliation: St. Petersburg Institute for Informatics and Automation, Russian Academy of Sciences
Address: Russian Federation, St. Petersburg
Journal nameVoprosy Jazykoznanija
EditionIssue 1
Pages104-123
Abstract

The present paper describes a multimedia database (MMDB) of the Russian Sign Language (RSL) and briefly sketches the outlines of its phonological annotation. This MMDB is unusual thanks to 3D format, which provides new opportunities for the lexicographical description of the vocabulary and enhances the study of RSL articulation features. The three-dimensional representation of the gesture is achieved by using a Kinect 2.0 camera that records data in high resolution (FullHD), in the depth map mode and in the infrared range. The presented DB comprises 164 lexical items and clauses, demonstrated by 13 signers and repeated 5 times at least. The total size of all the files is 3.8 Tb, the video duration is more than 8 hours. The article provides an overview of the existing RSL databases, reveals the content of the described dictionary, outlines principles of linguistic annotation of gestures, and gives an explanation the set of features used for annotation of the present MMDB. It is noted in the paper that some parameters of signs can vary significantly from informant to informant, e.g. handshape and localization. However, the reasons for such variability, as well as its scope and significance for describing RSL, are a matter of further research.

Keywordscomputational linguistics, databases, Russian Sign Language, sign languages
AcknowledgmentThe research is supported by the Ministry of Science and Higher Education of the Russian Federation, under the agreement No. 14.616.21.0095 (ID Number: RFMEFI61618X0095).
Publication date02.03.2020
Number of characters37201
Cite  
100 rub.
When subscribing to an article or issue, the user can download PDF, evaluate the publication or contact the author. Need to register.

Number of purchasers: 2, views: 2434

Readers community rating: votes 0 

1. Буркова 2012–2015 — Burkova S. I. Russkii zhestovyi yazyk: obshchie svedeniya [Russian Sign Language: General information]. URL: http://rsl.nstu.ru/site/signlang. 
2. Буркова, Варинова 2012 — Burkova S. I., Varinova O. A. On geographical and social variation in Russian Sign Language. Russkii zhestovyi yazyk: Pervaya lingvisticheskaya konferentsiya. Fedorova O. V. (ed.). Moscow: Buki Vedi, 2012, 127–143.
3. Воскресенский 2002 — Voskresenskii A. L. An unrecognized language (Sign language of the deaf and computational linguistics). Trudy Mezhdunarodnogo seminara po komp’yuternoi lingvistike i intellektual’nym tekhnologiyam «Dialog-2002». Vol. 2. Protvino, 2002, 100–106.
4. Воскресенский и др. 2009 — Voskresenskii A. L., Gulenko I. E., Khakhalin G. K. The RuSLED dictionary as an instrument for semantic research. Materialy Mezhdunarodnoi konferentsii «Dialog-2009». Issue 8(15). Moscow: Russian State Univ. for the Humanities, 2009, 64–68.
5. Гейльман 2001 — Geyl’man I. F. Znakom’tes’: ruchnaya rech’ [Introducing the manual speech]. Moscow: Zagrei, 2001.
6. Гейльман 2004 — Geyl’man I. F. Slovar’russkogo zhestovogo yazyka [A dictionary of Russian Sign Language]: In 2 vols. St. Petersburg: Prana, 2004.
7. Давиденко, Комарова 2006 — Davidenko T. P., Komarova A. A. A brief sketch of Russian Sign Language. Sovremennye aspekty zhestovogo yazyka. Komarova A. A. (comp.). Moscow: The All-Russian Deaf Society, 2006, 146–161.
8. Димскис 2002 — Dimskis L. S. Izuchaem zhestovyi yazyk: uchebnoe posobie [Learning the sign language: A manual]. Moscow: Academia, 2002.
9. Жесты 1995 — Zhesty: slovar’-spravochnik [Signs: A reference dictionary]. Moscow: Zagrei, 1995.
10. Заварицкий 2015 — Zavaritskii D. A. 100 fraz na russkom zhestovom yazyke. Razgovornik dlya svyashchennosluzhitelei [100 phrases in Russian Sign Language. A phrasebook for priests]. Moscow: Lepta Kniga, 2015.
11. Зайцева 2000 — Zaitseva G. L. Zhestovaya rech’. Daktilologiya [Sign speech. Dactilology]. A textbook for higher school students. Moscow: VLADOS Publ., 2000.
12. Захаров 2005 — Zakharov V. P. Korpusnaya lingvistika [Corpus linguistics]. A teaching guide. St. Petersburg: St. Petersburg State Univ., 2005.
13. Кагиров 2018 — Kagirov I. A. A concept of an automatical shopping cart supporting spoken Russian and Russian Sign Language user interfaces. Regional’naya informatika i informatsionnaya bezopasnost’: sbornik trudov. No. 5. Sovetov B. Ya., Yusupov R. M., Kasatkin V. V. (eds.). St. Petersburg: The St. Petersburg Society for Information Technology, Computation, Communication and Management Systems, 2018, 273–277.
14. Карпов 2011 — Karpov A. A. Computer analysis and synthesis of Russian Sign Language. Voprosy Jazykoznanija, 2011, 6: 41–54.
15. Карпов 2013 — Karpov A. A. Automatic synthesis of Russian fingerspelling from the text. Nauchno-tekhnicheskaya informatsiya. Seriya 2: Informatsionnye protsessy i sistemy, 2013, 1: 20–26.
16. Карпов, Кагиров 2011 — Karpov A. A., Kagirov I. A. Formalizing the lexicon of an automatical sign language synthesis system. Trudy SPIIRAN, 2011, 1(16): 123–140.
17. Киммельман 2010 — Kimmelman V. I. Bazovyi poryadok slov v russkom zhestovom yazyke [Basic word order in Russian Sign Language]. Graduation thesis. Moscow: Russian State Univ. for the Humanities, 2010.
18. Кипяткова, Карпов 2017 — Kipyatkova I. S., Karpov A. A. A study of neural-network models of Russian for ASR systems. Avtomatika i telemekhanika, 2017, 78(5): 110–122.
19. КРЖЯ — Russian Sign Language Corpus. URL: http://rsl.nstu.ru/. 
20. Львовская 2006 — L’vovskaya A. Analiz vidovoi i vremennoi sistemy glagola russkogo zhestovogo yazyka (RZhYa) na osnovanii raboty Ö. Dahl «Tense and Aspect Systems» [Analysis of aspect and tense system of the verb in Russian Sign Language based on Ö. Dahl’s Tense and Aspect Systems]. Term paper. Moscow: Moscow State Univ., 2006.
21. Ляшевская 2016 — Lyashevskaya O. N. Korpusnye instrumenty v grammaticheskikh issledovaniyakh russkogo yazyka [Corpus instruments in grammatical studies of Russian]. Moscow: YaSK Publishing House, 2016.
22. Мануева и др. 2014 — Manueva Yu. S., Grif M. G., Kozlov A. N. Developing a system of automatic translation of Russian speech into sign. Trudy SPIIRAN, 2014, 37(6): 170–187.
23. Правовые основы 2016 — Pravovye osnovy realizatsii Konventsii OON o pravakh invalidov v Rossiiskoi Federatsii [Legal foundations of the realization of the UN Convention on the Rights of Persons with Disabilities in Russian Federation]. Shestakov V. P. et al. (comp.). St. Petersburg: G. A. Albrecht Federal Scientific Center of Rehabilitation of the Disabled, 2016.
24. Прозорова 2007 — Russian Sign Language as a subject of linguistic study. Voprosy Jazykoznanija, 2007, 1: 44–61.
25. РЖЯБК 2001 — Russkii zhestovyi yazyk. Bazovyi kurs. Komp’yuternaya obuchayushchaya sistema [Russian Sign Language. Basic course. A digital learning system.]. Moscow: Istina, 2001.
26. Сурдосервер — Project “Surdoserver 2.0”. URL: http://surdoserver.ru. 
27. Сысоев, Гриф 2018 — Sysoev M. D., Grif M. G. Developing a remote sign-translation system for the deaf. Razrabotka sistemy udalennogo surdoperevoda dlya glukhikh. Innovatsionnye infokommunikatsii XXI veka: The 19th student conf. Klepikov S. I. et al. (eds.). Khabarovsk: Khabarovsk Institute for Telecommunication and Information Technology, 2018, 97–99.
28. Федорова 2012 — Fedorova O. V. (ed.). Russkii zhestovyi yazyk: Pervaya lingvisticheskaya konferentsiya [Russian Sign Language. The 1st linguistic conference]. Moscow: Buki Vedi, 2012.
29. Федорова 2018 — Russian gestures from a linguistic perspective (dedicated to the publication of Elena A. Grishina’s monograph). Voprosy Jazykoznanija, 2018, 5: 114–123.
30. Филимонова, Душкина 2018 — Filimonova E. V., Dushkina V. A. Uchebno-metodicheskoe posobie dlya izuchayushchikh russkii zhestovyi yazyk [A teaching guide for Russian Sign Language learners]. Moscow, 2018. URL: https://www.academia.edu/38047551/Учебно-методическое_пособие_для_изучающих_русский_жестовый_язык. 
31. Фрадкина 2001 — Fradkina R. N. Govoryashchie ruki: Tematicheskii slovar’ zhestovogo yazyka glukhikh Rossii [Speaking hands: Thematical dictionary of Russian Sign Language].
32. Шамаро 2006 — Shamaro E. Yu. Some facts from the tense-aspect system of the Russian Sign Language. Sovremennye aspekty zhestovogo yazyka. Komarova A. A. (comp.). Moscow: The All-Russian Deaf Society, 2006, 180–191.
33. AF 1996 — Arbeitsgruppe Fachgebärdenlexika. Fachgebärdenlexikon Psychologie. Hamburg: Signum, 1996.
34. Battison 1978 — Battison R. Lexical borrowing in American Sign Language. Silver Spring (MD): Linstok Press, 1978.
35. Bauer 2018 — Bauer A. Word articulation in Russian Sign Language. Deutsche Beiträge zum 16. Internationalet Slavistenkongress. Belgrad, 2018. Kempgen S. et al. (eds.). Wiesbaden: Harrassowitz Verlag, 2018, 35–44.
36. Brentari 1998 — Brentari D. A prosodic model of sign language phonology. Cambridge (MA): MIT Press, 1998.
37. Brentari et al. 2010 — Brentari D. (ed.). Sign languages. Cambridge: Cambridge Univ. Press, 2010.
38. Chomsky, Halle 1968 — Chomsky N., Halle M. The sound pattern of English. New York: Harper & Row, 1968.
39. Garcia, Sallandre 2013 — Garcia B., Sallandre M.-A. Transcription systems for sign languages: A sketch of the different graphical representations of sign language and their characteristics. Body — language — communication. An international handbook on multimodality in human interaction. Vol. 1. Müller C. et al. (eds.). Berlin: De Gruyter Mouton, 2013, 1125–1140.
40. Digitgestus — Digitgestus. URL: http://www.digitgestus.com. 
41. Frischberg et al. 2012 — Frischberg N., Hoiting N., Slobin D. I. Transcription. Sign language. An international handbook. Pfau R. et al. (eds.). Berlin: De Gruyter Mouton, 2012, 1045–1075.
42. Grenoble 1992 — Grenoble L. An overview of Russian Sign Language. Sign Language Studies, 1992, 77: 321–338.
43. Hanke 2004 — HamNoSys — Representing sign language data in language. Resources and language processing contexts. URL: https://www.sign-lang.uni-hamburg.de/dgs-korpus/files/inhalt_pdf/HankeLRECSLP2004_05.pdf. 
44. Hanke, Storz 2008 — Hanke T., Storz J. Ilex — a database tool for integrating sign language corpus linguistics and sign language lexicography. Proc. of LREC 2008, Sixth International Conf. on Language
45. Resources and Evaluation. Calzolari N. et al. (eds.). Paris: European Language Resources Association, 2008, 64–67.
46. Huang et al. 2017 — Huang J., Rathod V., Sun Ch., Zhu M., Korattikara A., Fathi A., Fischer I., Wojna Z., Song Y., Guadarrama S., Murphy K. Speed/accuracy trade-offs for modern convolutional object detectors. Proc. of 30th IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). Los Alamitos (CA): Institute of Electrical and Electronics Engineers, 2017, 3296–3297.
47. Johnston 2001 — Johnston T. The lexical database of Auslan (Australian Sign Language). Wilbur R. (ed.). Sign Language and Linguistics, 2001, 4(1/2): 145–169.
48. Karpov et al. 2014 — Karpov A., Markov K., Kipyatkova I., Vazhenina D., Ronzhin A. Large vocabulary Russian speech recognition using syntactico-statistical language modeling. Speech Communication, 2014, 56: 213–228.
49. Karpov et al. 2016 — Karpov A., Kipyatkova I., Zelezny M. Automatic technologies for processing spoken sign languages. Procedia Computer Science, 2016, 81: 201–207.
50. Kennedy et al. 1997 — Kennedy G., Arnold R., Dugdale P., Fahey S., Moskovitz D. (eds.). A dictionary of New Zealand Sign Language. Auckland: Auckland Univ. Press; Bridget Willam Books, 1997.
51. Kimmelman 2009 — Kimmelman V. Reflexive pronouns in Russian Sign Language and Sign Language of the Netherlands. MA thesis. Amsterdam: Univ. of Amsterdam, 2009.
52. Kimmelman 2014 — Kimmelman V. Information structure in Russian Sign Language and Sign Language of the Netherlands: Ph.D. diss. Amsterdam: Univ. of Amsterdam, 2014.
53. Language Archive — The Language Archive. URL: https://tla.mpi.nl/forums/software/elan. 
54. Liddell, Johnson 1983 — Liddell S., Johnson R. E. American Sign Language: The phonological base. Washington (D.C.): Ms. Gallaudet Univ., 1983.
55. Perlmutter 1992 — Perlmutter D. M. Sonority and syllable structure in American Sign Language. Linguistic Inquiry, 1992, 3(23): 407–442.
56. Prillwitz et al. 1989 — Prillwitz S., Leven R., Zienert H., Hanke T., Henning J. HamNoSys. Version 2.0; Hamburg Notation System for sign languages. An introductory guide. Hamburg: Signum Verlag, 1989.
57. Ryumin et al. 2019 — Ryumin D., Kagirov I., Ivanko D., Axyonov A., Karpov A. A. Automatic Detection and Recognition of 3D Manual Gestures for Human-machine Interaction. ISPRS — International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2019, XLII-2/ W12: 179–183.
58. RSL — “Russian Sign Language” at Ethnologue. URL: https://www.ethnologue.com/language/rsl.  Sandler 1986 — Sandler W. The spreading hand autosegment of American Sign Language. Sign Language Studies, 1986, 50: 1–28.
59. Sandler 2012 — Sandler W. The Phonological organization of sign languages. Language and Linguistic Compass, 2012, 6(3): 162–182.
60. Sandler, Lillo-Martin 2006 — Sandler W., Lillo-Martin D. Sign language and linguistic universals. Cambridge: Cambridge Univ. Press, 2006.
61. SLCN — Sign Linguistics Corpora Network. URL: https://www.ru.nl/slcn/. 
62. Spreadthesign — Spreadthesign. URL: https://www.spreadthesign.com/ru.ru/search. 
63. Stokoe 1960 — Stokoe W. C. Sign language structure: An outline of the visual communication systems of the American deaf. Buffalo: Dept. of Anthropology and Linguistics, Univ. of Buffalo, 1960.
64. Sutton 1995 — Sutton V. Lessons in sign writing. Textbook and workbook. La Jolla (CA): Deaf Action Committee for Sign Writing, 1995.
65. Thiessen 2011 — Thiessen S. A grammar of sign writing. MA thesis. Grand Forks (ND): Univ. of North Dakota, 2011.
66. Van der Hulst, Channon 2010 — van der Hulst H., Channon R. Notation systems. Sign languages. (Cambridge Language Surveys.) Brentari D. (ed.). Cambridge: Cambridge Univ. Press, 2010, 151–172.

Система Orphus

Loading...
Up