Grammar-Based Inductive Learning (GBIL) for Sign-Spotting in Continuous Sign Language Videos

In order to identify an Isolated Sign Word (ISW) in Continuous Sign Language Videos (CSLV) aka Sign-Spotting, we propose a Grammar-Based Inductive Learning (GBIL) framework utilizing a Grammar-Based Dictionary (GBD) that comprises of pre-defined syntactic structure of tokens for handshape, location, and movement related to every Isolated Sign Word. Through this GBIL we identify the start and end frames that match the grammar related to a particular ISW and detect the signed word in a sentence-level continuous sign language video. We observe that GBIL can improve cross-domain performance of sign spotting by integrating a grammar logic based inference on top of deep learning architectures.