CWG : Aixiu An

Aixiu An (LLF)
Representation of Constituents in Neural Language Models: coordination Phrase as a Case Study

Neural language models have achieved state-of-the-art performances on many NLP tasks, and recently have been shown to learn a number of hierarchically-sensitive syntactic dependencies between individual words. However, equally important for language processing is the ability to combine words into phrasal constituents, and use constituent-level features to drive downstream expectations. Here we investigate neural models' ability to represent constituent-level features, using coordinated Noun Phrases as a case study. We assess whether different neural language models trained on English and French represent phrase-level number and gender features, and use those features to drive downstream expectations. We find that all models tested are able to represent the basic facts Coordinated NP / Verb number agreement, but have less success with gender agreement. Models trained on large corpora perform best, and no advantage to models trained using explicit syntactic supervision. Our results indicate that neural models can compute constituent-level feature information on the fly and leverage fine-grained information about local syntactic context to drive humanlike behavior.