How Distributed Are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks

TitreHow Distributed Are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks
Publication TypeArticle dans des actes
Année de la conférence2022
AuthorsLi, Bingzhi, Guillaume Wisniewski, and Benoît Crabbé
Nom de la conférenceProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Pagination501–507
Date de publicationmay
PublisherAssociation for Computational Linguistics
Conference LocationDublin, Ireland
Abstract

This work addresses the question of the localization of syntactic information encoded in the transformers representations. We tackle this question from two perspectives, considering the object-past participle agreement in French, by identifying, first, in which part of the sentence and, second, in which part of the representation the syntactic information is encoded. The results of our experiments, using probing, causal analysis and feature selection method, show that syntactic information is encoded locally in a way consistent with the French grammar.