A mixture-of-experts model for learning multi-facet entity embeddings

Alshaikh, Rana; Bouraoui, Zied; Jeawak, Shelan; Schockaert, Steven

Abstract

Various methods have already been proposed for learning entity embeddings from text descriptions. Such embeddings are commonly used for inferring properties of entities, for recommendation and entity-oriented search, and for injecting background knowledge into neural architec-tures, among others. Entity embeddings essentially serve as a compact encoding of a similarity relation, but similarity is an inherently multi-faceted notion. By representing entities as single vectors, existing methods leave it to downstream applications to identify these different facets, and to select the most relevant ones. In this paper, we propose a model that instead learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. We use a mixture-of-experts formulation to jointly learn these facet-specific embeddings. The individual entity embeddings are learned using a variant of the GloVe model, which has the advantage that we can easily identify which properties are modelled well in which of the learned embeddings. This is exploited by an associated gating network, which uses pre-trained word vectors to encourage the properties that are modelled by a given embedding to be semantically coherent, i.e. to encourage each of the individual embeddings to capture a meaningful facet.

Citation

Alshaikh, R., Bouraoui, Z., Jeawak, S., & Schockaert, S. (2020). A mixture-of-experts model for learning multi-facet entity embeddings. In Proceedings of the 28th International Conference on Computational Linguistics (5124-5135)

• Conference Name: The 28th International Conference on Computational Linguistics
• Conference Location: Barcelona, Spain (Online)
• Start Date: Dec 8, 2020
• End Date: Dec 13, 2020
• Acceptance Date: Sep 30, 2020
• Publication Date: Dec 8, 2020
• Deposit Date: May 17, 2021
• Publicly Available Date: May 18, 2021
• Pages: 5124-5135
• Book Title: Proceedings of the 28th International Conference on Computational Linguistics
• Public URL: https://uwe-repository.worktribe.com/output/7336984
• Publisher URL: https://www.aclweb.org/anthology/2020.coling-main.0/