Existing evaluations of entity linking systems often say little about how the system is going to perform for a particular application. There are two fundamental reasons for this.
One is that many evaluations only use aggregate measures (like precision, recall, and F1 score), without a detailed error analysis or a closer look at the results.
The other is that all of the widely used benchmarks have strong biases and artifacts, in particular:
a strong focus on named entities, an unclear or missing specification of what else counts as an entity mention,
poor handling of ambiguities, and an over- or underrepresentation of certain kinds of entities.

We provide a more meaningful and fair in-depth evaluation of a variety of existing end-to-end entity linkers.
We characterize their strengths and weaknesses and also report on reproducibility aspects.
The detailed results of our evaluation can be inspected under https://elevant.cs.uni-freiburg.de/emnlp2023.
Our evaluation is based on several widely used benchmarks, which exhibit the problems mentioned above to various degrees,
as well as on two new benchmarks, which address the problems mentioned above.
The new benchmarks can be found under https://github.com/ad-freiburg/fair-entity-linking-benchmarks.

A Fair and In-Depth Evaluation of Existing End-to-End Entity Linking Systems

**Are you attending this poster session virtually?** 
Connect to presenters by clicking on the link that you can find in the 'Abstracts' tab of the posters.
 
**Are you attending this poster sessions in person?** 
Hybrid posters are displayed in the East Foyer.

PS3 (Hybrid) Posters, Demo, Industry

poster

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

Hannah Bast

1

Presentations

A Fair and In-Depth Evaluation of Existing End-to-End Entity Linking Systems

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES