Image-Text-Matching (ITM) is one of the defacto methods of learning generalized representations from a large corpus in Vision and Language (VL). However, due to the weak association between the web-collected image–text pairs, models fail to show fine-grained understanding of the combined semantics of these modalities. To this end, we propose Hard Negative Captions (HNC): an automatically created
dataset containing foiled hard negative captions for ITM training towards achieving fine-grained cross-modal comprehension in VL. Additionally, we provide a challenging manually-created test set for benchmarking models on a fine-grained cross-modal mismatch with varying levels of compositional complexity. Our results show the effectiveness of training on HNC by improving the models' zero-shot capabilities in detecting mismatches on diagnostic tasks and performing robustly under noisy visual input scenarios. Also, we demonstrate that HNC models yield a comparable or better initialization for fine-tuning. Our code and data are publicly available.

HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities

Affordance knowledge is a fundamental aspect of commonsense knowledge. Recent findings
indicate that world knowledge emerges through large-scale self-supervised pretraining, motivating our exploration of acquiring affordance knowledge from the visual domain. To this end,
we augment an existing instructional video resource to create the new Causal Action–Effect
(CAE) dataset and design two novel pretraining tasks—Masked Action Modeling (MAM)
and Masked Effect Modeling (MEM)—promoting the acquisition of two affordance properties
in models: behavior and entity equivalence, respectively. 
We empirically demonstrate the effectiveness of our proposed methods in learning
affordance properties. Furthermore, we show that a model pretrained on both tasks outperforms a strong image-based visual–linguistic foundation model (FLAVA) as well as pure linguistic models on a zero-shot physical reasoning probing task.

Implicit Affordance Acquisition via Causal Action–Effect Modeling in the Video Domain

Despite the recent success of pretrained lan-
guage models as on-the-fly knowledge sources
for various downstream tasks, they have been
shown to inadequately represent trivial com-
mon facts that vision typically captures. This
limits their application to natural language un-
derstanding tasks that require commonsense
knowledge. We seek to determine the capa-
bility of pretrained visual–linguistic models as
knowledge sources on demand. To this end,
we systematically compare language-only and
visual–linguistic models in a zero-shot com-
monsense question answering inference task.
We find that visual–linguistic models are highly
promising regarding their benefit for text-only
tasks on certain types of commonsense knowl-
edge associated with the visual world. Sur-
prisingly, this knowledge can be activated even
when no visual input is given during inference,
suggesting effective multimodal fusion during
pretraining. However, we also reveal that there
is still a huge scope for improvements towards
better cross-modal reasoning abilities and pre-
training strategies for event understanding.

Are Visual-Linguistic Models Commonsense Knowledge Bases?

**[[SIGNLL 2023 Part 2 Live Session Recording]](https://us06web.zoom.us/rec/share/Fb8BmKGoDzRS1ludb312jim4hY5ue8FRhX4Jp3IobAl9LfvhEYkHTv8OHU69N_FW.WP27Xv4DQ3y46hgv)** 
**[[SIGNLL 2023 Day 2 Live Session Recording]](https://us06web.zoom.us/rec/play/zVY0g0L3khSZ4NML4EjKw8M3TvMHEKl1LRouAtEyJl8WfTT7Qu1yavsyfW3VLuVB-70OPGzmUo2bW-51.lh2VfJJW-iHATvqw?canPlayFromShare=true&from=recording_mg&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Fus06web.zoom.us%2Frec%2Fshare%2FtW1mfqioXRyu4zINRhyliXQvFD9kPUfdDApBgQw6ytIez82oJzj2ISXXEAeK19AL.6RjVwcGUPKUowuEf)** 
**Workshop Organizers:** David Reitter and Jing Jiang
 *Coffee Break 10:30 am* 
*Lunch Break 12:30 pm* 
*Coffee Break 3:30 pm* [**Workshop Website**](https://www.conll.org/2023) 
CoNLL is a yearly conference organized by SIGNLL (ACL's Special Interest Group on Natural Language Learning), focusing on theoretically, cognitively and scientifically motivated approaches to computational linguistics.

W01: The SIGNLL Conference on Computational Natural Language Learning(CoNLL)

workshop paper

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

Session 3A: Multilingual and Multimodal Analysis

technical paper

### Welcome to IJCNLP-AACL 2023!
I am honored and happy to welcome you to IJCNLP-AACL 2023! The conference hosts both the 13th
International Joint Conference on Natural Language Processing (IJCNLP) and the 3rd Conference of
the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL). 
The joint conferences of IJCNLP and AACL have been organized by alternating leadership in the
Asia-Pacific region, in odd years by Asian Federation of Natural Language Processing (AFNLP) and
in even years by AACL. The conference is organized by AFNLP this year. 

Considering the current situation that we are not yet fully over the coronavirus pandemic, the
conference is offered in a hybrid mode, off-line in Nusa Dua, Bali, Indonesia, and on-line with the help
of Underline, during November 1-4, 2023. For those attending off-line, I hope that you will find time to
explore the exciting local area, which remains quite warm and pleasant at this time of the year. For
those attending on-line, I hope that the attendance is not much hampered, sometimes even enhanced
and comfortable, though the on-line arrangements have been made without ample time. 

Message from Jong C. Park, Korea Advanced Institute of Science and Technology (KAIST) 
IJCNLP-AACL 2023 General Chair 
###### [Conference Handbook](https://drive.google.com/file/d/1S2JSXa105xJxIkNBfhYmGH20Om6wARBx/view?usp=sharing)

In order to acces this event page, you need to register [here](http://www.ijcnlp-aacl2023.org/)

IJCNLP-AACL 2023

The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics

Click on the Live Session Recording to view the session

OS12 - Multimodal and Grounded Language Acquisition

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

Hsiu-Yu Yang

3

22

SHORT BIO

Presentations

HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities

Implicit Affordance Acquisition via Causal Action–Effect Modeling in the Video Domain

Are Visual-Linguistic Models Commonsense Knowledge Bases?

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES