Theory of Mind (ToM), the ability to understand people’s mental states, is an essential ingredient for developing machines with human-level social intelligence. Recent machine learning models, particularly large language models, seem to show some aspects of ToM understanding. However, existing ToM benchmarks use unimodal datasets – either video or text. Human ToM, on the other hand, is more than video or text understanding. People can flexibly reason about another person’s mind based on conceptual representations (e.g., goals, beliefs, plans) extracted from any available data. To address this, we introduce a multimodal Theory of Mind question answering (MMToM-QA) benchmark. MMToM-QA comprehensively evaluates machine ToM both on multimodal data and on different kinds of unimodal data about a person’s activity in a household environment. To engineer multimodal ToM capacity, we propose a novel method, BIP-ALM (Bayesian Inverse Planning Accelerated by Language Models). BIP-ALM extracts unified representations from multimodal data and utilizes language models for scalable Bayesian inverse planning. We conducted a systematic comparison of human performance, BIP-ALM, and state-of-the-art models, including GPT-4. The experiments demonstrate that large language models and large multimodal models still lack robust ToM capacity. BIP-ALM, on the other hand, shows promising results, by leveraging the power of both model-based mental inference and language models.

MMToM-QA: Multimodal Theory of Mind Question Answering

Recent work has considered whether large language models (LLMs) can function as planners: given a task, generate a plan. We investigate whether LLMs can serve as generalized planners: given a domain and training tasks, generate a program that efficiently produces plans for other tasks in the domain. In particular, we consider PDDL domains and use GPT-4 to synthesize Python programs. We also consider (1) Chain-of-Thought (CoT) summarization, where the LLM is prompted to summarize the domain and propose a strategy in words before synthesizing the program; and (2) automated debugging, where the program is validated with respect to the training tasks, and in case of errors, the LLM is re-prompted with four types of feedback. We evaluate this approach in seven PDDL domains and compare it to four ablations and four baselines. Overall, we find that GPT-4 is a surprisingly powerful generalized planner. We also conclude that automated debugging is very important, that CoT summarization has non-uniform impact, that GPT-4 is far superior to GPT-3.5, and that just two training tasks are often sufficient for strong generalization.

Generalized Planning in PDDL Domains with Pretrained Large Language Models | VIDEO

Multi-agent interactions, such as communication, teaching, and bluffing, often rely on higher-order social inference, i.e., understanding how others infer oneself. Such intricate reasoning can be effectively modeled through nested multi-agent reasoning. Nonetheless, the computational complexity escalates exponentially with each level of reasoning, posing a significant challenge. However, humans effortlessly perform complex social inferences as part of their daily lives. To bridge the gap between human-like inference capabilities and computational limitations, we propose a novel approach: leveraging neural networks to amortize high-order social inference, thereby expediting nested multi-agent reasoning. We evaluate our method in two challenging multi-agent interaction domains. The experimental results demonstrate that our method is computationally efficient while exhibiting minimal degradation in accuracy.

Neural Amortized Inference for Nested Multi-Agent Reasoning | VIDEO

Logical reasoning, i.e., deductively inferring the truth value of a conclusion from a set of premises, is an important task for artificial intelligence with wide potential impacts on science, mathematics, and society. While many prompting-based strategies have been proposed to enable Large Language Models (LLMs) to do such reasoning more effectively, they still appear unsatisfactory, often failing in subtle and unpredictable ways. In this work, we investigate the validity of instead reformulating such tasks as modular neurosymbolic programming, which we call LINC: Logical Inference via Neurosymbolic Computation. In LINC, the LLM acts as a semantic parser, translating premises and conclusions from natural language to expressions in first-order logic. These expressions are then offloaded to an external theorem prover, which symbolically performs deductive inference. Leveraging this approach, we observe significant performance gains on FOLIO and a balanced subset of ProofWriter for three different models in nearly all experimental conditions we evaluate. On ProofWriter, augmenting the comparatively small open-source StarCoder+ (15.5B parameters) with LINC even outperforms GPT-3.5 and GPT-4 with Chain-of-Thought (CoT) prompting by an absolute 38% and 10%, respectively. When used with GPT-4, LINC scores 26% higher than CoT on ProofWriter while performing comparatively on FOLIO. Further analysis reveals that although both methods on average succeed roughly equally often on this dataset, they exhibit distinct and complementary failure modes. We thus provide promising evidence for how logical reasoning over natural language can be tackled through jointly leveraging LLMs alongside symbolic provers. All corresponding code is publicly available.

LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers

Confidence is an important concept in cognitive science, as it integrates seamlessly with our beliefs, goals, and decisions. However, the dynamics of how beliefs about hypotheses (represented by a Bayesian posterior probability distribution) give rise to explicitly reported confidence judgments about predictions (which are typically measured on various numerical scales) in the human mind is not well understood. In this work, we report findings on this question from a concept learning and confidence judgment experiment. We compare how leading theories in the present literature stands against our data and propose some new model-based measurements and future directions for predicting humans' confidence judgments.

Towards a model of confidence judgements

The relationship between the language a speaker conveys and the meaning a listener recovers is often probabilistic and sensitive to context. Numerous strategies attempt to estimate such a mapping, often leveraging recursive Bayesian models of communication. In parallel, large language models (LLMs) have been increasingly applied to semantic parsing applications, tasked with inferring logical representations from NL. While existing LLM explorations have been largely restricted to literal language use, in this work, we evaluate the capacity of LLMs to make inferences over pragmatic utterances. Specifically, we explore the case of threshold estimation on gradable adjectives, e.g., “strong”, contextually conditioned on a strength prior, then extending to composition with qualification, negation, polarity inversion, and class comparison. We find that LLMs can derive context-grounded, human-like inferences over interpretations for complex utterances such as “very strong for a beginner”, but struggle with composing such amortized inferences with negation or polarity inversion.

LLMs are pretty strong pragmatic reasoners: Investigating amortized inferences about gradable adjectives in large language models

Many models of intuitive physical reasoning posit some kind of mental simulation mechanism, yet everyday environments frequently contain far more objects than people could plausibly represent with their limited cognitive capacity. What determines which objects are actually included in our representations? We asked participants to predict how a ball will bounce through a complex field of obstacles, and probed working memory for objects in the scene that were more and less likely to be relevant to the ball’s trajectory. We evaluate different accounts of relevance and find that successful object memory is best predicted by how frequently a ball’s trajectory is expected to contact that object under a probabilistic simulation model. This suggests that people construct representations for
mental simulation efficiently and dynamically, on the fly, by adding objects “just in time”: only when they are expected to become relevant for the next stage of simulation.

"Just In Time" Representations for Mental Simulation in Intuitive Physics

If we want to know whether a stack of blocks will fall, we might form our judgment by using mental simulation or simple heuristics. How do we know which strategy to use? Here, we test whether people’s strategy selection is “resource rational,” balancing their utilities and costs. We had participants judge how a balance beam will fall – a task that relies on both simulation and heuristics – and imposed visuospatial cognitive load to increase the cost of mental simulation, expecting the use of simulation to then decrease and the use of heuristics to increase. When cognitive load varied between participants, higher load led to less simulation. However, when load varied across trials for an individual participant, they instead used simulation at about the same rate as high-load-only participants. Thus strategy selection is sensitive to cognitive costs but it is also sticky, not accounting for rapidly fluctuating costs.

Strategy choice for physical reasoning is (partially) sensitive to cognitive costs

The relationship between communicated language and intended meaning is often probabilistic and sensitive to context. Numerous strategies attempt to estimate such a mapping, often leveraging recursive Bayesian models of communication. In parallel, large language models (LLMs) have been increasingly applied to semantic parsing applications, tasked with inferring logical representations from natural language. While existing LLM explorations have been largely restricted to literal language use, in this work, we evaluate the capacity of LLMs to infer the meanings of pragmatic utterances. Specifically, we explore the case of threshold estimation on the gradable adjective "strong", contextually conditioned on a strength prior, then extended to composition with qualification, negation, polarity inversion, and class comparison. We find that LLMs can derive context-grounded, human-like distributions over the interpretations of several complex pragmatic utterances, yet struggle composing with negation. These results inform the inferential capacity of statistical language models, and their use in pragmatic and semantic parsing applications.

Evaluating statistical language models as pragmatic reasoners

This paper investigates whether adding data of typologically closer languages improves the performance of transformer-based models for three different downstream tasks, namely Part-of-Speech tagging, Named Entity Recognition, and Sentiment Analysis, compared to a monolingual and plain multilingual language model. For the presented pilot study, we performed experiments for the use case of Slovene, a low(er)-resourced language belonging to the Slavic language group. The experiments were carried out in a controlled setting, where a monolingual model for Slovene was compared to combined language models containing Slovene, trained with the same amount of Slovene data. The experimental results show that adding typologically closer languages indeed improves the performance of the Slovene language model, and even succeeds in outperforming the large multilingual XLM-RoBERTa model for NER and PoS-tagging. We also reveal that, contrary to intuition, distant or unrelated languages also combine admirably with Slovene, often out-performing XLM-R as well. All the bilingual models used in the experiments are publicly available.

Too Many Cooks Spoil the Model: Are Bilingual Models for Slovene Better than a Large Multilingual Model?

Computer vision is one of the great AI success stories. Yet we are still far from having machine systems that can reliably and robustly see everything a human being sees in an image or in the real world. Despite rapid advances in self-supervised visual and multimodal representation learning, we are also far from having systems that can learn to see as richly as a human does, from so little data, or that can learn new visual concepts or adapt their representations as quickly as a human does. And even today’s remarkable generative image synthesis systems imagine the world in a very different and fundamentally less flexible way than human beings do. How can we close these gaps? I will describe several core insights from the study of human vision and visual cognitive development that run counter to the dominant trends in today's computer vision and machine learning world, but that can motivate and guide an alternative approach to building practical machine vision systems.

Technically, this approach rests on advances in differentiable and probabilistic programming: hybrids of neural, symbolic and probabilistic modeling and inference that can be more robust, more flexible and more data-efficient than purely neural approaches to learning to see. New probabilistic programming platforms offer to make these approaches scalable as well. Conceptually, this approach draws on classic proposals for understanding vision as ‘inverse graphics’, ‘analysis by synthesis’ or ‘inference to the best explanation’, and the notion that at least some high-level architecture for scene representation is built into the brain by evolution rather than learned from experience, reflecting invariant properties of the physical world. Learning then enables, enriches and extends these built-in representations; it does not create them from scratch. I will show a few examples of recent machine vision successes based on these ideas, from our group and others. But the hardest problems are still very open. I will highlight some ‘Grand Challenge’ tasks for building machines that learn to see like people: problems that far outstrip the abilities of any current system, and that I hope can inspire the next steps towards progress for computer vision researchers regardless of which approach they favor.

Learning to See the Human Way

We present a framework for learning useful subgoals that support efficient long-term planning to achieve novel goals.  At the core of our framework is a collection of rational subgoals (RSGs), which are essentially binary classifiers over the environmental states. RSGs can be learned from weakly-annotated data, in the form of unsegmented demonstration trajectories, paired with abstract task descriptions, which are composed of terms that are initially unknown to the agent (e.g., collect-wood then craft-boat then go-across-river). Our framework also discovers dependencies between RSGs, such as predicting "collect-wood" is a helpful subgoal for task "craft-boat". Given a goal description, the learned subgoals and the derived dependencies  facilitate off-the-shelf planning algorithms, such as A*and RRT, by setting helpful subgoals as waypoints to the planner. It significantly improves performance-time efficiency.

Learning Rational Subgoals from Demonstrations and Instructions

Humans and animals engage in rich social interactions. It is often theorized that a relatively small number of basic social interactions give rise to the full range of behavior observed. But no computational theory explaining how social interactions combine together has been proposed before. We do so here. We take a model, the Social MDP, which is able to express a range of social interactions, and extend it to represent linear combinations of social interactions. Practically for robotics applications, such models are now able to not just express that an agent should help another agent, but to express goal-centric social interactions. Perhaps an agent is helping someone get dressed, but preventing them from falling, and is happy to exchange stories in the meantime. How an agent responds socially, should depend on what it thinks the other agent is doing at that point in time. To encode this notion, we take linear combinations of social interactions as defined in Social MDPs, and compute the weights on those combinations on the fly depending on the estimated goals of other agents. This new model, the Linear Social MDP, enables zero-shot reasoning about complex social interactions, provides a mathematical basis for the long-standing intuition that social interactions should compose, and leads to interesting new behaviors that we validate using human observers. Complex social interactions are part of the future of intelligent agents, and having principled mathematical models built on a foundation like MDPs will make it possible to bring social interactions to every robotic application.

Zero-shot linear combinations of grounded social interactions with Linear Social MDPs

Efficient planning in continuous state and action spaces is fundamentally hard, even when the transition model is deterministic and known. One way to alleviate this challenge is to perform bilevel planning with abstractions, where a high-level search for abstract plans is used to guide planning in the original transition space. Previous work has shown that when state abstractions in the form of symbolic predicates are hand-designed, operators and samplers for bilevel planning can be learned from demonstrations. In this work, we propose an algorithm for learning predicates from demonstrations, eliminating the need for manually specified state abstractions. Our key idea is to learn predicates by optimizing a surrogate objective that is tractable but faithful to our real efficient-planning objective. We use this surrogate objective in a hill-climbing search over predicate sets drawn from a grammar. Experimentally, we show across four robotic planning environments that our learned abstractions are able to quickly solve held-out tasks, outperforming six baselines.

Predicate Invention for Bilevel Planning

Generalized planning accelerates classical planning by finding an algorithm-like policy that solves multiple instances of a task. A generalized plan can be learned from a few training examples and applied to an entire domain of problems. Generalized planning approaches perform well in discrete AI planning problems that involve large numbers of objects and extended action sequences to achieve the goal. In this paper, we propose an algorithm for learning features, abstractions, and generalized plans for continuous robotic task and motion planning (TAMP) and examine the unique difficulties that arise when forced to consider geometric and physical constraints as a part of the generalized plan. Additionally, we show that these simple generalized plans learned from only a handful of examples can be used to improve the search efficiency of TAMP solvers.

Discovering State and Action Abstractions for Generalized Task and Motion Planning

Recent advances in generative adversarial networks (GANs) have led to remarkable achievements in face image synthesis. While methods that use style-based GANs can generate strikingly photorealistic face images, it is often difficult to control the characteristics of the generated faces in a meaningful and disentangled way. Prior approaches aim to achieve such semantic control and disentanglement within the latent space of a previously trained GAN. In contrast, we propose a framework that a priori models physical attributes of the face such as 3D shape, albedo, pose, and lighting explicitly, thus providing disentanglement by design. Our method, MOST-GAN, integrates the expressive power and photorealism of style-based GANs with the physical disentanglement and flexibility of nonlinear 3D morphable models, which we couple with a state-of-the-art 2D hair manipulation network. MOST-GAN achieves photorealistic manipulation of portrait images with fully disentangled 3D control over their physical attributes, enabling extreme manipulation of lighting, facial expression, and pose variations up to full profile view.

MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation

This poster session includes Main Conference posters, TACL and Demos.

In-Person Poster Session 4

poster

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

Main Track - PRS

technical paper

We are pleased to announce the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24), which will be held in Vancouver, British Columbia at the Vancouver Convention Centre – West Building from 20-27 February, 2024.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-24 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

We expect for AAAI-24 to be an in-person conference – one author of all accepted papers will be expected to present work in person unless there are exceptional circumstances that prevent this.

In order to access the AAAI-24 event page you need to register [here](https://aaai.org/aaai-conference/registration/)

AAAI 2024

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. 

Main Track - Computer Management Systems

**Are you attending this poster session virtually?** 
In-person printed posters are available for in-person attendees only.
 
**Are you attending this poster session in person?** 
Hybrid posters are displayed in the East Foyer.

PS4 (In-person) Posters, Demo, Industry, Findings

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

Poster Session 3 

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference. 
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
 [**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
 

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

CogSci 2023

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

With recent scaling of large pre-trained Transformer language models (LLMs), the scope of feasible NLP tasks has broadened. Significant recent work has focused on tasks that require some kind of natural language reasoning. A trajectory in question answering has led us from extraction-oriented datasets like SQuAD to “multi-hop” reasoning datasets like HotpotQA and StrategyQA. Although LLMs have shown remarkable performance on most NLP tasks, it is often unclear why their answers follow from what they know. To address this gap, a new class of explanation techniques has emerged which play an integral part in structuring the reasoning necessary to solve these datasets. For example, the chain-of-thought paradigm leverages explanations as vehicles for LLMs to mimic human reasoning processes. Entailment trees offer a way to ground multi-step reasoning in a collection of verifiable steps. Frameworks like SayCan bridge high-level planning in language and with low-level action trajectories. As a result, we see a confluence of methods blending explainable machine learning/NLP, classical AI (especially theorem proving), and cognitive science (how do humans structure explanations?). This workshop aims to bring together a diverse set of perspectives from these different traditions and attempt to establish common ground for how these various kinds of explanation structures can tackle a broad class of reasoning problems in natural language and beyond. For more details visit our workshop website: https://nl-reasoning-workshop.github.io/
#### [Visit GatherTown](https://app.gather.town/app/5OscLswEEuYf2Psb/ACL2023)

W8: The 1st Workshop on Natural Language Reasoning and Structured Explanations

workshop paper

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

ACL 2023

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

**Organizers:** 
Jakub Piskorski, Polish Academy of Sciences, Poland
Bogdan Babych, Heidelberg University, Germany
Michał Marcińczuk, Wroclaw University of Science and Technology, Samurai Labs, Poland
Preslav Nakov, Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates
Maciej Ogrodniczuk, Polish Academy of Sciences, Poland
Senja Pollak, Jožef Stefan Institute, Slovenia
Pavel Přibáň, University of West Bohemia, Plzeň, Czech Republic
Josef Steinberger, University of West Bohemia, Plzeň, Czech Republic
Roman Yangarber, University of Helsinki, Finland
Piotr Rybak, Polish Academy of Sciences, Poland For more details, please visit our [workshop website](http://bsnlp.cs.helsinki.fi/index.html))

SlavNLP-2023: The 9th Biennial Workshop on Slavic NLP

### Welcome to the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL) 
Continuing its mission of expanding and involving the science community of all European countries, EACL had selected the Ukrainian community for the 16th EACL, which took place online due to the COVID pandemic. Unfortunately, the ongoing war made the organisation in Kyiv impossible. Considering the importance of physical interaction among researchers, especially after the restrictions imposed by the COVID pandemic, in addition to an online mode, the conference will be held in Dubrovnik, Croatia, from 2 to 6 of May, 2023. The original aim of strengthening the connection with the Ukrainian community will not change as our program will feature a dedicated session and a workshop to highlight work on Ukrainian language technologies. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

You need to register for the conference in order to access this site. Please visit https://2023.eacl.org/registration for more information.

EACL 2023

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

**Presenter:** [Josh Tenenbaum](https://underline.io/speakers/98677-josh-tenenbaum) (Massachusetts Institute of Technology, USA) 
**Talk Title:** Learning to see the human way 
 **Abstract:** Computer vision is one of the great AI success stories. Yet we are still far from having machine systems that can reliably and robustly see everything a human being sees in an image or in the real world. Despite rapid advances in self-supervised visual and multimodal representation learning, we are also far from having systems that can learn to see as richly as a human does, from so little data, or that can learn new visual concepts or adapt their representations as quickly as a human does. And even today’s remarkable generative image synthesis systems imagine the world in a very different and fundamentally less flexible way than human beings do. How can we close these gaps? I will describe several core insights from the study of human vision and visual cognitive development that run counter to the dominant trends in today&#39;s computer vision and machine learning world, but that can motivate and guide an alternative approach to building practical machine vision systems.

Technically, this approach rests on advances in differentiable and probabilistic programming: hybrids of neural, symbolic and probabilistic modeling and inference that can be more robust, more flexible and more data-efficient than purely neural approaches to learning to see. New probabilistic programming platforms offer to make these approaches scalable as well. Conceptually, this approach draws on classic proposals for understanding vision as ‘inverse graphics’, ‘analysis by synthesis’ or ‘inference to the best explanation’, and the notion that at least some high-level architecture for scene representation is built into the brain by evolution rather than learned from experience, reflecting invariant properties of the physical world. Learning then enables, enriches and extends these built-in representations; it does not create them from scratch. I will show a few examples of recent machine vision successes based on these ideas, from our group and others. But the hardest problems are still very open. I will highlight some ‘Grand Challenge’ tasks for building machines that learn to see like people: problems that far outstrip the abilities of any current system, and that I hope can inspire the next steps towards progress for computer vision researchers regardless of which approach they favor.

AAAI 2023 Invited talk Josh Tenenbaum

keynote

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

PRS: Planning/Scheduling and Learning

Posters & Demos III

PRS: Planning/Scheduling and Learning / ML: Online Learning & Reinforcement Learning

Main_Track

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

Josh Tenenbaum

36

245

SHORT BIO

OTHER AFFILIATIONS

Presentations