Canada

Decision Trees (DTs) are commonly used for many machine learning tasks due to their high degree of interpretability. However, learning a DT from data is a difficult optimization problem, as it is non-convex and non-differentiable. Therefore, common approaches learn DTs using a greedy growth algorithm that minimizes the impurity locally at each internal node. Unfortunately, this greedy procedure can lead to inaccurate trees. In this paper, we present a novel approach for learning hard, axis-aligned DTs with gradient descent. The proposed method uses backpropagation with a straight-through operator on a dense DT representation, to jointly optimize all tree parameters. Our approach outperforms existing methods on binary classification benchmarks and achieves competitive results for multi-class tasks. The implementation is available under: https://github.com/s-marton/GradTree

AAAI 2024

GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent

gradient descent

decision trees

tabular data

technical paper

We are pleased to announce the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24), which will be held in Vancouver, British Columbia at the Vancouver Convention Centre – West Building from 20-27 February, 2024.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-24 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

We expect for AAAI-24 to be an in-person conference – one author of all accepted papers will be expected to present work in person unless there are exceptional circumstances that prevent this.<br><br><br><br>

In order to access the AAAI-24 event page you need to register [here](https://aaai.org/aaai-conference/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. 

We study the problem of agent selection in causal strategic learning under multiple decision makers and address two key challenges that come with it. 
Firstly, while much of prior work focuses on studying a fixed pool of agents that remains static regardless of their evaluations, we consider the impact of selection procedure by which agents are not only evaluated, but also selected.
When each decision maker unilaterally selects agents by maximising their own utility, we show that the optimal selection rule is a trade-off between selecting the best agents and providing incentives to maximise the agents' improvement. 
Furthermore, this optimal selection rule relies on incorrect predictions of agents' outcomes. 
Hence, we study the conditions under which a decision maker's optimal selection rule will not lead to deterioration of agents' outcome nor cause unjust reduction in agents' selection chance. 
To that end, we provide an analytical form of the optimal selection rule and a mechanism to retrieve the causal parameters from observational data, under certain assumptions on agents' behaviour. 
Secondly, when there are multiple decision makers, the interference between selection rules introduces another source of biases in estimating the underlying causal parameters. 
To address this problem, we provide a cooperative protocol which all decision makers must collectively adopt to recover the true causal parameters. 
Lastly, we complement our theoretical results with simulation studies.
Our results highlight not only the importance of causal modeling as a strategy to mitigate the effect of gaming, as suggested by previous work, but also the need of a benevolent regulator to enable it.

Causal Strategic Learning with Competitive Selection

The maximum vertex-weighted clique problem (MVWCP) and the maximum edge-weighted clique problem (MEWCP) are two natural extensions of the fundamental maximum clique problem. 
In this paper, we systematically study MEWCP and make the following major contributions:
(1) We show that MEWCP is NP-hard even when the minimum degree of the graph is n-2, in contrast to MVWCP which is polynomial-time solvable when the minimum degree of the graph is at least n-3. This result distinguishes the complexity of the two problems for the first time.
(2) To address MEWCP, we develop an efficient branch-and-bound algorithm called MEWCat with both practical and theoretical performance guarantees. In practice, MEWCat utilizes a new upper bound tighter than existing ones, which allows for more efficient pruning of branches. In theory, we prove a running-time bound of O*(1.4423^n) for MEWCat, which breaks the trivial bound of O*(2^n) in the research line of practical exact MEWCP solvers for the first time.
(3) Empirically, we evaluate the performance of MEWCat on various benchmark instances. The experiments demonstrate that MEWCat outperforms state-of-the-art exact solvers significantly. For instance, on 16 DIMACS graphs that the state-of-the-art solver BBEWC fails to solve within 7200 seconds, MEWCat solves all of them with an average time of less than 1000 seconds. On real-world graphs, MEWCat achieves an average speedup of over 36x.

A Fast Exact Solver with Theoretical Analysis for the Maximum Edge-Weighted Clique Problem

As machine learning models, specifically neural networks, are becoming increasingly popular, there are concerns regarding their trustworthiness, specially in safety-critical applications, e.g. actions of an autonomous vehicle must be safe. There are approaches that can train neural networks where such domain requirements are enforced as constraints, but they either cannot guarantee that the constraint will be satisfied by all possible predictions (even on unseen data) or they are limited in the type of constraints that can be enforced. In this paper, we present an approach to train neural networks which can enforce a wide variety of constraints and guarantee that the constraint is satisfied by all possible predictions. The approach builds on earlier work where learning linear models is formulated as a constraint satisfaction problem (CSP). To make this idea applicable to neural networks, two crucial new elements are added: constraint propagation over the network layers, and weight updates based on a mix of gradient descent and CSP solving. Evaluation on various machine learning tasks demonstrates that our approach is flexible enough to enforce a wide variety of domain constraints and is able to guarantee them in neural networks.

DeepSaDe: Learning Neural Networks That Guarantee Domain Constraint Satisfaction

Boolean Networks (BNs) are widely used as a modeling formalism in several domains, notably systems biology and computer science. A fundamental problem in BN analysis is the enumeration of trap spaces, which are hypercubes in the state space that cannot be escaped once entered. Several methods have been proposed for enumerating trap spaces, however they often suffer from scalability and efficiency issues, particularly for large and complex models. To our knowledge, the most efficient and recent methods for the trap space enumeration all rely on Answer Set Programming (ASP), which has been widely applied to the analysis of BNs. Motivated by these considerations, our work proposes a new method for enumerating trap spaces in BNs using ASP. We evaluate the method on a mix of 250+ real-world and 400+ randomly generated BNs, showing that it enables analysis of models beyond the capabilities of existing tools (namely pyboolnet, mpbn, trappist, and trapmvn).

Scalable Enumeration of Trap Spaces in Boolean Networks via Answer Set Programming

Virtual Reality (VR) has emerged due to advancements in hardware and computer graphics. During the pandemic, conferences and exhibitions leveraging VR have gained attention. However, large-scale VR conferences, face a significant problem not yet studied in the
literature -- displaying too many irrelevant users on the screen which may negatively impact the user experience. To address this issue, we formulate a new research problem, {\it \underline{S}ocial-Aware \underline{V}R Conference \underline{G}roup \underline{D}isplay Configuration (SVGD)}. Accordingly, we design the {\it \underline{S}ocial Utility-Aware \underline{V}R \underline{C}onference Group Formation (SVC)} algorithm, which is a $2$-approximation algorithm to SVGD. We perform a user study on SVGD, and the results show that the configuration identified by SVC receives the highest satisfaction. Experiments on real metaverse datasets show that the proposed SVC outperforms 11 baselines by 75\% in terms of solution quality.

Social-Aware Group Display Configuration in VR Conference

Answer Set Programming (ASP) has emerged as a promising
paradigm in knowledge representation and automated reason-
ing owing to its ability to model hard combinatorial problems
from diverse domains in a natural way. Building on advances
in propositional SAT solving, the past two decades have wit-
nessed the emergence of well-engineered systems for solv-
ing the answer set satisfiability problem, i.e., finding mod-
els or answer sets for a given answer set program. In re-
cent years, there has been growing interest in problems be-
yond satisfiability, such as model counting, in the context of
ASP. Akin to the early days of propositional model count-
ing, state-of-the-art exact answer set counters do not scale
well beyond small instances. Exact ASP counters struggle
with handling larger input formulas. The primary contribu-
tion of this paper is a new ASP counting framework, called
sharpASP, which counts answer sets avoiding larger input
formulas. This relies on an alternative way of defining answer
sets that allows lifting of key techniques developed in the con-
text of propositional model counting. Our extensive empirical
analysis over 1470 benchmarks demonstrates significant per-
formance gain over current state-of-the-art exact answer set
counters. Specifically, by using sharpASP, we were able to
solve 1062 benchmarks with PAR2 score of 3082 whereas
using prior state-of-the-art, we could only solve 895 bench-
marks with PAR2 score of 4205, all other experimental con-
ditions being the same.

Exact ASP Counting with Compact Encodings

This paper is a contribution to neural network semantics, a foundational framework for neuro-symbolic AI. The key insight of this theory is that logical operators can be mapped to operators on neural network states. In this paper, we do this for a neural network \emph{learning} operator. We map a dynamic operator $\Hebbop{\varphi}$ to iterated Hebbian learning, a simple learning policy that updates a neural network by repeatedly applying Hebb's learning rule until the net reaches a fixed-point. Our main result is that we can "translate away" $\Hebbop{\varphi}$-formulas via reduction axioms. This means that completeness for the logic of iterated Hebbian learning follows from completeness of the base logic. These reduction axioms also provide (1) a human-interpretable description of iterated Hebbian learning as a kind of plausibility upgrade, and (2) an approach to building neural networks with guarantees on what they can learn.

What Do Hebbian Learners Learn? Reduction Axioms for Unstable Hebbian Learning

Graph neural networks (GNNs) have been demonstrated to perform well in graph representation learning, but always lacking of generalization capability when tackling out-of-distribution (OOD) data. Graph invariant learning methods, backed by the invariance principle among defined multiple environments, have shown effectiveness in dealing this issue. However, existing methods heavily rely on well predefined or accurately generated environment partitions, which are hard to be obtained in practice, leading to sub-optimal OOD generalization performances.
In this paper, we propose a novel graph invariant learning method based on invariant and variant patterns co-mixup strategy, which is capable of jointly generating mixed multiple environments and capturing invariant patterns from the mixed graph data. Specifically, we first adopt a subgraph extractor to identify invariant subgraphs. Subsequently, we design one novel co-mixup strategy, i.e., jointly conducting environment mixup and invariant mixup. For the environment mixup, we mix the variant environment-related subgraphs so as to generate sufficiently diverse multiple environments, which is important to guarantee the quality of the graph invariant learning. For the invariant mixup, we mix the invariant subgraphs, further encouraging to capture invariant patterns behind graphs while getting rid of spurious correlations for OOD generalization. We demonstrate that the proposed environment mixup and invariant mixup can mutually promote each other. Extensive experiments on both synthetic and real-world datasets demonstrate that our method significantly outperforms state-of-the-art under various distribution shifts.

Graph Invariant Learning with Subgraph Co-mixup for Out-of-Distribution Generalization

This paper presents BeliefFlow, a novel framework for representing how logical beliefs spread among interacting agents within a network. In a Belief Flow Network (BFN), agents communicate asynchronously. The agents' beliefs are represented using epistemic states, which encompass their current beliefs and conditional beliefs guiding future changes. When communication occurs between two connected agents, the receiving agent changes its epistemic state using an improvement operator, a well-known type of rational iterated belief change operator that generalizes belief revision operators. We show that BFNs satisfy appealing properties, leading to two significant outcomes. First, in any BFN with strong network connectivity, the beliefs of all agents converge towards a global consensus. Second, within any BFN, we show that it is possible to compute an optimal strategy for influencing the global beliefs. This strategy, which involves controlling the beliefs of a least number of agents through bribery, can be identified from the topology of the network and can be computed in polynomial time.

BeliefFlow: A Framework for Logic-Based Belief Diffusion via Iterated Belief Change | VIDEO

Anomaly detection (AD), also termed outlier detection, is a pivotal technique in machine learning (ML) and artificial intelligence (AI). In today’s interconnected landscape, AD’s importance is evident across diverse sectors, from national security to healthcare and security. For instance, NASA uses AD algorithms for spacecraft failure detection, the U.S. Navy applies it for enhanced space system autonomy, the University of Pittsburgh Medical Center uses AD for early patient prioritization, and companies like Google and Amazon employ AD for service disruption identification. Fundamentally, AD is invaluable in identifying rare yet significant data points in large datasets. My research focuses on using algorithmic and system methods to automate and accelerate AD. The goal is to develop automated, scalable, and reproducible detection algorithms and systems to support high-stakes research and applications. To sum up, my research has produced successful results: (1) Automated algorithms; (2) Scalable AD systems and (3) Reproducible AD benchmarks, which will be covered in the proposed presentation. Please see the attached cover letter and presentation schedule for details.

GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent

Downloads

Next from AAAI 2024

Causal Strategic Learning with Competitive Selection

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2024

Causal Strategic Learning with Competitive Selection

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads