Singapore

AI safety and alignment evaluations (benchmarks) have become de facto governance tools in the rapidly evolving landscape of artificial intelligence. However, these critical instruments themselves lack systematic governance oversight, raising the central question: &quot;Who evaluates the evaluators?&quot; Through a systematic mapping of 298 papers and in-depth thematic analysis of 50 core studies, this paper identifies fundamental governance challenges in current AI evaluation practices. The bibliometric analysis reveals explosive growth in this field since 2023, with persistent gaps in transparency, accountability, and institutional independence. This position paper synthesizes current knowledge on evaluation governance and proposes a future agenda grounded in three core principles: participatory design with adversarial scrutiny, radical transparency through &quot;Evaluation Cards,&quot; and institutional pluralism to ensure independent oversight. The paper argues for a paradigm shift from &quot;evaluating AI&quot; to &quot;governing evaluations,&quot; offering actionable recommendations for researchers, policymakers, and practitioners to build a trustworthy evaluation ecosystem.

AAAI 2026

Who Evaluates the Evaluators? Governance Challenges in AI Safety and Alignment Evaluations: A Review and Future Agenda

ai governance

ai safety

bibliometrics

systematic review

evaluation

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀

Please log in to explore this event.

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

NAML 2026

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



AI safety and alignment evaluations (benchmarks) have become de facto governance tools in the rapidly evolving landscape of artificial intelligence. However, these critical instruments themselves lack systematic governance oversight, raising the central question: "Who evaluates the evaluators?" Through a systematic mapping of 298 papers and in-depth thematic analysis of 50 core studies, this paper identifies fundamental governance challenges in current AI evaluation practices. The bibliometric analysis reveals explosive growth in this field since 2023, with persistent gaps in transparency, accountability, and institutional independence. This position paper synthesizes current knowledge on evaluation governance and proposes a future agenda grounded in three core principles: participatory design with adversarial scrutiny, radical transparency through "Evaluation Cards," and institutional pluralism to ensure independent oversight. The paper argues for a paradigm shift from "evaluating AI" to "governing evaluations," offering actionable recommendations for researchers, policymakers, and practitioners to build a trustworthy evaluation ecosystem.

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Offensive Yet Efficient: Semantic Summarization via Obscene Lexicon

Stop Harm Before It Starts: iCRAFT for Agentic LLM Governance

Gaurav Chawla, Caleb DeLeeuw, Vanessa Dietze, Aniket Sharma

The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind

Panatchakorn Anantaprayoon, Nataliia Babina, Jad Tarifi, Nima Asgharbeygi

Dynamic Alignment for Collective Agency: Toward a Scalable Self-Improving Framework for Open-Ended LLM Alignment

Navigating Healthcare AI Governance: Regulatory Divergence and the Harmonization Challenge Across Medical Domains

We propose Poincaré Gradient Descent (PGD), a first-order optimization method on the Poincaré ball. PGD replaces the exponential map with a projection-based retraction that is first-order equivalent while preserving geodesic structure, thereby reducing per-iteration computational cost. We establish convergence guarantees for smooth geodesically convex and strongly convex objectives, showing that PGD matches the iteration complexity of Riemannian Gradient Descent (RGD). A Möbius isometric initialization and step-size equivalence analysis further ensure stable and efficient updates. Numerical experiments confirm that PGD achieves the same theoretical convergence rate as RGD but runs significantly faster in practice, offering a simple and effective framework for scalable optimization in hyperbolic space.

Poincaré Gradient Descent: A Projection-Based Riemannian Method on the Poincaré Ball

Spiking neural networks (SNNs) have been proposed as an (energy-)efficient alternative to conventional artificial neural networks. However, the aspired benefits have not yet been realized in practice. To gain a better understanding of why this gap persists, we theoretically study both discrete-time and continuous-time models of leaky integrate-and-fire neurons.
In the discrete-time model, which is a widely used framework due to its amenability to conventional deep learning software and hardware approaches, we analyze the impact of explicit recurrent connections on the network size required to approximate continuously differentiable functions. We contrast this view by investigating the computational efficiency of digital systems that simulate spike-based computations in the continuous-time model. It turns out that even in well-behaved settings, the computational complexity of this task may grow super-polynomially in the prescribed accuracy.
Thereby, we exemplarily highlight the intricacies of realizing potential strengths in the biological context, namely recurrent connections and computational efficiency, of spike-based computations on digital systems.

How to realize efficient Spiking Neural Networks?

Bayesian neural networks (BNNs) require scalable sampling algorithms to approximate posterior distributions over parameters. Existing stochastic gradient Markov Chain Monte Carlo (SGMCMC) methods are highly sensitive to the choice of stepsize and adaptive variants such as pSGLD typically fail to sample the correct invariant measure without addition of a costly divergence correction term. In this work, we build on the recently proposed `SamAdams' framework for timestep adaptation (Leimkuhler, Lohmann, and Whalley 2025), introducing an adaptive scheme: SA-SGLD, which employs time rescaling to modulate the stepsize according to a monitored quantity (typically the local gradient norm). SA-SGLD can automatically shrink stepsizes in regions of high curvature and expand them in flatter regions, improving both stability and mixing without introducing bias. We show that our method can achieve more accurate posterior sampling than SGLD on high-curvature 2D toy examples and in image classification with BNNs using sharp priors.

Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks

We study the fundamental expressivity limits of transformer models. We formalize the notion of accessible sequences—those that a transformer can produce for some prompt—and characterize how accessibility depends on prompt length and model precision. By partitioning the embedding space via the decoder readout into next-token argmax regions and extending transformers to a mean-field map on probability measures, we derive theoretical upper bounds on the number and length of accessible output sequences. We prove that (i) the maximal length of accessible sequences grows linearly with the prompt length, and (ii) beyond a critical threshold, the proportion of reachable sequences decays exponentially with sequence length. These bounds hold even with unbounded context and computation time, linking the expressivity limits of transformers to the geometry of their embedding space and the finiteness of their representational precision. Experiments using a “cramming” procedure confirm both the linear scaling and the post-threshold exponential decay.

The Expressivity Limits of Transformers

Combinatorial and topological structures, such as graphs, simplicial complexes, and cell complexes, form the foundation of geometric and topological deep learning (GDL and TDL) architectures. These models aggregate signals over such domains, integrate local features, and generate representations for diverse real-world applications. However, the distribution and diffusion behavior of GDL and TDL features during training remains an open and underexplored problem. Motivated by this gap, we introduce a cellular sheaf theoretic framework for modeling and analyzing the local consistency and harmonicity of node features and edge weights in graph-based architectures. By tracking local feature alignments and agreements through sheaf structures, the framework offers a topological perspective on feature diffusion and aggregation. Furthermore, a multiscale extension inspired by topological data analysis (TDA) is proposed to capture hierarchical feature interactions in graph models. This approach enables a joint characterization of GDL and TDL architectures based on their underlying geometric and topological structures and the learned signals defined on them, providing insights for future studies on conventional tasks such as node classification, substructure detection, and community detection.

Downloads

Next from AAAI 2026

Offensive Yet Efficient: Semantic Summarization via Obscene Lexicon

Similar lecture

AI Governance at Sea

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES