United States

Packing and shuffling tokens is a common practice in training auto-regressive language models to prevent overfitting and improve efficiency. Documents are typically concatenated to chunks of maximum sequence length (MSL) and shuffled in chunks of tokens (atom-size chunk), possibly breaking context within documents. An alternative approach is padding, which only includes one document per chunk. To optimize both packing strategies (concatenation vs padding), we explored the optimal atom size for shuffling and compared performance and efficiency. We found that in the most common setup (where average document length is greater than MSL), matching atom size to MSL yields the lowest perplexity, controlling for dataset. Also, padding yields lower final perplexity than concatenation at the cost of lower efficiency. This trade-off informs the choice of shuffling and packing methods in training LMs.

ACL 2026

Optimizing Packing and Shuffling Strategies for Enhanced Performance in Generative Language Models

### Welcome to 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

Welcome to the 2025 meeting of the Nations of the Americas Chapter of the Association for Computational Linguistics! I am proud to help organize the first NAACL conference to carry the new name of our organization, one that emphasizes inclusion for all of the Americas. I am also pleased to welcome you to Albuquerque, New Mexico, a state whose unique blend of cultural influences will make for an excellent backdrop for NAACL 2025, especially with this year’s special theme on NLP in a Multicultural World. 
**[Continue reading...](https://drive.google.com/file/d/1jX-qGhqVSZZCIrAnJaz798pflu5Irrdn/view?usp=sharing)**

*- Colin Cherry, Google, NAACL 2025 General Chair* 

[![](https://assets.underline.io/markdown_image/1/image/b087f8a4dc5816d6e1a6514e59c59ac3.png)](https://drive.google.com/file/d/1T96GzPqObXrMln2BMByCSXTSizjTg69P/view?usp=sharing)

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

To access NAACL 2025 event page you are required to register. Please follow [**this link**](https://2025.naacl.org/registration/registration/) to register. Access will depend on your registration type.

Please register!

NAACL 2025

Welcome to 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

technical paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Large language models excel at technical problem solving in English but struggle when questions are posed in Bangla. While translation offers a practical solution, existing Bangla-English systems frequently mistranslate specialized terminology, altering problem semantics and degrading downstream performance. We present BanglaSTEM, a dataset of 5,000 Bangla-English sentence pairs covering computer science, mathematics, physics, chemistry, and biology. Our pipeline extracts matching passages from official bilingual curriculum textbooks using OCR, then uses LLMs to align sentences and mark technical terms. These aligned examples serve as few-shot prompts for generating over 12,000 new translation pairs from LLMs, avoiding copyright issues. Human evaluators then select the best 5,000 pairs that correctly preserve technical terminology. We also test a term-weighted BLEU metric that gives higher weight to technical words, since standard metrics treat terminology errors and common word errors equally. We show that our weighted metric correlates better with downstream accuracy in code generation and math solving, while standard BLEU gives high scores even for wrong translations. The full implementation, dataset, and model will be made publicly available.

BanglaSTEM: A Parallel Corpus and Term-Weighted Evaluation for Technical Bangla-English Translation

Aspect-Term Sentiment Analysis (ATSA) aims to predict sentiment polarity for specific aspect terms, a task complicated by conflicting sentiments and limited context in short texts. Existing graph-based approaches, including recent multi-graph methods, rely on predefined pairwise structures to capture different linguistic views. However, these approaches impose a fixed interaction granularity and depend on multiple predefined graphs, limiting their ability to adapt to sentence-specific sentiment composition and capture higher-order relationships. We propose HyperATSA, a dynamic hypergraph framework that constructs instance-specific aspect-opinion groupings directly from contextual token representations, without relying on external parsers. Hyperedges are induced via hierarchical agglomerative clustering over token embeddings, with a novel acceleration-based cutoff that identifies natural group boundaries per sentence. Experiments on Lap14, Rest14, and MAMS demonstrate consistent improvements over strong graph-based baselines, with controlled comparisons across BERT and RoBERTa backbones confirming that the gains stem from the hypergraph mechanism rather than encoder strength.

From Graphs to Hypergraphs: Enhancing Aspect-Term Sentiment Analysis via Multi-Level Relational Modeling

Faithfulness in NLP explanations is typically assessed via input perturbations, implicitly treating explanations as causal drivers of decisions, an assumption ill-suited to human judgment. In educational assessment, explanations are better understood as structured justifications accompanying holistic decisions. We propose an explainability-first framework that treats human explanations as structured artifacts independent of prediction. By modeling explanation structure explicitly, our approach enables diagnostic analyses of emphasis, consistency, and dependence. We find that scores remain invariant under evidence removal, while explanatory structure persists in tutor-specific and rubric dependent ways, revealing a mismatch between prevailing faithfulness metrics and human reasoning. This provides a model-free foundation for auditing explanations and designing NLP systems whose explanations align with human evaluative practice.

Faithfulness Beyond Plausibility: Auditing Human Explanations in Educational Assessment

Although existing frameworks for large language model (LLM) inference on CPUs are mature, they fail to fully exploit the computational potential of many-core CPU platforms. Many-core CPUs are widely deployed in web servers and high-end networking devices, and are typically organized into multiple NUMA nodes that group cores and memory. Current frameworks largely overlook the substantial overhead of cross-NUMA memory access, limiting inference scalability and intelligence enabling on such platforms. To address this limitation, we build ArcLight, a lightweight LLM inference architecture designed from the ground up for many-core CPUs. ArcLight integrates efficient memory management and thread scheduling, and introduces finely controlled tensor parallelism to mitigate the cross-node memory access wall. Experimental results show that ArcLight significantly surpasses the performance ceiling of mainstream frameworks, achieving up to 46% higher inference throughput. Moreover, ArcLight maintains compatibility with arbitrary CPU devices. ArcLight is publicly available at https://github.com/OpenBMB/ArcLight.

ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs

Public procurement data is legally open, yet practically locked inside thousands of unstructured PDFs and inconsistent portal metadata. pAtChWoRK starts with these fragmented, unstructured sources then leverages a hybrid pipeline (traditional NLP with LLM-based technologies) to restructure this information into a navigable knowledge base. Specifically, pAtChWoRK corrects manual classification errors, extracts complex unstructured fields such as award and solvency criteria and tenders' objectives, and assists users in easily navigating the tender landscape. This unified process enables more effective handling of the transparency bottlenecks that hinder competition and oversight in public administration. A user study with practitioners across different procurement

pAtChWoRK: Patching the Pieces of Public Procurement Documents

Vision-Language Models (VLMs) have become increasingly important in AI applications, yet interpretability tools for these models lag behind those available for text-only language models. While libraries like TransformerLens have enabled significant progress in understanding language models, existing tools for VLMs are limited to basic activation probing and saving. We introduce Spectra, a library specifically designed for mechanistic interpretability of VLMs that provides unified abstractions for activation patching, attention pattern analysis, and meta-functions across diverse VLM architectures. Built on HuggingFace's Transformers, our library handles architecture-specific complexities through per-checkpoint configurations while maintaining a simple, high-level interface. We demonstrate the library's capabilities by performing interpretability experiments on a counting task, showing how researchers can easily perform experiments that were previously cumbersome to do. The library currently supports Qwen2.5-VL, Qwen3-VL, LLaVA 1.5 and SmolVLM, with a design that facilitates extension to additional architectures. The library can be found at \url{github.com/clemneo/vlm-spectra}.

Spectra: A Mechanistic Interpretability Library for Vision-Language Models

We present OpenGlass, an open-source, privacy-preserving system for low-latency multimodal assistance in real-world deployments, with a primary focus on blind and low-vision users. Deploying large multimodal models for assistive use faces a practical tension between capability and interaction: cloud or server inference can provide strong visual understanding but often incurs multi-second delays and network jitter, while wearable devices can continuously sense the environment but have limited compute and power budgets. OpenGlass resolves this gap with a split local-first design: a lightweight glasses-side sensing unit captures visual context, while a nearby consumer-grade device performs multimodal inference and streams concise spoken feedback over local Wi-Fi or cellular connections, avoiding reliance on remote cloud services. We introduce a system-level evaluation suite covering response quality, end-to-end latency from query completion to audio playback, and safety-first abstention with auditable logs. OpenGlass supports three representative interaction patterns: mobile scene guidance for obstacle avoidance, user initiated interruption of speech output, and multi-turn question answering grounded in the current scene. In real deployment on consumer-grade hardware, OpenGlass achieves sub-2s system-level end-to-end latency from query completion to first audio playback.

OpenGlass: A Sensing-Computing Split Architecture for Local MLLM-Driven Real-Time Visual Assistance

Large language models (LLMs) have shown remarkable potential in scientific hypothesis discovery. However, existing approaches treat divergent exploratory ideation and convergent fine-grained refinement as isolated tasks, lacking a principled mechanism for human guidance. We present MOOSE-Copilot, the first unified framework to bridge this abstraction gap through a formalized human–AI interaction protocol. Our system empowers scientists to actively steer the automated process via three explicit guiding signals: initial blueprints, inter-stage routing, and regenerative feedback. Quantitative evaluations demonstrate that injecting these structured expert signals significantly outperforms purely autonomous baselines, establishing a theoretical upper bound for hypothesis quality through iterative refinement. Furthermore, we operationalize this paradigm into an accessible web-based interface featuring interactive tree visualization. By reducing the cognitively demanding discovery process into manageable subtasks of conceptual selection and critique, MOOSE-Copilot enables interdisciplinary researchers to effectively orchestrate and accelerate end-to-end scientific breakthroughs.

MOOSE-Copilot: A Web-Based Interactive Assistant for Unified Exploratory- and Fine-Grained Scientific Hypothesis Discovery

Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architectural modifications or optimizer adjustments. However, these approaches lack systematic optimization of weight patterns during training. Weight pattern refers to the distribution and relative magnitudes of weight parameters in a neural network. To address this issue, we propose a Weight Scaling method called WISCA to enhance training efficiency and model quality by strategically improving neural network weight patterns—without changing network structures. By rescaling weights while preserving model outputs, WISCA indirectly optimizes the model’s training trajectory. Experiments demonstrate that WISCA significantly improves convergence quality (measured by generalization capability and loss reduction), particularly in LLMs with Grouped Query Attention (GQA) architectures and LoRA fine-tuning tasks. Empirical results show 5.6% average improvement on zero-shot validation tasks and 2.12% average reduction in training perplexity across multiple architectures.

WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling

Role-playing models (RPMs) are widely used in real-world applications but underperform when deployed in the wild. This degradation can be attributed to distribution shifts, including user, character, and dialogue compositional shifts. Existing methods like LLM-as-a-judge fall short in providing a fine-grained diagnosis of how these shifts affect RPM generalization, and thus there lack formal frameworks to characterize RPM generalization behaviors. To bridge these gaps, we introduce an information-theoretic metric, named reasoning-based effective mutual information difference (R-EMID), to measure RPM performance degradation in an interpretable way. We also derive an upper bound on R-EMID to predict the worst-case generalization performance of RPMs and theoretically reveal how various shifts contribute to the RPM performance degradation. Moreover, we propose a co-evolving reinforcement learning framework to adaptively model the connection among user, character, and dialogue context and thus enhance the estimation of dialogue response generation probability, which is critical for calculating R-EMID. Finally, we evaluate the generalization performance of various RPMs using R-EMID, finding that user shift poses the highest risk among all shifts and reinforcement learning is the most effective approach for enhancing RPM generalization. Code and data are available at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/RPM-Generalization.

Downloads

Next from ACL 2026

BanglaSTEM: A Parallel Corpus and Term-Weighted Evaluation for Technical Bangla-English Translation

Similar lecture

Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

BanglaSTEM: A Parallel Corpus and Term-Weighted Evaluation for Technical Bangla-English Translation

Similar lecture

Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads