Large Language Models (LLMs) are proficient in natural language processing tasks, but their deployment is often restricted by extensive parameter sizes and computational demands. This paper focuses on post-training quantization (PTQ) in LLMs, specifically 4-bit weight and 8-bit activation (W4A8) quantization, to enhance computational efficiency—a topic less explored compared to weight-only quantization. We present two innovative techniques: activation-quantization-aware scaling (AQAS) and sequence-length-aware calibration (SLAC) to enhance PTQ by considering the combined effects on weights and activations and aligning calibration sequence lengths to target tasks. Moreover, we introduce dINT, a hybrid data format combining integer and denormal representations, to address the underflow issue in W4A8 quantization, where small values are rounded to zero. Through rigorous evaluations of LLMs, including OPT and LLaMA, we demonstrate that our techniques significantly boost task accuracies to levels comparable with full-precision models. By developing arithmetic units compatible with dINT, we further confirm that our methods yield a 2$\times$ hardware efficiency improvement compared to 8-bit integer MAC unit.

Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization

Automated Essay Scoring (AES) aims to automatically assess the quality of essays. Automation enables large-scale assessment, improvements in consistency, reliability, and standardization. Those characteristics are of particular relevance in the context of language certification exams. However, a major bottleneck in the development of AES systems is the availability of corpora, which, unfortunately, are scarce, especially for languages other than English. In this paper, we aim to foster the development of AES for French by providing the TCFLE-8 corpus, a corpus of 6.5k essays collected in the context of the \textit{Test de Connaissance du Français} (TCF - French Knowledge Test) certification exam. We report the strict quality procedure that led to the scoring of each essay by at least two raters according to the CEFR levels and to the creation of a balanced corpus. In addition, we describe how linguistic properties of the essays relate to the learners' proficiency in TCFLE-8. We also advance the state-of-the-art performance for the AES task in French by experimenting with two strong baselines (i.e. RoBERTa and feature-based). Finally, we discuss the challenges of AES using TCFLE-8.

TCFLE-8: a Corpus of Learner Written Productions for French as a Foreign Language and its Application to Automated Essay Scoring | VIDEO

Knowledge graph completion (KGC) aims to predict unseen edges in knowledge graphs (KGs), resulting in the discovery of new facts. A new class of methods have been proposed to tackle this problem by aggregating path information. These methods have shown tremendous ability in the task of KGC. However they are plagued by efficiency issues. Though there are a few recent attempts to address this through learnable path pruning, they often sacrifice the performance to gain efficiency. In this work, we identify two intrinsic limitations of these methods that affect the efficiency and representation quality. To address the limitations, we introduce a new method, TAGNet, which is able to efficiently propagate information. This is achieved by only aggregating paths in a fixed window for each source-target pair. We demonstrate that the complexity of TAGNet is independent of the number of layers. Extensive experiments demonstrate that TAGNet can cut down on the number of propagated messages by as much as $90$% while achieving competitive performance on multiple KG datasets.

Distance-Based Propagation for Efficient Knowledge Graph Reasoning | VIDEO

As the world tries to shift to cleaner energy, it is important to keep track of how each country manages to meet their energy needs while reducing their impacts on the planet. Denmark is one of the leading countries in the world in wind power penetration, reporting that 47% of their electricity consumption came from wind in 2019 (Reuters). My research looks at how they have achieved this statistic, what exactly it means, and how it may be misleading for other countries or societies seeking to emulate Denmark’s high wind integration. The intermittency of wind energy is a big problem because the intermittency destabilizes the energy grid when combined with reliable, dispatchable energy sources like fossil fuels. My conclusions regarding how Denmark has been able to integrate wind energy and maintain high grid stability indicate that Denmark’s success is quite unique because of a) geographical proximity to the naturally high-wind areas of the North and Baltic seas, b) geographical proximity and strong import/export relationships to energy-intensive neighbors like Germany, which produces a lot of energy for its own domestic needs, and neighbors like Norway and Sweden, who possess vast amounts of hydropower reserves; c) Denmark consumes relatively little energy compared to its neighbors, and d) Denmark has enjoyed enough financial wealth and political stability to invest in diversified interconnectors to its neighbors as well as grow its own wind industry. All of these factors play important roles in how Denmark has been able to integrate wind into its electric grid, but perhaps the most important factor is the robust import/export system, which acts as a stabilizing “functional battery” for Denmark’s electricity system. More quantitative research is needed to establish a clearer idea of wind energy’s role in Denmark’s electric grid and understand how that energy is distributed.

Analyzing the Outlook for Renewable Energy: Denmark Case Study

Magnetocaloric effect is a new nonconventional method that can be used to provide cooling power with several benefits compared to conventional gas-compression based cooling. Heusler alloys are promising candidates for micro-magnetic cooling applications thank their desirable behaviour. Preparation of Heulser alloys in a form of glass-coated microwires by the Taylor-Ulitovsky technique allows easy and fast production of microwires with protective glass coating and many additional advantages such as scalability of the size of cooling material. Previous works show the possibility of tuning the structural and magnetic transformation temperature of glass coated Heusler microwires by changing the chemical composition and valence electron concentration ratio e/a. This is one of the most favourable properties of glass coated Heusler microwires. Combining structural and magnetic transformation may enhance the amplitude of the magnetocaloric effect when both transformations overlap, the evaluation of their contribution may become complicated. Therefore, we would like to present the possible analyses which can provide information about the magnetocaloric effect or the transformation temperatures including both structural and magnetic transitions. Direct measurements of magnetocaloric effect, indirect measurement of magnetocaloric effect, resistivity measurements, Arrott plot method, universal curves, method of critical exponent n, Kouvel-Fisher method with Arrott-Noakes plots and many others can be used to study the magnetocaloric effect and transformations on a presented group of alloys. This research was supported by Slovak Grant Agency VEGA 1/0053/19, Slovak Grant Agency grant number APVV-16-0079.

Magnetocaloric Effect Study on Heusler Based Glass-Coated Microwires

Magnetic skyrmions are localized objects with a non-trivial topology. With their small sizes and stable magnetic properties, skyrmions are of great interest in particular due to their potential applications in future-generation magnetic storage devices. Multilayer ferromagnetic thin-film materials hosting room temperature (RT) skyrmions stabilized by interfacial Dzyaloshinskii-Moriya interactions in [1,2] allow a high tunability in terms of skyrmion size and density, making them very promising platforms for investigating functional skyrmions for future applications. However, several obstacles remain before implementing ferromagnetic skyrmions into applications. One of these problems is that the dipolar interactions in these materials makes it difficult to stabilize small skyrmions without an external field [3,4,5]. Another major difficulty concerns the skyrmion Hall effect which deflects the skyrmions to the device edges [6,7]. Both of these problems could potentially be solved by having skymions in antiferromagnetic (AF) materials, where the dipolar fields are minimized by having the two subsystems aligned antiparallel to each other, and the skyrmion Hall effect is compensated. This has been achieved by Legrand et al. in Ref. [5], where RT skyrmions at zero field were generated in a synthetic AF multilayer system and observed by magnetic force microscopy (MFM). Following the same general approach as in Ref. [5], we construct a system that can host high-density skyrmions at zero external field using interlayer exchange coupling and having Ir/Fe/Co/Pt multilayers as starting blocks. These skyrmion-generating layers are coupled with different bias layers that act as an external magnetic field. We will show magnetic characterization and high-resolution MFM imaging results of the impact of various bias layers and coupling strengths on the overall system properties.

Room Temperature Skyrmions at Zero Field in Ir/Fe/Co/Pt-based Systems with Antiferromagnetic Coupling

Room temperature magnetic skyrmions have the potential of enabling compact and energetically-efficient storage such as racetrack memories, albeit with two fundamental challenges: 1) Not being able to restrict their motion along straight paths, leading to skyrmions being lost at the device edge; one possible solution is the use of antiferromagnetically-coupled skyrmions [1]; and 2) Not having stable inter-skyrmion distances, leading to fluctuating distances among bits; a potential solution was revealed with the experimental observation of coexisting skyrmions and chiral bobbers [2], opening the possibility that a chain of binary data bits could be encoded by the two states. However, these observations were in B20-type crystalline materials and at low temperatures. In our previous work [3], we developed a ferro/ferri/ferromagnetic (FM/FI/FM) trilayer system that can host two distinct skyrmions at room temperature and can serve as a solution to the second challenge. The two phases represent a tubular skyrmion (running through the entire trilayer) and an incomplete skyrmion (existing in the FM layers only), as revealed from magnetic force microscopy (MFM) data and micromagnetic simulations. Such a two-skyrmion phase system exists for specific magnetic parameters of the individual sublayers. By exploring this parameter space, in this new study we reveal that the densities of the two skyrmions can be tuned via different routes: either by changing the magnetic properties of the FI or those of the FM. More specifically, we investigate the trilayer system when changing the thickness of the FM multilayers by using a combination of MFM, vibrating sample magnetometry and Brillouin light scattering experiments together with micromagnetic simulations. We show how the competition between different energies allows for the stabilization of either incomplete-only skyrmions, combination of both or mainly tubular skyrmions (Figure 1). Having a good understanding of the interplay between different magnetic properties can lead to better-tailored systems and possibly bring skyrmions a step closer to applications.

![](https://assets.underline.io/uploads/markdown_image/1/image/5ed95347749731b8a51d650ec9a59f11.jpg)

Fig. 1: Sample schematics and MFM images of three FM/FI/FM samples with varying Fe-layer thickness.

Tuning the Density of Incomplete and Tubular Skyrmions in Ferro/ferri/ferromagnetic Trilayers

Neuromorphic computing is attracting the attention of researchers for its usefulness in ultra-low-power computing applications1. Among the several magnetic and non-magnetic candidates to achieve low-power neuron and synaptic devices, the domain wall magnetic tunnel junctions (DW-MTJ) are the most promising contenders for energy efficiency. However, to realize the synaptic functions in neuromorphic devices, one must achieve multi-resistance states. The pinning sites can aid in achieving the required multi-resistance states2. Therefore, we propose meander DW devices, where segments of DW-MTJ wires join at an offset, which alters the energy landscape locally and helps in achieving deterministic DW motion. To study the proposed concept, we optimized the DW-MTJ stack with the perpendicular magnetic anisotropy to achieve a higher tunnel magnetoresistance (TMR) of 169% and a high RA (resistance-area product) of 241 Wmm2 suitable for DW devices with a large area. Subsequently, we measured the magnetic hysteresis (M-H) loops of the thin films to observe the layer-wise switching along with magnetic field variation (fig. 1 (a)). We have also measured the minor loop (fig. 1 (b)) and found that the coercivity of the free layer is ~21 Oe, which is suitable for energy-efficient DW motion. Next, we optimized the fabrication process for 3-terminal synaptic devices (fig. 1 (c)). We fabricated a sequence of devices with a DW-MTJ wire width w = 10 mm and corresponding offset (d) values ranging from 40% (of w) to 70% in the steps of 10%. Further, we fabricated three types of devices for every offset; (i) constant offset, increase in offset by (ii) 5%, and (iii) 10% over successive pinning sites. Finally, we performed field and current-driven DW motion experiments and witnessed the synaptic functions. Here we present the results of field-driven DW motion in fig. 1 (d). The stack details, process insights, and experimental findings will be presented in detail during the conference.

![](https://assets.underline.io/uploads/markdown_image/1/image/5676d1e7a6f58421533aa347a7748345.jpg)

Figure. (a) M-H loop of the full stack, (b) minor loop illustrating the switching of the free layer, (d) micrograph of the 3-terminal synaptic device, and (e) read-out TMR signal of field driven-DW motion.

Terminal Domain Wall Device Based Synapses for Neuromorphic Computing

Spin transfer torque[1] is an important phenomenon in order to develop of non-volatile magnetic memory. Using this spin transfer torque principle, it was predicted[2] and experimentally confirmed[3] that a voltage could be generated in the magnetic nanostructure. This effect is called spin motive force and is expected for next-generation power sources. It is not clear how the Faraday electromotive force[4] can be separated from the spin motive force, and there are few studies reported and the spin motive force in µV class. It is very small. To greatly increase the value, we fabricated Gd21(Fe75Co25)79 magnetic wire and succeeded in generating a spin motive force by domain wall motion. We prepared the following film stacks using rf and dc magnetron sputtering: Gd21 (Fe75Co25)79 (20)/SiN (10), the films are grown at the room temperature. The magnetic wires (3 µm wide and 125 µm long) and Hall crosses were micro-fabricated by an Electron-beam lithography (EBL) system and a lift-off method. Next, we prepared two types of samples, a sample with injection of domain wall and a sample without injection of domain wall. A magnetic field was applied to the sample, and the generated voltage was observed by an oscilloscope through a 100x amplifier. Spin motive force is determined by subtracting the induced electromotive force generated in the sample without injection of domain wall. Figure 1 (a) shows the spin motive force measurement system. Figure 1 (b) shows the value of the spin electromotive force by subtracting the induced electromotive force. As a result, a huge spin motive force of 12 mV was obtained. This is financially supported by KAKENHI 918023,918055,918043.

![](https://assets.underline.io/uploads/markdown_image/1/image/36414d6f67eeb09c3efa72deacf02c73.jpg)

![](https://assets.underline.io/uploads/markdown_image/1/image/f9d644bd049228bbbc961dd9398e36c1.jpg)

Milli volt class spin motive force due to domain wall motion in GdFeCo magnetic wire

Underline digital video library

Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization

TCFLE-8: a Corpus of Learner Written Productions for French as a Foreign Language and its Application to Automated Essay Scoring | VIDEO

Distance-Based Propagation for Efficient Knowledge Graph Reasoning | VIDEO

Analyzing the Outlook for Renewable Energy: Denmark Case Study

Magnetocaloric Effect Study on Heusler Based Glass-Coated Microwires

Room Temperature Skyrmions at Zero Field in Ir/Fe/Co/Pt-based Systems with Antiferromagnetic Coupling

Tuning the Density of Incomplete and Tubular Skyrmions in Ferro/ferri/ferromagnetic Trilayers

Terminal Domain Wall Device Based Synapses for Neuromorphic Computing

Milli volt class spin motive force due to domain wall motion in GdFeCo magnetic wire

Molecular Dynamics MOOC 11.1.1. Summary/Introduction

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES