In human-computer conversations, extracting entities such as names, street addresses and email addresses from speech is a challenging task. In this paper, we study the impact of fine-tuning pre-trained speech encoders on extracting spoken entities in human-readable form directly from speech without the need for text transcription. We illustrate that such a direct approach optimizes the encoder to transcribe only the entity relevant portions of speech ignoring the superfluous portions such as carrier phrases, or spell name entities. In the context of dialog from an enterprise virtual agent, we demonstrate that the 1-step approach outperforms the typical 2-step approach which first generates lexical transcriptions followed by text-based entity extraction for identifying spoken entities.

E2E Spoken Entity Extraction for Virtual Agents | VIDEO

We tackle Aspect Term Extraction (ATE), a task of automatically extracting aspect terms from sentences. The current Pretrained Language Model (PLM) based extractors have achieved significant improvements. They primarily benefit from context-aware encoding. However, a considerable number of sentences in ATE corpora contain uninformative or low-quality contexts. Such sentences frequently act as "troublemakers" during test. In this study, we explore the context-oriented quality improvement method. Specifically, we propose to automatically rewrite the sentences from the perspectives of virtual experts with different roles, such as a "chef" in the restaurant domain. On this basis, we perform ATE over the paraphrased sentences during test, using the well-trained extractors without any change. In the experiments, we leverage ChatGPT to determine virtual experts in the considered domains, and induce ChatGPT to generate paraphrases conditioned on the roles of virtual experts. We experiment on the benchmark SemEval datasets, including Laptop-domain L14 and Restaurant-domain R14-16. The experimental results show that our approach effectively recalls the inconspicuous aspect terms like "al di la", although it reduces the precision. In addition, it is proven that our approach can be substantially improved by redundancy elimination and multi-role voting. More importantly, our approach can be used to expand the predictions obtained on the original sentences. This yields state-of-the-art performance (i.e., F1-scores of 86.2%, 89.3%, 77.7%, 82.7% on L14 and R14-16) without retraining or fine-tuning the baseline extractors.

Smart “Chef”: Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction

Ensuring real-time properties on current heterogeneous multiprocessor systems on a chip is a challenging task. Furthermore, online artificial intelligent applications –which are routinely deployed on such chips– pose increasing pressure on the memory subsystem that becomes a source of unpredictability. Although techniques have been proposed to restore independent access to memory for concurrently executing virtual machines (VM), providing predictable inter-VM communication remains challenging. In this work, we tackle the problem of predictably transferring data between virtual machines and virtualized hardware resources on multiprocessor systems on chips under consideration of memory interference. We design a “broker-based” real-time communication framework for otherwise isolated virtual machines, provide a virtio-based reference implementation on top of the Jailhouse hypervisor, assess its overheads for FreeRTOS virtual machines, and formally analyze its communication flow schedulability under consideration of the implementation overheads. Furthermore, we define a methodology to assess the maximum DRAM memory saturation empirically, evaluate the framework’s performance and compare it with the theoretical schedulability.

A Real-Time virtio-based Framework for Predictable Inter-VM Communication

From LATAM, to Pakistan, to Nigeria to China, panelists tackle issues including barriers and solutions for mobile internet adoption, impact investing, partnership, social enterprise and national policies for eHealth. Speakers offer guidance on how to scale technology for end users, solve for manpower issues, connection flexibilities and offer uniform thinking for local successes.

Innovations in Virtual Healthcare: The Evolving Competitive Landscape Telehealth and Virtual Care – An International Perspective

Virtual reality (VR) technology continues to mature at a rapid clip, creating potential for its use as a means of administering "hands-on" physics lab curriculum in hybrid or remote learning contexts. This talk draws from experiences administering VR-enabled labs to classes of 120 students in a hybrid model and 37 students in a remote model to inform answers to key questions around use of VR in remote labs: What advantages does VR offer over traditional "flat" simulations? Do students recognize "hands-on" work done in an immersive 3D environment as being "real" science? How is collaboration between distant students affected by use of VR? What is the current and near-future feasibility of VR technology use in remote instruction? We argue that VR labs have a niche in the remote physics lab instructor's toolbox, and outline current and anticipated applications of VR to remote physics lab instruction.

Virtual Reality Enabling Remote Collaborative Physics Labs

The demand for intelligent virtual advisors in our rapidly advancing world is rising and, consequently, the need for understanding the reasoning process to answer why a particular piece of advice is provided to the user is directly increasing. Personalized explanation is regarded as a reliable way to improve the user's understanding and trust in the virtual advisor. So far, cognitive explainable agents utilize reason explanation by referring to their own mental state (beliefs and goals) to explain their own behaviour. However, when the explainable agent plays the role of a virtual advisor and recommends a behaviour for the human to perform, it is best to refer to the user's mental state, rather than the agent’s mental state, to form a reason explanation. In this paper, we are developing an explainable virtual advisor (XVA) that communicates with the user to elicit the user's beliefs and goals and then tailors its advice and explains it according to the user's mental state. We tested the proposed XVA with university students where the XVA provides tips to reduce the students' study stress. We measured the impact of receiving three different patterns of tailored explanations (belief-based, goal-based, and belief&goal-based explanation) in terms of the students' intentions to change their behaviours. The results showed that the intention to change is not only related to the explanation pattern but also to the user context, the relationship built with the agent, the type of behaviour recommended and the user’s current intention to do the behaviour.

Underline digital video library

E2E Spoken Entity Extraction for Virtual Agents | VIDEO

Smart “Chef”: Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction

A Real-Time virtio-based Framework for Predictable Inter-VM Communication

Innovations in Virtual Healthcare: The Evolving Competitive Landscape Telehealth and Virtual Care – An International Perspective

Virtual Reality Enabling Remote Collaborative Physics Labs

Reason Explanation for Encouraging Behaviour Change Intention

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES