For the last decade, convolutional neural networks (CNNs) have vastly superseded their predecessors in nearly all vision tasks in artificial intelligence, including object recognition. However, despite abundant advancements, they continue to pale in comparison to biological vision. This chasm has prompted the development of biologically-inspired models that have attempted to mimic the human visual system, primarily at a neural level, which is evaluated using standard dataset benchmarks. However, more work is needed to understand how these models perceive the visual world. This article proposes a state-of-the-art procedure that generates a new metric, Psychophysical-Score, which is grounded in visual psychophysics and is capable of reliably estimating perceptual responses across numerous models --- representing a large range in complexity and biological inspiration. We perform the procedure on twelve models that vary in degree of biological inspiration and complexity, we compare the results against the aggregated results of 2,390 Amazon Mechanical Turk workers who together provided $\sim2.7$ million perceptual responses. Each model's Psychophysical-Score is compared against the state-of-the-art neural activity-based metric, Brain-Score. Our study indicates that models with a high correlation to human perceptual behavior also have a high correlation with the corresponding neural activity.

Psychophysical-Score: A Behavioral Measure for Assessing the Biological Plausibility of Visual Recognition Models

Quantitative models of behavior are a fundamental tool in cognitive science. Typically, models are hand-crafted to implement specific cognitive mechanisms. Such "classic" models are interpretable by design, but may provide poor fit to experimental data. Artificial neural networks (ANNs), on the contrary, can fit arbitrary datasets at the cost of opaque mechanisms. Here, we adopt a hybrid approach, combining the predictive power of ANNs with the interpretability of classic models.
We apply this approach to Reinforcement Learning (RL), beginning with classic RL models and replacing their components one-by-one with ANNs. We find that hybrid models can provide similar fit to fully-general ANNs, while retaining the interpretability of classic cognitive models: 
They reveal reward-based learning mechanisms in humans that are strikingly similar to classic RL. They also reveal mechanisms not contained in classic models, including separate reward-blind mechanisms, and the specific memory contents relevant to reward-based and reward-blind mechanisms.

Predictive and Interpretable: Combining Artificial Neural Networks and Classic Cognitive Models to Understand Human Learning and Decision Making

Dependency tree is efficient for relation extraction model to exploit relations between words. Recent approaches have achieved promising performance while still suffering from inherent limitations. 
In this paper, we propose a novel relation extraction framework called Progressive Graph Learning over pruned dependency tree (PGLNet). PGLNet constructs a set of graphs by progressively adapting to input sentence. Specially, we implement the model to construct progressive weighted adjacency matrices by learning the relations among graph nodes with self-attention mechanism. Then, the model takes the learned weights as reference to prune dependency tree. Next, progressive convolution module is designed to encode the relations of entities and followed by relations classification. We evaluate our proposed model using public real-world datasets, experimental results demonstrate that the proposed model achieves state-of-the-art performance with consistency in all datasets. We conclude that the ability of PGLNet to progressively adapt to input data and enable the model with robustness.


Progressive Graph Learning over Pruned Dependency Trees For Relation Extraction

To overcome the von Neumann bottleneck and increase power efficiency in computation, neuromorphic computing has been considered which has the capability for parallel computing, long-term potentiation, and adaptive learning for developing energy efficient neural networks.1 Electric double layer (EDL) gated transistor uses ions within an electrolyte to induce high carrier density (1014 cm-2) in the channel, and its short term and long term ionic responses resemble a biological synapse, making it a promising platform for synaptic applications such as artificial synapses and neurons.2 The accuracy, computational demands, and simplicity of training neural networks depend significantly on a linear synaptic weight response (or plasticity), i.e. a proportionate synaptic weight update in response to input signals.3-6 However, achieving linear weight update in EDLT based synaptic devices have proven to be challenging with fixed magnitude input pulses.7-9 In this study, we first discussed the fundamental cause for nonlinear synaptic weight update in the aforementioned EDLT synaptic devices. With fixed magnitude input voltage pulses, while the amount of EDL build up during each pulse is constant, the amount of EDL dissipation in between pulses is dependent on the instantaneous EDL concentration and follows an exponential decay with respect to time. This leads to increasing amount of dissipation as pulses progress and EDL builds up nonlinearly. This behavior was confirmed with finite element modeling using modified Nernst-Poisson-Planck equations.10,11 A train of 10 pulses with frequencies ranging from 10 – 100 Hz (VG = 1 V) was applied to a parallel plate capacitor geometry consisting of PEO:LiClO4 as the electrolyte, and the ion concentration was calculated. As expected, the linear input pulses resulted in a non-linear response, where the maximum EDL concentration at 100 Hz eventually saturated at 9 x 1013 ions/cm2, more than 2 times smaller than the maximum EDL concentration at a steady 1 V (1.9 x 1014 ions/cm2). Then, we demonstrated that linear weight update can be achieved with varying input pulse widths and/or magnitudes (i.e., nonlinear inputs). A challenge for this approach is that the EDL concentration at the end of each pulse needs to be evaluated to predict the input voltage or width of the subsequent pulse. Based on the instantaneous change in EDL concentration with respect to time (dc/dt), we obtained an empirical relationship relating dc/dt, voltage, concentration, and time. Coupling this relation with the original COMSOL model, the input signals for a linear synaptic response can be predicted in a single computation. For a series of pulses with magnitude between 0.25 V and 3 V at a fixed frequency of 100 Hz, a linear synaptic weight update was obtained with R2 = 0.996. After 10 pulses the improved model resulted with a 13% increase in EDL concentration (to 1.02 x 1014 ions/cm2) and can continue to increase with subsequent pulses. These preliminary results provide insights into extending this model for an FET and developing an EDL gated synaptic network. As part of ongoing work, the group is currently working on experimentally demonstrating the model using parallel plate capacitors (metal-electrolyte-metal) and Graphene-based EDL transistors. 1. Schuman et al. arXiv abs/1705.06963, 2017; 2. Zhu et al. Adv. Mat., 2018, 30 (21); 3. Sung et al. Nanotech., 2018, 29 (11); 4. Lee at al. Adv. Fun. Mat., 2020, 30 (45); 5. Yang et al. Small Science, 2022, 2 (1); 6. Jacobs-Gedrim et al. IEEE (ICRC), 2017; 7. Jiang et al. Nanoscale, 2019, 11 (3); 8. Wan et al. Adv. Mat., 2016, 28; 9. Wen et al. ACS App. Mat. & Int., 2017, 9 (42); 10. Woeppel et al. ACS App. Mat. & Int., 2020, 12 (36); 11. Kilic et al. Phy. Rev., 2007, 75 (2)

Realizing a Linear Synaptic Weight Update in Electric-Double-Layer Gated Transistors for Achieving Spike-Timing-Dependent Plasticity in Neuromorphic Computing

Collecting large-scale medical datasets with fully annotated samples for training of deep networks is prohibitively expensive, especially for 3D volume data. Recent breakthroughs in self-supervised learning (SSL) offer the ability to overcome the lack of labeled training samples by learning feature representations from unlabeled data. However, most current SSL techniques in the medical field have been designed for either 2D images or 3D volumes. In practice, this restricts the capability to fully leverage unlabeled data from numerous sources, which may include both 2D and 3D data. Additionally, the use of these pre-trained networks is constrained to downstream tasks with compatible data dimensions.
In this paper, we propose a novel framework for unsupervised joint learning on 2D and 3D data modalities. Given a set of 2D images or 2D slices extracted from 3D volumes, we construct an SSL task based on a 2D contrastive clustering problem for distinct classes. The 3D volumes are exploited by computing vectored embedding at each slice and then assembling a holistic feature through deformable self-attention mechanisms in Transformer, allowing incorporating long-range dependencies between slices inside 3D volumes. These holistic features are further utilized to define a novel 3D clustering agreement-based SSL task and masking embedding prediction inspired by pre-trained language models. Experiments on downstream tasks, such as 3D brain segmentation, lung nodule detection, 3D heart structures segmentation, and abnormal chest X-ray detection, demonstrate the effectiveness of our joint 2D and 3D SSL approach. We improve plain 2D Deep-ClusterV2 and SwAV by a significant margin and also surpass various modern 2D and 3D SSL approaches.

Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering

Traffic congestion event prediction is an important yet challenging task in intelligent transportation systems. Many existing works about traffic prediction integrate various temporal encoders and graph convolution networks (GCNs), called spatio-temporal graph-based neural networks, which focus on predicting dense variables such as flow, speed and demand in time snapshots, but they can hardly forecast the traffic congestion events that are sparsely distributed on the continuous time axis. In recent years, neural point process (NPP) has emerged as an appropriate framework for event prediction in continuous time scenarios. However, most conventional works about NPP cannot model the complex spatio-temporal dependencies and congestion evolution patterns. To address these limitations, we propose a spatio-temporal graph neural point process framework, named STGNPP for traffic congestion event prediction. Specifically, we first design the spatio-temporal graph learning module to fully capture the long-range spatio-temporal dependencies from the historical traffic state data along with the road network. The extracted spatio-temporal hidden representation and congestion event information are then fed into a continuous gated recurrent unit to model the congestion evolution patterns. In particular, to fully exploit the periodic information, we also improve the intensity function calculation of the point process with a periodic gated mechanism. Finally, our model simultaneously predicts the occurrence time and duration of the next congestion. Extensive experiments on two real-world datasets demonstrate that our method achieves superior performance in comparison to existing state-of-the-art approaches.

Spatio-Temporal Graph Neural Point Process for Traffic Congestion Event Prediction

Despite that convolution neural networks (CNN) have recently demonstrated high-quality reconstruction for video super-resolution (VSR), efficiently training competitive VSR models remains a challenging problem. It usually takes an order of magnitude more time than training their counterpart image models, leading to long research cycles. Existing VSR methods typically train models with fixed spatial and temporal sizes from beginning to end. The fixed sizes are usually set to large values for good performance, resulting to slow training. However, is such a rigid training strategy necessary for VSR? In this work, we show that it is possible to gradually train video models from small to large spatial/temporal sizes, \ie, in an easy-to-hard manner. In particular, the whole training is divided into several stages and the earlier stage has smaller training spatial shape. Inside each stage, the temporal size also varies from short to long while the spatial size remains unchanged. Training is accelerated by such a multigrid training strategy, as most of computation is performed on smaller spatial and shorter temporal shapes. For further acceleration with GPU parallelization, we also investigate the large minibatch training without the loss in accuracy. Extensive experiments demonstrate that our method is capable of largely speeding up training (up to $6.2\times$ speedup in wall-clock training time) without performance drop for various VSR models.

Accelerating the Training of Video Super-Resolution Models

A core process in human cognition is analogical mapping: the ability to identify a similar relational structure between different situations.
We introduce a novel task, Visual Analogies of Situation Recognition, adapting the classical word-analogy task into the visual domain. Given a triplet of images, the task is to select an image candidate B' that completes the analogy (A to A' is like B to what?). Unlike previous work on visual analogy that focused on simple image transformations, we tackle complex analogies requiring understanding of scenes. 

We leverage situation recognition annotations and the CLIP model to generate a large set of 500k candidate analogies. Crowdsourced  annotations for a sample of the data indicate that humans agree with the dataset label ~80% of the time (chance level 25%). Furthermore, we use human annotations to create a gold-standard dataset of 3,820 validated analogies.
Our experiments demonstrate that state-of-the-art models do well when distractors are chosen randomly (~86%), but  struggle with carefully chosen distractors (~53%, compared to 90% human accuracy). We hope our dataset will encourage the development of new analogy-making models.

VASR: Visual Analogies of Situation Recognition

Graph Neural Networks (GNNs) have been successfully used in many problems involving graph-structured data, achieving state-of-the-art performance. 
GNNs typically employ a message-passing scheme, in which every node aggregates information from its neighbors using a permutation-invariant aggregation function.
Standard well-examined choices such as the mean or sum aggregation functions have limited capabilities, as they are not able to capture interactions among neighbors. 
In this work, we formalize these interactions using an information-theoretic framework that notably includes synergistic information. 
Driven by this definition, we introduce the Graph Ordering Attention (GOAT) layer, a novel GNN component that captures interactions between nodes in a neighborhood. 
This is achieved by learning local node orderings via an attention mechanism and processing the ordered representations using a recurrent neural network aggregator. 
This design allows us to make use of a permutation-sensitive aggregator while maintaining the permutation-equivariance of the proposed GOAT layer. 
The GOAT model demonstrates its increased performance in modeling graph metrics that capture complex information, such as the betweenness centrality and the effective size of a node. In practical use-cases, its superior modeling capability is confirmed through its success in several real-world node classification benchmarks. 

Graph Ordering Attention Networks

Deep Neural Networks (DNNs) possess powerful prediction capability thanks to their over-parameterization design, although the large model complexity makes it suffer from noisy supervision. Recent approaches seek to eliminate impacts from noisy labels by excluding data points with large loss values and showing promising performance. However, these approaches usually associate with significant computation overhead and lack of theoretical analysis. In this paper, we adopt a perspective to connect label noise with epistemic uncertainty. We design a simple, efficient, and theoretically provable robust algorithm named USDNL for DNNs with uncertainty-based Dropout. Specifically, we estimate the epistemic uncertainty of the network prediction after early training through single Dropout. The epistemic uncertainty is then combined with cross-entropy loss to select the clean samples during training. Finally, we theoretically show the equivalence of replacing selection loss with single cross-entropy loss. Compared to existing small-loss selection methods, USDNL features its simplicity for practical scenarios by only applying Dropout to a standard network, while still achieving high model accuracy. Extensive empirical results on both synthetic and real-world datasets show that USDNL outperforms other methods. Our code is available at https://github.com/kovelxyz/USDNL.

USDNL: Uncertainty-based Single Dropout in Noisy Label Learning

Learning from raw high dimensional data via interaction with a given environment has been effectively achieved through the utilization of deep neural networks. Yet the observed degradation in policy performance caused by imperceptible worst-case policy dependent translations along high sensitivity directions (i.e. adversarial perturbations) raises concerns on the robustness of deep reinforcement learning policies. In our paper, we show that these high sensitivity directions do not lie only along particular worst-case directions, but rather are more abundant in the deep neural policy landscape and can be found via more natural means in a black-box setting. Furthermore, we show that vanilla training techniques intriguingly result in learning more robust policies compared to the policies learnt via the state-of-the-art adversarial training techniques. We believe our work lays out intriguing properties of the deep reinforcement learning policy manifold and our results can help to build robust and generalizable deep reinforcement learning policies.

Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness

Adversarial attacks have threatened modern deep learning systems by crafting adversarial examples with small perturbations to fool the convolutional neural networks (CNNs). To alleviate that, ensemble training methods are proposed to facilitate better adversarial robustness by diversifying the vulnerabilities among the sub-models, simultaneously maintaining comparable natural accuracy as standard training. Previous practices also demonstrate that enlarging the ensemble can improve the robustness. However, conventional ensemble methods are with poor scalability, owing to the rapidly increasing complexity when containing more sub-models in the ensemble. Moreover, it is usually infeasible to train or deploy an ensemble with substantial sub-models, owing to the tight hardware resource budget and time limits. In this work, we propose Ensemble-in-One (EIO), a simple but effective method to efficiently enlarge the ensemble with a random gated network (RGN). EIO augments a candidate model by replacing the parametrized layers with multi-path random gated blocks (RGBs) to construct an RGN. The scalability is significantly boosted because the number of paths exponentially increases with the RGN depth. Then by learning from the vulnerabilities of numerous other paths within the RGN, every path obtains better adversarial robustness. Our experiments demonstrate that EIO consistently outperforms previous ensemble training methods with smaller computational overheads, simultaneously achieving better accuracy-robustness trade-offs than adversarial training methods under black-box transfer attacks.

Underline digital video library

Psychophysical-Score: A Behavioral Measure for Assessing the Biological Plausibility of Visual Recognition Models

Predictive and Interpretable: Combining Artificial Neural Networks and Classic Cognitive Models to Understand Human Learning and Decision Making

Progressive Graph Learning over Pruned Dependency Trees For Relation Extraction

Realizing a Linear Synaptic Weight Update in Electric-Double-Layer Gated Transistors for Achieving Spike-Timing-Dependent Plasticity in Neuromorphic Computing

Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering

Spatio-Temporal Graph Neural Point Process for Traffic Congestion Event Prediction

Accelerating the Training of Video Super-Resolution Models

VASR: Visual Analogies of Situation Recognition

Graph Ordering Attention Networks

USDNL: Uncertainty-based Single Dropout in Noisy Label Learning

Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness

Ensemble-in-One: Ensemble Learning within Random Gated Networks for Enhanced Adversarial Robustness

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES