Brazil

This work explores and compares the plethora of metrics for the performance evaluation of object-detection algorithms. Average precision (AP), for instance, is a popular metric for evaluating the accuracy of object detectors by estimating the area under the curve (AUC) of the precision × recall relationship. 
Depending on the point interpolation used in the plot, two different AP variants can be defined and, therefore, different results are generated. AP has six additional variants increasing the possibilities of benchmarking. The lack of consensus in different works and AP implementations is a problem faced by the academic and scientific communities. Metric implementations written in different computational languages and platforms are usually distributed with corresponding datasets sharing a given bounding box description. Such projects indeed help the community with evaluation tools, but demand extra work to be adapted for other datasets and bounding-box formats. This work reviews the most used metrics for object detection detaching their differences, applications, and main concepts. It also proposes a standard implementation that can be used as a benchmark among different datasets with minimum adaptation on the annotation files.

IWSSIP 2020

A Survey on Performance Metrics for Object-Detection Algorithms

bounding boxes

object detection challenges

average precision

object-detection metrics

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

technical paper

The 27th International Conference on Systems, Signals and Image Processing.

Deep learning-based classifiers for object recognition and classification have been used in the domain of plant disease detection, particularly lesions from leaf images. In such domains, as expected, deep neural networks perform better using balanced data sets than imbalanced ones, as they exhibit some inductive bias favoring balanced data from each class. However, data sets for plant disease detection are often imbalanced due to the rarity of disease lesions in real-world settings. While deep generative approaches such as generative adversarial networks (GANs) have been established as an effective means of augmenting high-dimensional image data, the literature lacks a detailed study of the effectiveness of GAN-based models on a plant disease detection task, compared to sampling-based approaches traditionally used to reduce the skewness of the data. In this paper, we comparatively evaluate an image classifier based on a dense convolutional neural network (CNN), trained using a GAN, versus the same CNN model used in tandem with undersampling, oversampling, and an adaptation of the Synthetic Minority Over-sampling Technique (SMOTE). The GAN-based approach is shown to attain significantly higher recall and hence F-measure and ROC AUC against each of these.

Addressing Class Imbalance in Image-Based Plant Disease Detection: Deep Generative vs. Sampling-Based Approaches

Breast cancer is one of the most common diseasesin women in the world. There are various imaging techniquesemployed in the diagnosis. The histological image analysis sup-ported by computational systems has proved to be quite effectivein diagnosing the disease. In this paper, we present an approachto quantify and classify tissue samples of the breast based on features extracted from the intensity histogram, co-occurrencematrix and the Shannon, Renyi, Tsallis and Kapoor entropies. The attribute set was employed to obtain the feature vectors which were evaluated as inputs to the random forest and sequential minimal optimization algorithms with the 10-fold cross-validation technique. In this study, we investigated the proposed approach with images obtained in four levels of magnification of the publicly available Breast Cancer Histopathological Database. In the feature selection stage, we investigated the correlation-Based feature selection, ReliefF, information gain, gain ratio, one-R and symmetrical uncertainty algorithms for evaluating the performance of theproposed approach. The proposed approach achieved significant results of AUC and accuracy for all cases analyzed. The proposed approach obtained 0.997 for AUC and 97.6% for the accuracy metric. These results are considered relevant and this approachis useful as an automated protocol for the diagnosis of breast histological tissue.

Analysis of Features for Breast Cancer Recognition in Different Magnifications of Histopathological Images

Borehole stability is one of the main factors influencing the success of an oil well drilling process. Fractures and beddings (sedimentary layers) are important stability features commonly identified in borehole imaging tools. Their visual presentation in the 2D Ultrasonic Borehole Imager (UBI) consists of sinusoidal curves, in which automatic detection is not trivial. Several previous works proposed different solutions, most of them based on the Hough transform (HT). Nevertheless, HT based solutions have a substantial computational cost. In this work, we propose an innovative approach to identify the harmonic curves related to fractures and sedimentary layers in borehole images based in the Iterative Local Search heuristic, reducing the computational cost of the task significantly.

Automatic Sinusoidal Curves Detection in Borehole Images Using the Iterated Local Search Algorithm

This paper presents an approach for evaluating CAD systems using concepts of Content-Based Image Retrieval and graphical oracles. After implementing feature descriptors and selecting three similarity functions, two metrics are proposed to measure the efficiency of CAD systems. A case study was conducted considering three simulated CAD systems to detect masses and calcifications in mammographic images. The results indicated that the our approach is as robust as traditional metrics with respect to performance evaluation. However, our approach is more flexible than traditional metrics because evaluators can choose the more adequate features to assess a particular CAD system.

Graphical Oracles to Assess Computer-Aided Diagnosis Systems: A Case Study in Mammogram Masses and Calcifications Detection

Active Shape Models (ASM) are widely explored for facial landmarks detection. The task is commonly approached in frontal perspective where target features such as eyes, mouth, and nose lie within the face’s contour. Conversely, few application proposals are aimed to find facial landmarks in a profile view. This paper addresses an issue with that technique applied to locate facial landmarks lying on the profile contour. Those boundary points present an extra challenge for ASM’s searching method when applied in images with background lighting variations. An ASM trained with profile images captured with different brightness backgrounds may lead the method to model texture profiles in segregated clusters. In those cases, it is inaccurate to assume that sampled texture profiles follow a Gaussian distribution, which is essential to estimate new landmarks locations with the Mahalanobis distance. We propose an alternative to improve ASM’s robustness in such a scenario by utilizing K- Means to identify those clusters and create specific texture models for each one. Then, during the searching stage, new estimates are compared with each model individually, assuming the texture profiles follow a normal distribution inside their corresponding clusters. We trained and tested our improved ASM with a private database of 670 face images in profile view captured in a controlled environment. We report an average error reduction of 46% for landmarks location compared to the classical method. We also demonstrate that, regarding this issue, our proposal drives ASM’s searching method to the point of convergence, whereas the original method does not.

Improving Active Shape Models Robustness Towards Locating Facial Landmarks In Profile Contour

In this paper, we proposed and evaluated the adoption of a capsule network-based (CapsNet-based) model rather than the convolutional neural network-based (CNN-based) models which are predominant in crowd counting tasks. The aim is to join the task of generating a high-quality density map from a single image along with producing a more precise estimate of the number of people. CapsNet-based model has a strong capacity of representation and a powerful dynamic routing mechanism that could address the drawback of a limited number of training samples. The replacement of the scalar values of CNN by vectors when using a CapsNet allows learning more discriminative features, which contributes to generating high-quality density maps, and thus a more precise number of individuals in crowd scenes. Experimental results show that our proposal presents competitive results concerning state-of-the-art, but with a 59.2% reduction in the number of parameters.

Dense Crowd Counting with Capsule Networks

Acute leukemia is a cancer-related to a bone marrow abnormality. It is more common in children and young adults. This type of leukemia generates unusual cell growth in a short period, requiring a quick start of treatment. Acute Lymphoid Leukemia (ALL) and Acute Myeloid Leukemia (AML) are the main responsible for deaths caused by this cancer. The classification of these two leukemia types on blood slide images is a vital process of and automatic system that can assist doctors in the selection of appropriate treatment. This work presents a convolutional neural networks (CNNs) architecture capable of differentiating blood slides with ALL, AML and Healthy Blood Slides (HBS). The experiments were performed using 16 datasets with 2,415 images, and the accuracy of 97.18% and a precision of 97.23% were achieved. The proposed model results were compared with the results obtained by the state of the art methods, including also based on CNNs.

Convolution Neural Network Models for Acute Leukemia Diagnosis

The time required for a vehicle to travel different routes in the daily traffic of large cities varies and changes constantly, impacting the daily lives of everyone dwelling in those cities. Trying to predict such time is essential to evolve in understanding the behavior of vehicle traffic. On the other hand, due to the vast amount of data generated in this context, it is necessary to use new ways of dealing with this problem. This paper presents an exploratory analysis of the behavior of batch and stream learning algorithms for predicting the trip duration time for vehicles going through different routes. We understand that batch learning algorithms are not necessarily adequate for being used in stream mining situations. However, we would like to have a testbed to analyze the behavior of stream learning algorithms. For our experimental analysis, we used real data from three specific routes. The results show that the use of data stream learning for this domain yields promising results

An Exploratory Study Using Stream Learning Algorithms to Predict Duration Time of Vehicle Routes

The prediction of secondary protein structures is one of the classic problems of bioinformatics and has several practical applications.In this work, we present an ensemble of bidirectional recurrent networks with random forests. The fusion was performed using weights for each of the classifiers and classes, found by searching through genetic algorithms. To evaluate our method, we used the PDB, CB6133 and CB513 datasets. We achieved 73.1%, 59.1% and 55.8% Q8 accuracy on PDB, CB6133 and CB513 proteins, respectively, whereas 81.5% Q3 accuracy on PDB proteins using only amino acid sequence information. In order to compare our results against the literature, we also evaluated our method using amino acid sequence and sequence profile features, achieving 73.4% and 68.9% Q8 accuracy on CB6133 and CB513. Our method yielded good results when using only the amino acid sequence and presents competitive results compared tothe literature when using amino acid sequence information and protein sequence similarity.

Ensemble of Bidirectional Recurrent Networks and Random Forests for Protein Secondary Structure Prediction

To make possible the exploration of oil and natural gas in maritime waters, it is necessary to develop high technology to support the projects of drilling and exploration of submarine wells. One of the equipment that enables offshore exploration is the flexible pipes. Considering the high cost and complexity of the flexible pipe manufacturing process, the objective of this work is to analyze the effect of the occurrence of whitening stress identified as bleaching observed in the polymeric material known as Polyfluoride of Vinylidene (PVDF), which is used in the manufacture of inner liner layers in flexible pipes. The proposed methodology is to develop a prediction model based on classification, which evaluates the whitening bleaching by extracting knowledge from an image database. Through the proposed methodology, the most relevant parameters associated with the study problem that can define classification rules areidentified. As a result, some relationships between some characters, fruits of image data extraction and product characteristics were identified through decision trees. The conclusion is that these relationships provides support for decision making when peformed by a domain expert.

Premium content

A Survey on Performance Metrics for Object-Detection Algorithms

Downloads

Next from IWSSIP 2020

Addressing Class Imbalance in Image-Based Plant Disease Detection: Deep Generative vs. Sampling-Based Approaches

Similar lecture

Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES