Your Data Anomaly Detection Can Be Better
- By CDOTrends editors
- February 22, 2023
Data is not always perfect — sometimes, unusual observations or "anomalies" can occur. Identifying these anomalies is important for various applications, such as detecting credit card fraud, predicting weather patterns, and recognizing medical conditions.
A team of researchers from Chung Ang University in Korea, including Professor Jason J. Jung and Dr. Gen Li, published a study in the journal Advanced Functional Materials that evaluated deep learning-based methods for detecting anomalies in multivariate time series data.
Multivariate time series data is a type of data that involves multiple variables collected over some time. This type of data is often used to analyze trends, make predictions and identify abnormalities. For example, if a stock market contains multiple stocks, their prices over time could be tracked through multivariate time series data.
“Our fundamental research topic is anomaly detection in multivariate time series. In this review, we have summarized the approaches, challenges, and applications for the same,” said Prof. Jung.
The team found that long short-term memory (LSTM) and autoencoders were the most effective deep learning-based methods for detecting abnormal time points and time intervals.
LSTM is a type of recurrent neural network that can remember past inputs and use that information to predict future outputs. This makes it well-suited for anomaly detection, as it can detect patterns and deviations in data sequences.
Autoencoders, meanwhile, are a type of neural network trained to compress data into a lower-dimensional representation and then reconstruct it back to its original form, allowing it to detect changes in data that might otherwise go unnoticed.
Dynamic graphs, or graphs that examine the relationships between time series data points, were also discussed as an alternative method. These graphs can
“In the field of science, people can easily find out the open access datasets and the corresponding state-of-art anomaly detection method in this paper. For industrial applications, the appropriate anomaly detection techniques to identify damages and faults could be conveniently found in this review," he added.
“The challenge is to identify the relationship between an abnormal time point and the time point leading to that anomaly,” says Prof. Jung. This implies that understanding the root causes of anomalies is critical to better anomaly detection.
Prof. Jung and Dr. Li have previously worked on time series anomaly detection for multiple variables. They have published their work on seizure detection, climate monitoring, and financial fraud monitoring, which culminated in this review.
Image credit: iStockphoto/canbedone