MIDT4PdM

Associated repository with complementary material to the paper A Multi-Instance Hoeffding Anytime Tree Approach for Interpretable Failure Detection in Industrial Systems, published in Advanced Engineering Informatics. In this repository you will find:

• Source code of the proposed Multi-Instance Hoeffding Anytime Tree
• Datasets used in the experimentation
• Complete tables of results

The aim of the proposal developed in this work is the Time Series Classification in a context of a industrial system under a performance degradation. Due to the intense data generation environment, the model learns in a weak supervised manner.

The code of the proposal has been depeloped in Python >=3.10, using the River library for the base implementation of the Hoeffding Anytime Tree. The complete list of libraries and dependencies of the project is specified in the file requirements.yml.

For a quick start tutorial of the proposal, the tutorial.ipynb describes the complete workflow to start using miHATT, from the loading data process, building and testing the model, and accessing to the interpretability resources.

The complete code generated for this proposal is available under the src folder following the structure:

src
│ requirements.yml
│ tutorial.ipynb
│
└───mihatt.py
│ │ MultiInstanceHATT > Implementation of the proposed method
│
└───utils.py > Auxiliary functions for loading data
│
└───dataprep > Auxiliary functions to prepare the original data for the problem
 │ cmapss_preprocess.py
 │ cwru_preprocess.py

This work has been tested in two public and popular predictive maintenance problems: NASA Ames Turbofan Engine Degradation Dataset and Case Western Reserve Univeristy Bearing Data Center Ball Bearing Dataset. While original data is available in their corresponding websites, this repository provides access to the processed versions used in this work to perform failure detection applying time series classification. Thus, for each of the two problems, four datasets are considered. These datasets are composed of time series obtained from the sensor measurements of turbofans or rotors during a working session, and in all the cases the target of the learning process will be the grade of degradation of the industrial system, which is a discrete state varying between 3 and 5 grades depending on the dataset.

The following tables show the characteristics of each dataset composing the two case studies. All the files are available for download under the folder datasets.

Both Turbofan and Ball bearing datasets have required transformations in order to use them for our problem of failure detection from temporal data using multi-instance learning.

In the case of the NASA Ames Turbofan Engine Degradation Dataset, the original learning target was the prediction of the remaining useful life, i.e. a regression task. Thus, the transformations go in the direction of generating a ground truth that associates each time series with a degradation state. The complete code for the transformation is available in cmapss_preprocess.py.

Attending to the Case Western Reserve University Bearing Data Center Ball Bearing Dataset, the original target is the fault identification directly, but the working sequences are not ready to use in a machine learning environment, as they are separated in different files corresponding to a single complete working sequence. Thus, the transformations consist of splitting the sequences into separate time series, mixing all the degradation states in a single pool, and creating separate sets for training and testing the machine learning models. The complete code for the transformation is availabe in cwru_preprocess.py.

This repository contains the complete tables of results from the experimental study presented in the paper, downloadable from the results folder.

The study is organized in three main parts:

1. Analysis of components of the proposed MIHATT
2. Comparative performance with the state of the art
3. Interpretability study

Each study is fully described in the paper, and below we describe the downloable materials for them.

This analysis evaluates the influence of the main components of the proposed framework: multi-instance formulation, the use of Hoeffding Anytime Tree, and the incremental feature extraction through iterative learning.

The table below contains the average results for each possible combination of components over the eight datasets considered.

From these results we can get statistics about the average performance across datasets:

bar plot of average performance for each component in MIHATT

And we can estimate as well the contribution of each component, measured as the average gain when introducing MIL, HATT and incremental learning respectively.

bar plot of the effect estimation of each component in MIHATT

The second study compares MIHATT performance against a selection of methods that have been used before for failure prediction in the context of predictive maintenance. These methods are configured following authors recommendations as listed below (more information in the paper).

The overall average obtained over the eight datasets for each of the 15 proposals included in the study is shown in the next table. In addition, the lower part of the table shows the results of the paired Wilcoxon test resulting from comparing each algorithm against our proposal: $R^+$ represents the times MIHATT wins while $R^-$, the times the other proposal wins. The last row is the $p$-value resulting from applying the statistical test.

The breakdown of the results for each metric is in the results folder, specifically:

• Accuracy results
• Hamming loss results
• Precision results
• Recall results
• F1 score results

MIHATT is naturally explainable thanks to two key aspects: (i) its base learner is a white-box decision tree that has the same input space as the original features, enabling users to trace the reasoning behind each prediction through explicit variable thresholds and decision paths; and (ii) the use of the MIL assumption — combined with the concept of identifying signature patterns within a sequence — makes it possible to highlight the most relevant temporal segment responsible for the predicted class, providing intuitive insight into when and why a failure is likely to occur.

The following table displays information about the trees generated by MIHATT for each studied problem.

As example, below there is the graph of the tree generated for the dataset FD001. The nodes display the feature for the decission, that matches the original input space since the model works with the unprocessed features. The branches show the threshold for the given feature at each moment to create the decission path to clasify the sensor data. At the leaves it is shown the probability for each class, i.e., health status considered that range from 0ドル$ (healthy system), to 1ドル$ (medium failure), and to 2ドル$ (critical failure).

tree sample

For more details about how to load MIHATT, to train it, and to get its interpretability insights, check the tutorial provided in this repository.