DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data

Python License Neuroplasticity OPE PIGL DMA Backbone IDF Analyzer Model Conference Citation Status

DynaTab is a neuro-inspired tabular deep learning model for high-dimensional tabular data that tackles the Column Permutation Problem by dynamically reordering features instead of treating them as a fixed set. It predicts when feature ordering is beneficial using an intrinsic-dimensionality-based IDF/FOE criterion, then applies dynamic feature ordering (DFO) to rewire feature graphs and produce a task-aware global sequence. This reordered input is processed by an order-aware fusion block combining positional embeddings (OPE), importance gating (PIGL), and dynamic masked attention (DMA) on top of a sequential backbone (Transformer, DAE, LSTM, Mamba, or DAE-MHA-LSTM). It also empirically group tabular datasets into 5 categories. Across 36 real-world datasets and over 45 baselines, DynaTab achieves strong, statistically significant gains, particularly in high-dimensional low-sample-size (HDLSS) and other complex regimes, positioning dynamic feature ordering as a powerful paradigm for order-sensitive backbones in tabular deep learning for high-dimensional tabular data.

Citation

Al Zadid Sultan Bin Habib, Gianfranco Doretto, and Donald A. Adjeroh. "DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data." In AAAI 2026 First International Workshop on Neuro for AI & AI for Neuro: Towards Multi-Modal Natural Intelligence (NeuroAI) Workshop Proceedings (PMLR), 2026.

Bibtex:

@InProceedings{dynatab,
 title = 	 {{DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data}},
 author = {Habib, Al Zadid Sultan Bin and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = 	 {{Proceedings of the First Workshop on NeuroAI Multimodal Intelligence @ AAAI 2026}},
 pages = {27--57},
 year = {2026},
 volume = {308},
 series = {{Proceedings of Machine Learning Research}},
 publisher = {PMLR},
 url = 	 {https://proceedings.mlr.press/v308/habib26a.html}
}

Paper Link: https://proceedings.mlr.press/v308/habib26a.html

arXiv preprint: https://arxiv.org/abs/2605.03430

Files and Repository Structure

Python package: `dynatab/`

This folder contains the core DynaTab implementation (15 Python modules):

__init__.py - Package initializer and high-level API exports.
model.py - Main DynaTab model definition and wiring of all sub-modules.
dfo.py - Dynamic Feature Ordering (DFO) module and clustering/graph construction.
ope.py - Order-Aware Positional Embedding (OPE) implementation.
pigl.py - Positional Importance Gating Layer (PIGL).
dma.py - Dynamic Masked Attention (DMA) block.
seqprobinary.py - Training loop / utilities for binary classification.
seqpromulti.py - Training loop / utilities for multiclass classification.
seqproregression.py - Training loop / utilities for regression.
preprocess.py - Data preprocessing and tabular input utilities (splits, scaling, etc.).
metrics.py - Evaluation metrics and helper functions.
estimator.py - High-level estimator wrapper for running experiments (sklearn-style API).
idf_analyzer.py - Intrinsic Dimensionality Factor (IDF) + FOE analyzer: "Feature Ordering – When to Use?".
customloss.py - Custom loss functions used by DynaTab.
trainer.py - Generic training / validation loop utilities shared across tasks.

Notebooks

DynaTab Dataset Complexity Analysis.ipynb
Contains the experiments for the "Feature Ordering – When to Use?" section, including IDF / FOE computation across datasets.
DynaTab IDF Analyzer.ipynb
Shows how to install/import the dynatab package and use TabularIDFAnalyzer to compute dataset complexity metrics with demo runs.
The code cells illustrate how to use DynaTab to assess when feature ordering is appropriate for a given dataset.
DynaTab_Experiment1.ipynb
Demonstrates how to use DynaTab for binary classification, multiclass classification, and regression, with or without Optuna-based hyperparameter tuning.
DynaTab_Experiment2.ipynb
Demonstrates DynaTab on the GLI-85 HDLSS dataset for binary classification, without Optuna tuning, using Mamba or LSTM as the sequential processor backbone.
DynaTab_PIP_Install_Check.ipynb
Demonstration of DynaTab in a Google Colab notebook using pip installation with a toy example.
N.B.: Demo runs only contain less number of epochs or Optuna trials. For complete run, please use proper number of Optuna trials to search and find optimum hyperparameters.

Other top-level files

requirements.txt - Python dependencies required to run the DynaTab package and notebooks.
DynaTab_Architecture.jpg - High-level architecture diagram of the DynaTab framework.
LICENSE - MIT license for this repository.
README.md - Project overview, usage instructions, and citation information.
.gitignore - Standard Git ignore rules for Python and Jupyter projects.

Tested Environment

Python 3.8+
torch 2.5.1+cu121 (CUDA 12.1)
numpy 1.26.4
pandas 2.2.3
scikit-learn 1.5.2
matplotlib 3.10.0
scipy 1.11.4
kmeans_gpu 0.0.5

Recommended PyTorch install (GPU, CUDA 12.1)

pip install "torch==2.5.1+cu121" --index-url https://download.pytorch.org/whl/cu121

Installation

You can install DynaTab in several ways depending on your workflow.

Option 1: Clone the Repository (Recommended for Development)

git clone https://github.com/zadid6pretam/DynaTab.git
cd DynaTab
pip install -r requirements.txt
pip install -e .

Option 2: Install Directly from GitHub (No Cloning Needed)

pip install "git+https://github.com/zadid6pretam/DynaTab.git"

Option 3: Use a Virtual Environment

python -m venv dynatab-env
source dynatab-env/bin/activate # On Windows: dynatab-env\Scripts\activate
git clone https://github.com/zadid6pretam/DynaTab.git
cd DynaTab
pip install -r requirements.txt
pip install -e .

Option 4: Local Install Without Editable Mode

git clone https://github.com/zadid6pretam/DynaTab.git
cd DynaTab
pip install -r requirements.txt
pip install .

Option 5: Install from PyPI (Planned)

pip install dynatab

Example Usage

Below are minimal examples for using DynaTab on standard binary, multiclass, and regression tasks.
For full HDLSS experiments and Optuna sweeps, see the accompanying Jupyter notebooks.

1. Binary Classification

import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from dynatab import (
 DynaTabClassifier,
 DFOConfig,
 TrainConfig,
 LossConfig,
)
# -----------------------------
# Data
# -----------------------------
data = load_breast_cancer()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = pd.Series(data.target) # 0/1 labels
X_train, X_test, y_train, y_test = train_test_split(
 X, y,
 test_size=0.2,
 stratify=y,
 random_state=42,
)
# -----------------------------
# DynaTab configs
# -----------------------------
dfo_cfg = DFOConfig(
 metric="manhattan",
 num_clusters=2,
 order="ascending",
 mutation_prob=0.0,
 tolerance=1e-3,
 seed=42,
)
train_cfg = TrainConfig(
 epochs=100,
 lr=1e-3,
 batch_size=256,
 print_every=20,
)
loss_cfg = LossConfig(
 loss_mode="DFO", # "standard" | "dispersion" | "DFO"
 lambda_disp=0.0,
 lambda_global=0.0,
)
# -----------------------------
# Model: DynaTabClassifier
# -----------------------------
clf = DynaTabClassifier(
 task="binary",
 backbone="Transformer", # or "LSTM", "DAE", "Mamba", ...
 embedding_dim=128,
 dfo_cfg=dfo_cfg,
 train_cfg=train_cfg,
 loss_cfg=loss_cfg,
 eval_metrics=["acc"],
 device=None, # auto-selects CUDA/CPU
 standardize=True, # train-only impute + standardize
)
clf.fit(X_train, y_train)
metrics = clf.score(X_test, y_test, metrics=["acc"])
print(f"Test Accuracy: {metrics['acc']:.4f}")

2. Multiclass Classification

import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from dynatab import (
 DynaTabClassifier,
 DFOConfig,
 TrainConfig,
 LossConfig,
)
data = load_iris()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = pd.Series(data.target) # 3 classes: 0,1,2
X_train, X_test, y_train, y_test = train_test_split(
 X, y,
 test_size=0.2,
 stratify=y,
 random_state=42,
)
dfo_cfg = DFOConfig(
 metric="variance",
 num_clusters=3,
 order="descending",
 mutation_prob=0.1,
 tolerance=1e-3,
 seed=42,
)
train_cfg = TrainConfig(
 epochs=80,
 lr=1e-3,
 batch_size=64,
 print_every=20,
)
loss_cfg = LossConfig(
 loss_mode="standard",
 lambda_disp=0.0,
 lambda_global=0.0,
)
clf = DynaTabClassifier(
 task="multiclass",
 num_classes=3,
 backbone="Transformer",
 embedding_dim=64,
 dfo_cfg=dfo_cfg,
 train_cfg=train_cfg,
 loss_cfg=loss_cfg,
 eval_metrics=["acc"],
 device=None,
 standardize=True,
)
clf.fit(X_train, y_train)
metrics = clf.score(X_test, y_test, metrics=["acc"])
print(f"Test Accuracy: {metrics['acc']:.4f}")

3. Regression

import pandas as pd
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from dynatab import (
 DynaTabRegressor,
 DFOConfig,
 TrainConfig,
 LossConfig,
)
data = load_diabetes()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = pd.Series(data.target)
X_train, X_test, y_train, y_test = train_test_split(
 X, y,
 test_size=0.2,
 random_state=42,
)
dfo_cfg = DFOConfig(
 metric="correlation",
 num_clusters=3,
 order="ascending",
 mutation_prob=0.1,
 tolerance=1e-3,
 seed=42,
)
train_cfg = TrainConfig(
 epochs=120,
 lr=1e-3,
 batch_size=128,
 print_every=20,
)
loss_cfg = LossConfig(
 loss_mode="standard", # for regression we typically keep it standard
 lambda_disp=0.0,
 lambda_global=0.0,
)
reg = DynaTabRegressor(
 backbone="Transformer",
 embedding_dim=64,
 dfo_cfg=dfo_cfg,
 train_cfg=train_cfg,
 loss_cfg=loss_cfg,
 eval_metrics=["r2"], # e.g., R^2
 device=None,
 standardize=True,
)
reg.fit(X_train, y_train)
metrics = reg.score(X_test, y_test, metrics=["r2"])
print(f"Test R2: {metrics['r2']:.4f}")

4. Advanced: 5-Fold CV + Optuna Hyperparameter Tuning

For full HDLSS experiments, repeated CV, and Optuna-based tuning (Transformer, LSTM, DAE, Mamba backbones) on real datasets such as AI-d_case5, ADNI_AD123, GLI-85, and others, see:

DynaTab_Experiment1.ipynb – Binary & multiclass classification and regression (with / without Optuna-based hyperparameter tuning).
DynaTab_Experiment2.ipynb – HDLSS case studies (e.g., GLI-85 with Mamba/LSTM backbones).
DynaTab Dataset Complexity Analysis.ipynb and DynaTab IDF Analyzer.ipynb – Intrinsic dimensionality and "when to use feature ordering" analysis.
You can tweak the metrics / epochs / DFO settings if you want them lighter or closer to the paper defaults.

Our Related Works

GOTabPFN (ICML 2026)

Our recent ICML 2026 Regular main conference paper on feature ordering and compression for tabular foundation models for high-dimensional low-sample-size tabular data:

GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
GitHub: https://github.com/zadid6pretam/GOTabPFN
Find it on ICML portal: https://icml.cc/virtual/2026/poster/62523
Project Webpage: https://www.zadidhabib.com/gotabpfn.html
OpenReview: https://openreview.net/forum?id=fpqfV3lCIB
Hugging Face Space: ZeroGPU Live Demo (recommended; faster GPU-backed testing) | CPU Backup Demo (use if ZeroGPU is unavailable) | ZeroGPU Space Repository | CPU Backup Space Repository

@inproceedings{habib2026gotabpfn,
 title = {GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna Kumar and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
 year = {2026}
}

iSyncTab (ECCV 2026)

Our neural synchrony-based cross-modal feature sequencing framework for multimodal learning with image and tabular data. iSyncTab addresses the image–tabular integration problem by aligning and sequencing cross-modal feature groups before structured multimodal representation learning.

iSyncTab: Learning Cross-Modal Feature Sequencing for Image-Tabular Data via Neural Synchrony
Accepted at the European Conference on Computer Vision (ECCV 2026)
GitHub: https://github.com/zadid6pretam/iSyncTab
Project Page: https://www.zadidhabib.com/isynctab.html (Under Construction)
ECCV Page: https://eccv.ecva.net/virtual/2026/poster/5442
Paper: https://doi.org/10.1007/978-3-032-37035-8 (In Press)
Hugging Face: https://huggingface.co/zadid6pretam/iSyncTab-HAM10000 (Model weights and checkpoint for the HAM10000 dataset)

@inproceedings{habib2026isynctab,
 title = {iSyncTab: Learning Cross-Modal Feature Sequencing for Image-Tabular Data via Neural Synchrony},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the European Conference on Computer Vision},
 year = {2026}
}

If you are interested in cross-modal feature sequencing, neural synchrony-guided image–tabular integration, and order-aware multimodal representation learning, please refer to the iSyncTab repository, project page, and paper.

BSTabDiff (ICLR 2026 DeLTa Workshop)

Our generative modeling framework for high-dimensional low-sample-size tabular data:

BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation

GitHub: https://github.com/zadid6pretam/BSTabDiff

@inproceedings{habib2026bstabdiff,
 title = {BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna Kumar and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {ICLR 2026 2nd Workshop on Deep Generative Models in Machine Learning: Theory, Principle and Efficacy (DeLTa)},
 year = {2026}
}

If you are interested in high-dimensional tabular synthesis, block-subunit generation, and diffusion/flow priors for HDLSS tabular data, please also refer to the BSTabDiff repository and paper.

iStructTab (ICPR 2026)

Our structured feature sequencing framework for multimodal learning with image and tabular data. This work is part of my PhD research on feature sequencing or ordering for multimodal image-tabular representation learning.

iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data

GitHub: https://github.com/zadid6pretam/iStructTab

@inproceedings{habib2026istructtab,
 title = {iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the 28th International Conference on Pattern Recognition},
 year = {2026},
 address = {Lyon, France}
}

If you are interested in structured feature sequencing, multimodal fusion of image and tabular data (the integration problem), and feature order-aware tabular representation learning, please also refer to the iStructTab repository and paper.

DynaTab (AAAI 2026 NeuroAI Workshop)

Our more recent work on learned feature ordering for high-dimensional tabular data:

DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data
GitHub: https://github.com/zadid6pretam/DynaTab
Paper: https://link.springer.com/chapter/10.1007/978-3-032-31404-8_43
arXiv: https://arxiv.org/abs/2608.04348

@inproceedings{habib2026dynatab,
 title = {{DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data}},
 author = {Habib, Al Zadid Sultan Bin and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the AAAI 2026 First International Workshop on Neuro for AI \& AI for Neuro: Towards Multi-Modal Natural Intelligence (NeuroAI)},
 year = {2026},
 series = {PMLR}
}

If you are interested in learned feature ordering, neural rewiring for high-dimensional tabular data, and sequential backbone design for HDLSS settings, please also refer to the DynaTab repository and paper.
DynaTab has completed camera-ready submission, and the public proceedings version is expected to appear online later.

TabSeq (ICPR 2024)

Our earlier work on sequential modeling for tabular data:

TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering

GitHub: https://github.com/zadid6pretam/TabSeq

Springer ICPR 2024 proceedings: https://link.springer.com/chapter/10.1007/978-3-031-78128-5_27

arXiv: https://arxiv.org/abs/2410.13203

@inproceedings{habib2024tabseq,
 title={TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering},
 author={Habib, Al Zadid Sultan Bin and Wang, Kesheng and Hartley, Mary-Anne and Doretto, Gianfranco and A. Adjeroh, Donald},
 booktitle={International Conference on Pattern Recognition},
 pages={418--434},
 year={2024},
 organization={Springer}
}

If you are interested in sequential ordering for tabular data, deep sequential backbones, and early feature-ordering-based tabular modeling, please also refer to the TabSeq repository and paper.

ZAYAN (ICPR 2026)

This repository corresponds to our separate collaborative work on tabular remote sensing and environmental data:

ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data
GitHub: https://github.com/zadid6pretam/ZAYAN
Paper: https://link.springer.com/chapter/10.1007/978-3-032-31397-3_1
arXiv: https://arxiv.org/abs/2604.27606

@inproceedings{habib2026zayan,
 title = {ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data},
 author = {Habib, Al Zadid Sultan Bin and Tasnim, Tanpia and Islam, Md. Ekramul and Tabasum, Muntasir},
 booktitle = {Proceedings of the 28th International Conference on Pattern Recognition},
 year = {2026},
 address = {Lyon, France}
}

ZAYAN focuses on feature-level contrastive learning and Transformer-based classification for tabular remote sensing and environmental datasets.
Unlike my PhD dissertation projects on high-dimensional tabular learning and HDLSS modeling, ZAYAN was developed as a separate collaboration.

Contact

For any questions, issues, or suggestions related to this repository, please feel free to contact us or open an issue on GitHub.

Name		Name	Last commit message	Last commit date
dynatab		dynatab
.gitignore		.gitignore
DynaTab Dataset Complexity Analysis.ipynb		DynaTab Dataset Complexity Analysis.ipynb
DynaTab IDF Analyzer.ipynb		DynaTab IDF Analyzer.ipynb
DynaTab_Architecture.jpg		DynaTab_Architecture.jpg
DynaTab_Experiment1.ipynb		DynaTab_Experiment1.ipynb
DynaTab_Experiment2.ipynb		DynaTab_Experiment2.ipynb
DynaTab_PIP_Install_Check.ipynb		DynaTab_PIP_Install_Check.ipynb
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
setup.cfg		setup.cfg

Latest commit

History

Folders and files

Repository files navigation