ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data (ICPR 2026 Oral || RRPR Badge)

Python License PyPI

Task Model Architecture Learning Objective Domain

ICPR 2026 Status RRPR Paper arXiv

ZAYAN Architecture

ZAYAN is a self-supervised, feature-centric contrastive learning framework for tabular remote sensing and environmental data. ZAYAN stands for Zero-Anchor dYnamic feAture eNcoding. Rather than applying contrastive learning at the sample or image-patch level, it learns representations at the feature level, where each feature is dynamically augmented, encoded, contrasted, and regularized to reduce redundancy. The learned feature embeddings are then used by a Transformer classifier that preserves the contrastive feature geometry for downstream prediction. This design makes ZAYAN especially suitable for heterogeneous tabular sensing data derived from satellite products, GIS layers, environmental indicators, and remote-sensing-driven prediction tasks. Across multiple remote-sensing and environmental tabular benchmarks, ZAYAN achieves strong classification performance, robustness, and generalization compared with classical machine learning, tree ensembles, tabular neural networks, and recent tabular foundation-style baselines.

Al Zadid Sultan Bin Habib, Tanpia Tasnim, Md. Ekramul Islam, and Muntasir Tabasum. "ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data." In Proceedings of the 28th International Conference on Pattern Recognition (ICPR), Lyon, France, 2026.

BibTeX:

@inproceedings{habib2026zayan,
 title = {ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data},
 author = {Habib, Al Zadid Sultan Bin and Tasnim, Tanpia and Islam, Md. Ekramul and Tabasum, Muntasir},
 booktitle = {Proceedings of the 28th International Conference on Pattern Recognition},
 year = {2026},
 address = {Lyon, France}
}

• Paper: https://link.springer.com/chapter/10.1007/978-3-032-31397-3_1
• arXiv: https://arxiv.org/abs/2604.27606v1

This folder contains the core ZAYAN implementation:

• __init__.py - Package initializer and high-level API exports.
• zayan.py - Main ZAYAN implementation, including ZAYAN_CL, ZAYAN_T, and the high-level ZAYAN wrapper for contrastive pretraining, Transformer-based supervised training, and evaluation.

This folder contains the Urban Land Cover dataset in csv format (urbantraining.csv and urbantesting.csv)

• ZAYAN_Experiment.ipynb
  Contains the main experiment notebook for ZAYAN. The notebook includes an Optuna-tuned run on the Urban Land Cover dataset, along with data preprocessing, ZAYAN-CL feature-level contrastive pretraining, ZAYAN-T supervised Transformer training, evaluation, and diagnostic analysis.

• ZAYAN_PIP_Install_Check.ipynb
  Demonstration of ZAYAN using pip installation, including simple toy examples for importing the package, initializing ZAYAN_CL, ZAYAN_T, and ZAYAN, and running a minimal classification workflow.

• requirements.txt - Python dependencies required to run the ZAYAN package and notebooks.
• ZAYAN_Architecture.png - High-level architecture diagram of the ZAYAN framework.
• LICENSE - MIT license for this repository.
• README.md - Project overview, installation, usage instructions, and citation information.
• .gitignore - Standard Git ignore rules for Python and Jupyter projects.
• pyproject.toml - Build system and packaging metadata for installation and PyPI upload.
• setup.cfg - Optional package configuration and installation metadata, if used alongside pyproject.toml.

• Python 3.10.13
• torch 2.0.0+
• numpy 1.23.0+
• pandas 1.5.0+
• scikit-learn 1.2.0+
• matplotlib 3.7.0+
• optuna 3.6.0+
• jupyterlab 4.0.0+

You can install ZAYAN in several ways depending on your workflow.

git clone https://github.com/zadid6pretam/ZAYAN.git
cd ZAYAN
pip install -r requirements.txt
pip install -e .

pip install "git+https://github.com/zadid6pretam/ZAYAN.git"

python -m venv zayan-env
source zayan-env/bin/activate # On Windows: zayan-env\Scripts\activate
git clone https://github.com/zadid6pretam/ZAYAN.git
cd ZAYAN
pip install -r requirements.txt
pip install -e .

git clone https://github.com/zadid6pretam/ZAYAN.git
cd ZAYAN
pip install -r requirements.txt
pip install .

pip install zayan

Below is a minimal example showing how to train ZAYAN on a dummy tabular classification dataset using contrastive pretraining followed by Transformer-based supervised classification.

import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from zayan import ZAYAN
# Dummy tabular classification data
np.random.seed(42)
X, y = make_classification(
 n_samples=300,
 n_features=40,
 n_informative=15,
 n_redundant=10,
 n_classes=3,
 random_state=42
)
X = X.astype(np.float32)
y = y.astype(np.int64)
# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
 X,
 y,
 test_size=0.2,
 random_state=42,
 stratify=y
)
# Standardize features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train).astype(np.float32)
X_test = scaler.transform(X_test).astype(np.float32)
# ZAYAN-CL hyperparameters
cl_params = {
 "emb_dim": 128,
 "hidden_dim": 256,
 "tau": 0.1,
 "lambd": 1.0,
 "sigma": 0.1,
 "mask_prob": 0.1,
 "dropout": 0.1,
}
# ZAYAN-T hyperparameters
t_params = {
 "nhead": 4,
 "num_layers": 2,
 "gamma": 1.0,
 "dropout": 0.1,
}
# Initialize ZAYAN
model = ZAYAN(
 cl_params=cl_params,
 t_params=t_params,
 classification_type="multiclass",
 num_classes=len(np.unique(y_train)),
 device="cpu" # use "cuda" if a GPU is available
)
# Fit ZAYAN
results = model.fit(
 X_train,
 y_train,
 X_test=X_test,
 y_test=y_test,
 cl_epochs=50,
 cl_lr=1e-3,
 cl_weight_decay=1e-4,
 t_epochs=30,
 t_lr=1e-4,
 t_weight_decay=1e-4,
 batch_size=32
)
print("Evaluation results:", results)

• The returned results dictionary contains classification metrics. For multiclass classification, ZAYAN reports:

{
 "test": {
 "accuracy": ...,
 "macro_precision": ...,
 "macro_recall": ...,
 "macro_f1": ...
 }
}

• For binary classification, set:

classification_type="binary"
num_classes=2

• and the model reports accuracy, precision, recall, and F1-score.

• ZAYAN_Experiment.ipynb

This notebook contains the Urban Land Cover experiment from the ZAYAN workflow, including Optuna-based hyperparameter tuning, training diagnostics, evaluation metrics, and analysis plots.

ZAYAN is part of our broader research and collaboration portfolio on tabular deep learning, feature-centric representation learning, and structured data modeling. However, ZAYAN is not part of my PhD dissertation work; it is a separate collaborative project focused on tabular remote sensing and environmental data. My PhD research primarily focuses on high-dimensional tabular learning, feature ordering, HDLSS modeling, and tabular generative frameworks.

Our recent ICML 2026 Regular main conference paper on feature ordering and compression for tabular foundation models for high-dimensional low-sample-size tabular data:

• GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data

• GitHub: https://github.com/zadid6pretam/GOTabPFN

• Find it on ICML portal: https://icml.cc/virtual/2026/poster/62523

• Project Webpage: https://www.zadidhabib.com/gotabpfn.html

• OpenReview: https://openreview.net/forum?id=fpqfV3lCIB

• Hugging Face Space: ZeroGPU Live Demo (recommended; faster GPU-backed testing) | CPU Backup Demo (use if ZeroGPU is unavailable) | ZeroGPU Space Repository | CPU Backup Space Repository

@inproceedings{habib2026gotabpfn,
 title = {GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna Kumar and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
 year = {2026}
}

Our neural synchrony-based cross-modal feature sequencing framework for multimodal learning with image and tabular data. iSyncTab addresses the image–tabular integration problem by aligning and sequencing cross-modal feature groups before structured multimodal representation learning.

• iSyncTab: Learning Cross-Modal Feature Sequencing for Image-Tabular Data via Neural Synchrony
• Accepted at the European Conference on Computer Vision (ECCV 2026)
• GitHub: https://github.com/zadid6pretam/iSyncTab
• Project Page: https://www.zadidhabib.com/isynctab.html (Under Construction)
• ECCV Page: https://eccv.ecva.net/virtual/2026/poster/5442
• Paper: https://doi.org/10.1007/978-3-032-37035-8 (In Press)
• Hugging Face: https://huggingface.co/zadid6pretam/iSyncTab-HAM10000 (Model weights and checkpoint for the HAM10000 dataset)

@inproceedings{habib2026isynctab,
 title = {iSyncTab: Learning Cross-Modal Feature Sequencing for Image-Tabular Data via Neural Synchrony},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the European Conference on Computer Vision},
 year = {2026}
}

• If you are interested in cross-modal feature sequencing, neural synchrony-guided image–tabular integration, and order-aware multimodal representation learning, please refer to the iSyncTab repository, project page, and paper.

Our generative modeling framework for high-dimensional low-sample-size tabular data:

• BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation

• GitHub: https://github.com/zadid6pretam/BSTabDiff

• OpenReview: https://openreview.net/forum?id=RKNDy0KhGT

@inproceedings{habib2026bstabdiff,
 title = {BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna Kumar and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {ICLR 2026 2nd Workshop on Deep Generative Models in Machine Learning: Theory, Principle and Efficacy (DeLTa)},
 year = {2026}
}

• If you are interested in high-dimensional tabular synthesis, block-subunit generation, and diffusion/flow priors for HDLSS tabular data, please also refer to the BSTabDiff repository and paper.

Our structured feature sequencing framework for multimodal learning with image and tabular data. This work involves feature sequencing or ordering for multimodal image-tabular representation learning.

• iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data

• GitHub: https://github.com/zadid6pretam/iStructTab

• Paper: https://link.springer.com/chapter/10.1007/978-3-032-31404-8_43

• arXiv: https://arxiv.org/abs/2608.04348

@inproceedings{habib2026istructtab,
 title = {iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data},
 author = {Habib, Al Zadid Sultan Bin and Ahamed, Md Younus and Gyawali, Prashnna and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = {Proceedings of the 28th International Conference on Pattern Recognition},
 year = {2026},
 address = {Lyon, France}
}

• If you are interested in structured feature sequencing, multimodal fusion of image and tabular data (the integration problem), and feature order-aware tabular representation learning, please also refer to the iStructTab repository and paper.

One of our older works on learned feature ordering for high-dimensional tabular data:

• DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data

• GitHub: https://github.com/zadid6pretam/DynaTab

• Paper Link: https://proceedings.mlr.press/v308/habib26a.html

Bibtex:

@InProceedings{dynatab,
 title = 	 {{DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data}},
 author = {Habib, Al Zadid Sultan Bin and Doretto, Gianfranco and Adjeroh, Donald A.},
 booktitle = 	 {{Proceedings of the First Workshop on NeuroAI Multimodal Intelligence @ AAAI 2026}},
 pages = {27--57},
 year = {2026},
 volume = {308},
 series = {{Proceedings of Machine Learning Research}},
 publisher = {PMLR},
 url = 	 {https://proceedings.mlr.press/v308/habib26a.html}
}

• If you are interested in learned feature ordering, neural rewiring for high-dimensional tabular data, and sequential backbone design for HDLSS settings, please also refer to the benchmark study in DynaTab repository and paper.

Our earlier work on sequential modeling for tabular data:

• TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering

• GitHub: https://github.com/zadid6pretam/TabSeq

• Springer ICPR 2024 proceedings: https://link.springer.com/chapter/10.1007/978-3-031-78128-5_27

• arXiv: https://arxiv.org/abs/2410.13203

@inproceedings{habib2024tabseq,
 title={TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering},
 author={Habib, Al Zadid Sultan Bin and Wang, Kesheng and Hartley, Mary-Anne and Doretto, Gianfranco and A. Adjeroh, Donald},
 booktitle={International Conference on Pattern Recognition},
 pages={418--434},
 year={2024},
 organization={Springer}
}

• If you are interested in sequential feature ordering for tabular data, deep sequential backbones, and early feature ordering-based tabular modeling, please also refer to the TabSeq repository and paper.

This repository corresponds to our separate collaborative work on tabular remote sensing and environmental data:

• ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data
• GitHub: https://github.com/zadid6pretam/ZAYAN

@inproceedings{habib2026zayan,
 title = {ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data},
 author = {Habib, Al Zadid Sultan Bin and Tasnim, Tanpia and Islam, Md. Ekramul and Tabasum, Muntasir},
 booktitle = {Proceedings of the 28th International Conference on Pattern Recognition},
 year = {2026},
 address = {Lyon, France}
}

• ZAYAN focuses on feature-level contrastive learning and Transformer-based classification for tabular remote sensing and environmental datasets.
• Unlike my PhD dissertation projects on high-dimensional tabular learning and HDLSS modeling, ZAYAN was developed as a separate collaboration.

For any questions, issues, or suggestions related to this repository, please feel free to contact us or open an issue on GitHub.