October 1, 2026
Project overview
Some anomalous records contain individually plausible values but violate relationships between variables. This project investigates whether interventions in a tabular model’s use of reference examples can expose these inconsistencies more effectively than reconstruction errors alone. It extends the Lusis Chair’s work on NPT-AD and retrieval-augmented anomaly detection [2, 3].
Students will construct self-supervised tasks by masking a variable and predicting it from the remaining columns and an explicit memory of examples. The initial detector will be developed without annotated anomalies. For the same query, they will compare a coherent context with a control context whose target–predictor associations have been broken while preserving marginal values. The resulting differences in internal activations will guide relational steering, with the model’s weights kept fixed. Responses to these interventions will form signatures scored against reference signatures through a nearest-neighbour method.
NPT-AD provides a controlled setting for understanding interactions between features and observations, while a frozen TabPFN is the main inference platform [2, 6, 7]. Extensions may examine TabICLv2 and T-JEPA representations for neighbour selection [8, 21].
The evaluation will combine synthetic relational anomalies and real tabular datasets. It will distinguish clean reference data from contaminated unlabelled data, prevent masked-target leakage, and compare steering with retrieval alone, conditional prediction errors, output contrasts and random interventions. Detection quality, robustness to rare normal patterns, calibration and computational cost will be assessed. Deliverables are a reproducible Python/PyTorch prototype and a research manuscript documenting the ablations, limitations and conditions under which steering helps; an improvement remains a hypothesis to test.
References
[1] Kossen, J., Band, N., Lyle, C., Gomez, A. N., Rainforth, T., & Gal, Y. (2021). Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning. NeurIPS.
[2] Thimonier, H., Popineau, F., Rimmel, A., & Doan, B.-L. (2024). Beyond Individual Input for Deep Anomaly Detection on Tabular Data. ICML, PMLR 235, 48097–48123.
[3] Thimonier, H., Popineau, F., Rimmel, A., & Doan, B.-L. (2024). Retrieval Augmented Deep Anomaly Detection for Tabular Data. CIKM, 2250–2259.
[4] Qiu, C., Pfrommer, T., Kloft, M., Mandt, S., & Rudolph, M. (2021). Neural Transformation Learning for Deep Anomaly Detection Beyond Images. ICML, PMLR 139, 8703–8714.
[5] Shenkar, T., & Wolf, L. (2022). Anomaly Detection for Tabular Data with Internal Contrastive Learning. ICLR.
[6] Hollmann, N., Müller, S., Purucker, L., Krishnakumar, A., Körfer, M., Hoo, S. B., Schirrmeister, R. T., & Hutter, F. (2025). Accurate predictions on small data with a tabular foundation model. Nature, 637, 319–326.
[7] Grinsztajn, L., et al. (2025). TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models. Technical report, arXiv:2511.08667; version 2, February 2026.
[8] Qu, J., Holzmüller, D., Varoquaux, G., & Le Morvan, M. (2026). TabICLv2: A better, faster, scalable, and open tabular foundation model. ICML; arXiv:2602.11139, version 2, September 2026.
[9] Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., & Turner, A. (2024). Steering Llama 2 via Contrastive Activation Addition. ACL, 15504–15522.
[10] Postmus, J., & Abreu, S. (2024). Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering. MINT workshop, NeurIPS 2024; arXiv:2410.16314, version 4, May 2025.
[11] Gupta, A., Kumar, D., Mandal, M., & Deshpande, S. (2026). Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function. Foundation Models for Structured Data workshop, ICML 2026; arXiv:2606.12917.
[12] Heimersheim, S., & Nanda, N. (2024). How to use and interpret activation patching. Preprint, arXiv:2404.15255.
[13] Marszałek, P., Kuśmierczyk, T., & Śmieja, M. (2026). TACTIC for Navigating the Unknown: Tabular Anomaly deteCTion via In-Context inference. Preprint, arXiv:2603.14171.
[14] Wei, J. Y., & Armanfard, N. (2026). ICLAD: In-Context Learning for Unified Tabular Anomaly Detection Across Supervision Regimes. Preprint, arXiv:2603.19497.
[15] Lu, S., Liu, J., Peters, S., Le, T. D., Xie, C., Liu, L., & Li, J. (2026). uLEAD-TabPFN: Uncertainty-aware Dependency-based Anomaly Detection with TabPFN. Preprint, arXiv:2604.20255.
[16] Han, S., Hu, X., Huang, H., Jiang, M., & Zhao, Y. (2022). ADBench: Anomaly Detection Benchmark. NeurIPS, Datasets and Benchmarks.
[17] Bates, S., Candès, E., Lei, L., Romano, Y., & Sesia, M. (2023). Testing for Outliers with Conformal p-values. Annals of Statistics, 51(1), 149–178.
[18] Ye, H.-J., Liu, S.-Y., & Chao, W.-L. (2025). A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities. Preprint, arXiv:2502.17361, version 2, June 2025.
[19] Prior Labs. TabPFN Extensions, including the unsupervised module. Official software repository; cited in the project brief as an experimental extension.
[20] Thimonier, H., et al. NPT-AD. Official software repository accompanying reference [2].
[21] Thimonier, H., De Melo Costa, J. L., Popineau, F., Rimmel, A., & Doan, B.-L. (2025). T-JEPA: Augmentation-Free Self-Supervised Learning for Tabular Data. ICLR.