In-context learning for small-sample soil mid-infrared spectroscopy: evaluating TabPFN across an integrated soil-acidity panel
Abstract
Soil acidity diagnosis requires several interdependent laboratory measurements, limiting throughput. Mid-infrared (MIR) spectroscopy can estimate this panel from one scan, but practical use depends on robust calibration. Deep learning has surpassed conventional methods in many data-rich fields; however, adapted deep learning architectures such as convolutional neural networks (CNNs) are data-hungry and can overfit when regional datasets contain only a few hundred labelled soils. Supervised transfer learning and self-supervised learning (SSL) address this scarcity by reusing representations learned from large spectral libraries, whereas TabPFN, a newer deep-learning method designed for small tabular datasets, applies a pretrained in-context prediction rule. We tested whether TabPFN provides a stronger regional calibration strategy than these deep-learning alternatives and standard soil-spectroscopy methods. Partial least squares regression (PLSR), Cubist, 2 CNN architectures trained from scratch (SmallCNN and LargeCNN), supervised transfer, SSL and TabPFN were evaluated for ten acidity-related properties in target-specific cohorts of 418 acidic soils from eastern North America. The Open Soil Spectral Library (OSSL) supported supervised and SSL pretraining and a seven-property large-library benchmark of TabPFN and LargeCNN, with published PCA-Cubist results as an external reference. Regionally, TabPFN achieved the highest mean R² (0.83), followed by Cubist (0.81) and fine-tuned SSL (0.79). It led on nine properties and exceeded PLSR and both from-scratch CNNs on all ten. For the seven regional targets with corresponding or closely related measurements in OSSL, TabPFN and LargeCNN achieved mean R² values of 0.85 and 0.75; on OSSL, the corresponding values were 0.96 and 0.94. TabPFN's advantage over LargeCNN therefore narrowed from 0.11 regionally to 0.02 at library scale, consistent with abundant labels reducing the CNN deficit, while TabPFN remained strongest on six of seven OSSL properties. TabPFN thus offers a sample-efficient route to integrated MIR acidity assessment while retaining strong large-library performance without sample subsampling or spectral compression.
Related
- Near-infrared spectroscopy for rapid and non-destructive assessment of PFAS effects in blue mussels (Mytilus spp.): a proof-of-concept study
- Monitoring a Polyurethane Synthesis by Fiber-Coupled Attenuated Total Reflection Fourier Transform Infrared Spectroscopy and Multivariate Curve Resolution-Alternating Least Squares
- Solvent-Dependent Self-Association of Chiral 2-Acetoxy-2-Phenylacetic Acid Revealed by Infrared and Vibrational Circular Dichroism Spectroscopy
- Gas-Phase Far-Infrared and Rotational Spectroscopy of 1-and 2-Cyanonaphthalene: Experiment and Theory
- Comprehensive assessment of prediction of pH in red meat based on near-infrared spectroscopy (NIRS)
BibTeX
@misc{jouichat2026incontext,
title = {{In-context learning for small-sample soil mid-infrared spectroscopy: evaluating TabPFN across an integrated soil-acidity panel}},
author = {Hamza Jouichat and Lotfi Khiari},
howpublished = {EarthArXiv},
year = {2026},
month = sep,
doi = {10.31223/x55f73},
url = {https://eartharxiv.org/repository/view/15130/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - In-context learning for small-sample soil mid-infrared spectroscopy: evaluating TabPFN across an integrated soil-acidity panel
AU - Hamza Jouichat
AU - Lotfi Khiari
PY - 2026
DA - 2026/09/21
PB - EarthArXiv
DO - 10.31223/x55f73
UR - https://eartharxiv.org/repository/view/15130/
AB - Soil acidity diagnosis requires several interdependent laboratory measurements, limiting throughput. Mid-infrared (MIR) spectroscopy can estimate this panel from one scan, but practical use depends on robust calibration. Deep learning has surpassed conventional methods in many data-rich fields; however, adapted deep learning architectures such as convolutional neural networks (CNNs) are data-hungry and can overfit when regional datasets contain only a few hundred labelled soils. Supervised transfer learning and self-supervised learning (SSL) address this scarcity by reusing representations learned from large spectral libraries, whereas TabPFN, a newer deep-learning method designed for small tabular datasets, applies a pretrained in-context prediction rule. We tested whether TabPFN provides a stronger regional calibration strategy than these deep-learning alternatives and standard soil-spectroscopy methods. Partial least squares regression (PLSR), Cubist, 2 CNN architectures trained from scratch (SmallCNN and LargeCNN), supervised transfer, SSL and TabPFN were evaluated for ten acidity-related properties in target-specific cohorts of 418 acidic soils from eastern North America. The Open Soil Spectral Library (OSSL) supported supervised and SSL pretraining and a seven-property large-library benchmark of TabPFN and LargeCNN, with published PCA-Cubist results as an external reference. Regionally, TabPFN achieved the highest mean R² (0.83), followed by Cubist (0.81) and fine-tuned SSL (0.79). It led on nine properties and exceeded PLSR and both from-scratch CNNs on all ten. For the seven regional targets with corresponding or closely related measurements in OSSL, TabPFN and LargeCNN achieved mean R² values of 0.85 and 0.75; on OSSL, the corresponding values were 0.96 and 0.94. TabPFN's advantage over LargeCNN therefore narrowed from 0.11 regionally to 0.02 at library scale, consistent with abundant labels reducing the CNN deficit, while TabPFN remained strongest on six of seven OSSL properties. TabPFN thus offers a sample-efficient route to integrated MIR acidity assessment while retaining strong large-library performance without sample subsampling or spectral compression.
ER -