Conditioned Direct Feedback Alignment via Activity and Error Geometry
Abstract
Direct feedback alignment (DFA) trains hidden layers with fixed random projections of the output error. Even when this feedback provides useful credit, unequal scales across activity or error directions can distort the local update. We study conditioned DFA (nDFA), a family that adapts established inverse-moment preconditioning to either side of this update. An aligned linear analysis describes how activity conditioning changes spectral learning rates and early-stopping risk. Synthetic experiments show gains from both stabilization and changes in update direction. Confirmatory CIFAR-10 experiments show that activity conditioning, including its FOOF formulation, improves tuned DFA at matched measured training time. Controls support a role for centered correlations beyond the tested mean and diagonal alternatives, while early-only conditioning retains most of the benefit with less training work. Conditioning also benefits backpropagation, and timing interventions do not establish a mechanism specific to random feedback. Error conditioning improves short-budget models and changes update directions, but its small additional improvement under stable training does not survive correction for multiple comparisons. Predicted benefits on image-background benchmarks are not confirmed. These findings establish practical benefits and important limits of conditioning learning with fixed random feedback.
Related
- Protein Language Model-Conditioned Graph Neural Networks for Multitask GPCR Ligand Activity Prediction
- Set mediates chromosome alignment and Cohesin cleavage independently of direct PP2A-B56-binding in oocyte meiosis
- Direct cortical feedforward and feedback pathways differentially shape visual responses and selectively engage looped circuits
- Cargo levels control endosomal PI(3)P homeostasis through direct feedback between SNX17 and MTM1
- Neural-DISCO: Source-Conditioned Counterfactual Editing of Neural Population Activity
BibTeX
@misc{safaai2026conditioned,
title = {{Conditioned Direct Feedback Alignment via Activity and Error Geometry}},
author = {Houman Safaai and Varun Reddy and Bernardo L. Sabatini},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.18574},
eprint = {2607.18574},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.18574},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Conditioned Direct Feedback Alignment via Activity and Error Geometry
AU - Houman Safaai
AU - Varun Reddy
AU - Bernardo L. Sabatini
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2607.18574
UR - https://arxiv.org/abs/2607.18574
AB - Direct feedback alignment (DFA) trains hidden layers with fixed random projections of the output error. Even when this feedback provides useful credit, unequal scales across activity or error directions can distort the local update. We study conditioned DFA (nDFA), a family that adapts established inverse-moment preconditioning to either side of this update. An aligned linear analysis describes how activity conditioning changes spectral learning rates and early-stopping risk. Synthetic experiments show gains from both stabilization and changes in update direction. Confirmatory CIFAR-10 experiments show that activity conditioning, including its FOOF formulation, improves tuned DFA at matched measured training time. Controls support a role for centered correlations beyond the tested mean and diagonal alternatives, while early-only conditioning retains most of the benefit with less training work. Conditioning also benefits backpropagation, and timing interventions do not establish a mechanism specific to random feedback. Error conditioning improves short-budget models and changes update directions, but its small additional improvement under stable training does not survive correction for multiple comparisons. Predicted benefits on image-background benchmarks are not confirmed. These findings establish practical benefits and important limits of conditioning learning with fixed random feedback.
ER -