Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization
Abstract
Deep learning problems rarely involve objectives that are equal in importance. A primary objective defines the goal, whilst secondary objectives, such as sparsity, compression, or robustness constrain the solution. While existing multi-objective methods have proven effective in practice, they have a clear symmetry problem and neglect the inherent objective hierarchy built into these objective spaces. We introduce Priority-Constrained Descent (PCD), a gradient-based optimization framework designed to explicitly exploit hierarchical objective structures. PCD preserves the direction of primary descent whilst allowing for the minimal distortion necessary to guarantee progress on secondary objectives, controlled by a single τ∈ [0, 1] that dictates the strength of the distortion. The resulting formulation is invariant to objective scaling and admits exact closed-form solutions for problems with two and three objectives. We evaluate PCD within structured network compression settings, unstructured sparsity and low-rankness, and across a variety of synthetic experiments, showing Pareto dominance and better per-objective performance with secondary progress guarantees over existing methods, further exhibiting the interpretable trade-off that τ provides.
Related
- Portfolio-Based Constrained Multi-Objective Bayesian Optimization for Materials Design
- Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates
- Learn Feasibility Once, Optimize All Objectives: Derivative-Free Diffusion Models for Chance-Constrained Programming
- Automated Priority Rule Design for the Resource-Constrained Project Scheduling Problem: A Large Language Model-Guided Population-Based Search
- Sustainable portfolio optimization with four objectives: Assessing the role of diversification
BibTeX
@misc{varam2026objectives,
title = {{Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization}},
author = {Dara Varam and Mohamed I. Alhajri},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.29521},
eprint = {2606.29521},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.29521},
}
RIS
TY - UNPB
TI - Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization
AU - Dara Varam
AU - Mohamed I. Alhajri
PY - 2026
DA - 2026/09/21
PB - arXiv
DO - 10.48550/arxiv.2606.29521
UR - https://arxiv.org/abs/2606.29521
AB - Deep learning problems rarely involve objectives that are equal in importance. A primary objective defines the goal, whilst secondary objectives, such as sparsity, compression, or robustness constrain the solution. While existing multi-objective methods have proven effective in practice, they have a clear symmetry problem and neglect the inherent objective hierarchy built into these objective spaces. We introduce Priority-Constrained Descent (PCD), a gradient-based optimization framework designed to explicitly exploit hierarchical objective structures. PCD preserves the direction of primary descent whilst allowing for the minimal distortion necessary to guarantee progress on secondary objectives, controlled by a single τ∈ [0, 1] that dictates the strength of the distortion. The resulting formulation is invariant to objective scaling and admits exact closed-form solutions for problems with two and three objectives. We evaluate PCD within structured network compression settings, unstructured sparsity and low-rankness, and across a variety of synthetic experiments, showing Pareto dominance and better per-objective performance with secondary progress guarantees over existing methods, further exhibiting the interpretable trade-off that τ provides.
ER -