MatrixReward: Reward from Rubric Matrix for Open-Ended Generation
Abstract
Open-ended query generation lacks standard answers, thus necessitating an effective reward mechanism. Pointwise scoring rubrics provide limited information about the relative quality of sample answers under the same prompt; merging multiple rubric judgments into a single score may also mask the differences between these answers. We propose MatrixReward, which constructs rewards from a rollout-by-rubric win-rate matrix obtained by comparing every pair of sampled responses under each rubric. The spread of each matrix column captures how strongly that rubric distinguishes the current rollouts, while correlations between columns reveal rubric repetition; together, these statistics yield data-dependent rubric weights. We combine these weights with the prior weights of rubrics. After column normalization and weighting, the observed per-rubric maxima and minima define positive and negative ideal profiles. Each rollout's distances to these two ideals determine its relative-closeness quality reward. Evaluated using Qwen3-8B on four open-ended query-answering benchmarks, MatrixReward achieves an average score of 63.02, outperforming the strongest baseline by approximately 2.0%. These results support the idea that matrices derived from relative comparisons can be used to construct rewards more reasonably for open-ended generative reinforcement learning.
Related
- A MATLAB Tool for the Stable Generation of Matrix Polynomial Evaluation Schemes with Two-Product Savings
- Optimizing Open-Ended Interview Assessment Using Response-Based Semantic Indices
- Nonamenable subforests of multi‐ended quasi‐pmp graphs
- Open-ended Scientific Discovery with Possibilistic Reasoning
- ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery
BibTeX
@misc{shen2026matrixreward,
title = {{MatrixReward: Reward from Rubric Matrix for Open-Ended Generation}},
author = {Zihan Shen and Qi Liu and Zixuan Yang and Yiqun Chen and Chenglong Zhao and Xiaozhao Wang and Lei He},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.00389},
eprint = {2610.00389},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.00389},
}
RIS
TY - UNPB
TI - MatrixReward: Reward from Rubric Matrix for Open-Ended Generation
AU - Zihan Shen
AU - Qi Liu
AU - Zixuan Yang
AU - Yiqun Chen
AU - Chenglong Zhao
AU - Xiaozhao Wang
AU - Lei He
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.00389
UR - https://arxiv.org/abs/2610.00389
AB - Open-ended query generation lacks standard answers, thus necessitating an effective reward mechanism. Pointwise scoring rubrics provide limited information about the relative quality of sample answers under the same prompt; merging multiple rubric judgments into a single score may also mask the differences between these answers. We propose MatrixReward, which constructs rewards from a rollout-by-rubric win-rate matrix obtained by comparing every pair of sampled responses under each rubric. The spread of each matrix column captures how strongly that rubric distinguishes the current rollouts, while correlations between columns reveal rubric repetition; together, these statistics yield data-dependent rubric weights. We combine these weights with the prior weights of rubrics. After column normalization and weighting, the observed per-rubric maxima and minima define positive and negative ideal profiles. Each rollout's distances to these two ideals determine its relative-closeness quality reward. Evaluated using Qwen3-8B on four open-ended query-answering benchmarks, MatrixReward achieves an average score of 63.02, outperforming the strongest baseline by approximately 2.0%. These results support the idea that matrices derived from relative comparisons can be used to construct rewards more reasonably for open-ended generative reinforcement learning.
ER -