Cost-Effective Deferral Policies in Multi-Model Cascade
发布者：梁慧丽发布时间：2026-10-08浏览次数：10
郑莹
法国国家信息与自动化研究所（INRIA）博士后
2026年10月8日 星期四
中午 12:00-13:00
Deep neural networks (DNNs) have achieved remarkable performance across a wide range of real-world tasks. As modern DNNs become increasingly large and complex to tackle more challenging tasks, their inference costs, such as energy consumption, also grow substantially. Model cascade provides a promising solution by organizing multiple DNNs in increasing order of inference cost. With an appropriate deferral policy, easy samples can be handled by smaller models, while more difficult samples are deferred to larger and more powerful models, thereby significantly reducing the overall inference cost. However, most existing deferral policies lack a unified theoretical framework and a principled characterization of the optimal policy for multi-model cascade, often resulting in suboptimal performance. We bridge this gap by using optimal stopping theory as the methodological foundation for multi-model cascade and develop a new approach to approximate the optimal deferral policy. We train the deferral policy at each stage in a backward manner,enabling each deferral decision to account for the outcomes of downstream stages. Experimental results demonstrate the effectiveness of our proposed approach.郑莹，法国国家信息与自动化研究所（INRIA）博士后，由Dr. Giovanni Neglia和Dr. Chuan Xu联合指导。郑莹于2024年在复旦大学计算机系取得博士学位，导师是王新教授和徐跃东教授。她的研究内容是人工智能，优化，和计算机网络的交叉领域。目前她关注Efficient AI，从算法的角度设计策略，在保证大模型推理准确率的同时降低推理过程的开销，以此促进AI服务的普及与环境可持续性。她的成果发表于INFOCOM, JSAC, ToN, ICPP等国际会议和期刊上。
