When to Reason: Semantic Router for vLLM
- Chen Wang
- Xunzhuo Liu
- et al.
- 2025
- NeurIPS 2025
Workshop paper
Publications
8 results for Yue Zhu
Networking for Stateful LLM Inference
- Junchen Jiang
- Yuhan Liu
- et al.
- 2025
- SIGCOMM 2025
TutorialTowards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
- Yue Zhu
- Hao Yu
- et al.
- 2025
- CLOUD 2025
Short paperScalable and Efficient LLM Serving with the vLLM Production Stack
- Junchen Jiang
- Yue Zhu
- 2025
- OSSNA 2025
TalkHow Low Can LoRA Go: System-Level Throughput, Energy, and Model Quality Tradeoffs when Fine-Tuning Adapters
- Connor Espenshade
- Umesh Deshpande
- et al.
- 2025
- ISCA 2025
Workshop paperOptimizing GPU Multiplexing for Efficient and Cost-Effective Access to Diverse Large Language Models in GPU Clusters
- Yue Zhu
- Chen Wang
- et al.
- 2024
- MASCOTS 2024
Conference paperGPU OPTIMIZATIONS FOR EFFICIENT AND COST-EFFECTIVE ACCESS TO DIVERSE LARGE LANGUAGE MODELS IN RESEARCH CLUSTER
- Chen Wang
- Yue Zhu
- et al.
- 2024
- MLSys 2024
PosterTowards Pareto Optimal Throughput in Small Language Model Serving
- Pol G. Recasens
- Yue Zhu
- et al.
- 2024
- EuroSys 2024
Workshop paper