Skip to content

Fix ep qps #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 198 commits into
base: main
Choose a base branch
from
Open

Fix ep qps #3

wants to merge 198 commits into from

Conversation

whybeyoung
Copy link

Motivation

Fix deepseek-ai/DeepEP#134

BearBiscuit05 and others added 30 commits April 16, 2025 19:56
@BearBiscuit05 @zhaochenyang20
[verl] Modify the update_weights func to align with verl's resharding (…
90faf90 
…sgl-project#5345)

Co-authored-by: Chayenne <zhaochen20@outlook.com>
@yyihuang @yhyang201
@hnyls2002 @zhaochenyang20 @zhyncs
[Model Support] unsloth/Phi-4-mini bnb model (sgl-project#4982)
8f783c1 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@didier-durand
Update attention_backend.md: plural form (sgl-project#5489)
92d1561
@Fridge003
Add test for flash_attn_varlen_func kernel (sgl-project#5484)
81c8911
@Fridge003
Deprecate disable-mla (sgl-project#5481)
4fb0558
@Fridge003
Deprecate enable-flashinfer-mla and enable-flashmla (sgl-project#5480)
6fb29ff
@woodx9
Feat/support encoder model (like bert) (sgl-project#4887)
3bface1
@CatherineSue
Enable local attention during decode (sgl-project#5479)
c776234
@fzyzcjy
Refactor DeepSeek decoder layer branches (sgl-project#5205)
8beb356
@windsonsea
Fix a link in sgl-kernel/README.md (sgl-project#5493)
22c2a79
@XucSh
[Bug fix] use correct func path in deepseek (sgl-project#5496)
06d0a3d 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
@minleminzui
Doc: fix problems of the 'Execute Notebooks / run-all-notebooks' ci c…
f13d65a 
…aused by the unstability of deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (sgl-project#5503)
@yinfan98 @zhyncs
[Feat] Update sgl-kernel flashinfer to latest main version (sgl-proje…
c08a717 
…ct#5500)

Co-authored-by: zhyncs <me@zhyncs.com>
@u4lr451
Fix: Incorrect parameters passed to forward_batch_generation (sgl-pro…
211c7b3 
…ject#5506) (sgl-project#5511)
@minleminzui @ocss884
Fix: fix the exception 'the memory capacity is unbalanced. Some GPUs … (
27e9538 
sgl-project#5426)

Co-authored-by: ocss884 <ocss.lin@gmail.com>
@windsonsea @Fridge003
[docs] Fix several consistency issues in sampling_params.md (sgl-proj…
a0fc5bc 
…ect#5373)

Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
@michaelfeil
Configuration qwen2_moe.py - qkv_bias now in transformers (sgl-projec…
1effba4 
…t#5512)
@fzyzcjy
Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 …
53dcf38 
…+ 4x8xH100 (sgl-project#4836)
@BBuf
Sgl kernel fused_moe_gate support n_shared_experts (sgl-project#5440)
8e09b37
@zhyncs
chore: bump sgl-kernel 0.0.9.post2 (sgl-project#5518)
f28d829
@strgrb
use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion…
b2a189d 
… kernel (sgl-project#5473)

Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
@BBuf
fix kimi vl running bug after rebase main (sgl-project#5461)
bed0587
@liwenju0
fix bug of VLLM_AVAILABLE not defined (sgl-project#5497)
e465b08
@Edenzzzz @Fridge003
Avoid computing lse in Ragged Prefill when there's no prefix. (sgl-pr…
bfa3922 
…oject#5476)

Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
@yhyang201
[Model] Adding Qwen3 and Qwen3MoE (sgl-project#4693)
4db463b
@zhyncs
fix util import (sgl-project#5542)
08b518d
@zhyncs
Revert "Avoid computing lse in Ragged Prefill when there's no prefix.… (
a6f892e 
sgl-project#5544)
@zhyncs
chore: upgrade sgl-kernel 0.0.9.post2 (sgl-project#5540)
2c11f9c
@fzyzcjy
Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (s…
1e0806f 
…gl-project#5340)
@fzyzcjy
Make profiler output file names consistent (sgl-project#5548)
f6a7113
congcongke and others added 30 commits April 26, 2025 23:18
@congcongke
add switch to disable open api doc (sgl-project#3744)
0210206 
Signed-off-by: congcongke <zhanweidu@163.com>
@merrymercy
Revert "fix: import vllm_rotary_embedding error when head_size not in…
8ba3133 
… 64, 128, 256, 512" (sgl-project#5772)
@merrymercy
Fix eagle test case (sgl-project#5776)
981a261
@Fridge003
Split local attention test from fa3 test (sgl-project#5774)
a45a4b2
@merrymercy
Revert "Revert "fix: import vllm_rotary_embedding error when head_siz…
6e313c1 
…e not in 64, 128, 256, 512"" (sgl-project#5777)
@merrymercy
Simplify FA3 tests (sgl-project#5779)
4d23ba0
@merrymercy
Revert "[fix] fix bench_one_batch_server" (sgl-project#5785)
005aad3
@merrymercy
Revert "Use device_id in dist init to reduce NCCL communicator warmup…
9c08882 
… & creation overhead" (sgl-project#5786)
@merrymercy
[CI] Tune threshold (sgl-project#5787)
3c4e0ee
@merrymercy
[CI] fix port conflicts (sgl-project#5789)
35ca04d
@merrymercy
[CI] Fix ci tests (sgl-project#5769)
621e96b
@hnyls2002
[PD]Reduce kv transfer threads (sgl-project#5791)
beb65c7
@merrymercy
[CI] Fix test case (sgl-project#5790)
a38f693
@Fridge003 @merrymercy
Add 8-GPU Test for Deepseek-V3 (sgl-project#5691)
f9fb33e 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
@Fridge003
Release v0.4.6 (sgl-project#5795)
84022c0
@merrymercy
Update nightly-test.yml (sgl-project#5797)
ded04b2
@merrymercy
[CI] Improve github summary & enable fa3 for more models (sgl-project…
daed453 
…#5796)
@PopSoda2002 @zhudianGG
[Docs] update grafana setup guide in production metrics (sgl-project#…
86317c0 
…5643)

Co-authored-by: NoahM <88418672+zhudianGG@users.noreply.github.com>
@slin1237
[Misc] add structure logging, write to file and log tracing for SGL R…
f036582 
…outer
@hnyls2002
Improve overlap scheduling (sgl-project#5788)
40d9b8a
@trevor-m
Add Cutlass MLA attention backend (sgl-project#5390)
84810da
@zhyncs
chore: upgrade sgl-kernel 0.1.0 (sgl-project#5690)
41ac0c6
@BBuf
Dockerfile.dev pip scikit_build_core (sgl-project#5807)
ad4df30
@BBuf
Add a doc to fix sgl-kernel build link error in py39 with ccache (sgl…
ef15dcd 
…-project#5809)
@merrymercy
Turn on overlap scheduler for multimodal models (sgl-project#5771)
3029889
@fzyzcjy
Tiny refactor DefaultModelLoader.Source (sgl-project#5482)
644ed40
@windsonsea
[Docs] Replace lists with tables for cleanup and readability in serve…
966eb90 
…r_arguments (sgl-project#5276)
@merrymercy
Revert "Tiny refactor DefaultModelLoader.Source" (sgl-project#5825)
693723d
@whybeyoung
[PD] Fix deepseek-ai/DeepEP#134
b0d1bfd
@whybeyoung
fix lint
12825c7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Assertion failed: /DeepEP/csrc/kernels/internode.cu:483, condition: ibgda_get_state()->num_rc_per_pe >= num_channels