Skip to content

Issues: triton-inference-server/server

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

673 Open 3,245 Closed
673 Open 3,245 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

build.py fails during onnxruntime backend installation
#8126 opened Apr 3, 2025 by davidhalascsak
3
All counter metrics reports 0 while the xxx_summary_us_count is not 0
#8125 opened Apr 3, 2025 by chunyanlv
4
[Question] Reloading models and latency spikes
#8117 opened Apr 2, 2025 by msyulia
Incorrect Correlation ID Data Type for Sequence Batching with Warmup Request
#8110 opened Apr 1, 2025 by simonzgx
Vllm backend : GRPC async stream_infer error
#8108 opened Mar 28, 2025 by fsw152
FasterRCNN object detection model config issue
#8107 opened Mar 26, 2025 by TopAgrume
Performance issues on Windows
#8106 opened Mar 26, 2025 by sk-iss-rs
How can I release the GPU memory used by triton_python_backend_stub when using the Python backend?
#8102 opened Mar 25, 2025 by lzcchl
1
Failed to bind model using PyTriton
#8101 opened Mar 25, 2025 by smartnet-club
Clarification on Request Queuing and Dynamic Batching Behavior in Triton Inference Server
#8094 opened Mar 23, 2025 by TanayJoshi2k
Unable to use --model-config-name with an ensemble model?
#8093 opened Mar 23, 2025 by AlJohri
AWS IRSA instead credential propagation.
#8090 opened Mar 22, 2025 by yuriyyurov
2
Readiness probes not working for onnx-tensorrt models
#8085 opened Mar 21, 2025 by samueltrautwein
--no-container-build not work when build with --backend=onnxruntime option
#8084 opened Mar 21, 2025 by JamesPoon
GPU VRAM Leak with Python Backend BLS Requests to ORT Backend
#8083 opened Mar 21, 2025 by WoodieDudy
genai-perf out of bounds error when choices array is null when setting "include_usage": true
#8082 opened Mar 21, 2025 by sre42
Cannot specify target platform/os for compose.py
#8081 opened Mar 21, 2025 by haruyama480
TRITON_AWS_MOUNT_DIRECTORY becomes useless because of the random directory name
#8077 opened Mar 19, 2025 by ShuaiShao93
Tokens streamed out of order from OpenAI compatible frontend
#8073 opened Mar 14, 2025 by njaramish
Suggesting using SavedModelBundleLite to reduce RAM usage by 40% in Tensorflow backend
#8067 opened Mar 13, 2025 by vdel
build triton from source --no-container-build Error
#8066 opened Mar 13, 2025 by ronghuaiyang
1
feature request: distinct prometheus metrics for streamed vs non-streamed requests
#8063 opened Mar 12, 2025 by MadDanWithABox
Building triton server python_backend from source
#8060 opened Mar 10, 2025 by mritunjaysharma394
[feature request] Real-time streaming inference load generation by perf_analyzer
#8059 opened Mar 8, 2025 by vadimkantorov
OpenAI Frontend Batch Support
#8058 opened Mar 7, 2025 by Loc8888
ProTip! Type g i on any issue or pull request to go back to the issue listing page.