Code

Parallel sampling with threadpool#1252

Code

Open

mzegla wants to merge 13 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino.genai:masterfrom mzegla:parallel_sampling_threadpoolmzegla/openvino.genai:parallel_sampling_threadpoolCopy head branch name to clipboard

+281-134

Commits

Commits on Jan 14, 2025

• extract sampling for single sequence group and call it asynchronously

  Show description for 8b7a92e
  mzegla

  committed
  8b7a92e

  Copy full SHA for 8b7a92e

• fix

  mzegla

  committed
  1ef8716

  Copy full SHA for 1ef8716

Commits on Jan 15, 2025

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  09c894f

  Copy full SHA for 09c894f

• tmp CI verbose

  mzegla

  committed
  77b62e3

  Copy full SHA for 77b62e3

• include internal state update for non-sampled requests

  mzegla

  committed
  013fbf1

  Copy full SHA for 013fbf1

• CI echo prediction for SD

  mzegla

  committed
  030f093

  Copy full SHA for 030f093

• bring back timer comment

  mzegla

  committed
  06bdc78

  Copy full SHA for 06bdc78

Commits on Jan 16, 2025

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  e82f429

  Copy full SHA for e82f429

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  c7ca805

  Copy full SHA for c7ca805

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  6b0e034

  Copy full SHA for 6b0e034

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  4eb8696

  Copy full SHA for 4eb8696

Commits on Jan 17, 2025

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  ce8053c

  Copy full SHA for ce8053c

• Merge branch 'master' into parallel_sampling_threadpool

  mzegla

  authored
  b9034b2

  Copy full SHA for b9034b2