Code

add generation time metrics#613

Code

Merged

andrei-kochin merged 12 commits intoopenvinotoolkit:releases/2024/3openvinotoolkit/openvino.genai:releases/2024/3from pavel-esir:add_perf_counterspavel-esir/openvino.genai:add_perf_countersCopy head branch name to clipboardJul 26, 2024

+744-30

Commits

Commits on Jul 22, 2024

• [Continuous batching] Replace standard max_element call with custom loop for greedy sampling (openvinotoolkit#607)

  Show description for cb100cb
  

  mzegla

  authored and

  pavel-esir

  committed
  cb100cb

  Copy full SHA for cb100cb

• wip

  pavel-esir

  committed
  f0e4190

  Copy full SHA for f0e4190

• add detokenization metric; refactor split to perf_conter & perf_metrics

  pavel-esir

  committed
  7cab496

  Copy full SHA for 7cab496

• refactor structure, add python sample

  pavel-esir

  committed
  bb1113c

  Copy full SHA for bb1113c

• add more preicise durations

  pavel-esir

  committed
  0a8f0d9

  Copy full SHA for 0a8f0d9

Commits on Jul 23, 2024

• add cpp Readme, ensured correct batch processing, add PerfMetrics to Readme

  pavel-esir

  committed
  90320f4

  Copy full SHA for 90320f4

• use MeanStdPair

  pavel-esir

  committed
  aeec730

  Copy full SHA for aeec730

Commits on Jul 26, 2024

• Merge remote-tracking branch 'upstream/releases/2024/3' into add_perf_counters

  pavel-esir

  committed
  c45aed5

  Copy full SHA for c45aed5

• resolve conflicts

  pavel-esir

  committed
  be2fdaf

  Copy full SHA for be2fdaf

• apply comments

  pavel-esir

  committed
  b00bcd8

  Copy full SHA for b00bcd8

• uset getter and cache evaluate results

  pavel-esir

  committed
  60e7188

  Copy full SHA for 60e7188

• update Readme's

  pavel-esir

  committed
  e553ef5

  Copy full SHA for e553ef5