From beb721a78c9f92dfa0102bd666f7e4f4e23aaef3 Mon Sep 17 00:00:00 2001 From: jeffail Date: Fri, 13 Apr 2018 17:09:42 +0100 Subject: [PATCH] Refer back to pipelines in docs --- resources/docs/README.md | 13 ++----------- resources/docs/pipeline.md | 18 +++++++++--------- 2 files changed, 11 insertions(+), 20 deletions(-) diff --git a/resources/docs/README.md b/resources/docs/README.md index 2571756503..3b12aa58fe 100644 --- a/resources/docs/README.md +++ b/resources/docs/README.md @@ -429,17 +429,8 @@ use. If the number of inputs is less than or close to the number of processing threads then it is also important to use a [buffer][buffers] in order to decouple those inputs. -Here are some examples to illustrate: - -``` yaml -TODO: config -``` - -Is a config that creates pipelines like this: - -``` -TODO: diagram -``` +Please refer [to the documentation regarding pipelines][pipeline] for some +examples. [default-conf]: ../../config/everything.yaml [pipeline]: ./pipeline.md diff --git a/resources/docs/pipeline.md b/resources/docs/pipeline.md index 0756e51aaa..42c15dbeec 100644 --- a/resources/docs/pipeline.md +++ b/resources/docs/pipeline.md @@ -9,12 +9,12 @@ output. If you have processors that are heavy on CPU and aren't specific to a certain input or output they are best suited for the pipeline section. It is advantageous to use the pipeline section as it allows you to set an explicit -number of parallel threads of execution which, should ideally match the number -of available logical CPU cores. +number of parallel threads of execution which should ideally match the number of +available logical CPU cores. If [a buffer is chosen][buffers] these processors are applied to messages read -from it. It is therefore possible to use buffers to distribute messages from a -single input across multiple parallel processing threads. +from it. It is therefore possible to use buffers as a way of distributing +messages from a single input across multiple parallel processing threads. The following are some examples of how to get good performance out of your processing pipelines. @@ -59,7 +59,8 @@ output: type: bar ``` -With this config the block diagram of our Benthos instance might look like this: +With this config the pipeline within our Benthos instance would look something +like the following: ``` foo -> memory buffer ---> processor ---> bar @@ -95,9 +96,7 @@ input: inputs: - type: baz buffer: - type: memory - memory: - limit: 5000000 + type: none pipeline: threads: 4 processors: @@ -109,7 +108,8 @@ output: type: bar ``` -With this config the block diagram of our Benthos instance might look like this: +With this config the pipeline within our Benthos instance would look something +like the following: ``` baz -\