Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Included documentation of secondary files #318

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open
Changes from 11 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
81bb17f
added Optional Inputs Parameters
Mackenzie-OO7 Oct 18, 2022
0496edc
Update inputs.md
Mackenzie-OO7 Oct 19, 2022
491c147
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Oct 19, 2022
ce48733
Merge branch 'main' into restructure-content
Mackenzie-OO7 Oct 19, 2022
bdafc41
Merge branch 'main' into restructure-content
Mackenzie-OO7 Oct 19, 2022
5bab0ab
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Oct 20, 2022
636ebc7
Merge branch 'main' into restructure-content
Mackenzie-OO7 Oct 20, 2022
7383dbc
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Oct 20, 2022
142c3c2
updated secondaryFiles parameter
Mackenzie-OO7 Oct 20, 2022
c08106b
Merge branch 'main' into restructure-content
tetron Oct 24, 2022
b074013
Merge branch 'main' into restructure-content
tetron Oct 24, 2022
ecdb21e
Merge branch 'main' into restructure-content
Mackenzie-OO7 Oct 24, 2022
ceab4f2
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Oct 24, 2022
b72b74a
Merge branch 'main' into restructure-content
tetron Oct 28, 2022
961cc8d
Update inputs.md
Mackenzie-OO7 Oct 28, 2022
b97cd46
Merge branch 'main' into restructure-content
Mackenzie-OO7 Nov 21, 2022
96a3d28
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Dec 16, 2022
1833260
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Jan 3, 2023
3f4f552
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Jan 10, 2023
d03dfaf
update documentation of secondaryFiles
Mackenzie-OO7 Jan 13, 2023
e4c3a48
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Jan 24, 2023
b292e30
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Jan 24, 2023
0236149
Merge branch 'main' into restructure-content
Mackenzie-OO7 Jan 25, 2023
f422a50
update secondaryFiles documentation
Mackenzie-OO7 Jan 26, 2023
fd51dc7
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Jan 26, 2023
895aae7
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Jan 31, 2023
6b7e8b7
update secondaryFiles documentation
Mackenzie-OO7 Feb 6, 2023
69909d4
Merge branch 'common-workflow-language:main' into restructure-content
Mackenzie-OO7 Feb 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 64 additions & 22 deletions src/topics/inputs.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ and *null*; complex types are *array* and *record*; in addition there are
special types *File*, *Directory* and *Any*.

The following example demonstrates some input parameters with different
types and appearing on the command line in different ways.
types and how they can appear on the command line.

First, create a file called `inp.cwl`, containing the following:

Expand All @@ -31,7 +31,7 @@ Create a file called `inp-job.yml`:

````{note}
You can use `cwltool` to create a template input object. That saves you from having
to type all the input parameters in a input object file:
to type all the input parameters in an input object file:

```{runcmd} cwltool --make-template inp.cwl
:working-directory: src/_includes/cwl/inputs
Expand All @@ -50,7 +50,7 @@ Next, create a whale.txt using [touch] by typing `touch whale.txt` on the comman
$ touch whale.txt
```

Now invoke `cwltool` with the tool description and the input object on the command line,
Now, invoke `cwltool` with the tool description and the input object on the command line,
using the command `cwltool inp.cwl inp-job.yml`. The following boxed text describes these
two commands and the expected output from the command line:

Expand All @@ -67,8 +67,8 @@ the tools aren't accidentally accessing files that were not explicitly
specified
```

The field `inputBinding` is optional and indicates whether and how the
input parameter should appear on the tool's command line. If
The field `inputBinding` is optional and indicates if and how the
input parameter should appear on the tool's command line. If
`inputBinding` is missing, the parameter does not appear on the command
line. Let's look at each example in detail.

Expand All @@ -80,9 +80,9 @@ example_flag:
prefix: -f
```

Boolean types are treated as a flag. If the input parameter
Boolean types are treated as a flag. If the input parameter
"example_flag" is "true", then `prefix` will be added to the
command line. If false, no flag is added.
command line. If false, no flag is added.

```cwl
example_string:
Expand All @@ -92,9 +92,9 @@ example_string:
prefix: --example-string
```

String types appear on the command line as literal values. The `prefix`
String types appear on the command line as literal values. The `prefix`
is optional, if provided, it appears as a separate argument on the
command line before the parameter . In the example above, this is
command line before the parameter. In the example above, this is
rendered as `--example-string hello`.

```cwl
Expand All @@ -107,9 +107,9 @@ example_int:
```

Integer (and floating point) types appear on the command line with
decimal text representation. When the option `separate` is false (the
decimal text representation. When the option `separate` is false (the
default value is true), the prefix and value are combined into a single
argument. In the example above, this is rendered as `-i42`.
argument. In the example above, this is rendered as `-i42`.


```cwl
Expand All @@ -121,17 +121,17 @@ example_file:
position: 4
```

File types appear on the command line as the path to the file. When the
File types appear on the command line as the path to the file. When the
parameter type ends with a question mark `?` it indicates that the
parameter is optional. In the example above, this is rendered as
`--file=/tmp/random/path/whale.txt`. However, if the "example_file"
parameter is optional. In the example above, this is rendered as
`--file=/tmp/random/path/whale.txt`. However, if the "example_file"
parameter were not provided in the input, nothing would appear on the
command line.

Input files are read-only. If you wish to update an input file, you must
Input files are read-only. If you wish to update an input file, you must
[first copy it to the output directory](staging-input-files.md).

The value of `position` is used to determine where parameter should
The value of `position` is used to determine where the parameter should
appear on the command line. Positions are relative to one another, not
absolute. As a result, positions do not have to be sequential, three
parameters with positions 1, 3, 5 will result in the same command
Expand All @@ -143,13 +143,56 @@ The `baseCommand` field will always appear in the final command line before the

[touch]: http://www.linfo.org/touch.html

### Optional Input Parameters
Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved

Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved
Optional input parameters must include `label` and `secondaryFiles`.
`label` is a short, human-readable description for the parameter.

```cwl
inputs:
example_file:
type: File
label: Use case for label parameter
inputBinding:
position: 1
```

`secondaryFiles` is an optional input parameter that provides a pattern of specifying files
or directories that must be included alongside the primary file.
Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved
The following example demonstrates the `secondaryFiles` input parameter.

```cwl
inputs:
example_file:
type: File
secondaryFiles: [example_file.txt]
Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved
```

Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved
Also, a file object listed in `secondaryFiles` may contain nested `secondaryFiles` as shown below:
Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved

```{code-block} cwl
inputs:
example_file:
type: File
secondaryFiles: [
{
example_file.txt
secondaryFiles: [example_file_2.txt]
}
]
```

Note that secondary files are only valid when `type` has a value `File`, or is an array of `items: File`.
All listed secondary files must be present in the same directory as the primary file,
since an implementation may fail workflow execution if a listed secondary file is not present.

## Array Inputs

It is easy to add arrays of input parameters represented to the command
line. There are two ways to specify an array parameter. First is to provide
`type` field with `type: array` and `items` defining the valid data types
that may appear in the array. Alternatively, brackets `[]` may be added after
the type name to indicate that input parameter is array of that type.
the `type: name` to indicate that the input parameter is an array of that type.
Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved

```{literalinclude} /_includes/cwl/inputs/array-inputs.cwl
:language: cwl
Expand All @@ -163,7 +206,7 @@ the type name to indicate that input parameter is array of that type.
:name: array-inputs-job.yml
```

Now invoke `cwltool` providing the tool description and the input object
Now invoke `cwltool` by providing the tool description and the input object
on the command line:

```{runcmd} cwltool array-inputs.cwl array-inputs-job.yml
Expand Down Expand Up @@ -191,7 +234,7 @@ You can specify arrays of arrays, arrays of records, and other complex types.
## Inclusive and Exclusive Inputs

Sometimes an underlying tool has several arguments that must be provided
together (they are dependent) or several arguments that cannot be provided
together, (they are dependent) or several arguments that cannot be provided
together (they are exclusive). You can use records and type unions to group
parameters together to describe these two conditions.

Expand Down Expand Up @@ -270,7 +313,7 @@ that accepts `null` (i.e. no value provided), or any value from an enum.

Note how the JavaScript expression uses the value of the exclusive input parameter
without taking into consideration a `null` value. If you provide a valid value,
such as “fasta” (one of the values of the enum), your command should execute
such as “fasta” (one of the values of the enum), your command should be executed
successfully:

```{runcmd} cwltool exclusive-parameter-expressions.cwl --file_format fasta
Expand All @@ -286,7 +329,7 @@ output field (a `string`), resulting in failure when running your workflow.
:emphasize-lines: 5-10
```

To correct it, you must remember to use an or operator in your JavaScript expression
To correct it, you must remember to use an **or** operator in your JavaScript expression
when using exclusive parameters, or any parameter that allows `null`. For example,
the expression could be changed to `$(inputs.file_format || 'auto')`, to have
a default value if none was provided in the command line or job input file.
Expand All @@ -298,4 +341,3 @@ a default value if none was provided in the command line or job input file.
% - Several ways of defining inputs/arguments to tools and workflows - https://github.com/common-workflow-language/user_guide/issues/33
% - Using an input output in another input - https://github.com/common-workflow-language/user_guide/issues/90
% - How to use linkMerge - https://github.com/common-workflow-language/user_guide/issues/117 (or maybe move to Advanced?)
% - Secondary files - https://github.com/common-workflow-language/common-workflow-language/issues/270
Mackenzie-OO7 marked this conversation as resolved.
Show resolved Hide resolved