You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using the `--from` flag, existing partition data is preserved by default. This means that data will not be recollected for time ranges that have already been collected. To recollect data for a time range, use the `--overwrite` flag.
67
+
54
68
Subsequent collection runs occur chronologically, resuming from the last collection by default, so there are no time gaps while the data is being collected.
Copy file name to clipboardExpand all lines: docs/reference/cli/collect.md
+9-3Lines changed: 9 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ title: tailpipe collect
7
7
[Run collection](/docs/collect/collect).
8
8
9
9
10
-
To improve the first-run experience for collection, Tailpipe will only collect the last 7 days during the [initial collection](/docs/collect/collect#initial-collection) (though you can override this behavior woth the `--from`argument). Subsequent collection runs occur chronologically, resuming from the last collection by default, so there are no time gaps while the data is being collected.
10
+
To improve the first-run experience for collection, Tailpipe will only collect the last 7 days during the [initial collection](/docs/collect/collect#initial-collection) (though you can override this behavior with the `--from`and `--to` arguments). Subsequent collection runs occur chronologically, resuming from the last collection by default, so there are no time gaps while the data is being collected.
11
11
12
12
13
13
@@ -23,7 +23,9 @@ To improve the first-run experience for collection, Tailpipe will only collect t
23
23
| `--compact` | Compact the Parquet files after collection (default true)
24
24
| `--from string` | Collect days newer than a relative or absolute date.
25
25
| `--help` | Help for collect
26
+
| `--overwrite` | Overwrite existing data for the specified time range
26
27
| `--progress` | Show active progress of collection, set to `false` to disable (default `true`)
28
+
| `--to string` | Collect days older than a relative or absolute date (use with `--from` for time ranges)
27
29
28
30
29
31
@@ -53,13 +55,17 @@ Collect all partitions in the `aws_cloudtrail_log` table for the last 45 days.
53
55
tailpipe collect aws_cloudtrail_log --from T-45d
54
56
```
55
57
56
-
<!--
57
58
Collect all partitions in the `aws_cloudtrail_log` between January and June.
0 commit comments