[Fleet] Add documentation for remote ES output integrations syncing #952
Conversation
|
@kilfoyle Could you please review this draft? I eventually found it more logical to add this information to the already existing page for remote ES outputs, as integrations syncing is specific to this use case. |
| As a final step before using the remote {{es}} output, you need to make sure that for any integrations that have been [added to your {{agent}} policy](/reference/ingestion-tools/fleet/add-integration-to-policy.md), the integration assets have been installed on the remote {{es}} cluster. Refer to [Install and uninstall {{agent}} integration assets](/reference/ingestion-tools/fleet/install-uninstall-integration-assets.md) for the steps. | ||
| If you have chosen not to automatically synchronize integrations, you need to make sure that for any integrations that have been [added to your {{agent}} policy](/reference/ingestion-tools/fleet/add-integration-to-policy.md), the integration assets have been installed on the remote {{es}} cluster. Refer to [Install and uninstall {{agent}} integration assets](/reference/ingestion-tools/fleet/install-uninstall-integration-assets.md) for the steps. | ||
|
|
||
| ## Automatic integrations synchronization |
There was a problem hiding this comment.
I can't add it as a suggestion because the backticks get interpolated, but if we update as follows:
## Automatic integrations synchronization
three backtics{applies_to}
stack: ga 9.1
three backticks
The output will give us this nice "Elastic Stack (9.1.0)" tag.
There was a problem hiding this comment.
Great, thanks for letting me know! I left the original Beginning in version 8.12.0 at the top of the page, but that can be changed too if it makes sense.
There was a problem hiding this comment.
Oh, that's a good point. If you don't mind removing the Beginning in version 8.12.0 at the top, I think that would be best.
There was a problem hiding this comment.
Removed in the last commit 🙂
There was a problem hiding this comment.
Thanks a lot @jillguyonnet! I think it looks superb. ⛵
I added just a few super small suggestions.
|
Oops. I didn't notice the build errors. Let me check on those... It seems to me that the errors are unrelated to the changes in this PR so hopefully a rebase can resolve that. 🤞 |
jillguyonnet
force-pushed
the
fleet-add-integrations-sync-doc
branch
from
2ef9160 to
e56e4a3
Compare
|
Thanks a lot @kilfoyle, I implemented your feedback and rebased the branch 🤞 Consistency question: "synchronize" and "sync" seem to be used somewhat interchangeably in this context, do you have any concerns about that? Another question: do we need to add a label, feature flag or something else to target 9.1.0? |
|
@criamico FYI - has the possibility of having to set up CCS on the main cluster discarded? Edit: |
I'm going according to the original plan as I wrote in that comment. This design was finalized for long time so I think it would be really rushed at this point to change it and use something that doesn't offer the same flexibility (ccs seems to only work with indices). |
Nope! No concerns. We try and have a very casual tone in the docs, so "sync" is just fine, and switching between the two is also okay.
I think we're okay here as is since it's indicated clearly that the feature isn't available until 9.1. But I'll come back to this content if the guidance changes around how we should handle post 9.0 content. It's nice to see that green CI check! 🟢 |
|
By the way, a preview of the changes is available here. |
|
We will add CCS in this feature: elastic/kibana#209947 |
| 1. In the remote cluster, open the {{kib}} menu and go to **Stack Management > Remote Clusters**. | ||
| 2. Refer to [Remote clusters](https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters.html) to add your main cluster (where the remote {{es}} output is configured) as a remote cluster. | ||
| 3. Go to **Stack Management > Cross-Cluster Replication**. | ||
| 4. Create a follower index named `fleet-synced-integrations-ccr-<output name>` that replicates the `fleet-synced-integrations` leader index on the main cluster. |
There was a problem hiding this comment.
We could add one more step to Resume replication after adding the follower index.
| 2. In the remote cluster, open the {{kib}} menu and go to **Management > Dev Tools**. | ||
| 3. Run the API request. | ||
| 4. Copy the encoded value of the generated API key. | ||
| 5. Back in the main cluster, paste the value you copied into the output **Remote Kibana API Key** field. |
There was a problem hiding this comment.
| 5. Back in the main cluster, paste the value you copied into the output **Remote Kibana API Key** field. | |
| 5. Back in the main cluster, paste the value you copied into the **Remote Kibana API Key** field. |
|
Hey @juliaElastic 👋 I was just checking some details before merging this and I'm wondering if I left something out: is the user expected to configure an output in the remote cluster as well? It sounds counterintuitive, but seems required for the feature to work? I think this question might be related to https://github.com/elastic/kibana/pull/212371/files#r1975507442. |
No, on the remote cluster only the default elasticsearch output is needed (the host has to match the remote ES output host on the main cluster) |
I see, thanks. Should we add a note for self-managed stacks? These wouldn't necessary have the default output configured or am I mistaken? |
Default output is created in self-managed too: https://github.com/elastic/kibana/blob/f270bd5956ed2cadf391fea323fc965fc2db123a/x-pack/platform/plugins/shared/fleet/server/services/setup.ts#L226 |
Closes elastic/kibana#206246
This PR updates the documentation about remote Elasticsearch outputs to detail the requirements and configuration steps of syncing integrations on the remote cluster.
Preview here
Note that in-product instructions were added in elastic/kibana#211997 (tiny typo fix in elastic/kibana#216149).