From 64de4b49b0c45f77eb739fc14cd7346805886027 Mon Sep 17 00:00:00 2001 From: Jagadisha V Date: Thu, 24 Jul 2025 20:03:01 +0530 Subject: [PATCH 1/9] Partitions FAQ --- docs/manage/partitions/faq.md | 67 +++++++++++++++++++++++++++++++++ docs/manage/partitions/index.md | 6 +++ sidebars.ts | 1 + 3 files changed, 74 insertions(+) create mode 100644 docs/manage/partitions/faq.md diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md new file mode 100644 index 0000000000..3d63b217e9 --- /dev/null +++ b/docs/manage/partitions/faq.md @@ -0,0 +1,67 @@ +--- +id: faq +title: Partitions FAQ +sidebar_label: Partitions FAQ +description: Answers to frequently asked questions about Sumo Logic Partitions. +--- + +### What is Default Scope? + +For the tier-based pricing model: + +- All partitions in the continuous tier are considered a part of the default scope. +- Decommissioned partitions are also included. +- Partitions in the Infrequent and Frequent tier are not part of the default scope. + +For Flex Customers: + +- The customer can configure at the partition level to include or exclude it in the default scope. +- All partitions which are not excluded explicitly will then be included in the default scope. + +## How does Sumo Logic decide on which partitions to scan? + +1. For any query, the first step is determining the scope of the query. If your query does not explicitly mention the `index/view` clause in the source expression, Sumo Logic will consider all partitions in the default scope. You can override the scope of the query by mentioning the specific `index/view` in the source expression `(_index=partitionA)` or adding other tier partitions in the scope by using `_dataTier` modifier like `_dataTier=Infrequent or _dataTier=All`. +2. Then apply a **[Partition Selection Process](#what-happens-in-the-partition-selection-process)** as mentioned below that helps with the final list of partitions that will scan. + +## What happens in the partition selection process? + +Sumo Logic applies the partition selection process purely based on the routing expressions of the partitions and matches with the search expression of the query. Sumo Logic excludes any partitions that can; the rest are scanned. + +### How does the partition selection process work? + +The system tries to match the source expression of the query with the routing expressions of all the partitions in the scope of the query. + +- If the partition matches the source expression, then all other partitions are filtered out and only the matching partition is considered for scan (for example, source expression: `_collector=service1_all_logs AND _sourceCategory=another_category` and routing expression of a partition: `_collector=service1_all_logs`). +- If the routing expression of a partition and the source expression of the query are not overlapping, that partition will be excluded from the scan (for example, source expression: `_sourceCategory=prod_logs` and routing expression of a partition: `_sourceCategory=staging_logs`). +- The matching logic is filtered with the complex boolean expressions in either the routing expressions or the source expression of the query. +- The system will stumble on the side of caution in case there is ambiguity to guarantee functional correctness of the query results. +- The system does not have any information about the data other than routing expressions for this selection process. +- The number of partitions scanned is directly related to how well a query source expression matches the routing expressions. Therefore, it is recommended to use simpler expressions and preferably, use the same dimensions for partition routing as you would generally use in your queries. + +### What happens when you change the routing expression of partition? + +Sumo Logic maintains a historical record of the routing expressions of the partitions. It also considers the historical routing expressions with the timerange to filter in or out the partitions. + +### Is there a way to find out which partitions are being scanned? + +Currently, there is no way to extract this list for a given query, but Sumo Logic can expose it on demand on your log search page. You can look at the corresponding audit log in the search query audit index to check the number of partitions being scanned in each query. + +### Why are decommissioned partitions being scanned? + +Disabling a partition does not remove the existing data in it. Disabling simply means that we will no longer ingest data into that partition. The data already present in a partition will continue to be served in queries and will still need to be scanned. + +### What happens when you set `_dataTier=All`? + +When you specify `_dataTier=All`, the query will include partitions from both the *infrequent* and *frequent* tiers. The partition selection process will still be applied to minimize the number of partitions that need to be scanned. + +### What happens when you do not include `_dataTier=Continuous`? + +By default, `_dataTier=Continuous` is the scope, and you do not need to specify this explicitly. + +### What happens if you specify an index or view which is not a partition in the query, like a scheduled view, audit index, or security indexes? + +If the query's scope does not include any partitions, the partition selection process is not executed, and all indexes/views within the scope are scanned for the query's time range. + +### Are all the data scanned in the selected partitions? + +No data outside the specified timerange and intervalType (Message Time or Receipt Time) is scanned for the query. diff --git a/docs/manage/partitions/index.md b/docs/manage/partitions/index.md index 2662976b6a..84b04457d1 100644 --- a/docs/manage/partitions/index.md +++ b/docs/manage/partitions/index.md @@ -84,5 +84,11 @@ In this section, we'll introduce the following concepts:

Learn how to decommission a Partition to keep it from being started.

+
+
+ icon

Partitions FAQs

 +

Answers to frequently asked questions about Sumo Logic Partitions.

+
+
diff --git a/sidebars.ts b/sidebars.ts index 3746240a08..ce659de210 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -903,6 +903,7 @@ module.exports = { 'manage/partitions/edit-data-forwarding-destinations-partition', 'manage/partitions/manage-indexes-variable-retention', 'manage/partitions/decommission-partition', + 'manage/partitions/faq', ] }, { From 20213e268ed1f6b9bf830fe28a4ea7f0b9a4a379 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:48:42 +0530 Subject: [PATCH 2/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index 3d63b217e9..7a7a4cac2d 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -5,7 +5,7 @@ sidebar_label: Partitions FAQ description: Answers to frequently asked questions about Sumo Logic Partitions. --- -### What is Default Scope? +### What is the default scope? For the tier-based pricing model: From 1ba659012bed28b394579acc48ce83cc74e4859c Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:48:52 +0530 Subject: [PATCH 3/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index 7a7a4cac2d..4d2dd37885 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -52,7 +52,7 @@ Disabling a partition does not remove the existing data in it. Disabling simply ### What happens when you set `_dataTier=All`? -When you specify `_dataTier=All`, the query will include partitions from both the *infrequent* and *frequent* tiers. The partition selection process will still be applied to minimize the number of partitions that need to be scanned. +When you specify `_dataTier=All`, the query will include partitions from both the *Infrequent* and *Frequent* tiers. The partition selection process will still be applied to minimize the number of partitions that need to be scanned. ### What happens when you do not include `_dataTier=Continuous`? From 21f085366879191112fe33dfad7370548e1eb080 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:48:59 +0530 Subject: [PATCH 4/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index 4d2dd37885..e8e6bc3fa7 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -13,7 +13,7 @@ For the tier-based pricing model: - Decommissioned partitions are also included. - Partitions in the Infrequent and Frequent tier are not part of the default scope. -For Flex Customers: +For Flex customers: - The customer can configure at the partition level to include or exclude it in the default scope. - All partitions which are not excluded explicitly will then be included in the default scope. From 53f5c19f558f72063ab6c5aec55d4dd501de25d5 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:49:07 +0530 Subject: [PATCH 5/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index e8e6bc3fa7..3cdce4a4c9 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -21,7 +21,7 @@ For Flex customers: ## How does Sumo Logic decide on which partitions to scan? 1. For any query, the first step is determining the scope of the query. If your query does not explicitly mention the `index/view` clause in the source expression, Sumo Logic will consider all partitions in the default scope. You can override the scope of the query by mentioning the specific `index/view` in the source expression `(_index=partitionA)` or adding other tier partitions in the scope by using `_dataTier` modifier like `_dataTier=Infrequent or _dataTier=All`. -2. Then apply a **[Partition Selection Process](#what-happens-in-the-partition-selection-process)** as mentioned below that helps with the final list of partitions that will scan. +2. Then apply a **[partition selection process](#what-happens-in-the-partition-selection-process)** as mentioned below that helps with the final list of partitions that will scan. ## What happens in the partition selection process? From 2b82ea190a308a92151074b169c1fdd5d53c9df5 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:49:17 +0530 Subject: [PATCH 6/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index 3cdce4a4c9..25b3f564a4 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -34,7 +34,7 @@ The system tries to match the source expression of the query with the routing ex - If the partition matches the source expression, then all other partitions are filtered out and only the matching partition is considered for scan (for example, source expression: `_collector=service1_all_logs AND _sourceCategory=another_category` and routing expression of a partition: `_collector=service1_all_logs`). - If the routing expression of a partition and the source expression of the query are not overlapping, that partition will be excluded from the scan (for example, source expression: `_sourceCategory=prod_logs` and routing expression of a partition: `_sourceCategory=staging_logs`). - The matching logic is filtered with the complex boolean expressions in either the routing expressions or the source expression of the query. -- The system will stumble on the side of caution in case there is ambiguity to guarantee functional correctness of the query results. +- The system will err on the side of caution in case there is ambiguity to guarantee functional correctness of the query results. - The system does not have any information about the data other than routing expressions for this selection process. - The number of partitions scanned is directly related to how well a query source expression matches the routing expressions. Therefore, it is recommended to use simpler expressions and preferably, use the same dimensions for partition routing as you would generally use in your queries. From 56eb1bbf889ea36a387e2f8f6874edfc8e404975 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:49:23 +0530 Subject: [PATCH 7/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index 25b3f564a4..c4c07bd5b9 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -36,7 +36,7 @@ The system tries to match the source expression of the query with the routing ex - The matching logic is filtered with the complex boolean expressions in either the routing expressions or the source expression of the query. - The system will err on the side of caution in case there is ambiguity to guarantee functional correctness of the query results. - The system does not have any information about the data other than routing expressions for this selection process. -- The number of partitions scanned is directly related to how well a query source expression matches the routing expressions. Therefore, it is recommended to use simpler expressions and preferably, use the same dimensions for partition routing as you would generally use in your queries. +- The number of partitions scanned is directly related to how well a query source expression matches the routing expressions. Therefore, it is recommended to use simpler expressions, and preferably, use the same dimensions for partition routing as you would generally use in your queries. ### What happens when you change the routing expression of partition? From de4068264fffe325b8586504550b3008052fb4d8 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:49:31 +0530 Subject: [PATCH 8/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index c4c07bd5b9..7a9cebf1f7 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -38,9 +38,9 @@ The system tries to match the source expression of the query with the routing ex - The system does not have any information about the data other than routing expressions for this selection process. - The number of partitions scanned is directly related to how well a query source expression matches the routing expressions. Therefore, it is recommended to use simpler expressions, and preferably, use the same dimensions for partition routing as you would generally use in your queries. -### What happens when you change the routing expression of partition? +### What happens when you change the routing expression of a partition? -Sumo Logic maintains a historical record of the routing expressions of the partitions. It also considers the historical routing expressions with the timerange to filter in or out the partitions. +Sumo Logic maintains a historical record of the routing expressions of the partitions. It also considers the historical routing expressions with the timerange to filter the partitions. ### Is there a way to find out which partitions are being scanned? From 61a61e7065f06fe5a57bdbe8da9ef1bc82b3c3a3 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:49:42 +0530 Subject: [PATCH 9/9] Update docs/manage/partitions/faq.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/manage/partitions/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/manage/partitions/faq.md b/docs/manage/partitions/faq.md index 7a9cebf1f7..95469d25a7 100644 --- a/docs/manage/partitions/faq.md +++ b/docs/manage/partitions/faq.md @@ -25,7 +25,7 @@ For Flex customers: ## What happens in the partition selection process? -Sumo Logic applies the partition selection process purely based on the routing expressions of the partitions and matches with the search expression of the query. Sumo Logic excludes any partitions that can; the rest are scanned. +Sumo Logic applies the partition selection process purely based on the routing expressions of the partitions and matches with the search expression of the query. Sumo Logic excludes any partitions that it can; the rest are scanned. ### How does the partition selection process work?