Added index for certificates by subject

Author: Artemkaaas

.github/workflows/verify.yml

@@ -125,7 +125,7 @@ jobs:
             ${{ runner.os }}-go-
       - name: Run integration tests
         shell: bash
-        run: integration_tests/run-all.sh upgrade
+        run: integration_tests/run-all.sh
   lint:
     if: needs.changes.outputs.go == 'true'
     name: Check linter issues with golangci-lint tool

README-DEV.md

@@ -125,15 +125,15 @@ Start a local pool as described above, and then just execute
 dcld
 ```
 
-Have a look at [How To](docs/how-to.md) and [transactions](docs/transactions/transactions.md) for instructions how to configure and use the CLI.
+Have a look at [How To](docs/how-to.md) and [transactions](docs/transactions.md) for instructions how to configure and use the CLI.
 
 ## REST
 
 Start a local pool as described above.
 
 Every node exposes a REST API at `http://<node-host>:1317` (see <https://docs.cosmos.network/v0.45/core/grpc_rest.html>).
 
-Have a look at [transactions](docs/transactions/transactions.md) for a full list of REST endpoints.
+Have a look at [transactions](docs/transactions.md) for a full list of REST endpoints.
 
 ## Contributing
 

README.md

@@ -22,8 +22,8 @@ DC Ledger is based on [CometBFT](https://cometbft.com/) and [Cosmos SDK](https:/
 
 DC Ledger is a public permissioned ledger in the following sense:
 
-- Anyone can read from the ledger (that's why it's public). See [How to read from the Ledger](docs/transactions/transactions.md#how-to-read-from-the-ledger).
-- Writes to the ledger are permissioned. See [How to write to the Ledger](docs/transactions/transactions.md#how-to-write-to-the-ledger) for details.
+- Anyone can read from the ledger (that's why it's public). See [How to read from the Ledger](docs/transactions.md#how-to-read-from-the-ledger).
+- Writes to the ledger are permissioned. See [How to write to the Ledger](docs/transactions.md#how-to-write-to-the-ledger) for details.
 - PoA (proof of authority) approach is used for adding new validator nodes to the network
  (see [Add New Node Use Case](docs/use_cases/use_cases_add_validator_node.png)) and
   [Running Node](docs/running-node.md).
@@ -117,7 +117,7 @@ See [Run local pool](README-DEV.md#run-local-pool) section in [README-DEV.md](RE
 ### CLI
 
 - The same `dcld` binary as a Node
-- A full list of all CLI commands can be found there: [transactions.md](docs/transactions/transactions.md).
+- A full list of all CLI commands can be found there: [transactions.md](docs/transactions.md).
 - CLI can be used for write and read requests.
 - Please configure the CLI before using (see [how-to.md](docs/how-to.md#cli-configuration)).
 - **If there are no trusted Observer or Validator nodes to connect a CLI, then a [Light Client Proxy](#light-client-proxy) can be used.**
@@ -140,20 +140,20 @@ See [Run Light Client Proxy](docs/running-light-client-proxy.md) for details how
 - **There are no state proofs in REST, so REST queries should be sent to trusted Validator or Observer nodes only.**
 - OpenAPI specification: <https://zigbee-alliance.github.io/distributed-compliance-ledger/>.
 - Any running node exposes a REST API at port `1317`. See <https://docs.cosmos.network/v0.47/learn/advanced/grpc_rest>.
-- See [transactions](docs/transactions/transactions.md) for a full list of endpoints.
+- See [transactions](docs/transactions.md) for a full list of endpoints.
 - REST HTTP(S) queries can be directly used for read requests.
-  See [How to read from the Ledger](docs/transactions/transactions.md#how-to-read-from-the-ledger).
+  See [How to read from the Ledger](docs/transactions.md#how-to-read-from-the-ledger).
 - REST HTTP(S) queries can be directly used to broadcast generated and signed transaction.
 - Generation and signing of transactions need to be done in code or via CLI.
-  See [How to write to the Ledger](docs/transactions/transactions.md#how-to-write-to-the-ledger).
+  See [How to write to the Ledger](docs/transactions.md#how-to-write-to-the-ledger).
 
 ### gRPC
 
 - **There are no state proofs in gRPC, so gRPC queries should be sent to trusted Validator or Observer nodes only.**
 - Any running node exposes a REST API at port `9090`. See <https://docs.cosmos.network/v0.47/learn/advanced/grpc_rest>.
 - A client code can be generated for all popular languages from the proto files [proto](proto), see <https://grpc.io/docs/languages/>.
 - The generated client code can be used for read and write requests, i.e. generation and signing of transactions
-  See [How to read from the Ledger](docs/transactions/transactions.md#how-to-read-from-the-ledger) and [How to write to the Ledger](docs/transactions/transactions.md#how-to-write-to-the-ledger) for details.
+  See [How to read from the Ledger](docs/transactions.md#how-to-read-from-the-ledger) and [How to write to the Ledger](docs/transactions.md#how-to-write-to-the-ledger) for details.
 
 ### CometBFT RPC and Light Client
 
@@ -209,7 +209,7 @@ the following instructions from [how-to.md](docs/how-to.md) can be used for ever
 
 - [OpenAPI specification](https://zigbee-alliance.github.io/distributed-compliance-ledger/)
 - [Quick Start](docs/quickStartGuide.adoc)
-- [List of Transactions, Queries, CLI command, REST API](docs/transactions/transactions.md)
+- [List of Transactions, Queries, CLI command, REST API](docs/transactions.md)
 - [How To Guide](docs/how-to.md)
 - [Use Case Diagrams](docs/use_cases)
   - [PKI](docs/use_cases/use_cases_pki.png)

docs/design/noc-root-cert-design.md

@@ -96,7 +96,7 @@ Retrieve NOC root certificates associated with a specific VID.
 - CLI Command:
   - `dcld query pki get_noc_x509_root_certs --vid=<uint16>`
 - REST API:
-  - GET `/dcl/pki/noc-root-certificates/{vid}`
+  - GET `/dcl/pki/noc-vid-root-certificates/{vid}`
 
 ### GET_ALL_NOC_X509_ROOT_CERTS
 

docs/how-to.md

@@ -6,7 +6,7 @@ This document contains tutorials demonstrating how to accomplish common tasks us
 - If write requests to the Ledger needs to be sent, please make sure that you have
 an Account created on the Ledger with an appropriate role (see [Getting Account](#getting-account) section in [how-to.md](how-to.md)).
 - Sending read requests to the Ledger doesn't require an Account (Ledger is public for reads).
-- A full list of all CLI commands with all parameters can be found there: [transactions](transactions/transactions.md).
+- A full list of all CLI commands with all parameters can be found there: [transactions](transactions.md).
 - After the CLI is configured and Account with an appropriate role is created,
 the following instructions can be used for every role (see [Use Case Diagrams](use_cases)):
   - [Trustee](#trustee-instructions)

docs/pool-upgrade-how-to.md

@@ -60,7 +60,7 @@ One of the trustees proposes the upgrade using the following steps:
       ```
       Please note, that it must be called against the `dcld` binary, not the platform archive itself. So, for Ubuntu, either take a `dcld` binary from the root folder of the release, or extract it from ` dcld.ubuntu.tar.gz`.
 
-   2. Sends [`ProposeUpgrade`](transactions/transactions.md#propose_upgrade) transaction
+   2. Sends [`ProposeUpgrade`](transactions.md#propose_upgrade) transaction
       with the name of the new upgrade handler, the chosen ledger height and the
       info containing URLs of the new application version binaries for supported
       platforms with the calculated checksums.
@@ -92,7 +92,7 @@ Each of them uses the following steps to accomplish this:
          ```
        Please note, that it must be called against the `dcld` binary, not the platform archive itself. So, for Ubuntu, either take a `dcld` binary from the root folder of the release, or extract it from ` dcld.ubuntu.tar.gz`.
 
-2. Sends [`ApproveUpgrade`](transactions/transactions.md#approve_upgrade) transaction
+2. Sends [`ApproveUpgrade`](transactions.md#approve_upgrade) transaction
     with the name of the proposed upgrade.
 
     For example:

docs/pool-upgrade.md

@@ -27,10 +27,10 @@ Any upgrade plan has the following fields:
 ## Workflow
 
 Initially an upgrade plan is proposed by one of trustees using
-[propose-upgrade](transactions/transactions.md#propose_upgrade) command. Then the proposed
+[propose-upgrade](transactions.md) command. Then the proposed
 upgrade plan has to be approved by the majority of trustees (2/3 including the
 trustee who has proposed it) using
-[approve-upgrade](transactions/transactions.md#approve_upgrade) command. When the necessary
+[approve-upgrade](transactions.md#approve_upgrade) command. When the necessary
 count of approvals is gathered, the upgrade plan turns into the approved state
 and is actually scheduled.
 
@@ -81,5 +81,5 @@ checksum when the URL is provided in the specified format. If the downloaded
 file checksum does not equal to the checksum provided in the URL, `go-getter`
 reports that checksums did not match. To view `Info` field value of an upgrade
 plan, just execute an appropriate query command from `dclupgrade` or `upgrade`
-module. See [Upgrade CLI commands reference](transactions/transactions.md#upgrade) for
+module. See [Upgrade CLI commands reference](transactions.md#upgrade) for
 details.

docs/static/openapi.yml

@@ -10209,7 +10209,7 @@ paths:
           type: string
       tags:
         - Query
-  /dcl/pki/noc-certificates/{subject}:
+  /dcl/pki/all-noc-certificates/{subject}:
     get:
       summary: Queries a NocCertificatesBySubject by index.
       operationId: NocCertificatesBySubject
@@ -10256,7 +10256,7 @@ paths:
           type: string
       tags:
         - Query
-  /dcl/pki/noc-certificates/{subject}/{subjectKeyId}:
+  /dcl/pki/all-noc-certificates/{subject}/{subjectKeyId}:
     get:
       summary: Queries a NocCertificates by index.
       operationId: NocCertificates
@@ -10384,7 +10384,7 @@ paths:
           type: string
       tags:
         - Query
-  /dcl/pki/noc-certificates/{vid}/{subjectKeyId}:
+  /dcl/pki/noc-vid-certificates/{vid}/{subjectKeyId}:
     get:
       summary: Queries a NocCertificatesByVidAndSkid by index.
       operationId: NocCertificatesByVidAndSkid
@@ -10716,7 +10716,7 @@ paths:
           type: boolean
       tags:
         - Query
-  /dcl/pki/noc-ica-certificates/{vid}:
+  /dcl/pki/noc-vid-ica-certificates/{vid}:
     get:
       summary: Queries a NocIcaCertificates by index.
       operationId: NocIcaCertificates
@@ -11039,7 +11039,7 @@ paths:
           type: boolean
       tags:
         - Query
-  /dcl/pki/noc-root-certificates/{vid}:
+  /dcl/pki/noc-vid-root-certificates/{vid}:
     get:
       summary: Queries a NocRootCertificates by index.
       operationId: NocRootCertificates

docs/transactions/transactions.md docs/transactions.md

@@ -21,7 +21,7 @@ See use case sequence diagrams for the examples of how transaction can be used.
 
 - Every writer to the Ledger must  
   - Have a private/public key pair.
-  - Have an Account created on the ledger via `ACCOUNT` transaction (see [Use Case Txn Auth](../use_cases/use_cases_txn_auth.puml)).
+  - Have an Account created on the ledger via `ACCOUNT` transaction (see [Use Case Txn Auth](use_cases/use_cases_txn_auth.puml)).
     - The Account stores the public part of the key
     - The Account has an associated role. The role is used for authorization policies.
   - Sign every transaction by the private key.
@@ -38,9 +38,9 @@ an Account or sign the request.
 
 - Local CLI
   - Configure the CLI before using.
-      See `CLI Configuration` section in [how-to.md](../how-to.md#cli-configuration).
+      See `CLI Configuration` section in [how-to.md](how-to.md#cli-configuration).
   - Generate and store a private key for the Account to be used for sending.
-      See `Getting Account` section in [how-to.md](../how-to.md#getting-account).
+      See `Getting Account` section in [how-to.md](how-to.md#getting-account).
   - Send transactions to the ledger from the Account (`--from`).
     - it will automatically build a request, sign it by the account's key, and broadcast to the ledger.
   - See `CLI` sub-sections for every write request (transaction).
@@ -54,9 +54,9 @@ an Account or sign the request.
     - Broadcast transaction by CLI 1: `dcld tx broadcast txn.json`
     - To get the actual result of transaction, `dcld query tx=txHash` call must be executed, where `txHash` is the hash of previously executed transaction.
 - gRPC:
-  - Generate a client code from the proto files [proto](../../proto) for the client language (see <https://grpc.io/docs/languages/>)
+  - Generate a client code from the proto files [proto](../proto) for the client language (see <https://grpc.io/docs/languages/>)
   - Build, sign, and broadcast the message (transaction).
-      See [grpc/rest integration tests](../../integration_tests/grpc_rest) as an example.
+      See [grpc/rest integration tests](../integration_tests/grpc_rest) as an example.
 - REST API
   - Build and sign a transaction by one of the following ways
     - In code via gRPC (see above)
@@ -81,25 +81,25 @@ Please make sure that TLS is enabled in gRPC, REST or Light Client Proxy for sec
 
 - Local CLI
   - See `CLI` section for every read request.
-  - If there are no trusted Observer or Validator nodes to connect a CLI, then a [Light Client Proxy](../running-light-client-proxy.md) can be used.
+  - If there are no trusted Observer or Validator nodes to connect a CLI, then a [Light Client Proxy](running-light-client-proxy.md) can be used.
 - REST API
   - OpenAPI specification: <https://zigbee-alliance.github.io/distributed-compliance-ledger/>.
   - Any running node exposes a REST API at port `1317`. See <https://docs.cosmos.network/v0.45/core/grpc_rest.html>.
   - See `REST API` section for every read request.
-  - See [grpc/rest integration tests](../../integration_tests/grpc_rest) as an example.
+  - See [grpc/rest integration tests](../integration_tests/grpc_rest) as an example.
   - There are no state proofs in REST, so REST queries should be sent to trusted Validator or Observer nodes only.
 - gRPC
   - Any running node exposes a REST API at port `9090`. See <https://docs.cosmos.network/v0.45/core/grpc_rest.html>.
-  - Generate a client code from the proto files [proto](../../proto) for the client language (see <https://grpc.io/docs/languages/>).
-  - See [grpc/rest integration tests](../../integration_tests/grpc_rest) as an example.
+  - Generate a client code from the proto files [proto](../proto) for the client language (see <https://grpc.io/docs/languages/>).
+  - See [grpc/rest integration tests](../integration_tests/grpc_rest) as an example.
   - There are no state proofs in gRPC, so gRPC queries should be sent to trusted Validator or Observer nodes only.
 - Tendermint RPC
   - Tendermint RPC OpenAPI specification can be found in <https://zigbee-alliance.github.io/distributed-compliance-ledger/>.
   - Tendermint RPC is exposed by every running node  at port `26657`. See <https://docs.cosmos.network/v0.45/core/grpc_rest.html#tendermint-rpc>.
   - Tendermint RPC supports state proofs. Tendermint's Light Client library can be used to verify the state proofs.
     So, if Light Client API is used, then it's possible to communicate with non-trusted nodes.
   - Please note, that multi-value queries don't have state proofs support and should be sent to trusted nodes only.
-  - Refer to [this doc](../cometbft-rpc.md) to see how to [subscribe](../cometbft-rpc.md#subscribe) to a Tendermint WebSocket based events and/or [query](../cometbft-rpc.md#querying-application-components) an application components. 
+  - Refer to [this doc](cometbft-rpc.md) to see how to [subscribe](cometbft-rpc.md#subscribe) to a Tendermint WebSocket based events and/or [query](cometbft-rpc.md#querying-application-components) an application components. 
 
 `NotFound` (404 code) is returned if an entry is not found on the ledger.
 
@@ -1650,7 +1650,7 @@ Use [GET_ALL_REVOKED_NOC_ROOT](#get_all_revoked_noc_root-rcacs) to get a list of
 - CLI Command:
   - `dcld query pki noc-x509-root-certs --vid=<uint16>`
 - REST API:
-  - GET `/dcl/pki/noc-root-certificates/{vid}`
+  - GET `/dcl/pki/noc-vid-root-certificates/{vid}`
 
 #### GET_NOC_BY_VID_AND_SKID (RCACs/ICACs)
 
@@ -1670,7 +1670,7 @@ Use [GET_ALL_REVOKED_NOC_ICA](#get_all_revoked_noc_ica-icacs) to get a list of a
 - CLI Command:
   - `dcld query pki noc-x509-certs --vid=<uint16> --subject-key-id=<hex string>`
 - REST API:
-  - GET `/dcl/pki/noc-certificates/{vid}/{subject_key_id}`
+  - GET `/dcl/pki/noc-vid-certificates/{vid}/{subject_key_id}`
 
 #### GET_NOC_ICA_BY_VID (ICACs)
 
@@ -1687,7 +1687,7 @@ Use [GET_ALL_REVOKED_CERT](#get_all_revoked_certs) to get a list of all revoked
 - CLI Command:
   - `dcld query pki noc-x509-ica-certs --vid=<uint16>`
 - REST API:
-  - GET `/dcl/pki/noc-ica-certificates/{vid}`
+  - GET `/dcl/pki/noc-vid-ica-certificates/{vid}`
 
 #### GET_NOC_CERTS_BY_SUBJECT
 
@@ -1858,6 +1858,24 @@ Should be sent to trusted nodes only.
 - REST API:
   - GET `/dcl/pki/all-certificates`
 
+#### GET_CERTS_BY_SUBJECT
+
+**Status: Implemented**
+
+Gets all certificates associated with a subject. This query works for all types of certificates (PAA, PAI, RCAC, ICAC).
+
+Revoked certificates are not returned.
+Use [GET_ALL_REVOKED_DA_CERTS](#get_all_revoked_da_certs) to get a list of all revoked DA certificates.
+Use [GET_ALL_REVOKED_NOC_ROOT_CERTS](#get_all_revoked_noc_root-rcacs) to get a list of all revoked Noc Root certificates.
+Use [GET_ALL_REVOKED_NOC_ICA_CERTS](#get_all_revoked_noc_ica-icacs) to get a list of all revoked Noc ICA certificates.
+
+- Parameters:
+  - subject: `string`  - certificates's `Subject` is base64 encoded subject DER sequence bytes
+- CLI command:
+  - `dcld query pki all-subject-certs --subject=<base64 string>`
+- REST API:
+  - GET `/dcl/pki/all-certificates/{subject}`
+
 #### GET_CHILD_CERTS
 
 **Status: Implemented**

integration_tests/cli/pki-combine-certs.sh

@@ -283,6 +283,18 @@ response_does_not_contain "$result" "\"subjectKeyId\": \"$da_root_subject_key_id
 
 test_divider
 
+echo "Request DA certificates by subject using global command"
+result=$(dcld query pki all-subject-certs --subject=$da_root_subject)
+echo $result | jq
+check_response "$result" "\"$da_root_subject\""
+check_response "$result" "\"$da_root_subject_key_id\""
+
+echo "Request NOC certificates by subject using global command"
+result=$(dcld query pki all-subject-certs --subject=$noc_root_subject)
+echo $result | jq
+check_response "$result" "\"$noc_root_subject\""
+check_response "$result" "\"$noc_root_subject_key_id\""
+
 echo "Request all DA certificates by subject must be empty"
 result=$(dcld query pki all-subject-x509-certs --subject="$da_root_subject")
 echo $result | jq

integration_tests/grpc_rest/pki/noc_cert_helpers.go

@@ -190,7 +190,7 @@ func GetNocX509RootCerts(suite *utils.TestSuite, vendorID int32) (*pkitypes.NocR
 	var res pkitypes.NocRootCertificates
 	if suite.Rest {
 		var resp pkitypes.QueryGetNocRootCertificatesResponse
-		err := suite.QueryREST(fmt.Sprintf("/dcl/pki/noc-root-certificates/%v", vendorID), &resp)
+		err := suite.QueryREST(fmt.Sprintf("/dcl/pki/noc-vid-root-certificates/%v", vendorID), &resp)
 		if err != nil {
 			return nil, err
 		}
@@ -218,7 +218,7 @@ func GetNocX509CertsByVidAndSkid(suite *utils.TestSuite, vendorID int32, subject
 	var res pkitypes.NocCertificatesByVidAndSkid
 	if suite.Rest {
 		var resp pkitypes.QueryGetNocCertificatesByVidAndSkidResponse
-		err := suite.QueryREST(fmt.Sprintf("/dcl/pki/noc-certificates/%v/%s", vendorID, url.QueryEscape(subjectKeyID)), &resp)
+		err := suite.QueryREST(fmt.Sprintf("/dcl/pki/noc-vid-certificates/%v/%s", vendorID, url.QueryEscape(subjectKeyID)), &resp)
 		if err != nil {
 			return nil, err
 		}
@@ -246,7 +246,7 @@ func GetNocX509IcaCerts(suite *utils.TestSuite, vendorID int32) (*pkitypes.NocIc
 	var res pkitypes.NocIcaCertificates
 	if suite.Rest {
 		var resp pkitypes.QueryGetNocIcaCertificatesResponse
-		err := suite.QueryREST(fmt.Sprintf("/dcl/pki/noc-ica-certificates/%v", vendorID), &resp)
+		err := suite.QueryREST(fmt.Sprintf("/dcl/pki/noc-vid-ica-certificates/%v", vendorID), &resp)
 		if err != nil {
 			return nil, err
 		}

proto/zigbeealliance/distributedcomplianceledger/pki/all_certificates_by_subject.proto

@@ -0,0 +1,12 @@
+syntax = "proto3";
+package zigbeealliance.distributedcomplianceledger.pki;
+
+option go_package = "github.com/zigbee-alliance/distributed-compliance-ledger/x/pki/types";
+
+message AllCertificatesBySubject {
+  string subject = 1; 
+  repeated string subjectKeyIds = 2; 
+  uint32 schemaVersion = 3;
+  
+}
+