Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backend integration tests for handle topic create #706

Merged
merged 20 commits into from
May 17, 2023
Merged

Backend integration tests for handle topic create #706

merged 20 commits into from
May 17, 2023

Conversation

bojand
Copy link
Member

@bojand bojand commented May 17, 2023

Builds on #704.

Changes

  • Refactors existing integration test utility functions into a new testutil package. This package some common utility functions for creating topics, test data, and alike that can be used generally for integration tests.
  • Adds integration tests for create topic API endpoint (POST /api/topics).

bojand added 19 commits May 12, 2023 09:45
@bojand
console: initial pass at console integration tests using testcontainers
662ff07
@bojand
console: add gotest github action workflow
557dbfc
@bojand
console: add working-directory for test step in gotest GitHub action job
5bf600c
@bojand
console: add remove -failfast and -json in test step in gotest GitHub…
0166ed5 
… action job
@bojand
console: update setup-go action to v4 and unify lint and test steps i…
f95faa0 
…nto a single verify job
@bojand
console: initial WIP on handle topic create API integration tests
1ed74bf
@bojand
console: more test for handle topic create API
4d39974
@bojand
console: add custom hooks to api tests
cb50f55
@bojand
console: rename github verify workflow
c732fb3
@bojand
console: add create topic api test for handling failure
d2b8f66
@bojand
console: add verify task and add testing notes to CONTRIBUTING guide
178f099
@bojand
Merge branch 'master' into backend_integration_tests
9f60b50
@bojand
console: add license comments, use dev logger in tests, and address s…
028a0bb 
…ome minor PR feedback.
@bojand
Backend integration tests suite (#704)
6a3d176 
* console: moving integration tests to use testify suites

* console: migrate existing integration tests to use testify test suite
@bojand
Merge branch 'master' into backend_integration_tests
f86fc1e
@bojand
Merge branch 'backend_integration_tests' of https://github.com/redpan…
7cbce30 
…da-data/console into backend_integration_tests
@bojand
Merge branch 'backend_integration_tests' into backend_integration_tes…
2abb19d 
…ts_handle_topic_create
@bojand
console: add license info to new files
4d20ace
@bojand
Merge branch 'master' into backend_integration_tests_handle_topic_create
b7fd346
@bojand bojand requested a review from weeco May 17, 2023 00:08
weeco
weeco reviewed May 17, 2023
View reviewed changes
backend/pkg/api/handle_topic_create_integration_test.go Outdated
log, err := logCfg.Build()
require.NoError(err)

port := rand.Intn(50000) + 10000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can clash with the exposed testcontainers port I think which would cause a "flaky" (even though very unlikely) test. I think the alternative would be to use port 0 and Go will find a listener on a random port. However, you wouldn't be able to figure out what the randomly chosen port was afterwards. So I think it's reasonable to keep it as is, but have it in mind.

backend/pkg/api/handle_topic_create_integration_test.go Outdated Show resolved Hide resolved
backend/pkg/api/handle_topic_create_integration_test.go Outdated Show resolved Hide resolved
backend/pkg/api/handle_topic_create_integration_test.go Outdated Show resolved Hide resolved
backend/pkg/api/handle_topic_create_integration_test.go Outdated Show resolved Hide resolved
backend/pkg/api/handle_topic_create_integration_test.go Outdated Show resolved Hide resolved
backend/pkg/api/handle_topic_create_integration_test.go Outdated Show resolved Hide resolved
@bojand bojand merged commit 512798d into master May 17, 2023
@bojand bojand deleted the backend_integration_tests_handle_topic_create branch May 17, 2023 15:31
rikimaru0345 added a commit that referenced this pull request Jun 15, 2023
@rikimaru0345
Squashed commit of the following:
50f1319 
commit f7e38a0
Author: Santiago Jimenez Giraldo <santiago@redpanda.com>
Date:   Wed Jun 14 01:13:20 2023 +0200

    feat(frontend): handle unauthorized access on console

    Update console frontend to handle unauthorized when embedded:
    - remove unnecessary internal switch for routes
    - redirect to the unauthorized page in cloud UI
    - leave an unauthorized page in the repository for reference

    Related to: redpanda-data/cloudv2#6318

commit 0e24b03
Merge: 86d06d7 0973993
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Tue Jun 13 19:39:01 2023 +0200

    Merge pull request #730 from redpanda-data/mb/auto_offsets_reset_mm2

    connectors: changed auto.offset.reset value to 'none'

commit 0973993
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Tue Jun 13 18:23:10 2023 +0200

    connectors: changed auto.offset.reset value to 'none'

commit 86d06d7
Merge: 7896c8c 9ee2a32
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Tue Jun 13 15:33:18 2023 +0200

    Merge pull request #729 from redpanda-data/mb/auto_offsets_reset_mm2

    connectors: fixed 'auto.offset.reset' MM2 property default value

commit 9ee2a32
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Tue Jun 13 15:19:08 2023 +0200

    connectors: fixed 'auto.offset.reset' MM2 property default value

commit 7896c8c
Merge: baf3b2f ccaf962
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Mon Jun 12 22:11:54 2023 +0200

    Merge pull request #728 from redpanda-data/mb/auto_offsets_reset_mm2

commit ccaf962
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Mon Jun 12 14:20:46 2023 +0200

    connectors: added support for 'auto.offset.reset' MM2 property

commit baf3b2f
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Fri Jun 9 12:36:53 2023 +0100

    Multi kafka support (#726)

    * backend: create interface for console service

    * backend: move kafka service into the console pkg

    * backend: allow creation of multiple kafka services

    * backend: make eager kafka connectivity check configurable

    * backend: make console service creation configurable

commit 0ba44b2
Author: Santiago Jimenez Giraldo <santiago@redpanda.com>
Date:   Tue Jun 6 19:37:23 2023 +0200

    fix(frontend): for over null/undefined

    there where some null/undefined mapping, when the request is
    unauthorized

    Related to: redpanda-data/cloudv2#6318

    Signed-off-by: Santiago Jimenez Giraldo <santiago@redpanda.com>

commit 20c97bd
Author: Moritz Staudinger <rikimaru0345@users.noreply.github.com>
Date:   Tue Jun 6 14:14:31 2023 +0200

    Update frontend/src/EmbeddedApp.tsx

commit d1b4fed
Author: Santiago Jimenez Giraldo <santiago@redpanda.com>
Date:   Tue Jun 6 09:38:08 2023 +0200

    feat(frontend): handle unauthorized in console embedded

    update console embedded to handle unauthorized tokens and redirect to a
    page with some more context on the issue

    Resolves: redpanda-data/cloudv2#6318

commit de36be4
Author: Santiago Jimenez Giraldo <santiago@redpanda.com>
Date:   Tue Jun 6 09:23:13 2023 +0200

    deps(frontend): update redpanda UI to latest

    update redpanda-data/ui and fix tabs incompatibility issues

commit 5a0823f
Merge: 19430fb 5f7f77a
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Tue Jun 6 09:09:15 2023 +0200

    Merge pull request #722 from redpanda-data/mb/bigquery_retry

    connectors: added support BigQuery sink connector retries

commit 5f7f77a
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Mon Jun 5 14:37:30 2023 +0200

    connectors: added support BigQuery sink connector retries

commit 19430fb
Merge: 9e013cd ec66bfb
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Mon Jun 5 09:33:01 2023 +0200

    Merge pull request #721 from redpanda-data/connectors-unit-tests

    backend: Add tests for connectors

commit 9e013cd
Merge: dcaa6fd 66e05df
Author: Bojan <bojan@redpanda.com>
Date:   Fri Jun 2 11:48:49 2023 -0300

    Merge pull request #720 from redpanda-data/backend_connector_status_task_unassigned

    backend: if there are unassigned tasks on a running connector, set co…

commit ec66bfb
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Fri Jun 2 11:56:07 2023 +0200

    backend: Add tests for connectors

commit dcaa6fd
Merge: 3191733 d05e3b7
Author: tomasz-sadura <tomasz@redpanda.com>
Date:   Fri Jun 2 11:01:15 2023 +0200

    Merge pull request #716 from redpanda-data/mm2-fixes

    MirrorSourceConnector fixes

commit 66e05df
Author: Bojan Djurkovic <bojan@redpanda.com>
Date:   Thu Jun 1 13:35:39 2023 -0300

    backend: if there are unassigned tasks on a running connector, set connector status to unassigned

commit 3191733
Merge: 6cb2673 9e6e742
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Thu Jun 1 17:36:15 2023 +0300

    Merge pull request #718 from redpanda-data/mb/connect_max_req_size

    connectors: added support for max.request.size UI property for MM2 and Debezium connectors

commit 9e6e742
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Thu Jun 1 14:44:30 2023 +0200

    connectors: added support for max.request.size UI property for MM2 and Debezium connectors

commit d05e3b7
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Thu Jun 1 14:04:15 2023 +0200

    backend: Fix for wiping out credentials in case of editing imported MM2 config

commit e82af57
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Thu Jun 1 14:01:30 2023 +0200

    backend: Add sizing to MirrorSourceConnector

commit 6cb2673
Merge: 5d4175f 3572afe
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Thu May 25 21:55:35 2023 +0300

    Merge pull request #713 from redpanda-data/mb/cloudevents_debezium

    connectors: added support for CloudEvents converter in Debezium connectors

commit 3572afe
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Thu May 25 15:07:40 2023 +0200

    connectors: added support for CloudEvents converter avro serializer in Debezium connectors

commit 5d4175f
Merge: 9d39251 ce805de
Author: tomasz-sadura <tomasz@redpanda.com>
Date:   Thu May 25 13:40:27 2023 +0200

    Merge pull request #697 from redpanda-data/remove-trailing-dot-from-connector-documentation

    Remove trailing dot from connector documentation

commit ce805de
Merge: ec2f0f8 9d39251
Author: tomasz-sadura <tomasz@redpanda.com>
Date:   Thu May 25 13:29:07 2023 +0200

    Merge branch 'master' into remove-trailing-dot-from-connector-documentation

commit 9d39251
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Wed May 24 14:12:03 2023 +0100

    chore: replace context import

commit f4b9e6f
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Wed May 24 14:11:26 2023 +0100

    task: run backend:fmt

commit 13ccda5
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Wed May 24 14:09:56 2023 +0100

    task: add goimports to fmt task

commit ef40108
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Wed May 24 15:02:11 2023 +0200

    connectors: added support for CloudEvents converter in Debezium connectors

commit 3ed8a89
Author: Bojan <bojan@redpanda.com>
Date:   Tue May 23 08:52:42 2023 -0300

    Backend get topics tests (#711)

    * backend: WIP on get topics integration tests

    * backend: WIP on get topics integration tests

    * backend: WIP on get topics integration tests

    * backend: add assertCallReturnValue to mocked hooks in tests

    * backend: undo skip

    * backend: add test for allow topic action error

    * backend: minor format fixes

    * backend: send a more realistic unauthorized status code in AllowedTopicActions test

commit a84472e
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Fri May 19 13:14:31 2023 +0100

    backend: update dependencies

commit 79c28d5
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Fri May 19 11:49:27 2023 +0100

    build: Impersonate Redpanda vbot user

commit c13ca56
Author: Bojan Djurkovic <bojan@redpanda.com>
Date:   Wed May 17 22:20:55 2023 -0300

    backend: in getConsumerGroupOffsets() properly check for existing group data in groupOffsets map

commit 512798d
Author: Bojan <bojan@redpanda.com>
Date:   Wed May 17 12:31:10 2023 -0300

    Backend integration tests for handle topic create (#706)

    * console: initial pass at console integration tests using testcontainers

    * console: add gotest github action workflow

    * console: add working-directory for test step in gotest GitHub action job

    * console: add remove -failfast and -json in test step in gotest GitHub action job

    * console: update setup-go action to v4 and unify lint and test steps into a single verify job

    * console: initial WIP on handle topic create API integration tests

    * console: more test for handle topic create API

    * console: add custom hooks to api tests

    * console: rename github verify workflow

    * console: add create topic api test for handling failure

    * console: add verify task and add testing notes to CONTRIBUTING guide

    * console: add license comments, use dev logger in tests, and address some minor PR feedback.

    * Backend integration tests suite (#704)

    * console: moving integration tests to use testify suites

    * console: migrate existing integration tests to use testify test suite

    * console: add license info to new files

    * console: address PR feedback

commit a9cb6aa
Author: Bojan <bojan@redpanda.com>
Date:   Tue May 16 20:59:37 2023 -0300

    Console backend integration tests (#701)

    * console: initial pass at console integration tests using testcontainers

    * console: add gotest github action workflow

    * console: add working-directory for test step in gotest GitHub action job

    * console: add remove -failfast and -json in test step in gotest GitHub action job

    * console: update setup-go action to v4 and unify lint and test steps into a single verify job

    * console: rename github verify workflow

    * console: add verify task and add testing notes to CONTRIBUTING guide

    * console: add license comments, use dev logger in tests, and address some minor PR feedback.

    * Backend integration tests suite (#704)

    * console: moving integration tests to use testify suites

    * console: migrate existing integration tests to use testify test suite

commit 0a973f3
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Tue May 16 14:53:21 2023 +0100

    chore: Update referenc config for v2.3.0

commit 1c00c2d
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Tue May 16 14:53:12 2023 +0100

    chore: Add changelog for v2.3.0

commit 90c9fe4
Author: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Date:   Tue May 16 12:19:07 2023 +0100

    backend: add support for TLS termination (#702)

    * backend: add support for TLS termination

commit 3544ecc
Merge: 3a6bc2a b95cf0f
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Tue May 16 14:15:06 2023 +0300

    Merge pull request #703 from redpanda-data/mb/jdbc_connect_fixes

    connectors: improved documentations in jdbc connector, replaced Kafka with Redpanda

commit b95cf0f
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Tue May 16 12:23:10 2023 +0200

    connectors: improved documentations in jdbc connector, replaced Kafka with Redpanda in docs and property names

commit ec2f0f8
Merge: d6b9d03 3a6bc2a
Author: tomasz-sadura <tomasz@redpanda.com>
Date:   Mon May 15 14:31:03 2023 +0200

    Merge branch 'master' into remove-trailing-dot-from-connector-documentation

commit 3a6bc2a
Merge: fce386e 3a7bfb4
Author: tomasz-sadura <tomasz@redpanda.com>
Date:   Mon May 15 14:20:22 2023 +0200

    Merge pull request #696 from redpanda-data/mongodb-ui-enhancements

    Add enhancements to MongoDB display names and documentation

commit fce386e
Merge: 821ed67 95b0e9f
Author: Martin Bochenek <1389015+bochenekmartin@users.noreply.github.com>
Date:   Mon May 15 13:22:09 2023 +0300

    Merge pull request #700 from redpanda-data/mb/connect_debezium_improved

    connectors: improved description and documentations in debezium connectors

commit 95b0e9f
Author: Martin Bochenek <bochenek.martin@gmail.com>
Date:   Fri May 12 18:43:11 2023 +0200

    connectors: improved description and documentations in debezium connectors

commit 3a7bfb4
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Thu May 11 13:04:27 2023 +0200

    backend: Remove trailing dot from connector errors

commit 8b6672c
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Wed May 10 15:47:44 2023 +0200

    backend: go mod tidy

commit d6b9d03
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Wed May 10 15:46:29 2023 +0200

    backend: go mod tidy

commit fa6b1ba
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Wed May 10 15:23:59 2023 +0200

    backend: Remove trailing dot from connector documentation

commit 71942cd
Author: Tomasz Sadura <tomasz@redpanda.com>
Date:   Wed May 10 15:10:09 2023 +0200

    backend: Add enhancements to MongoDB display names and documentation

commit 821ed67
Author: Chase Higgins <chasehx@gmail.com>
Date:   Wed May 10 10:21:21 2023 +0100

    exclude docs dir fom kics (#693)
@bojand bojand restored the backend_integration_tests_handle_topic_create branch April 12, 2024 00:25
@bojand bojand deleted the backend_integration_tests_handle_topic_create branch April 12, 2024 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weeco weeco weeco approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bojand @weeco