Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ui): run web console tests with context path #372

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions .github/workflows/tests_with_context_path.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
name: Run Tests With Context Path

on:
push:

jobs:
run_tests_with_context_path:
name: Running Tests With Context Path
runs-on: ubuntu-latest

env:
QDB_HTTP_CONTEXT_PATH: "/context"

steps:
- uses: actions/checkout@v4
with:
submodules: "recursive"

- uses: actions/setup-java@v4
with:
java-version: "17"
distribution: "temurin"
cache: maven

- name: Build QuestDB
run: mvn clean package -f packages/browser-tests/questdb/pom.xml -DskipTests -P build-binaries

- name: Extract QuestDB
run: tar -xzf packages/browser-tests/questdb/core/target/questdb-*-rt-linux-x86-64.tar.gz -C tmp/

- name: Create DB Root
run: mkdir tmp/dbroot

- name: Start QuestDB
run: ./tmp/questdb-*-rt-linux-x86-64/bin/questdb.sh start -d ./tmp/dbroot
env:
QDB_DEV_MODE_ENABLED: "true"

- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "yarn"

- name: Install dependencies
run: yarn install --immutable --immutable-cache

- name: Build @questdb/react-components
run: yarn workspace @questdb/react-components run build

- name: Build @questdb/web-console
run: yarn workspace @questdb/web-console run build

- name: Run bundle watcher on @questdb/web-console
run: yarn workspace @questdb/web-console bundlewatch

- name: Run @questdb/web-console unit tests
run: yarn workspace @questdb/web-console run test:prod

- name: Run browser-tests test - auth
run: node packages/web-console/serve-dist.js & yarn workspace browser-tests test:auth

- name: Stop QuestDB
run: ./tmp/questdb-*-rt-linux-x86-64/bin/questdb.sh stop

- name: Start QuestDB, set auth credentials
run: ./tmp/questdb-*-rt-linux-x86-64/bin/questdb.sh start -d ./tmp/dbroot
env:
QDB_DEV_MODE_ENABLED: "true"
QDB_HTTP_USER: "admin"
QDB_HTTP_PASSWORD: "quest"

- name: Run browser-tests test
run: yarn workspace browser-tests test

- name: Print Log Files
if: success() || failure()
run: cat tmp/dbroot/log/*

- name: Save QuestDB log on test failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: questdb-log
path: tmp/dbroot/log/*

- name: Stop QuestDB
if: success() || failure()
run: ./tmp/questdb-*-rt-linux-x86-64/bin/questdb.sh stop
5 changes: 4 additions & 1 deletion packages/browser-tests/cypress.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,16 @@ const {
addMatchImageSnapshotPlugin,
} = require("@simonsmith/cypress-image-snapshot/plugin");

const contextPath = process.env.QDB_HTTP_CONTEXT_PATH || ""
const baseUrl = `http://localhost:9999${contextPath}`;

module.exports = defineConfig({
defaultCommandTimeout: 10000,
e2e: {
defaultCommandTimeout: 30000,
screenshotOnRunFailure: false,
video: false,
baseUrl: "http://localhost:9999",
baseUrl: baseUrl,
viewportWidth: 1280,
viewportHeight: 720,
specPattern: "cypress/integration/**/*.spec.js",
Expand Down
3 changes: 2 additions & 1 deletion packages/browser-tests/cypress/commands.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ addMatchImageSnapshotCommand({

const { ctrlOrCmd, escapeRegExp } = require("./utils");

const baseUrl = "http://localhost:9999";
const contextPath = process.env.QDB_HTTP_CONTEXT_PATH || ""
const baseUrl = `http://localhost:9999${contextPath}`;

const tableSchemas = {
btc_trades:
Expand Down
3 changes: 2 additions & 1 deletion packages/browser-tests/cypress/integration/auth/auth.spec.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";
const contextPath = process.env.QDB_HTTP_CONTEXT_PATH || ""
const baseUrl = `http://localhost:9999${contextPath}`;

const interceptSettings = (payload) => {
cy.intercept({ method: "GET", url: `${baseUrl}/settings` }, payload).as(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";
const contextPath = process.env.QDB_HTTP_CONTEXT_PATH || ""
const baseUrl = `http://localhost:9999${contextPath}`;

const getTabDragHandleByTitle = (title) =>
`.chrome-tab[data-tab-title="${title}"] .chrome-tab-drag-handle`;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";

const rowHeight = 30;

const assertRowCount = () => {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";

describe("Sidebar tests", () => {
beforeEach(() => {
cy.loadConsoleWithAuth();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";

describe("questdb charts", () => {
beforeEach(() => {
cy.loadConsoleWithAuth();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";
const contextPath = process.env.QDB_HTTP_CONTEXT_PATH || ""
const baseUrl = `http://localhost:9999${contextPath}`;

const tables = [
"btc_trades",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";

const toggleTelemetry = (enabled) => {
// expected dataset format of the first row:
// [id, enabled, version, os, package]
cy.interceptQuery("telemetry_config LIMIT -1", "telemetryConfig", (req) => {
return req.continue((res) => {
// enable telemetry to kick start the process on the client side
// enable telemetry to kick-start the process on the client side
res.body.dataset[0][1] = enabled;
return res;
});
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/// <reference types="cypress" />

const baseUrl = "http://localhost:9999";

describe("System configuration - no warnings", () => {
before(() => {
cy.loadConsoleWithAuth(true);
Expand Down
2 changes: 1 addition & 1 deletion packages/browser-tests/questdb
Submodule questdb updated 77 files
+21 −31 benchmarks/src/main/java/org/questdb/PGWireInsertSelectBenchmark.java
+10 −0 ci/templates/hosted-cover-jobs.yml
+22 −2 ci/templates/hosted-jobs.yml
+2 −4 compat/pom.xml
+1 −1 core/rust/qdbr/src/parquet_write/varchar.rs
+4 −3 core/src/main/java/io/questdb/Bootstrap.java
+22 −2 core/src/main/java/io/questdb/PropServerConfiguration.java
+2 −0 core/src/main/java/io/questdb/PropertyKey.java
+5 −3 core/src/main/java/io/questdb/Telemetry.java
+0 −29 core/src/main/java/io/questdb/cairo/AlterTableUtils.java
+25 −5 core/src/main/java/io/questdb/cairo/CairoTable.java
+2 −0 core/src/main/java/io/questdb/cairo/MetadataCache.java
+54 −2 core/src/main/java/io/questdb/cairo/PartitionBy.java
+1 −0 core/src/main/java/io/questdb/cairo/TableReader.java
+8 −0 core/src/main/java/io/questdb/cairo/TableReaderMetadata.java
+10 −0 core/src/main/java/io/questdb/cairo/TableStructure.java
+9 −0 core/src/main/java/io/questdb/cairo/TableUtils.java
+77 −31 core/src/main/java/io/questdb/cairo/TableWriter.java
+1 −1 core/src/main/java/io/questdb/cairo/TableWriterAPI.java
+11 −0 core/src/main/java/io/questdb/cairo/TableWriterMetadata.java
+3 −1 core/src/main/java/io/questdb/cairo/TxReader.java
+9 −2 core/src/main/java/io/questdb/cairo/wal/MetadataService.java
+1 −4 core/src/main/java/io/questdb/cairo/wal/WalWriter.java
+10 −6 core/src/main/java/io/questdb/cairo/wal/seq/MetadataServiceStub.java
+10 −0 core/src/main/java/io/questdb/cutlass/http/DefaultHttpServerConfiguration.java
+2 −0 core/src/main/java/io/questdb/cutlass/http/HttpFullFatServerConfiguration.java
+12 −9 core/src/main/java/io/questdb/cutlass/http/HttpServer.java
+5 −0 core/src/main/java/io/questdb/cutlass/http/HttpServerConfigurationWrapper.java
+2 −0 core/src/main/java/io/questdb/cutlass/http/processors/LineHttpProcessorConfiguration.java
+40 −16 core/src/main/java/io/questdb/cutlass/http/processors/StaticContentProcessor.java
+6 −1 core/src/main/java/io/questdb/cutlass/line/tcp/TableStructureAdapter.java
+4 −2 core/src/main/java/io/questdb/cutlass/pgwire/modern/PGConnectionContextModern.java
+27 −7 core/src/main/java/io/questdb/griffin/SqlCompilerImpl.java
+7 −0 core/src/main/java/io/questdb/griffin/SqlKeywords.java
+90 −20 core/src/main/java/io/questdb/griffin/SqlParser.java
+38 −0 core/src/main/java/io/questdb/griffin/engine/functions/catalogue/TablesFunctionFactory.java
+24 −1 core/src/main/java/io/questdb/griffin/engine/ops/AlterOperation.java
+9 −0 core/src/main/java/io/questdb/griffin/engine/ops/AlterOperationBuilder.java
+10 −1 core/src/main/java/io/questdb/griffin/engine/ops/CreateTableOperationBuilderImpl.java
+11 −1 core/src/main/java/io/questdb/griffin/engine/ops/CreateTableOperationImpl.java
+36 −4 core/src/main/java/io/questdb/griffin/engine/table/ShowCreateTableRecordCursorFactory.java
+10 −3 core/src/main/java/io/questdb/mp/SCSequence.java
+53 −1 core/src/main/java/io/questdb/std/datetime/microtime/Timestamps.java
+4 −0 core/src/main/java/io/questdb/tasks/AbstractTelemetryTask.java
+11 −1 core/src/main/java/io/questdb/tasks/TelemetryTask.java
+11 −1 core/src/main/java/io/questdb/tasks/TelemetryWalTask.java
+23 −5 core/src/test/java/io/questdb/test/ServerMainForeignTableTest.java
+2 −0 core/src/test/java/io/questdb/test/ServerMainTest.java
+37 −37 core/src/test/java/io/questdb/test/cairo/MetadataCacheTest.java
+11 −0 core/src/test/java/io/questdb/test/cairo/TableModel.java
+25 −0 core/src/test/java/io/questdb/test/cairo/TableWriterTest.java
+632 −0 core/src/test/java/io/questdb/test/cairo/TtlTest.java
+6 −3 core/src/test/java/io/questdb/test/cairo/fuzz/AbstractFuzzTest.java
+6 −2 core/src/test/java/io/questdb/test/cairo/fuzz/CheckpointFuzzTest.java
+46 −19 core/src/test/java/io/questdb/test/cairo/fuzz/DedupInsertFuzzTest.java
+6 −4 core/src/test/java/io/questdb/test/cairo/fuzz/FrameAppendFuzzTest.java
+41 −16 core/src/test/java/io/questdb/test/cairo/fuzz/FuzzRunner.java
+4 −3 core/src/test/java/io/questdb/test/cairo/fuzz/O3MaxLagFuzzTest.java
+3 −2 core/src/test/java/io/questdb/test/cairo/fuzz/O3ParquetPartitionFuzzTest.java
+2 −1 core/src/test/java/io/questdb/test/cairo/fuzz/ReaderReloadFuzzTest.java
+240 −16 core/src/test/java/io/questdb/test/cairo/fuzz/WalWriterFuzzTest.java
+21 −7 core/src/test/java/io/questdb/test/cairo/mig/EngineMigrationTest.java
+17 −5 core/src/test/java/io/questdb/test/cairo/wal/WalTableFailureTest.java
+17 −8 core/src/test/java/io/questdb/test/cutlass/http/WebConsoleLoadingTest.java
+1 −2 core/src/test/java/io/questdb/test/cutlass/line/tcp/SymbolCacheTest.java
+54 −10 core/src/test/java/io/questdb/test/fuzz/FuzzDropCreateTableOperation.java
+76 −0 core/src/test/java/io/questdb/test/fuzz/FuzzSetTtlOperation.java
+30 −6 core/src/test/java/io/questdb/test/fuzz/FuzzTransactionGenerator.java
+1 −1 core/src/test/java/io/questdb/test/fuzz/TableStructMetadataAdapter.java
+2 −2 core/src/test/java/io/questdb/test/griffin/AlterTableAddColumnTest.java
+7 −2 core/src/test/java/io/questdb/test/griffin/AlterTableAlterColumnTest.java
+2 −2 core/src/test/java/io/questdb/test/griffin/AlterTableDetachPartitionTest.java
+2 −2 core/src/test/java/io/questdb/test/griffin/AlterTableDropColumnTest.java
+9 −2 core/src/test/java/io/questdb/test/griffin/AlterTableO3MaxLagTest.java
+2 −2 core/src/test/java/io/questdb/test/griffin/AlterTableRenameColumnTest.java
+11 −11 core/src/test/java/io/questdb/test/griffin/KeywordAsTableNameTest.java
+96 −0 core/src/test/java/io/questdb/test/griffin/ShowCreateTableTest.java
6 changes: 3 additions & 3 deletions packages/web-console/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ If the server has a context path configured with the `http.context.path` option,
the dev server is aware of it, and it proxies requests accordingly.\
We can set the context path with the following environment variable before starting dev server:
```
CONTEXT_PATH=/instance2
export CONTEXT_PATH
QDB_HTTP_CONTEXT_PATH=/instance2
export QDB_HTTP_CONTEXT_PATH
yarn workspace @questdb/web-console start
```

Expand All @@ -46,7 +46,7 @@ After the above the web console is available on [localhost:9999/instance2/](http
If the context path is removed from the server configuration, we also need to clear the environment variable,
and restart the dev server:
```
unset CONTEXT_PATH
unset QDB_HTTP_CONTEXT_PATH
yarn workspace @questdb/web-console start
```

Expand Down
21 changes: 13 additions & 8 deletions packages/web-console/serve-dist.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,25 @@ const url = require("url")
const fs = require("fs")
const path = require("path")

const contextPath = process.env.QDB_HTTP_CONTEXT_PATH || ""

const server = http.createServer((req, res) => {
const { method } = req
const urlData = url.parse(req.url)
const baseUrl = "http://" + req.headers.host + contextPath;
const reqUrl = new url.URL(req.url, baseUrl);
const reqPath = reqUrl.pathname + reqUrl.search;
const reqPathName = reqUrl.pathname;

if (
urlData.pathname.startsWith("/exec") ||
urlData.pathname.startsWith("/settings") ||
urlData.pathname.startsWith("/warnings")
reqPathName.startsWith("/exec") ||
reqPathName.startsWith("/settings") ||
reqPathName.startsWith("/warnings")
) {
// proxy /exec requests to localhost:9000
const options = {
hostname: "localhost",
port: 9000,
path: urlData.path,
path: contextPath + reqPath,
method,
headers: req.headers,
}
Expand All @@ -34,16 +39,16 @@ const server = http.createServer((req, res) => {
"packages",
"web-console",
"dist",
["", "/"].some((p) => urlData.pathname === p)
["", "/"].some((p) => reqPathName === p)
? "index.html"
: urlData.pathname,
: reqPathName,
)
const fileStream = fs.createReadStream(filePath)

fileStream.on("error", (err) => {
if (err.code === "ENOENT") {
res.statusCode = 404
res.end(`File not found: ${urlData}`)
res.end(`File not found: ${reqPath}`)
} else {
res.statusCode = 500
res.end(`Server error: ${err}`)
Expand Down
2 changes: 1 addition & 1 deletion packages/web-console/webpack.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ const config = {
port: 9999,
backendUrl: "http://127.0.0.1:9000",
isProduction: process.env.NODE_ENV === "production",
contextPath: process.env.CONTEXT_PATH || "",
contextPath: process.env.QDB_HTTP_CONTEXT_PATH || "",
}

module.exports = {
Expand Down
28 changes: 25 additions & 3 deletions run_browser_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ yarn workspace @questdb/web-console run build

# Start proxy
node packages/web-console/serve-dist.js &
PID="$!"
echo "Proxy started, PID=$PID"
PID1="$!"
echo "Proxy started, PID=$PID1"

# Switch dev mode on
export QDB_DEV_MODE_ENABLED=true
Expand All @@ -41,6 +41,8 @@ export QDB_DEV_MODE_ENABLED=true
yarn workspace browser-tests test:auth
./tmp/questdb-*/bin/questdb.sh stop

read -p "Press any key to continue... " -n1 -s

# Switch authentication on
export QDB_HTTP_USER=admin
export QDB_HTTP_PASSWORD=quest
Expand All @@ -50,5 +52,25 @@ export QDB_HTTP_PASSWORD=quest
yarn workspace browser-tests test
./tmp/questdb-*/bin/questdb.sh stop

read -p "Press any key to continue... " -n1 -s

# Set context path
export QDB_HTTP_CONTEXT_PATH=/context1

# Restart proxy to pickup context path
kill -SIGTERM $PID1
sleep 1
node packages/web-console/serve-dist.js &
PID2="$!"
echo "Proxy started, PID=$PID2"

# Cleanup snapshots for second run
rm -rf packages/browser-tests/cypress/snapshots/*

# Running tests with context path
./tmp/questdb-*/bin/questdb.sh start -d tmp/dbroot
yarn workspace browser-tests test
./tmp/questdb-*/bin/questdb.sh stop

# Stop proxy
kill -SIGTERM $PID
kill -SIGTERM $PID2
Loading