Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Benchmark Test Run Report

DateSep 19, 2023 7:37
Duration4h 19m
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/22.8.15.25.altinitystable/6231175917/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/265/6b52da82b6ca003e8ebd34ead5f4eabbc45d1cd6/package_aarch64/clickhouse-common-static_22.8.15.25.altinitystable_arm64.deb
version22.8.15.25.altinitystable
user.nameMyroTk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashc23ebf091e566342377fb79fbd78ac3f7a42e678
job.id6231175917
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/6231175917
archaarch64
nodeclickhouse1
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/265/6b52da82b6ca003e8ebd34ead5f4eabbc45d1cd6/package_aarch64/clickhouse-common-static_22.8.15.25.altinitystable_arm64.deb
stressFalse
collect_service_logsTrue
storages['aws_s3']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')
formatNone

Summary

97.2%OK
2.8%Error

Statistics

Units OK Error
Modules
1
1
Suites
2
2
Features
2
2
Scenarios
208
207
1
Steps
1464
1461
3

Fails

Test NameResultMessage
/benchmarkError 4h 19m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.003s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 315, in feature
    scenario(format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 265, in s3
    benchmark(table_name="s3", table_settings=table_settings, format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 238, in benchmark
    node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 735, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 3600.000s for '(bash# )\|(\n)'
/benchmark/aws_s3Error 4h 19m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.003s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 315, in feature
    scenario(format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 265, in s3
    benchmark(table_name="s3", table_settings=table_settings, format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 238, in benchmark
    node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 735, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 3600.000s for '(bash# )\|(\n)'
/benchmark/aws_s3/queriesError 4h 16m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.003s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 315, in feature
    scenario(format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 265, in s3
    benchmark(table_name="s3", table_settings=table_settings, format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 238, in benchmark
    node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 735, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 3600.000s for '(bash# )\|(\n)'
/benchmark/aws_s3/queries/s3Error 1h 35m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.003s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 315, in feature
    scenario(format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 265, in s3
    benchmark(table_name="s3", table_settings=table_settings, format=format)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 238, in benchmark
    node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 735, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 3600.000s for '(bash# )\|(\n)'

Results

Test Name Result Duration
/benchmark Error 4h 19m
/benchmark/aws_s3 Error 4h 19m
/benchmark/aws_s3/queries Error 4h 16m
/benchmark/aws_s3/queries/default OK 12m 53s
/benchmark/aws_s3/queries/s3 Error 1h 35m
/benchmark/aws_s3/queries/s3 tiered OK 1h 25m
/benchmark/aws_s3/queries/zero copy replication OK 1h 2m

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122