Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Benchmark Test Run Report

DateApr 13, 2023 13:33
Duration2h 44m
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/22.8.15.25.altinitystable/4688213790/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/22.8/a454ebb2761c2732ab39211459f19fa026dc8527/package_release/clickhouse-common-static_22.8.15.25.altinitystable_amd64.deb
version22.8.15.25.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash4023e0eafd6319edc79b82d79ab800c0c3cf20ec
job.id4688213790
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/4688213790
archx86_64
nodeclickhouse1
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/22.8/a454ebb2761c2732ab39211459f19fa026dc8527/package_release/clickhouse-common-static_22.8.15.25.altinitystable_amd64.deb
stressFalse
collect_service_logsTrue
storages['aws_s3']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriNone
gcs_key_idNone
gcs_key_secretNone
formatNone

Summary

96.7%OK
3.3%Error

Statistics

Units OK Error
Modules
1
1
Suites
2
2
Features
2
2
Scenarios
208
206
2
Steps
1305
1299
6

Fails

Test NameResultMessage
/benchmarkError 2h 44m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.090s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 312, in feature
    scenario(format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 276, in s3_tiered
    benchmark(table_name="tiered", table_settings=table_settings, format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 235, in benchmark
    node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 721, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 1200.000s for '(bash# )\|(\n)'
/benchmark/aws_s3Error 2h 44m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.090s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 312, in feature
    scenario(format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 276, in s3_tiered
    benchmark(table_name="tiered", table_settings=table_settings, format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 235, in benchmark
    node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 721, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 1200.000s for '(bash# )\|(\n)'
/benchmark/aws_s3/queriesError 2h 41m
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.090s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 312, in feature
    scenario(format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 276, in s3_tiered
    benchmark(table_name="tiered", table_settings=table_settings, format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 235, in benchmark
    node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 721, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 1200.000s for '(bash# )\|(\n)'
/benchmark/aws_s3/queries/s3 tieredError 48m 55s
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.090s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 312, in feature
    scenario(format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 276, in s3_tiered
    benchmark(table_name="tiered", table_settings=table_settings, format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 235, in benchmark
    node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 721, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 1200.000s for '(bash# )\|(\n)'
/benchmark/aws_s3/queries/zero copy replicationError 57m 28s
ExpectTimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 223, in read
    d = self.queue.get(timeout=timeleft)
  File "/usr/lib/python3.10/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 187, in expect
    data = self.read(timeout=min(timeleft, 0.1), raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/testflows/uexpect/uexpect.py", line 235, in read
    raise TimeoutError(timeout)
testflows.uexpect.uexpect.TimeoutError: Timeout 0.068s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 171, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/benchmark.py", line 165, in regression
    Feature(test=load("ontime_benchmark.tests.benchmark", "feature"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 312, in feature
    scenario(format=format)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 298, in zero_copy_replication
    benchmark(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../ontime_benchmark/tests/benchmark.py", line 235, in benchmark
    node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ontime_benchmark/../helpers/cluster.py", line 721, in query
    r = self.cluster.bash(self.name)(command, *args, **kwargs)
testflows.uexpect.uexpect.ExpectTimeoutError: Timeout 1200.000s for '(bash# )\|(\n)'

Results

Test Name Result Duration
/benchmark Error 2h 44m
/benchmark/aws_s3 Error 2h 44m
/benchmark/aws_s3/queries Error 2h 41m
/benchmark/aws_s3/queries/default OK 6m 49s
/benchmark/aws_s3/queries/s3 OK 48m 22s
/benchmark/aws_s3/queries/s3 tiered Error 48m 55s
/benchmark/aws_s3/queries/zero copy replication Error 57m 28s

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122