Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateOct 05, 2023 14:54
Duration51m 18s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/23.3.13.7.altinitystable/6416627213/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/f040635d5f373c5e4be08e5fbec2622a9889adde/package_release/clickhouse-common-static_23.3.13.7.altinitystable_amd64.deb
version23.3.13.7.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash53dd0b4af71fdeac6704e70f4ff79eacbae2159a
job.id6416627213
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/6416627213
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/f040635d5f373c5e4be08e5fbec2622a9889adde/package_release/clickhouse-common-static_23.3.13.7.altinitystable_amd64.deb
stressFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.7%OK
19.8%Known

Statistics

Units Skip OK Fail XFail
Modules
1
1
Suites
9
9
Features
9
9
Scenarios
141
3
98
40
Examples
42
42
Steps
12682
12675
7

Known Fails

Test NameResultMessage
/s3/minio disk/cacheXFail 848us
Under development for 22.8 and newer.
None
/s3/minio disk/cache defaultXFail 749us
Under development for 22.8 and newer.
None
/s3/minio disk/cache pathXFail 786us
Under development for 22.8 and newer.
None
/s3/minio disk/low cardinality offsetXFail 38s 36ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 251, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2787, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2742, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2711, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2711 in 'low_cardinality_offset'

2703\|                          "1",
2704\|                      ),
2705\|                      (
2706\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2707\|                          "1",
2708\|                      ),
2709\|                  ],
2710\|              ).output
2711\|>             assert output == "23999\n", error()
2712\|  
2713\|      finally:
2714\|          with Finally(f"I remove the table {name}"):
/s3/minio invalid disk/cache path conflictXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio zero copy replication/alterXFail 1m 39s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 863, in alter
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/alter repeatXFail 1m 37s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1025, in alter_repeat
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/deleteXFail 40s 429ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1314, in delete
    assert size_before >= size - 5, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_before >= size - 5, error()

Assertion values
  assert size_before >= size - 5, error()
         ^ is 0
  assert size_before >= size - 5, error()
                        ^ is 6306511
  assert size_before >= size - 5, error()
                             ^ is = 6306506
  assert size_before >= size - 5, error()
                     ^ is = False
  assert size_before >= size - 5, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 1314 in 'delete'

1306\|              ):
1307\|                  size = get_bucket_size(
1308\|                      name=bucket_name,
1309\|                      prefix=bucket_path,
1310\|                      minio_enabled=minio_enabled,
1311\|                      access_key=self.context.secret_access_key,
1312\|                      key_id=self.context.access_key_id,
1313\|                  )
1314\|>                 assert size_before >= size - 5, error()
1315\|                  assert size_before <= size + 5, error()
1316\|  
1317\|          finally:
/s3/minio zero copy replication/ttl deleteXFail 41s 498ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1721, in ttl_delete
    assert size_before <= size + 5, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_before <= size + 5, error()

Assertion values
  assert size_before <= size + 5, error()
         ^ is 60275769
  assert size_before <= size + 5, error()
                        ^ is 53969258
  assert size_before <= size + 5, error()
                             ^ is = 53969263
  assert size_before <= size + 5, error()
                     ^ is = False
  assert size_before <= size + 5, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 1721 in 'ttl_delete'

1713\|              size = get_bucket_size(
1714\|                  name=bucket_name,
1715\|                  prefix=bucket_path,
1716\|                  minio_enabled=minio_enabled,
1717\|                  access_key=self.context.secret_access_key,
1718\|                  key_id=self.context.access_key_id,
1719\|              )
1720\|              assert size_before >= size - 5, error()
1721\|>             assert size_before <= size + 5, error()
1722\|  
1723\|  
1724\|  @TestScenario

Results

Test Name Result Duration
/s3 OK 51m 18s
/s3/minio table function OK 1m 9s
/s3/minio table function/auto OK 6s 16ms
/s3/minio table function/compression OK 5s 867ms
/s3/minio table function/credentials OK 682ms
/s3/minio table function/data format OK 8s 547ms
/s3/minio table function/multipart OK 4s 292ms
/s3/minio table function/multiple columns OK 866ms
/s3/minio table function/partition OK 247ms
/s3/minio table function/remote host filter OK 38s 612ms
/s3/minio table function/syntax OK 623ms
/s3/minio table function/wildcard OK 4s 135ms
/s3/minio backup OK 2m 38s
/s3/minio backup/local and s3 disk OK 51s 704ms
/s3/minio backup/local and s3 volumes OK 52s 605ms
/s3/minio backup/s3 disk OK 53s 856ms
/s3/minio invalid table function OK 2s 1ms
/s3/minio invalid table function/empty path OK 257ms
/s3/minio invalid table function/empty structure OK 242ms
/s3/minio invalid table function/invalid compression OK 254ms
/s3/minio invalid table function/invalid credentials OK 267ms
/s3/minio invalid table function/invalid format OK 467ms
/s3/minio invalid table function/invalid path OK 233ms
/s3/minio invalid table function/invalid structure OK 265ms
/s3/minio disk OK 25m 26s
/s3/minio disk/access OK 37s 928ms
/s3/minio disk/access skip check OK 39s 283ms
/s3/minio disk/add storage OK 1m 18s
/s3/minio disk/alter move OK 45s 646ms
/s3/minio disk/alter on cluster modify ttl OK 1m 20s
/s3/minio disk/cache XFail 848us
/s3/minio disk/cache default XFail 749us
/s3/minio disk/cache path XFail 786us
/s3/minio disk/compact parts OK 39s 958ms
/s3/minio disk/config over restart OK 55s 913ms
/s3/minio disk/default move factor OK 58s 538ms
/s3/minio disk/delete OK 1m 36s
/s3/minio disk/download appropriate disk OK 1m 6s
/s3/minio disk/environment credentials Skip 1ms
/s3/minio disk/exports OK 39s 761ms
/s3/minio disk/generic url Skip 1ms
/s3/minio disk/imports OK 38s 639ms
/s3/minio disk/low cardinality offset XFail 38s 36ms
/s3/minio disk/max single part upload size syntax OK 38s 15ms
/s3/minio disk/mergetree OK 2m 53s
/s3/minio disk/mergetree collapsing OK 43s 185ms
/s3/minio disk/mergetree versionedcollapsing OK 43s 382ms
/s3/minio disk/metadata OK 41s 502ms
/s3/minio disk/min bytes for seek syntax OK 38s 205ms
/s3/minio disk/multiple storage OK 43s 517ms
/s3/minio disk/multiple storage query OK 42s 572ms
/s3/minio disk/perform ttl move on insert OK 1m 17s
/s3/minio disk/perform ttl move on insert default OK 43s 717ms
/s3/minio disk/performance ttl move OK 58s 89ms
/s3/minio disk/remote host filter OK 1m 15s
/s3/minio disk/restart OK 11s 808ms
/s3/minio disk/specific url Skip 1ms
/s3/minio disk/syntax OK 40s 676ms
/s3/minio disk/wide parts OK 39s 139ms
/s3/minio invalid disk OK 1m 50s
/s3/minio invalid disk/access default OK 15s 536ms
/s3/minio invalid disk/access failed OK 15s 546ms
/s3/minio invalid disk/access failed skip check OK 42s 713ms
/s3/minio invalid disk/cache path conflict XFail 1ms
/s3/minio invalid disk/empty endpoint OK 8s 509ms
/s3/minio invalid disk/invalid endpoint OK 9s 529ms
/s3/minio invalid disk/invalid type OK 19s 90ms
/s3/minio sanity OK 48s 628ms
/s3/minio sanity/sanity OK 7s 454ms
/s3/minio reconnect OK 2m 18s
/s3/minio reconnect/local and s3 disk OK 47s 391ms
/s3/minio reconnect/local and s3 volumes OK 45s 478ms
/s3/minio reconnect/s3 disk OK 45s 430ms
/s3/minio zero copy replication OK 14m 26s
/s3/minio zero copy replication/add replica OK 38s 139ms
/s3/minio zero copy replication/alter XFail 1m 39s
/s3/minio zero copy replication/alter repeat XFail 1m 37s
/s3/minio zero copy replication/default value OK 11s 165ms
/s3/minio zero copy replication/delete XFail 40s 429ms
/s3/minio zero copy replication/delete all OK 38s 372ms
/s3/minio zero copy replication/drop alter replica OK 54s 505ms
/s3/minio zero copy replication/drop replica OK 53s 615ms
/s3/minio zero copy replication/global setting OK 41s 580ms
/s3/minio zero copy replication/insert multiple replicas OK 38s 294ms
/s3/minio zero copy replication/lost data during mutation OK 1s 41ms
/s3/minio zero copy replication/metadata OK 38s 762ms
/s3/minio zero copy replication/performance alter OK 1m 4s
/s3/minio zero copy replication/performance insert OK 1m 2s
/s3/minio zero copy replication/performance select OK 1m 3s
/s3/minio zero copy replication/ttl delete XFail 41s 498ms
/s3/minio zero copy replication/ttl move OK 40s 323ms
/s3/cit OK 724ms
/s3/cit/test empty put OK 170ms
/s3/cit/test put OK 189ms
/s3/cit/test put csv OK 207ms
/s3/cit/test put get with redirect OK 135ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122