Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateJul 22, 2024 9:28
Duration2h 4m
Framework TestFlows 2.0.240705.1133204

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/b6642564dbf296d6e55b907d7af6c8087470b672/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/24.3/b6642564dbf296d6e55b907d7af6c8087470b672/package_release/clickhouse-common-static_24.3.5.47.altinitystable_amd64.deb
version24.3.5.47.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashc5e1513a2214ee33696c29717935e0a94989ac2a
job.id10037417827
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/10037417827
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/24.3/b6642564dbf296d6e55b907d7af6c8087470b672/package_release/clickhouse-common-static_24.3.5.47.altinitystable_amd64.deb
keeper_binary_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerFalse
reuse_envFalse
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

87.4%OK
11.8%Known

Statistics

Units Skip OK Fail XFail
Modules
2
2
Features
13
13
Scenarios
184
3
142
39
Checks
15
15
Examples
71
70
1
Steps
175354
175328
12
14

Known Fails

Test NameResultMessage
/s3/minio/disk/cacheXFail 2ms
Under development for 22.8 and newer.
None
/s3/minio/disk/cache defaultXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/disk/cache pathXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/disk/low cardinality offsetXFail 41s 232ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 323, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2461, in minio
    disk_tests(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2431, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2394, in low_cardinality_offset
    assert output == "23999\n", error()
           ^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2394 in 'low_cardinality_offset'

2386\|                      "1",
2387\|                  ),
2388\|                  (
2389\|                      "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2390\|                      "1",
2391\|                  ),
2392\|              ],
2393\|          ).output
2394\|>         assert output == "23999\n", error()
2395\|  
2396\|  
2397\|  @TestFeature
/s3/minio/disk/no restartXFail 2ms
https://github.com/ClickHouse/ClickHouse/issues/58924
None
/s3/minio/invalid disk/cache path conflictXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/zero copy replication/deleteXFail 10m 33s
test is slow to clean up, needs investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 332, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1720, in minio
    outline(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1692, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 826, in delete
    check_bucket_size(expected_size=size_before, tolerance=5)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 814, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 19082064
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 122553608
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = -103471544
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 103471544
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 814 in 'check_bucket_size'

806\|  ):
807\|      current_size = get_bucket_size(
808\|          name=name,
809\|          prefix=prefix,
810\|          minio_enabled=minio_enabled,
811\|          access_key=self.context.secret_access_key,
812\|          key_id=self.context.access_key_id,
813\|      )
814\|>     assert abs(current_size - expected_size) <= tolerance, error()
815\|  
816\|  
817\|  @TestStep(When)
/s3/minio/zero copy replication/ttl deleteXFail 1m 46s
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 332, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1720, in minio
    outline(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1692, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1042, in ttl_delete
    check_query_node(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 720, in check_query_node
    assert r == expected, error()
           ^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r == expected, error()

Assertion values
  assert r == expected, error()
         ^ is '1441794'
  assert r == expected, error()
              ^ is '1310721'
  assert r == expected, error()
           ^ is = False
    @@ -1 +1 @@
    -1441794
    +1310721
  assert r == expected, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 720 in 'check_query_node'

712\|  
713\|  @TestStep(Then)
714\|  def check_query_node(self, node, num, query, expected):
715\|      node = current().context.node
716\|  
717\|      with By(f"executing query {num}", description=query):
718\|          r = node.query(query).output.strip()
719\|          with Then(f"result should match the expected", description=expected):
720\|>             assert r == expected, error()
721\|  
722\|  
723\|  def get_s3_file_content(cluster, bucket, filename, decode=True):

Results

Test Name Result Duration
/s3 OK 2h 4m
/s3/minio OK 2h 4m
/s3/minio/sanity OK 46s 585ms
/s3/minio/sanity/sanity OK 6s 549ms
/s3/minio/table function OK 2m 4s
/s3/minio/table function/auto OK 7s 800ms
/s3/minio/table function/compression OK 7s 915ms
/s3/minio/table function/credentials OK 893ms
/s3/minio/table function/data format OK 8s 262ms
/s3/minio/table function/measure file size OK 43s 246ms
/s3/minio/table function/multipart OK 4s 69ms
/s3/minio/table function/multiple columns OK 1s 283ms
/s3/minio/table function/partition OK 464ms
/s3/minio/table function/remote host filter OK 40s 797ms
/s3/minio/table function/syntax OK 1s 344ms
/s3/minio/table function/wildcard OK 7s 964ms
/s3/minio/backup OK 5m 40s
/s3/minio/backup/local and s3 disk OK 1m 52s
/s3/minio/backup/local and s3 volumes OK 1m 54s
/s3/minio/backup/s3 disk OK 1m 54s
/s3/minio/invalid table function OK 3s 61ms
/s3/minio/invalid table function/empty path OK 374ms
/s3/minio/invalid table function/empty structure OK 334ms
/s3/minio/invalid table function/invalid compression OK 387ms
/s3/minio/invalid table function/invalid credentials OK 533ms
/s3/minio/invalid table function/invalid format OK 708ms
/s3/minio/invalid table function/invalid path OK 348ms
/s3/minio/invalid table function/invalid structure OK 358ms
/s3/minio/disk OK 37m 8s
/s3/minio/disk/access OK 39s 963ms
/s3/minio/disk/access skip check OK 40s 736ms
/s3/minio/disk/add storage OK 1m 22s
/s3/minio/disk/alter move OK 46s 820ms
/s3/minio/disk/alter on cluster modify ttl OK 1m 21s
/s3/minio/disk/cache XFail 2ms
/s3/minio/disk/cache default XFail 1ms
/s3/minio/disk/cache path XFail 1ms
/s3/minio/disk/compact parts OK 40s 704ms
/s3/minio/disk/config over restart OK 54s 420ms
/s3/minio/disk/default move factor OK 52s 929ms
/s3/minio/disk/delete OK 1m 40s
/s3/minio/disk/download appropriate disk OK 1m 6s
/s3/minio/disk/drop sync OK 1m 12s
/s3/minio/disk/environment credentials Skip 2ms
/s3/minio/disk/exports OK 40s 924ms
/s3/minio/disk/generic url Skip 2ms
/s3/minio/disk/imports OK 40s 975ms
/s3/minio/disk/log OK 4m 11s
/s3/minio/disk/low cardinality offset XFail 41s 232ms
/s3/minio/disk/max single part upload size syntax OK 41s 359ms
/s3/minio/disk/mergetree OK 7m 26s
/s3/minio/disk/mergetree collapsing OK 1m 24s
/s3/minio/disk/mergetree versionedcollapsing OK 1m 25s
/s3/minio/disk/metadata OK 43s 174ms
/s3/minio/disk/min bytes for seek syntax OK 40s 618ms
/s3/minio/disk/multiple storage OK 45s 394ms
/s3/minio/disk/multiple storage query OK 44s 739ms
/s3/minio/disk/no restart XFail 2ms
/s3/minio/disk/perform ttl move on insert OK 1m 18s
/s3/minio/disk/perform ttl move on insert default OK 43s 726ms
/s3/minio/disk/performance ttl move OK 1m 1s
/s3/minio/disk/remote host filter OK 1m 19s
/s3/minio/disk/specific url Skip 1ms
/s3/minio/disk/syntax OK 41s 341ms
/s3/minio/disk/wide parts OK 38s 892ms
/s3/minio/invalid disk OK 3m 46s
/s3/minio/invalid disk/access default OK 11s 927ms
/s3/minio/invalid disk/access failed OK 12s 932ms
/s3/minio/invalid disk/access failed skip check OK 40s 277ms
/s3/minio/invalid disk/cache path conflict XFail 1ms
/s3/minio/invalid disk/empty endpoint OK 9s 937ms
/s3/minio/invalid disk/invalid endpoint OK 2m 11s
/s3/minio/invalid disk/invalid type OK 19s 818ms
/s3/minio/alter OK 1m 10s
/s3/minio/alter/attach from OK 1s 944ms
/s3/minio/alter/columns OK 1s 795ms
/s3/minio/alter/detach OK 2s 369ms
/s3/minio/alter/drop OK 4s 918ms
/s3/minio/alter/fetch OK 3s 173ms
/s3/minio/alter/freeze OK 1s 577ms
/s3/minio/alter/index OK 6s 91ms
/s3/minio/alter/move to table OK 2s 159ms
/s3/minio/alter/order by OK 671ms
/s3/minio/alter/projection OK 1s 172ms
/s3/minio/alter/replace OK 3s 315ms
/s3/minio/alter/sample by OK 710ms
/s3/minio/alter/update delete OK 1s 155ms
/s3/minio/combinatoric table OK 3m 47s
/s3/minio/combinatoric table/engine=MergeTree,replicated=False,n_cols=500,n_tables=1,part_type=wide OK 7s 693ms
/s3/minio/combinatoric table/engine=ReplacingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=unspecified OK 20s 162ms
/s3/minio/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=10,n_tables=1,part_type=wide OK 6s 319ms
/s3/minio/combinatoric table/engine=CollapsingMergeTree,replicated=False,n_cols=2000,n_tables=3,part_type=unspecified OK 6s 243ms
/s3/minio/combinatoric table/engine=VersionedCollapsingMergeTree,replicated=False,n_cols=10,n_tables=3,part_type=compact OK 1s 570ms
/s3/minio/combinatoric table/engine=AggregatingMergeTree,replicated=False,n_cols=500,n_tables=1,part_type=unspecified OK 1s 154ms
/s3/minio/combinatoric table/engine=SummingMergeTree,replicated=False,n_cols=500,n_tables=1,part_type=compact OK 7s 565ms
/s3/minio/combinatoric table/engine=MergeTree,replicated=True,n_cols=10,n_tables=3,part_type=unspecified OK 7s 843ms
/s3/minio/combinatoric table/engine=ReplacingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 18s 869ms
/s3/minio/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=compact OK 57s 630ms
/s3/minio/combinatoric table/engine=VersionedCollapsingMergeTree,replicated=True,n_cols=500,n_tables=1,part_type=unspecified OK 6s 729ms
/s3/minio/combinatoric table/engine=AggregatingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 18s 914ms
/s3/minio/combinatoric table/engine=SummingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 18s 907ms
/s3/minio/combinatoric table/engine=SummingMergeTree,replicated=True,n_cols=10,n_tables=1,part_type=unspecified OK 6s 31ms
/s3/minio/reconnect OK 2m 22s
/s3/minio/reconnect/local and s3 disk OK 47s 462ms
/s3/minio/reconnect/local and s3 volumes OK 47s 411ms
/s3/minio/reconnect/s3 disk OK 47s 825ms
/s3/minio/zero copy replication OK 37m 11s
/s3/minio/zero copy replication/add remove one replica OK 37s 241ms
/s3/minio/zero copy replication/add remove replica parallel OK 39s 668ms
/s3/minio/zero copy replication/add replica global setting OK 1m 14s
/s3/minio/zero copy replication/add replica local setting OK 34s 704ms
/s3/minio/zero copy replication/alter OK 4m 25s
/s3/minio/zero copy replication/bad detached part OK 59s 287ms
/s3/minio/zero copy replication/check refcount after mutation OK 1m 11s
/s3/minio/zero copy replication/consistency during conflicting mutation OK 1m 13s
/s3/minio/zero copy replication/consistency during double mutation OK 1m 13s
/s3/minio/zero copy replication/default value OK 12s 19ms
/s3/minio/zero copy replication/delete XFail 10m 33s
/s3/minio/zero copy replication/delete all OK 23s 245ms
/s3/minio/zero copy replication/drop replica OK 35s 929ms
/s3/minio/zero copy replication/global setting OK 1m 3s
/s3/minio/zero copy replication/insert multiple replicas OK 2m 4s
/s3/minio/zero copy replication/metadata OK 1m 2s
/s3/minio/zero copy replication/migration OK 1m 5s
/s3/minio/zero copy replication/offline alter replica OK 35s 385ms
/s3/minio/zero copy replication/performance alter OK 29s 196ms
/s3/minio/zero copy replication/performance insert OK 49s 379ms
/s3/minio/zero copy replication/performance select OK 50s 539ms
/s3/minio/zero copy replication/stale alter replica OK 1m 3s
/s3/minio/zero copy replication/ttl delete XFail 1m 46s
/s3/minio/zero copy replication/ttl move OK 1m 47s
/s3/minio/cit OK 1s 184ms
/s3/minio/cit/test empty put OK 301ms
/s3/minio/cit/test put OK 341ms
/s3/minio/cit/test put csv OK 296ms
/s3/minio/cit/test put get with redirect OK 214ms
/s3/minio/settings OK 12m 55s
/s3/minio/settings/setting combinations OK 12m 14s
/s3/minio/table function performance OK 14m 0s
/s3/minio/table function performance/setup OK 9m 49s
/s3/minio/table function performance/wildcard OK 4m 10s

Generated by TestFlows Open-Source Test Framework v2.0.240705.1133204