Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateMar 19, 2024 17:46
Duration51m 46s
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/0a8ac3b092733da37e3e2a0079c486938a36790d/package_release/clickhouse-common-static_23.3.19.34.altinityfips_amd64.deb
version23.3.19.34.altinityfips
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash6da94b78dc53cb8965ab56c04a89ebf54ed04cbc
job.id8247806903
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8247806903
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/0a8ac3b092733da37e3e2a0079c486938a36790d/package_release/clickhouse-common-static_23.3.19.34.altinityfips_amd64.deb
stressFalse
allow_vfsFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.6%OK
19.9%Known

Statistics

Units Skip OK Fail XFail
Modules
3
3
Features
9
9
Scenarios
144
3
101
40
Checks
3
3
Examples
42
42
Steps
14099
14092
7

Known Fails

Test NameResultMessage
/s3/minio/normal/disk/cacheXFail 925us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache defaultXFail 639us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache pathXFail 656us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/low cardinality offsetXFail 35s 493ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 287, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2791, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2746, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2715, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2715 in 'low_cardinality_offset'

2707\|                          "1",
2708\|                      ),
2709\|                      (
2710\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2711\|                          "1",
2712\|                      ),
2713\|                  ],
2714\|              ).output
2715\|>             assert output == "23999\n", error()
2716\|  
2717\|      finally:
2718\|          with Finally(f"I remove the table {name}"):
/s3/minio/normal/invalid disk/cache path conflictXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/normal/zero copy replication/bad detached partXFail 31s 955ms
https://github.com/ClickHouse/ClickHouse/pull/58333
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1663, in bad_detached_part
    r = nodes[0].query(f"SELECT * FROM {table_name}")
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Received exception from server (version 23.3.19):
Code: 499. DB::Exception: Received from localhost:9000. DB::Exception: The specified key does not exist.: while reading key: data/zero-copy-replication/pex/ieoimgrzdvbwdtfyopyouitmwsdhz, from bucket: root: (while reading column d): (while reading from part /var/lib/clickhouse/disks/external/store/e7b/e7b10216-593a-4dbc-8b31-2f831c4cc2a0/all_0_0_0/ from mark 0 with max_rows_to_read = 1): While executing MergeTreeInOrder. (S3_ERROR)
(query: SELECT * FROM detach_table
)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/s3/minio/normal/zero copy replication/deleteXFail 32s 409ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1217, in delete
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 203293655
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 196987144
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = 6306511
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 6306511
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)
/s3/minio/normal/zero copy replication/ttl deleteXFail 35s 915ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1590, in ttl_delete
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 156789498
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 208536535
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = -51747037
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 51747037
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)
/s3/minio/normal/zero copy replication/ttl moveXFail 35s 936ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1444, in ttl_move
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 53748330
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 106029047
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = -52280717
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 52280717
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)

Results

Test Name Result Duration
/s3 OK 51m 46s
/s3/minio OK 51m 46s
/s3/minio/normal OK 49m 32s
/s3/minio/normal/table function OK 1m 0s
/s3/minio/normal/table function/auto OK 5s 775ms
/s3/minio/normal/table function/compression OK 5s 663ms
/s3/minio/normal/table function/credentials OK 650ms
/s3/minio/normal/table function/data format OK 4s 962ms
/s3/minio/normal/table function/multipart OK 2s 491ms
/s3/minio/normal/table function/multiple columns OK 859ms
/s3/minio/normal/table function/partition OK 263ms
/s3/minio/normal/table function/remote host filter OK 35s 173ms
/s3/minio/normal/table function/syntax OK 634ms
/s3/minio/normal/table function/wildcard OK 4s 229ms
/s3/minio/normal/backup OK 2m 29s
/s3/minio/normal/backup/local and s3 disk OK 49s 168ms
/s3/minio/normal/backup/local and s3 volumes OK 49s 418ms
/s3/minio/normal/backup/s3 disk OK 51s 315ms
/s3/minio/normal/invalid table function OK 2s 32ms
/s3/minio/normal/invalid table function/empty path OK 262ms
/s3/minio/normal/invalid table function/empty structure OK 239ms
/s3/minio/normal/invalid table function/invalid compression OK 262ms
/s3/minio/normal/invalid table function/invalid credentials OK 251ms
/s3/minio/normal/invalid table function/invalid format OK 477ms
/s3/minio/normal/invalid table function/invalid path OK 268ms
/s3/minio/normal/invalid table function/invalid structure OK 258ms
/s3/minio/normal/disk OK 23m 9s
/s3/minio/normal/disk/access OK 39s 655ms
/s3/minio/normal/disk/access skip check OK 39s 449ms
/s3/minio/normal/disk/add storage OK 1m 19s
/s3/minio/normal/disk/alter move OK 44s 8ms
/s3/minio/normal/disk/alter on cluster modify ttl OK 1m 14s
/s3/minio/normal/disk/cache XFail 925us
/s3/minio/normal/disk/cache default XFail 639us
/s3/minio/normal/disk/cache path XFail 656us
/s3/minio/normal/disk/compact parts OK 39s 371ms
/s3/minio/normal/disk/config over restart OK 49s 314ms
/s3/minio/normal/disk/default move factor OK 41s 908ms
/s3/minio/normal/disk/delete OK 1m 29s
/s3/minio/normal/disk/download appropriate disk OK 57s 282ms
/s3/minio/normal/disk/environment credentials Skip 675us
/s3/minio/normal/disk/exports OK 35s 104ms
/s3/minio/normal/disk/generic url Skip 1ms
/s3/minio/normal/disk/imports OK 36s 108ms
/s3/minio/normal/disk/low cardinality offset XFail 35s 493ms
/s3/minio/normal/disk/max single part upload size syntax OK 35s 546ms
/s3/minio/normal/disk/mergetree OK 2m 29s
/s3/minio/normal/disk/mergetree collapsing OK 37s 583ms
/s3/minio/normal/disk/mergetree versionedcollapsing OK 37s 901ms
/s3/minio/normal/disk/metadata OK 38s 113ms
/s3/minio/normal/disk/min bytes for seek syntax OK 35s 601ms
/s3/minio/normal/disk/multiple storage OK 39s 299ms
/s3/minio/normal/disk/multiple storage query OK 39s 537ms
/s3/minio/normal/disk/no restart OK 11s 435ms
/s3/minio/normal/disk/perform ttl move on insert OK 1m 12s
/s3/minio/normal/disk/perform ttl move on insert default OK 38s 37ms
/s3/minio/normal/disk/performance ttl move OK 49s 730ms
/s3/minio/normal/disk/remote host filter OK 1m 10s
/s3/minio/normal/disk/specific url Skip 965us
/s3/minio/normal/disk/syntax OK 37s 286ms
/s3/minio/normal/disk/wide parts OK 35s 680ms
/s3/minio/normal/invalid disk OK 1m 54s
/s3/minio/normal/invalid disk/access default OK 17s 603ms
/s3/minio/normal/invalid disk/access failed OK 18s 564ms
/s3/minio/normal/invalid disk/access failed skip check OK 40s 179ms
/s3/minio/normal/invalid disk/cache path conflict XFail 1ms
/s3/minio/normal/invalid disk/empty endpoint OK 9s 511ms
/s3/minio/normal/invalid disk/invalid endpoint OK 9s 507ms
/s3/minio/normal/invalid disk/invalid type OK 18s 996ms
/s3/minio/normal/sanity OK 40s 6ms
/s3/minio/normal/sanity/sanity OK 3s 707ms
/s3/minio/normal/reconnect OK 2m 4s
/s3/minio/normal/reconnect/local and s3 disk OK 41s 503ms
/s3/minio/normal/reconnect/local and s3 volumes OK 41s 629ms
/s3/minio/normal/reconnect/s3 disk OK 41s 691ms
/s3/minio/normal/zero copy replication OK 18m 11s
/s3/minio/normal/zero copy replication/bucket should be empty before test begins OK 4ms
/s3/minio/normal/zero copy replication/add replica OK 31s 700ms
/s3/minio/normal/zero copy replication/alter OK 1m 2s
/s3/minio/normal/zero copy replication/alter repeat OK 6m 37s
/s3/minio/normal/zero copy replication/bad detached part XFail 31s 955ms
/s3/minio/normal/zero copy replication/check refcount after mutation OK 1s 82ms
/s3/minio/normal/zero copy replication/consistency during conflicting mutation OK 14s 778ms
/s3/minio/normal/zero copy replication/consistency during double mutation OK 12s 104ms
/s3/minio/normal/zero copy replication/default value OK 10s 224ms
/s3/minio/normal/zero copy replication/delete XFail 32s 409ms
/s3/minio/normal/zero copy replication/delete all OK 33s 471ms
/s3/minio/normal/zero copy replication/drop alter replica OK 44s 406ms
/s3/minio/normal/zero copy replication/drop replica OK 43s 910ms
/s3/minio/normal/zero copy replication/global setting OK 31s 614ms
/s3/minio/normal/zero copy replication/insert multiple replicas OK 34s 865ms
/s3/minio/normal/zero copy replication/metadata OK 34s 39ms
/s3/minio/normal/zero copy replication/performance alter OK 55s 649ms
/s3/minio/normal/zero copy replication/performance insert OK 54s 215ms
/s3/minio/normal/zero copy replication/performance select OK 56s 287ms
/s3/minio/normal/zero copy replication/ttl delete XFail 35s 915ms
/s3/minio/normal/zero copy replication/ttl move XFail 35s 936ms
/s3/minio/normal/cit OK 776ms
/s3/minio/normal/cit/test empty put OK 180ms
/s3/minio/normal/cit/test put OK 217ms
/s3/minio/normal/cit/test put csv OK 200ms
/s3/minio/normal/cit/test put get with redirect OK 156ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232