Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateMar 01, 2024 21:26
Duration54m 40s
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/19a57e6f4a8ca8dd7737ac12f3ba472b37ac166c/package_release/clickhouse-common-static_23.3.19.33.altinitystable_amd64.deb
version23.3.19.33.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash6da94b78dc53cb8965ab56c04a89ebf54ed04cbc
job.id8115174679
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8115174679
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/19a57e6f4a8ca8dd7737ac12f3ba472b37ac166c/package_release/clickhouse-common-static_23.3.19.33.altinitystable_amd64.deb
stressFalse
allow_vfsFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.6%OK
19.9%Known

Statistics

Units Skip OK Fail XFail
Modules
3
3
Features
9
9
Scenarios
144
3
101
40
Checks
3
3
Examples
42
42
Steps
14099
14092
7

Known Fails

Test NameResultMessage
/s3/minio/normal/disk/cacheXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache defaultXFail 954us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache pathXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/low cardinality offsetXFail 39s 969ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 287, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2791, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2746, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2715, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2715 in 'low_cardinality_offset'

2707\|                          "1",
2708\|                      ),
2709\|                      (
2710\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2711\|                          "1",
2712\|                      ),
2713\|                  ],
2714\|              ).output
2715\|>             assert output == "23999\n", error()
2716\|  
2717\|      finally:
2718\|          with Finally(f"I remove the table {name}"):
/s3/minio/normal/invalid disk/cache path conflictXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/normal/zero copy replication/bad detached partXFail 34s 557ms
https://github.com/ClickHouse/ClickHouse/pull/58333
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1663, in bad_detached_part
    r = nodes[0].query(f"SELECT * FROM {table_name}")
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Received exception from server (version 23.3.19):
Code: 499. DB::Exception: Received from localhost:9000. DB::Exception: The specified key does not exist.: while reading key: data/zero-copy-replication/clm/svoydvjditeondreidlovhbnmusng, from bucket: root: (while reading column d): (while reading from part /var/lib/clickhouse/disks/external/store/084/08411cdd-c8c5-4e1f-92e9-98a1b203d771/all_0_0_0/ from mark 0 with max_rows_to_read = 1): While executing MergeTreeInOrder. (S3_ERROR)
(query: SELECT * FROM detach_table
)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/s3/minio/normal/zero copy replication/deleteXFail 35s 930ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1217, in delete
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 197841533
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 191535022
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = 6306511
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 6306511
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)
/s3/minio/normal/zero copy replication/ttl deleteXFail 36s 408ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1590, in ttl_delete
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 53852267
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 106134097
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = -52281830
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 52281830
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)
/s3/minio/normal/zero copy replication/ttl moveXFail 35s 566ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 299, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2278, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1444, in ttl_move
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 47545756
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 53852267
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = -6306511
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 6306511
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)

Results

Test Name Result Duration
/s3 OK 54m 40s
/s3/minio OK 54m 40s
/s3/minio/normal OK 52m 14s
/s3/minio/normal/table function OK 1m 16s
/s3/minio/normal/table function/auto OK 8s 743ms
/s3/minio/normal/table function/compression OK 8s 974ms
/s3/minio/normal/table function/credentials OK 1s 79ms
/s3/minio/normal/table function/data format OK 7s 363ms
/s3/minio/normal/table function/multipart OK 3s 673ms
/s3/minio/normal/table function/multiple columns OK 1s 300ms
/s3/minio/normal/table function/partition OK 363ms
/s3/minio/normal/table function/remote host filter OK 37s 760ms
/s3/minio/normal/table function/syntax OK 950ms
/s3/minio/normal/table function/wildcard OK 6s 451ms
/s3/minio/normal/backup OK 2m 37s
/s3/minio/normal/backup/local and s3 disk OK 52s 180ms
/s3/minio/normal/backup/local and s3 volumes OK 52s 506ms
/s3/minio/normal/backup/s3 disk OK 52s 568ms
/s3/minio/normal/invalid table function OK 3s 22ms
/s3/minio/normal/invalid table function/empty path OK 384ms
/s3/minio/normal/invalid table function/empty structure OK 387ms
/s3/minio/normal/invalid table function/invalid compression OK 405ms
/s3/minio/normal/invalid table function/invalid credentials OK 405ms
/s3/minio/normal/invalid table function/invalid format OK 705ms
/s3/minio/normal/invalid table function/invalid path OK 370ms
/s3/minio/normal/invalid table function/invalid structure OK 342ms
/s3/minio/normal/disk OK 24m 59s
/s3/minio/normal/disk/access OK 38s 151ms
/s3/minio/normal/disk/access skip check OK 37s 944ms
/s3/minio/normal/disk/add storage OK 1m 15s
/s3/minio/normal/disk/alter move OK 44s 649ms
/s3/minio/normal/disk/alter on cluster modify ttl OK 1m 16s
/s3/minio/normal/disk/cache XFail 1ms
/s3/minio/normal/disk/cache default XFail 954us
/s3/minio/normal/disk/cache path XFail 1ms
/s3/minio/normal/disk/compact parts OK 40s 742ms
/s3/minio/normal/disk/config over restart OK 51s 940ms
/s3/minio/normal/disk/default move factor OK 46s 451ms
/s3/minio/normal/disk/delete OK 1m 34s
/s3/minio/normal/disk/download appropriate disk OK 1m 4s
/s3/minio/normal/disk/environment credentials Skip 1ms
/s3/minio/normal/disk/exports OK 42s 146ms
/s3/minio/normal/disk/generic url Skip 1ms
/s3/minio/normal/disk/imports OK 42s 304ms
/s3/minio/normal/disk/low cardinality offset XFail 39s 969ms
/s3/minio/normal/disk/max single part upload size syntax OK 40s 157ms
/s3/minio/normal/disk/mergetree OK 2m 50s
/s3/minio/normal/disk/mergetree collapsing OK 42s 787ms
/s3/minio/normal/disk/mergetree versionedcollapsing OK 42s 946ms
/s3/minio/normal/disk/metadata OK 42s 323ms
/s3/minio/normal/disk/min bytes for seek syntax OK 40s 207ms
/s3/minio/normal/disk/multiple storage OK 44s 414ms
/s3/minio/normal/disk/multiple storage query OK 42s 42ms
/s3/minio/normal/disk/no restart OK 12s 167ms
/s3/minio/normal/disk/perform ttl move on insert OK 1m 16s
/s3/minio/normal/disk/perform ttl move on insert default OK 40s 634ms
/s3/minio/normal/disk/performance ttl move OK 54s 588ms
/s3/minio/normal/disk/remote host filter OK 1m 15s
/s3/minio/normal/disk/specific url Skip 1ms
/s3/minio/normal/disk/syntax OK 40s 771ms
/s3/minio/normal/disk/wide parts OK 38s 313ms
/s3/minio/normal/invalid disk OK 1m 51s
/s3/minio/normal/invalid disk/access default OK 14s 829ms
/s3/minio/normal/invalid disk/access failed OK 15s 774ms
/s3/minio/normal/invalid disk/access failed skip check OK 41s 519ms
/s3/minio/normal/invalid disk/cache path conflict XFail 1ms
/s3/minio/normal/invalid disk/empty endpoint OK 9s 773ms
/s3/minio/normal/invalid disk/invalid endpoint OK 9s 757ms
/s3/minio/normal/invalid disk/invalid type OK 19s 466ms
/s3/minio/normal/sanity OK 41s 380ms
/s3/minio/normal/sanity/sanity OK 5s 508ms
/s3/minio/normal/reconnect OK 2m 10s
/s3/minio/normal/reconnect/local and s3 disk OK 42s 843ms
/s3/minio/normal/reconnect/local and s3 volumes OK 42s 815ms
/s3/minio/normal/reconnect/s3 disk OK 44s 732ms
/s3/minio/normal/zero copy replication OK 18m 33s
/s3/minio/normal/zero copy replication/bucket should be empty before test begins OK 7ms
/s3/minio/normal/zero copy replication/add replica OK 35s 927ms
/s3/minio/normal/zero copy replication/alter OK 1m 8s
/s3/minio/normal/zero copy replication/alter repeat OK 6m 17s
/s3/minio/normal/zero copy replication/bad detached part XFail 34s 557ms
/s3/minio/normal/zero copy replication/check refcount after mutation OK 1s 210ms
/s3/minio/normal/zero copy replication/consistency during conflicting mutation OK 14s 262ms
/s3/minio/normal/zero copy replication/consistency during double mutation OK 12s 208ms
/s3/minio/normal/zero copy replication/default value OK 10s 580ms
/s3/minio/normal/zero copy replication/delete XFail 35s 930ms
/s3/minio/normal/zero copy replication/delete all OK 35s 887ms
/s3/minio/normal/zero copy replication/drop alter replica OK 46s 25ms
/s3/minio/normal/zero copy replication/drop replica OK 47s 750ms
/s3/minio/normal/zero copy replication/global setting OK 34s 271ms
/s3/minio/normal/zero copy replication/insert multiple replicas OK 36s 401ms
/s3/minio/normal/zero copy replication/metadata OK 33s 328ms
/s3/minio/normal/zero copy replication/performance alter OK 1m 0s
/s3/minio/normal/zero copy replication/performance insert OK 57s 781ms
/s3/minio/normal/zero copy replication/performance select OK 1m 0s
/s3/minio/normal/zero copy replication/ttl delete XFail 36s 408ms
/s3/minio/normal/zero copy replication/ttl move XFail 35s 566ms
/s3/minio/normal/cit OK 1s 200ms
/s3/minio/normal/cit/test empty put OK 277ms
/s3/minio/normal/cit/test put OK 314ms
/s3/minio/normal/cit/test put csv OK 336ms
/s3/minio/normal/cit/test put get with redirect OK 245ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232