Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateApr 28, 2024 17:01
Duration50m 17s
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.8/8e9be16153250c8307730c9b88c8517181f494ce/package_release/clickhouse-common-static_23.8.11.29.altinitystable_amd64.deb
version23.8.11.29.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash17a81c07fc1f41fbee651e0ef0ca4b44e537e5b1
job.id8868426423
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8868426423
archx86_64
allow_experimental_analyzerFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.8/8e9be16153250c8307730c9b88c8517181f494ce/package_release/clickhouse-common-static_23.8.11.29.altinitystable_amd64.deb
stressFalse
allow_vfsFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.7%OK
19.8%Known

Statistics

Units Skip OK Fail XFail
Modules
3
3
Features
9
9
Scenarios
143
3
101
39
Checks
3
3
Examples
44
43
1
Steps
13734
13723
11

Known Fails

Test NameResultMessage
/s3/minio/normal/disk/cacheXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache defaultXFail 526us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache pathXFail 804us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/low cardinality offsetXFail 39s 288ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 284, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2425, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2380, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2353, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2353 in 'low_cardinality_offset'

2345\|                      "1",
2346\|                  ),
2347\|                  (
2348\|                      "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2349\|                      "1",
2350\|                  ),
2351\|              ],
2352\|          ).output
2353\|>         assert output == "23999\n", error()
2354\|  
2355\|  
2356\|  @TestFeature
/s3/minio/normal/invalid disk/cache path conflictXFail 796us
Under development for 22.8 and newer.
None
/s3/minio/normal/zero copy replication/bad detached partXFail 31s 734ms
https://github.com/ClickHouse/ClickHouse/pull/58333
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 296, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1317, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1271, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 796, in bad_detached_part
    r = nodes[0].query(f"SELECT * FROM {table_name}")
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Received exception from server (version 23.8.11):
Code: 499. DB::Exception: Received from localhost:9000. DB::Exception: The specified key does not exist.: while reading key: data/zero-copy-replication/pxi/zplqnhghisaaesfvxrhnahitvhgtx, from bucket: root: (while reading column d): (while reading from part /var/lib/clickhouse/disks/external/store/588/588664db-f5e3-426b-8ccf-79c8bef2f8e0/all_0_0_0/ in table default.detach_table (588664db-f5e3-426b-8ccf-79c8bef2f8e0) located on disk external of type s3, from mark 0 with max_rows_to_read = 1): While executing MergeTreeInOrder. (S3_ERROR)
(query: SELECT * FROM detach_table
)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/s3/minio/normal/zero copy replication/deleteXFail 33s 442ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 296, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1317, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1271, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 466, in delete
    check_bucket_size(expected_size=size_before, tolerance=0)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 783, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 174389539
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 161785001
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = 12604538
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 12604538
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 0
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 783 in 'check_bucket_size'

775\|  ):
776\|      current_size = get_bucket_size(
777\|          name=name,
778\|          prefix=prefix,
779\|          minio_enabled=minio_enabled,
780\|          access_key=self.context.secret_access_key,
781\|          key_id=self.context.access_key_id,
782\|      )
783\|>     assert abs(current_size - expected_size) <= tolerance, error()
784\|  
785\|  
786\|  @TestStep(When)
/s3/minio/normal/zero copy replication/ttl deleteXFail 1m 16s
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 296, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1317, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1271, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 27, in measure_buckets_before_and_after
    check_stable_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 853, in check_stable_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 118545868
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 163249477
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = -44703609
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 44703609
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 5
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 853 in 'check_stable_bucket_size'

845\|      current_size = get_stable_bucket_size(
846\|          name=name,
847\|          prefix=prefix,
848\|          minio_enabled=minio_enabled,
849\|          access_key=self.context.secret_access_key,
850\|          key_id=self.context.access_key_id,
851\|          delay=delay,
852\|      )
853\|>     assert abs(current_size - expected_size) <= tolerance, error()
854\|  
855\|  
856\|  @TestStep(Given)

Results

Test Name Result Duration
/s3 OK 50m 17s
/s3/minio OK 50m 17s
/s3/minio/normal OK 49m 0s
/s3/minio/normal/table function OK 55s 175ms
/s3/minio/normal/table function/auto OK 3s 598ms
/s3/minio/normal/table function/compression OK 3s 806ms
/s3/minio/normal/table function/credentials OK 469ms
/s3/minio/normal/table function/data format OK 4s 514ms
/s3/minio/normal/table function/multipart OK 2s 125ms
/s3/minio/normal/table function/multiple columns OK 599ms
/s3/minio/normal/table function/partition OK 237ms
/s3/minio/normal/table function/remote host filter OK 35s 902ms
/s3/minio/normal/table function/syntax OK 782ms
/s3/minio/normal/table function/wildcard OK 3s 121ms
/s3/minio/normal/backup OK 2m 18s
/s3/minio/normal/backup/local and s3 disk OK 45s 38ms
/s3/minio/normal/backup/local and s3 volumes OK 45s 445ms
/s3/minio/normal/backup/s3 disk OK 47s 759ms
/s3/minio/normal/invalid table function OK 1s 387ms
/s3/minio/normal/invalid table function/empty path OK 158ms
/s3/minio/normal/invalid table function/empty structure OK 160ms
/s3/minio/normal/invalid table function/invalid compression OK 173ms
/s3/minio/normal/invalid table function/invalid credentials OK 252ms
/s3/minio/normal/invalid table function/invalid format OK 320ms
/s3/minio/normal/invalid table function/invalid path OK 155ms
/s3/minio/normal/invalid table function/invalid structure OK 157ms
/s3/minio/normal/disk OK 23m 11s
/s3/minio/normal/disk/access OK 36s 107ms
/s3/minio/normal/disk/access skip check OK 35s 207ms
/s3/minio/normal/disk/add storage OK 1m 8s
/s3/minio/normal/disk/alter move OK 41s 579ms
/s3/minio/normal/disk/alter on cluster modify ttl OK 1m 11s
/s3/minio/normal/disk/cache XFail 1ms
/s3/minio/normal/disk/cache default XFail 526us
/s3/minio/normal/disk/cache path XFail 804us
/s3/minio/normal/disk/compact parts OK 35s 161ms
/s3/minio/normal/disk/config over restart OK 45s 846ms
/s3/minio/normal/disk/default move factor OK 41s 851ms
/s3/minio/normal/disk/delete OK 1m 29s
/s3/minio/normal/disk/download appropriate disk OK 57s 665ms
/s3/minio/normal/disk/environment credentials Skip 1ms
/s3/minio/normal/disk/exports OK 35s 629ms
/s3/minio/normal/disk/generic url Skip 2ms
/s3/minio/normal/disk/imports OK 36s 503ms
/s3/minio/normal/disk/low cardinality offset XFail 39s 288ms
/s3/minio/normal/disk/max single part upload size syntax OK 39s 211ms
/s3/minio/normal/disk/mergetree OK 2m 37s
/s3/minio/normal/disk/mergetree collapsing OK 40s 117ms
/s3/minio/normal/disk/mergetree versionedcollapsing OK 37s 333ms
/s3/minio/normal/disk/metadata OK 39s 599ms
/s3/minio/normal/disk/min bytes for seek syntax OK 39s 282ms
/s3/minio/normal/disk/multiple storage OK 37s 988ms
/s3/minio/normal/disk/multiple storage query OK 42s 30ms
/s3/minio/normal/disk/no restart OK 13s 405ms
/s3/minio/normal/disk/perform ttl move on insert OK 1m 15s
/s3/minio/normal/disk/perform ttl move on insert default OK 41s 631ms
/s3/minio/normal/disk/performance ttl move OK 50s 142ms
/s3/minio/normal/disk/remote host filter OK 1m 10s
/s3/minio/normal/disk/specific url Skip 829us
/s3/minio/normal/disk/syntax OK 37s 986ms
/s3/minio/normal/disk/wide parts OK 35s 391ms
/s3/minio/normal/invalid disk OK 3m 52s
/s3/minio/normal/invalid disk/access default OK 18s 596ms
/s3/minio/normal/invalid disk/access failed OK 18s 527ms
/s3/minio/normal/invalid disk/access failed skip check OK 36s 826ms
/s3/minio/normal/invalid disk/cache path conflict XFail 796us
/s3/minio/normal/invalid disk/empty endpoint OK 9s 496ms
/s3/minio/normal/invalid disk/invalid endpoint OK 2m 10s
/s3/minio/normal/invalid disk/invalid type OK 18s 969ms
/s3/minio/normal/sanity OK 37s 733ms
/s3/minio/normal/sanity/sanity OK 3s 567ms
/s3/minio/normal/reconnect OK 2m 1s
/s3/minio/normal/reconnect/local and s3 disk OK 41s 164ms
/s3/minio/normal/reconnect/local and s3 volumes OK 39s 328ms
/s3/minio/normal/reconnect/s3 disk OK 41s 378ms
/s3/minio/normal/zero copy replication OK 16m 0s
/s3/minio/normal/zero copy replication/bucket should be empty before test begins OK 13ms
/s3/minio/normal/zero copy replication/add replica OK 33s 551ms
/s3/minio/normal/zero copy replication/alter OK 3m 54s
/s3/minio/normal/zero copy replication/bad detached part XFail 31s 734ms
/s3/minio/normal/zero copy replication/check refcount after mutation OK 914ms
/s3/minio/normal/zero copy replication/consistency during conflicting mutation OK 13s 145ms
/s3/minio/normal/zero copy replication/consistency during double mutation OK 10s 696ms
/s3/minio/normal/zero copy replication/default value OK 9s 969ms
/s3/minio/normal/zero copy replication/delete XFail 33s 442ms
/s3/minio/normal/zero copy replication/delete all OK 30s 786ms
/s3/minio/normal/zero copy replication/drop alter replica OK 41s 176ms
/s3/minio/normal/zero copy replication/drop replica OK 44s 613ms
/s3/minio/normal/zero copy replication/global setting OK 33s 588ms
/s3/minio/normal/zero copy replication/insert multiple replicas OK 52s 863ms
/s3/minio/normal/zero copy replication/metadata OK 33s 721ms
/s3/minio/normal/zero copy replication/performance alter OK 55s 786ms
/s3/minio/normal/zero copy replication/performance insert OK 56s 825ms
/s3/minio/normal/zero copy replication/performance select OK 55s 300ms
/s3/minio/normal/zero copy replication/ttl delete XFail 1m 16s
/s3/minio/normal/zero copy replication/ttl move OK 1m 15s
/s3/minio/normal/cit OK 668ms
/s3/minio/normal/cit/test empty put OK 147ms
/s3/minio/normal/cit/test put OK 169ms
/s3/minio/normal/cit/test put csv OK 175ms
/s3/minio/normal/cit/test put get with redirect OK 157ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232