Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateApr 28, 2024 17:17
Duration1h 2m
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.8/8e9be16153250c8307730c9b88c8517181f494ce/package_aarch64/clickhouse-common-static_23.8.11.29.altinitystable_arm64.deb
version23.8.11.29.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash17a81c07fc1f41fbee651e0ef0ca4b44e537e5b1
job.id8868426423
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8868426423
archaarch64
allow_experimental_analyzerFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.8/8e9be16153250c8307730c9b88c8517181f494ce/package_aarch64/clickhouse-common-static_23.8.11.29.altinitystable_arm64.deb
stressFalse
allow_vfsFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

79.2%OK
19.3%Known

Statistics

Units Skip OK Fail XFail
Modules
3
3
Features
9
9
Scenarios
143
3
102
38
Checks
3
3
Examples
44
43
1
Steps
13730
13721
9

Known Fails

Test NameResultMessage
/s3/minio/normal/disk/cacheXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache defaultXFail 766us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/cache pathXFail 633us
Under development for 22.8 and newer.
None
/s3/minio/normal/disk/low cardinality offsetXFail 44s 92ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 284, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2425, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2380, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2353, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2353 in 'low_cardinality_offset'

2345\|                      "1",
2346\|                  ),
2347\|                  (
2348\|                      "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2349\|                      "1",
2350\|                  ),
2351\|              ],
2352\|          ).output
2353\|>         assert output == "23999\n", error()
2354\|  
2355\|  
2356\|  @TestFeature
/s3/minio/normal/invalid disk/cache path conflictXFail 2ms
Under development for 22.8 and newer.
None
/s3/minio/normal/zero copy replication/bad detached partXFail 41s 981ms
https://github.com/ClickHouse/ClickHouse/pull/58333
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 296, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1317, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1271, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 796, in bad_detached_part
    r = nodes[0].query(f"SELECT * FROM {table_name}")
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Received exception from server (version 23.8.11):
Code: 499. DB::Exception: Received from localhost:9000. DB::Exception: The specified key does not exist.: while reading key: data/zero-copy-replication/usu/mwhhbewagaimscikatjpxqqgyxapx, from bucket: root: (while reading column d): (while reading from part /var/lib/clickhouse/disks/external/store/4db/4db392a2-7de9-4a91-b6c3-6a0473552dfd/all_0_0_0/ in table default.detach_table (4db392a2-7de9-4a91-b6c3-6a0473552dfd) located on disk external of type s3, from mark 0 with max_rows_to_read = 1): While executing MergeTreeInOrder. (S3_ERROR)
(query: SELECT * FROM detach_table
)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/s3/minio/normal/zero copy replication/deleteXFail 43s 584ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 513, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 504, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 296, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1317, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1271, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 466, in delete
    check_bucket_size(expected_size=size_before, tolerance=0)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 783, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 187430739
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 174826201
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = 12604538
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 12604538
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 0
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 783 in 'check_bucket_size'

775\|  ):
776\|      current_size = get_bucket_size(
777\|          name=name,
778\|          prefix=prefix,
779\|          minio_enabled=minio_enabled,
780\|          access_key=self.context.secret_access_key,
781\|          key_id=self.context.access_key_id,
782\|      )
783\|>     assert abs(current_size - expected_size) <= tolerance, error()
784\|  
785\|  
786\|  @TestStep(When)

Results

Test Name Result Duration
/s3 OK 1h 2m
/s3/minio OK 1h 2m
/s3/minio/normal OK 1h 0m
/s3/minio/normal/table function OK 1m 26s
/s3/minio/normal/table function/auto OK 7s 193ms
/s3/minio/normal/table function/compression OK 7s 228ms
/s3/minio/normal/table function/credentials OK 778ms
/s3/minio/normal/table function/data format OK 12s 551ms
/s3/minio/normal/table function/multipart OK 6s 123ms
/s3/minio/normal/table function/multiple columns OK 1s 170ms
/s3/minio/normal/table function/partition OK 414ms
/s3/minio/normal/table function/remote host filter OK 43s 964ms
/s3/minio/normal/table function/syntax OK 1s 263ms
/s3/minio/normal/table function/wildcard OK 5s 713ms
/s3/minio/normal/backup OK 2m 49s
/s3/minio/normal/backup/local and s3 disk OK 56s 136ms
/s3/minio/normal/backup/local and s3 volumes OK 56s 301ms
/s3/minio/normal/backup/s3 disk OK 57s 251ms
/s3/minio/normal/invalid table function OK 2s 958ms
/s3/minio/normal/invalid table function/empty path OK 338ms
/s3/minio/normal/invalid table function/empty structure OK 330ms
/s3/minio/normal/invalid table function/invalid compression OK 358ms
/s3/minio/normal/invalid table function/invalid credentials OK 521ms
/s3/minio/normal/invalid table function/invalid format OK 704ms
/s3/minio/normal/invalid table function/invalid path OK 347ms
/s3/minio/normal/invalid table function/invalid structure OK 342ms
/s3/minio/normal/disk OK 28m 53s
/s3/minio/normal/disk/access OK 46s 225ms
/s3/minio/normal/disk/access skip check OK 47s 336ms
/s3/minio/normal/disk/add storage OK 1m 30s
/s3/minio/normal/disk/alter move OK 53s 415ms
/s3/minio/normal/disk/alter on cluster modify ttl OK 1m 29s
/s3/minio/normal/disk/cache XFail 1ms
/s3/minio/normal/disk/cache default XFail 766us
/s3/minio/normal/disk/cache path XFail 633us
/s3/minio/normal/disk/compact parts OK 43s 30ms
/s3/minio/normal/disk/config over restart OK 1m 1s
/s3/minio/normal/disk/default move factor OK 1m 9s
/s3/minio/normal/disk/delete OK 1m 48s
/s3/minio/normal/disk/download appropriate disk OK 1m 11s
/s3/minio/normal/disk/environment credentials Skip 934us
/s3/minio/normal/disk/exports OK 43s 29ms
/s3/minio/normal/disk/generic url Skip 1ms
/s3/minio/normal/disk/imports OK 45s 967ms
/s3/minio/normal/disk/low cardinality offset XFail 44s 92ms
/s3/minio/normal/disk/max single part upload size syntax OK 43s 396ms
/s3/minio/normal/disk/mergetree OK 3m 14s
/s3/minio/normal/disk/mergetree collapsing OK 49s 660ms
/s3/minio/normal/disk/mergetree versionedcollapsing OK 47s 777ms
/s3/minio/normal/disk/metadata OK 49s 63ms
/s3/minio/normal/disk/min bytes for seek syntax OK 43s 243ms
/s3/minio/normal/disk/multiple storage OK 49s 621ms
/s3/minio/normal/disk/multiple storage query OK 49s 847ms
/s3/minio/normal/disk/no restart OK 17s 66ms
/s3/minio/normal/disk/perform ttl move on insert OK 1m 27s
/s3/minio/normal/disk/perform ttl move on insert default OK 49s 494ms
/s3/minio/normal/disk/performance ttl move OK 1m 1s
/s3/minio/normal/disk/remote host filter OK 1m 26s
/s3/minio/normal/disk/specific url Skip 1ms
/s3/minio/normal/disk/syntax OK 47s 492ms
/s3/minio/normal/disk/wide parts OK 43s 451ms
/s3/minio/normal/invalid disk OK 3m 51s
/s3/minio/normal/invalid disk/access default OK 18s 928ms
/s3/minio/normal/invalid disk/access failed OK 14s 816ms
/s3/minio/normal/invalid disk/access failed skip check OK 44s 862ms
/s3/minio/normal/invalid disk/cache path conflict XFail 2ms
/s3/minio/normal/invalid disk/empty endpoint OK 8s 867ms
/s3/minio/normal/invalid disk/invalid endpoint OK 2m 6s
/s3/minio/normal/invalid disk/invalid type OK 17s 536ms
/s3/minio/normal/sanity OK 56s 124ms
/s3/minio/normal/sanity/sanity OK 11s 64ms
/s3/minio/normal/reconnect OK 2m 43s
/s3/minio/normal/reconnect/local and s3 disk OK 51s 742ms
/s3/minio/normal/reconnect/local and s3 volumes OK 55s 807ms
/s3/minio/normal/reconnect/s3 disk OK 55s 826ms
/s3/minio/normal/zero copy replication OK 19m 37s
/s3/minio/normal/zero copy replication/bucket should be empty before test begins OK 7ms
/s3/minio/normal/zero copy replication/add replica OK 43s 211ms
/s3/minio/normal/zero copy replication/alter OK 4m 12s
/s3/minio/normal/zero copy replication/bad detached part XFail 41s 981ms
/s3/minio/normal/zero copy replication/check refcount after mutation OK 1s 221ms
/s3/minio/normal/zero copy replication/consistency during conflicting mutation OK 21s 21ms
/s3/minio/normal/zero copy replication/consistency during double mutation OK 18s 583ms
/s3/minio/normal/zero copy replication/default value OK 12s 596ms
/s3/minio/normal/zero copy replication/delete XFail 43s 584ms
/s3/minio/normal/zero copy replication/delete all OK 41s 576ms
/s3/minio/normal/zero copy replication/drop alter replica OK 56s 54ms
/s3/minio/normal/zero copy replication/drop replica OK 58s 552ms
/s3/minio/normal/zero copy replication/global setting OK 41s 990ms
/s3/minio/normal/zero copy replication/insert multiple replicas OK 1m 8s
/s3/minio/normal/zero copy replication/metadata OK 49s 327ms
/s3/minio/normal/zero copy replication/performance alter OK 1m 14s
/s3/minio/normal/zero copy replication/performance insert OK 1m 13s
/s3/minio/normal/zero copy replication/performance select OK 1m 14s
/s3/minio/normal/zero copy replication/ttl delete OK 1m 9s
/s3/minio/normal/zero copy replication/ttl move OK 1m 28s
/s3/minio/normal/cit OK 1s 165ms
/s3/minio/normal/cit/test empty put OK 296ms
/s3/minio/normal/cit/test put OK 319ms
/s3/minio/normal/cit/test put csv OK 326ms
/s3/minio/normal/cit/test put get with redirect OK 203ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232