Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateOct 05, 2023 14:31
Duration54m 10s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/23.3.13.7.altinitystable/6416627213/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/f040635d5f373c5e4be08e5fbec2622a9889adde/package_aarch64/clickhouse-common-static_23.3.13.7.altinitystable_arm64.deb
version23.3.13.7.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash19e8624c5e4ccc65b128d27b19836c0570e53991
job.id6416627213
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/6416627213
archaarch64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/f040635d5f373c5e4be08e5fbec2622a9889adde/package_aarch64/clickhouse-common-static_23.3.13.7.altinitystable_arm64.deb
stressFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.7%OK
19.8%Known

Statistics

Units Skip OK Fail XFail
Modules
1
1
Suites
9
9
Features
9
9
Scenarios
141
3
98
40
Examples
42
42
Steps
12645
12636
9

Known Fails

Test NameResultMessage
/s3/minio disk/cacheXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio disk/cache defaultXFail 995us
Under development for 22.8 and newer.
None
/s3/minio disk/cache pathXFail 782us
Under development for 22.8 and newer.
None
/s3/minio disk/low cardinality offsetXFail 41s 990ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 483, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 451, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 260, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2787, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2742, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2711, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2711 in 'low_cardinality_offset'

2703\|                          "1",
2704\|                      ),
2705\|                      (
2706\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2707\|                          "1",
2708\|                      ),
2709\|                  ],
2710\|              ).output
2711\|>             assert output == "23999\n", error()
2712\|  
2713\|      finally:
2714\|          with Finally(f"I remove the table {name}"):
/s3/minio invalid disk/cache path conflictXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio zero copy replication/alterXFail 1m 39s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 483, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 451, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 272, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 863, in alter
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/alter repeatXFail 1m 40s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 483, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 451, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 272, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1025, in alter_repeat
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/deleteXFail 40s 632ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 483, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 451, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 272, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1314, in delete
    assert size_before >= size - 5, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_before >= size - 5, error()

Assertion values
  assert size_before >= size - 5, error()
         ^ is 0
  assert size_before >= size - 5, error()
                        ^ is 6306511
  assert size_before >= size - 5, error()
                             ^ is = 6306506
  assert size_before >= size - 5, error()
                     ^ is = False
  assert size_before >= size - 5, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 1314 in 'delete'

1306\|              ):
1307\|                  size = get_bucket_size(
1308\|                      name=bucket_name,
1309\|                      prefix=bucket_path,
1310\|                      minio_enabled=minio_enabled,
1311\|                      access_key=self.context.secret_access_key,
1312\|                      key_id=self.context.access_key_id,
1313\|                  )
1314\|>                 assert size_before >= size - 5, error()
1315\|                  assert size_before <= size + 5, error()
1316\|  
1317\|          finally:
/s3/minio zero copy replication/ttl deleteXFail 42s 6ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 483, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 451, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 272, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1655, in ttl_delete
    check_query_node(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 695, in check_query_node
    assert r == expected, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r == expected, error()

Assertion values
  assert r == expected, error()
         ^ is '1441794'
  assert r == expected, error()
              ^ is '1310721'
  assert r == expected, error()
           ^ is = False
    @@ -1 +1 @@
    -1441794
    +1310721
  assert r == expected, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 695 in 'check_query_node'

687\|  
688\|  @TestStep(Then)
689\|  def check_query_node(self, node, num, query, expected):
690\|      node = current().context.node
691\|  
692\|      with By(f"executing query {num}", description=query):
693\|          r = node.query(query).output.strip()
694\|          with Then(f"result should match the expected", description=expected):
695\|>             assert r == expected, error()
696\|  
697\|  
698\|  def get_s3_file_content(cluster, bucket, filename, decode=True):

Results

Test Name Result Duration
/s3 OK 54m 10s
/s3/minio table function OK 1m 29s
/s3/minio table function/auto OK 9s 736ms
/s3/minio table function/compression OK 9s 887ms
/s3/minio table function/credentials OK 1s 61ms
/s3/minio table function/data format OK 10s 560ms
/s3/minio table function/multipart OK 5s 277ms
/s3/minio table function/multiple columns OK 1s 398ms
/s3/minio table function/partition OK 431ms
/s3/minio table function/remote host filter OK 42s 872ms
/s3/minio table function/syntax OK 1s 102ms
/s3/minio table function/wildcard OK 7s 56ms
/s3/minio backup OK 2m 44s
/s3/minio backup/local and s3 disk OK 55s 398ms
/s3/minio backup/local and s3 volumes OK 54s 750ms
/s3/minio backup/s3 disk OK 54s 759ms
/s3/minio invalid table function OK 3s 281ms
/s3/minio invalid table function/empty path OK 430ms
/s3/minio invalid table function/empty structure OK 405ms
/s3/minio invalid table function/invalid compression OK 414ms
/s3/minio invalid table function/invalid credentials OK 453ms
/s3/minio invalid table function/invalid format OK 758ms
/s3/minio invalid table function/invalid path OK 411ms
/s3/minio invalid table function/invalid structure OK 392ms
/s3/minio disk OK 27m 21s
/s3/minio disk/access OK 41s 801ms
/s3/minio disk/access skip check OK 42s 31ms
/s3/minio disk/add storage OK 1m 22s
/s3/minio disk/alter move OK 45s 659ms
/s3/minio disk/alter on cluster modify ttl OK 1m 24s
/s3/minio disk/cache XFail 1ms
/s3/minio disk/cache default XFail 995us
/s3/minio disk/cache path XFail 782us
/s3/minio disk/compact parts OK 42s 898ms
/s3/minio disk/config over restart OK 57s 613ms
/s3/minio disk/default move factor OK 1m 1s
/s3/minio disk/delete OK 1m 46s
/s3/minio disk/download appropriate disk OK 1m 10s
/s3/minio disk/environment credentials Skip 1ms
/s3/minio disk/exports OK 41s 446ms
/s3/minio disk/generic url Skip 2ms
/s3/minio disk/imports OK 42s 128ms
/s3/minio disk/low cardinality offset XFail 41s 990ms
/s3/minio disk/max single part upload size syntax OK 41s 912ms
/s3/minio disk/mergetree OK 3m 0s
/s3/minio disk/mergetree collapsing OK 47s 184ms
/s3/minio disk/mergetree versionedcollapsing OK 44s 808ms
/s3/minio disk/metadata OK 44s 711ms
/s3/minio disk/min bytes for seek syntax OK 43s 310ms
/s3/minio disk/multiple storage OK 49s 494ms
/s3/minio disk/multiple storage query OK 49s 671ms
/s3/minio disk/perform ttl move on insert OK 1m 27s
/s3/minio disk/perform ttl move on insert default OK 49s 132ms
/s3/minio disk/performance ttl move OK 1m 1s
/s3/minio disk/remote host filter OK 1m 20s
/s3/minio disk/restart OK 15s 565ms
/s3/minio disk/specific url Skip 1ms
/s3/minio disk/syntax OK 44s 265ms
/s3/minio disk/wide parts OK 40s 215ms
/s3/minio invalid disk OK 1m 48s
/s3/minio invalid disk/access default OK 15s 861ms
/s3/minio invalid disk/access failed OK 14s 872ms
/s3/minio invalid disk/access failed skip check OK 41s 557ms
/s3/minio invalid disk/cache path conflict XFail 1ms
/s3/minio invalid disk/empty endpoint OK 9s 844ms
/s3/minio invalid disk/invalid endpoint OK 8s 828ms
/s3/minio invalid disk/invalid type OK 17s 856ms
/s3/minio sanity OK 47s 615ms
/s3/minio sanity/sanity OK 8s 476ms
/s3/minio reconnect OK 2m 25s
/s3/minio reconnect/local and s3 disk OK 48s 414ms
/s3/minio reconnect/local and s3 volumes OK 47s 954ms
/s3/minio reconnect/s3 disk OK 48s 682ms
/s3/minio zero copy replication OK 14m 58s
/s3/minio zero copy replication/add replica OK 39s 120ms
/s3/minio zero copy replication/alter XFail 1m 39s
/s3/minio zero copy replication/alter repeat XFail 1m 40s
/s3/minio zero copy replication/default value OK 11s 796ms
/s3/minio zero copy replication/delete XFail 40s 632ms
/s3/minio zero copy replication/delete all OK 40s 234ms
/s3/minio zero copy replication/drop alter replica OK 55s 430ms
/s3/minio zero copy replication/drop replica OK 54s 161ms
/s3/minio zero copy replication/global setting OK 41s 986ms
/s3/minio zero copy replication/insert multiple replicas OK 42s 656ms
/s3/minio zero copy replication/lost data during mutation OK 1s 232ms
/s3/minio zero copy replication/metadata OK 42s 714ms
/s3/minio zero copy replication/performance alter OK 1m 7s
/s3/minio zero copy replication/performance insert OK 1m 8s
/s3/minio zero copy replication/performance select OK 1m 5s
/s3/minio zero copy replication/ttl delete XFail 42s 6ms
/s3/minio zero copy replication/ttl move OK 44s 657ms
/s3/cit OK 1s 368ms
/s3/cit/test empty put OK 282ms
/s3/cit/test put OK 340ms
/s3/cit/test put csv OK 401ms
/s3/cit/test put get with redirect OK 316ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122