Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateJul 14, 2023 19:52
Duration52m 27s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/23.3.8.22.altinitystable/5553084583/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/b2217a3881bc514b436b6e2c0e21eb899b4d4aee/package_release/clickhouse-common-static_23.3.8.22.altinitystable_amd64.deb
version23.3.8.22.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashb13df10b2ee4ce720b5635d1e0208debff3f6ca8
job.id5553084583
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/5553084583
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/b2217a3881bc514b436b6e2c0e21eb899b4d4aee/package_release/clickhouse-common-static_23.3.8.22.altinitystable_amd64.deb
stressFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.7%OK
19.8%Known

Statistics

Units Skip OK Fail XFail
Modules
1
1
Suites
9
9
Features
9
9
Scenarios
141
3
98
40
Examples
42
42
Steps
12646
12637
9

Known Fails

Test NameResultMessage
/s3/minio disk/cacheXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio disk/cache defaultXFail 688us
Under development for 22.8 and newer.
None
/s3/minio disk/cache pathXFail 680us
Under development for 22.8 and newer.
None
/s3/minio disk/low cardinality offsetXFail 43s 936ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 486, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 454, in regression
    minio_regression(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 261, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2787, in minio
    disk_tests()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2742, in disk_tests
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2711, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2711 in 'low_cardinality_offset'

2703\|                          "1",
2704\|                      ),
2705\|                      (
2706\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2707\|                          "1",
2708\|                      ),
2709\|                  ],
2710\|              ).output
2711\|>             assert output == "23999\n", error()
2712\|  
2713\|      finally:
2714\|          with Finally(f"I remove the table {name}"):
/s3/minio invalid disk/cache path conflictXFail 644us
Under development for 22.8 and newer.
None
/s3/minio zero copy replication/alterXFail 1m 41s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 486, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 454, in regression
    minio_regression(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 273, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 863, in alter
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/alter repeatXFail 1m 38s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 486, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 454, in regression
    minio_regression(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 273, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1025, in alter_repeat
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/deleteXFail 38s 546ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 486, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 454, in regression
    minio_regression(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 273, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1314, in delete
    assert size_before >= size - 5, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_before >= size - 5, error()

Assertion values
  assert size_before >= size - 5, error()
         ^ is 0
  assert size_before >= size - 5, error()
                        ^ is 6306511
  assert size_before >= size - 5, error()
                             ^ is = 6306506
  assert size_before >= size - 5, error()
                     ^ is = False
  assert size_before >= size - 5, error()
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 1314 in 'delete'

1306\|              ):
1307\|                  size = get_bucket_size(
1308\|                      name=bucket_name,
1309\|                      prefix=bucket_path,
1310\|                      minio_enabled=minio_enabled,
1311\|                      access_key=self.context.secret_access_key,
1312\|                      key_id=self.context.access_key_id,
1313\|                  )
1314\|>                 assert size_before >= size - 5, error()
1315\|                  assert size_before <= size + 5, error()
1316\|  
1317\|          finally:
/s3/minio zero copy replication/ttl deleteXFail 43s 231ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 486, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 454, in regression
    minio_regression(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/regression.py", line 273, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2179, in minio
    outline()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2133, in outline
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1655, in ttl_delete
    check_query_node(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 696, in check_query_node
    assert r == expected, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r == expected, error()

Assertion values
  assert r == expected, error()
         ^ is '1441794'
  assert r == expected, error()
              ^ is '1310721'
  assert r == expected, error()
           ^ is = False
    @@ -1 +1 @@
    -1441794
    +1310721
  assert r == expected, error()
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 696 in 'check_query_node'

688\|  
689\|  @TestStep(Then)
690\|  def check_query_node(self, node, num, query, expected):
691\|      node = current().context.node
692\|  
693\|      with By(f"executing query {num}", description=query):
694\|          r = node.query(query).output.strip()
695\|          with Then(f"result should match the expected", description=expected):
696\|>             assert r == expected, error()
697\|  
698\|  
699\|  def get_s3_file_content(cluster, bucket, filename, decode=True):

Results

Test Name Result Duration
/s3 OK 52m 27s
/s3/minio table function OK 1m 16s
/s3/minio table function/auto OK 7s 807ms
/s3/minio table function/compression OK 7s 621ms
/s3/minio table function/credentials OK 863ms
/s3/minio table function/data format OK 9s 364ms
/s3/minio table function/multipart OK 4s 629ms
/s3/minio table function/multiple columns OK 1s 120ms
/s3/minio table function/partition OK 333ms
/s3/minio table function/remote host filter OK 38s 512ms
/s3/minio table function/syntax OK 833ms
/s3/minio table function/wildcard OK 5s 583ms
/s3/minio backup OK 2m 35s
/s3/minio backup/local and s3 disk OK 52s 213ms
/s3/minio backup/local and s3 volumes OK 51s 677ms
/s3/minio backup/s3 disk OK 51s 871ms
/s3/minio invalid table function OK 2s 376ms
/s3/minio invalid table function/empty path OK 285ms
/s3/minio invalid table function/empty structure OK 291ms
/s3/minio invalid table function/invalid compression OK 295ms
/s3/minio invalid table function/invalid credentials OK 326ms
/s3/minio invalid table function/invalid format OK 573ms
/s3/minio invalid table function/invalid path OK 303ms
/s3/minio invalid table function/invalid structure OK 286ms
/s3/minio disk OK 26m 45s
/s3/minio disk/access OK 39s 803ms
/s3/minio disk/access skip check OK 39s 726ms
/s3/minio disk/add storage OK 1m 19s
/s3/minio disk/alter move OK 45s 618ms
/s3/minio disk/alter on cluster modify ttl OK 1m 23s
/s3/minio disk/cache XFail 1ms
/s3/minio disk/cache default XFail 688us
/s3/minio disk/cache path XFail 680us
/s3/minio disk/compact parts OK 39s 716ms
/s3/minio disk/config over restart OK 54s 141ms
/s3/minio disk/default move factor OK 55s 550ms
/s3/minio disk/delete OK 1m 40s
/s3/minio disk/download appropriate disk OK 1m 8s
/s3/minio disk/environment credentials Skip 853us
/s3/minio disk/exports OK 43s 524ms
/s3/minio disk/generic url Skip 1ms
/s3/minio disk/imports OK 43s 601ms
/s3/minio disk/low cardinality offset XFail 43s 936ms
/s3/minio disk/max single part upload size syntax OK 43s 931ms
/s3/minio disk/mergetree OK 3m 10s
/s3/minio disk/mergetree collapsing OK 47s 596ms
/s3/minio disk/mergetree versionedcollapsing OK 48s 215ms
/s3/minio disk/metadata OK 46s 89ms
/s3/minio disk/min bytes for seek syntax OK 43s 932ms
/s3/minio disk/multiple storage OK 46s 384ms
/s3/minio disk/multiple storage query OK 43s 790ms
/s3/minio disk/perform ttl move on insert OK 1m 19s
/s3/minio disk/perform ttl move on insert default OK 43s 595ms
/s3/minio disk/performance ttl move OK 57s 677ms
/s3/minio disk/remote host filter OK 1m 19s
/s3/minio disk/restart OK 14s 70ms
/s3/minio disk/specific url Skip 3ms
/s3/minio disk/syntax OK 43s 635ms
/s3/minio disk/wide parts OK 39s 915ms
/s3/minio invalid disk OK 1m 51s
/s3/minio invalid disk/access default OK 14s 653ms
/s3/minio invalid disk/access failed OK 15s 685ms
/s3/minio invalid disk/access failed skip check OK 42s 428ms
/s3/minio invalid disk/cache path conflict XFail 644us
/s3/minio invalid disk/empty endpoint OK 9s 657ms
/s3/minio invalid disk/invalid endpoint OK 9s 655ms
/s3/minio invalid disk/invalid type OK 19s 295ms
/s3/minio sanity OK 48s 326ms
/s3/minio sanity/sanity OK 7s 787ms
/s3/minio reconnect OK 2m 24s
/s3/minio reconnect/local and s3 disk OK 48s 393ms
/s3/minio reconnect/local and s3 volumes OK 48s 422ms
/s3/minio reconnect/s3 disk OK 47s 580ms
/s3/minio zero copy replication OK 14m 45s
/s3/minio zero copy replication/add replica OK 38s 167ms
/s3/minio zero copy replication/alter XFail 1m 41s
/s3/minio zero copy replication/alter repeat XFail 1m 38s
/s3/minio zero copy replication/default value OK 11s 616ms
/s3/minio zero copy replication/delete XFail 38s 546ms
/s3/minio zero copy replication/delete all OK 39s 908ms
/s3/minio zero copy replication/drop alter replica OK 53s 337ms
/s3/minio zero copy replication/drop replica OK 52s 948ms
/s3/minio zero copy replication/global setting OK 41s 420ms
/s3/minio zero copy replication/insert multiple replicas OK 41s 930ms
/s3/minio zero copy replication/lost data during mutation OK 1s 903ms
/s3/minio zero copy replication/metadata OK 40s 41ms
/s3/minio zero copy replication/performance alter OK 1m 6s
/s3/minio zero copy replication/performance insert OK 1m 6s
/s3/minio zero copy replication/performance select OK 1m 8s
/s3/minio zero copy replication/ttl delete XFail 43s 231ms
/s3/minio zero copy replication/ttl move OK 40s 677ms
/s3/cit OK 1s 24ms
/s3/cit/test empty put OK 228ms
/s3/cit/test put OK 254ms
/s3/cit/test put csv OK 266ms
/s3/cit/test put get with redirect OK 202ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122