Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateDec 26, 2023 12:46
Duration50m 35s
Framework TestFlows 2.0.231130.1212236

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.8/57762a45dd8d7579a37ae9643654cde6896312dc/package_release/clickhouse-common-static_23.8.8.21.altinitystable_amd64.deb
version23.8.8.21.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash09db6160aac30e37169797b73fac77b4cbca41c6
job.id7328846900
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/7328846900
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.8/57762a45dd8d7579a37ae9643654cde6896312dc/package_release/clickhouse-common-static_23.8.8.21.altinitystable_amd64.deb
stressFalse
collect_service_logsTrue
storages['minio']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78.9%OK
19.6%Known

Statistics

Units Skip OK Fail XFail
Modules
1
1
Suites
9
9
Features
9
9
Scenarios
142
3
99
40
Checks
1
1
Examples
42
42
Steps
13706
13699
7

Known Fails

Test NameResultMessage
/s3/minio disk/cacheXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio disk/cache defaultXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio disk/cache pathXFail 715us
Under development for 22.8 and newer.
None
/s3/minio disk/low cardinality offsetXFail 39s 127ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 251, in minio_regression
    Feature(test=load("s3.tests.disk", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2787, in minio
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2742, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2711, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2711 in 'low_cardinality_offset'

2703\|                          "1",
2704\|                      ),
2705\|                      (
2706\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2707\|                          "1",
2708\|                      ),
2709\|                  ],
2710\|              ).output
2711\|>             assert output == "23999\n", error()
2712\|  
2713\|      finally:
2714\|          with Finally(f"I remove the table {name}"):
/s3/minio invalid disk/cache path conflictXFail 1ms
Under development for 22.8 and newer.
None
/s3/minio zero copy replication/alter repeatXFail 2m 9s
https://github.com/ClickHouse/ClickHouse/issues/22516
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2250, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2204, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1021, in alter_repeat
    assert False, "data in S3 has grown by more than 50%"
AssertionError: data in S3 has grown by more than 50%
/s3/minio zero copy replication/deleteXFail 35s 711ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2250, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2204, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1299, in delete
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 773, in check_bucket_size
    assert current_size <= upper_bound, error(msg)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert current_size <= upper_bound, error(msg)

Description
  119326959 is not within 113024690±5

Assertion values
  assert current_size <= upper_bound, error(msg)
         ^ is 119326959
  assert current_size <= upper_bound, error(msg)
                         ^ is 113024695
  assert current_size <= upper_bound, error(msg)
                      ^ is = False
  assert current_size <= upper_bound, error(msg)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 773 in 'check_bucket_size'

765\|      )
766\|      if tolerance is None or tolerance == 0:
767\|          assert expected_size == current_size, error()
768\|      else:
769\|          msg = f"{current_size} is not within {expected_size}±{tolerance}"
770\|          lower_bound = expected_size - tolerance
771\|          upper_bound = expected_size + tolerance
772\|          assert current_size >= lower_bound, error(msg)
773\|>         assert current_size <= upper_bound, error(msg)
774\|  
775\|  
776\|  @TestStep(Given)
/s3/minio zero copy replication/ttl deleteXFail 35s 123ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2250, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2204, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1701, in ttl_delete
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 772, in check_bucket_size
    assert current_size >= lower_bound, error(msg)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert current_size >= lower_bound, error(msg)

Description
  22887522 is not within 119326959±5

Assertion values
  assert current_size >= lower_bound, error(msg)
         ^ is 22887522
  assert current_size >= lower_bound, error(msg)
                         ^ is 119326954
  assert current_size >= lower_bound, error(msg)
                      ^ is = False
  assert current_size >= lower_bound, error(msg)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 772 in 'check_bucket_size'

764\|          key_id=self.context.access_key_id,
765\|      )
766\|      if tolerance is None or tolerance == 0:
767\|          assert expected_size == current_size, error()
768\|      else:
769\|          msg = f"{current_size} is not within {expected_size}±{tolerance}"
770\|          lower_bound = expected_size - tolerance
771\|          upper_bound = expected_size + tolerance
772\|>         assert current_size >= lower_bound, error(msg)
773\|          assert current_size <= upper_bound, error(msg)
774\|  
775\|
/s3/minio zero copy replication/ttl moveXFail 36s 285ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 456, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 424, in regression
    minio_regression(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 263, in minio_regression
    Feature(test=load("s3.tests.zero_copy_replication", "minio"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2250, in minio
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2204, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1547, in ttl_move
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 772, in check_bucket_size
    assert current_size >= lower_bound, error(msg)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert current_size >= lower_bound, error(msg)

Description
  16585253 is not within 22887522±5

Assertion values
  assert current_size >= lower_bound, error(msg)
         ^ is 16585253
  assert current_size >= lower_bound, error(msg)
                         ^ is 22887517
  assert current_size >= lower_bound, error(msg)
                      ^ is = False
  assert current_size >= lower_bound, error(msg)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 772 in 'check_bucket_size'

764\|          key_id=self.context.access_key_id,
765\|      )
766\|      if tolerance is None or tolerance == 0:
767\|          assert expected_size == current_size, error()
768\|      else:
769\|          msg = f"{current_size} is not within {expected_size}±{tolerance}"
770\|          lower_bound = expected_size - tolerance
771\|          upper_bound = expected_size + tolerance
772\|>         assert current_size >= lower_bound, error(msg)
773\|          assert current_size <= upper_bound, error(msg)
774\|  
775\|

Results

Test Name Result Duration
/s3 OK 50m 35s
/s3/minio table function OK 1m 6s
/s3/minio table function/auto OK 6s 400ms
/s3/minio table function/compression OK 6s 765ms
/s3/minio table function/credentials OK 716ms
/s3/minio table function/data format OK 6s 774ms
/s3/minio table function/multipart OK 3s 380ms
/s3/minio table function/multiple columns OK 1s 16ms
/s3/minio table function/partition OK 324ms
/s3/minio table function/remote host filter OK 35s 530ms
/s3/minio table function/syntax OK 790ms
/s3/minio table function/wildcard OK 4s 987ms
/s3/minio backup OK 2m 28s
/s3/minio backup/local and s3 disk OK 48s 623ms
/s3/minio backup/local and s3 volumes OK 50s 227ms
/s3/minio backup/s3 disk OK 50s 9ms
/s3/minio invalid table function OK 1s 941ms
/s3/minio invalid table function/empty path OK 212ms
/s3/minio invalid table function/empty structure OK 244ms
/s3/minio invalid table function/invalid compression OK 292ms
/s3/minio invalid table function/invalid credentials OK 242ms
/s3/minio invalid table function/invalid format OK 447ms
/s3/minio invalid table function/invalid path OK 228ms
/s3/minio invalid table function/invalid structure OK 255ms
/s3/minio disk OK 24m 7s
/s3/minio disk/access OK 36s 798ms
/s3/minio disk/access skip check OK 36s 809ms
/s3/minio disk/add storage OK 1m 13s
/s3/minio disk/alter move OK 42s 583ms
/s3/minio disk/alter on cluster modify ttl OK 1m 16s
/s3/minio disk/cache XFail 1ms
/s3/minio disk/cache default XFail 1ms
/s3/minio disk/cache path XFail 715us
/s3/minio disk/compact parts OK 38s 733ms
/s3/minio disk/config over restart OK 50s 837ms
/s3/minio disk/default move factor OK 50s 99ms
/s3/minio disk/delete OK 1m 34s
/s3/minio disk/download appropriate disk OK 1m 2s
/s3/minio disk/environment credentials Skip 2ms
/s3/minio disk/exports OK 40s 383ms
/s3/minio disk/generic url Skip 749us
/s3/minio disk/imports OK 40s 521ms
/s3/minio disk/low cardinality offset XFail 39s 127ms
/s3/minio disk/max single part upload size syntax OK 36s 885ms
/s3/minio disk/mergetree OK 2m 40s
/s3/minio disk/mergetree collapsing OK 40s 542ms
/s3/minio disk/mergetree versionedcollapsing OK 40s 395ms
/s3/minio disk/metadata OK 40s 250ms
/s3/minio disk/min bytes for seek syntax OK 37s 110ms
/s3/minio disk/multiple storage OK 40s 574ms
/s3/minio disk/multiple storage query OK 41s 27ms
/s3/minio disk/perform ttl move on insert OK 1m 12s
/s3/minio disk/perform ttl move on insert default OK 40s 647ms
/s3/minio disk/performance ttl move OK 54s 177ms
/s3/minio disk/remote host filter OK 1m 13s
/s3/minio disk/restart OK 8s 228ms
/s3/minio disk/specific url Skip 1ms
/s3/minio disk/syntax OK 40s 676ms
/s3/minio disk/wide parts OK 37s 20ms
/s3/minio invalid disk OK 3m 55s
/s3/minio invalid disk/access default OK 17s 765ms
/s3/minio invalid disk/access failed OK 18s 753ms
/s3/minio invalid disk/access failed skip check OK 39s 705ms
/s3/minio invalid disk/cache path conflict XFail 1ms
/s3/minio invalid disk/empty endpoint OK 8s 710ms
/s3/minio invalid disk/invalid endpoint OK 2m 10s
/s3/minio invalid disk/invalid type OK 19s 279ms
/s3/minio sanity OK 41s 314ms
/s3/minio sanity/sanity OK 5s 380ms
/s3/minio reconnect OK 2m 10s
/s3/minio reconnect/local and s3 disk OK 42s 520ms
/s3/minio reconnect/local and s3 volumes OK 42s 431ms
/s3/minio reconnect/s3 disk OK 45s 726ms
/s3/minio zero copy replication OK 13m 26s
/s3/minio zero copy replication/Bucket should be empty before test begins OK 5ms
/s3/minio zero copy replication/add replica OK 33s 381ms
/s3/minio zero copy replication/alter OK 1m 8s
/s3/minio zero copy replication/alter repeat XFail 2m 9s
/s3/minio zero copy replication/check refcount after mutation OK 1s 158ms
/s3/minio zero copy replication/consistency during double mutation OK 12s 998ms
/s3/minio zero copy replication/default value OK 10s 400ms
/s3/minio zero copy replication/delete XFail 35s 711ms
/s3/minio zero copy replication/delete all OK 34s 419ms
/s3/minio zero copy replication/drop alter replica OK 44s 447ms
/s3/minio zero copy replication/drop replica OK 45s 791ms
/s3/minio zero copy replication/global setting OK 35s 589ms
/s3/minio zero copy replication/insert multiple replicas OK 36s 180ms
/s3/minio zero copy replication/metadata OK 34s 220ms
/s3/minio zero copy replication/performance alter OK 59s 979ms
/s3/minio zero copy replication/performance insert OK 56s 594ms
/s3/minio zero copy replication/performance select OK 1m 0s
/s3/minio zero copy replication/ttl delete XFail 35s 123ms
/s3/minio zero copy replication/ttl move XFail 36s 285ms
/s3/cit OK 963ms
/s3/cit/test empty put OK 219ms
/s3/cit/test put OK 227ms
/s3/cit/test put csv OK 285ms
/s3/cit/test put get with redirect OK 202ms

Generated by TestFlows Open-Source Test Framework v2.0.231130.1212236