Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateMar 19, 2024 17:46
Duration57m 34s
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/0a8ac3b092733da37e3e2a0079c486938a36790d/package_release/clickhouse-common-static_23.3.19.34.altinityfips_amd64.deb
version23.3.19.34.altinityfips
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash6da94b78dc53cb8965ab56c04a89ebf54ed04cbc
job.id8247806903
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8247806903
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/0a8ac3b092733da37e3e2a0079c486938a36790d/package_release/clickhouse-common-static_23.3.19.34.altinityfips_amd64.deb
stressFalse
allow_vfsFalse
collect_service_logsTrue
storages['aws_s3']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

78%OK
20.3%Known

Statistics

Units Skip OK Fail XFail
Modules
3
3
Features
11
3
8
Scenarios
105
93
12
Checks
3
2
1
Examples
60
36
24
Steps
14191
14015
154
22

Known Fails

Test NameResultMessage
/s3/aws s3/normal/disk/cacheXFail 816us
Under development for 22.8 and newer.
None
/s3/aws s3/normal/disk/cache defaultXFail 551us
Under development for 22.8 and newer.
None
/s3/aws s3/normal/disk/cache pathXFail 584us
Under development for 22.8 and newer.
None
/s3/aws s3/normal/disk/generic urlXFail 3ms
not yet supported
Generic URL is treated as invalid configuration, ClickHouse
             will not start if config is added
/s3/aws s3/normal/disk/low cardinality offsetXFail 42s 368ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 361, in aws_s3_regression
    Feature(test=load("s3.tests.disk", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2759, in aws_s3
    disk_tests()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2746, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2715, in low_cardinality_offset
    assert output == "23999\n", error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2715 in 'low_cardinality_offset'

2707\|                          "1",
2708\|                      ),
2709\|                      (
2710\|                          "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2711\|                          "1",
2712\|                      ),
2713\|                  ],
2714\|              ).output
2715\|>             assert output == "23999\n", error()
2716\|  
2717\|      finally:
2718\|          with Finally(f"I remove the table {name}"):
/s3/aws s3/normal/invalid disk/cache path conflictXFail 836us
Under development for 22.8 and newer.
None
/s3/aws s3/normal/zero copy replication/bucket should be empty before test beginsXFail 3s 232ms
Data cleanup not working as expected
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 370, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2247, in aws_s3
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2224, in outline
    check_bucket_size(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 774, in check_bucket_size
    assert abs(current_size - expected_size) <= tolerance, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert abs(current_size - expected_size) <= tolerance, error()

Assertion values
  assert abs(current_size - expected_size) <= tolerance, error()
             ^ is 228462742
  assert abs(current_size - expected_size) <= tolerance, error()
                            ^ is 0
  assert abs(current_size - expected_size) <= tolerance, error()
                          ^ is = 228462742
  assert abs(current_size - expected_size) <= tolerance, error()
         ^ is = 228462742
  assert abs(current_size - expected_size) <= tolerance, error()
                                              ^ is 50
  assert abs(current_size - expected_size) <= tolerance, error()
                                           ^ is = False
  assert abs(current_size - expected_size) <= tolerance, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 774 in 'check_bucket_size'

766\|  ):
767\|      current_size = get_bucket_size(
768\|          name=name,
769\|          prefix=prefix,
770\|          minio_enabled=minio_enabled,
771\|          access_key=self.context.secret_access_key,
772\|          key_id=self.context.access_key_id,
773\|      )
774\|>     assert abs(current_size - expected_size) <= tolerance, error()
775\|  
776\|  
777\|  @TestStep(Given)
/s3/aws s3/normal/zero copy replication/add replicaXFail 48s 309ms
Data cleanup not working as expected
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 370, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2247, in aws_s3
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 355, in add_replica
    assert size - size_after == 1, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size - size_after == 1, error()

Assertion values
  assert size - size_after == 1, error()
         ^ is 228462742
  assert size - size_after == 1, error()
                ^ is 228462742
  assert size - size_after == 1, error()
              ^ is = 0
  assert size - size_after == 1, error()
                           ^ is = False
  assert size - size_after == 1, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 355 in 'add_replica'

347\|          ):
348\|              size = get_bucket_size(
349\|                  name=bucket_name,
350\|                  prefix=bucket_path,
351\|                  access_key=self.context.secret_access_key,
352\|                  key_id=self.context.access_key_id,
353\|                  minio_enabled=self.context.minio_enabled,
354\|              )
355\|>             assert size - size_after == 1, error()
356\|  
357\|          with And("I check simple queries on the first node"):
358\|              check_query_node(
/s3/aws s3/normal/zero copy replication/bad detached partXFail 39s 863ms
Data cleanup not working as expected
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 370, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2247, in aws_s3
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1663, in bad_detached_part
    r = nodes[0].query(f"SELECT * FROM {table_name}")
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Received exception from server (version 23.3.19):
Code: 499. DB::Exception: Received from localhost:9000. DB::Exception: The specified key does not exist.: while reading key: data/zero-copy-replication/ldd/cdolynlmulmsytdfdnbxnmwokivfs, from bucket: [masked]:Secret(name='aws_s3_bucket'): (while reading column d): (while reading from part /var/lib/clickhouse/disks/external/store/404/40482c63-8dfa-47b0-91ba-0e904b443a8b/all_0_0_0/ from mark 0 with max_rows_to_read = 1): While executing MergeTreeInOrder. (S3_ERROR)
(query: SELECT * FROM detach_table
)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/s3/aws s3/normal/zero copy replication/delete allXFail 48s 444ms
Data cleanup not working as expected
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 370, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2247, in aws_s3
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1291, in delete_all
    assert size_now > size_before, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_now > size_before, error()

Assertion values
  assert size_now > size_before, error()
         ^ is 228462742
  assert size_now > size_before, error()
                    ^ is 228462742
  assert size_now > size_before, error()
                  ^ is = False
  assert size_now > size_before, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 1291 in 'delete_all'

1283\|              size_now = get_bucket_size(
1284\|                  name=bucket_name,
1285\|                  prefix=bucket_path,
1286\|                  minio_enabled=self.context.minio_enabled,
1287\|                  access_key=self.context.secret_access_key,
1288\|                  key_id=self.context.access_key_id,
1289\|              )
1290\|  
1291\|>             assert size_now > size_before, error()
1292\|  
1293\|      finally:
1294\|          with Finally("I drop the table on each node"):
/s3/aws s3/normal/zero copy replication/insert multiple replicasXFail 50s 363ms
Data cleanup not working as expected
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 370, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2247, in aws_s3
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1117, in insert_multiple_replicas
    assert added_size >= expected * 0.99, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert added_size >= expected * 0.99, error()

Assertion values
  assert added_size >= expected * 0.99, error()
         ^ is 0
  assert added_size >= expected * 0.99, error()
                       ^ is 6306510
  assert added_size >= expected * 0.99, error()
                                ^ is = 6243444.9
  assert added_size >= expected * 0.99, error()
                    ^ is = False
  assert added_size >= expected * 0.99, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py', line 1117 in 'insert_multiple_replicas'

1109\|                  name=bucket_name,
1110\|                  prefix=bucket_path,
1111\|                  access_key=self.context.secret_access_key,
1112\|                  key_id=self.context.access_key_id,
1113\|                  minio_enabled=self.context.minio_enabled,
1114\|              )
1115\|              added_size = current_size - size_before
1116\|  
1117\|>             assert added_size >= expected * 0.99, error()
1118\|              assert added_size <= expected * 1.01, error()
1119\|  
1120\|      finally:
/s3/aws s3/normal/zero copy replication/ttl deleteXFail 45s 939ms
https://github.com/ClickHouse/ClickHouse/issues/22679
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 515, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 506, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 370, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2247, in aws_s3
    outline()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 2232, in outline
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/zero_copy_replication.py", line 1532, in ttl_delete
    check_query_node(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 701, in check_query_node
    assert r == expected, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r == expected, error()

Assertion values
  assert r == expected, error()
         ^ is '1441794'
  assert r == expected, error()
              ^ is '1310721'
  assert r == expected, error()
           ^ is = False
    @@ -1 +1 @@
    -1441794
    +1310721
  assert r == expected, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 701 in 'check_query_node'

693\|  
694\|  @TestStep(Then)
695\|  def check_query_node(self, node, num, query, expected):
696\|      node = current().context.node
697\|  
698\|      with By(f"executing query {num}", description=query):
699\|          r = node.query(query).output.strip()
700\|          with Then(f"result should match the expected", description=expected):
701\|>             assert r == expected, error()
702\|  
703\|  
704\|  def get_s3_file_content(cluster, bucket, filename, decode=True):

Results

Test Name Result Duration
/s3 OK 57m 34s
/s3/aws s3 OK 57m 34s
/s3/aws s3/normal OK 55m 16s
/s3/aws s3/normal/table function OK 2m 54s
/s3/aws s3/normal/table function/auto OK 11s 475ms
/s3/aws s3/normal/table function/compression OK 11s 867ms
/s3/aws s3/normal/table function/credentials OK 1s 298ms
/s3/aws s3/normal/table function/data format OK 7s 761ms
/s3/aws s3/normal/table function/multipart OK 4s 551ms
/s3/aws s3/normal/table function/multiple columns OK 1s 531ms
/s3/aws s3/normal/table function/partition OK 1s 732ms
/s3/aws s3/normal/table function/remote host filter OK 35s 307ms
/s3/aws s3/normal/table function/syntax OK 1s 344ms
/s3/aws s3/normal/table function/wildcard OK 20s 860ms
/s3/aws s3/normal/table function/ssec encryption check Skip 610us
/s3/aws s3/normal/table function/ssec OK 1m 17s
/s3/aws s3/normal/table function/ssec/auto OK 7s 951ms
/s3/aws s3/normal/table function/ssec/compression OK 7s 896ms
/s3/aws s3/normal/table function/ssec/credentials OK 1s 113ms
/s3/aws s3/normal/table function/ssec/data format OK 5s 46ms
/s3/aws s3/normal/table function/ssec/multipart OK 2s 756ms
/s3/aws s3/normal/table function/ssec/multiple columns OK 1s 345ms
/s3/aws s3/normal/table function/ssec/partition OK 1s 109ms
/s3/aws s3/normal/table function/ssec/remote host filter OK 36s 283ms
/s3/aws s3/normal/table function/ssec/syntax OK 1s 131ms
/s3/aws s3/normal/table function/ssec/wildcard OK 3s 437ms
/s3/aws s3/normal/invalid table function OK 2s 132ms
/s3/aws s3/normal/invalid table function/empty path OK 242ms
/s3/aws s3/normal/invalid table function/empty structure OK 232ms
/s3/aws s3/normal/invalid table function/invalid compression OK 246ms
/s3/aws s3/normal/invalid table function/invalid credentials OK 424ms
/s3/aws s3/normal/invalid table function/invalid format OK 462ms
/s3/aws s3/normal/invalid table function/invalid path OK 248ms
/s3/aws s3/normal/invalid table function/invalid structure OK 266ms
/s3/aws s3/normal/disk OK 29m 50s
/s3/aws s3/normal/disk/access OK 40s 660ms
/s3/aws s3/normal/disk/access skip check OK 37s 490ms
/s3/aws s3/normal/disk/add storage OK 1m 24s
/s3/aws s3/normal/disk/alter move OK 52s 460ms
/s3/aws s3/normal/disk/alter on cluster modify ttl OK 1m 26s
/s3/aws s3/normal/disk/cache XFail 816us
/s3/aws s3/normal/disk/cache default XFail 551us
/s3/aws s3/normal/disk/cache path XFail 584us
/s3/aws s3/normal/disk/compact parts OK 40s 332ms
/s3/aws s3/normal/disk/config over restart OK 1m 13s
/s3/aws s3/normal/disk/default move factor OK 48s 692ms
/s3/aws s3/normal/disk/delete OK 2m 22s
/s3/aws s3/normal/disk/download appropriate disk OK 1m 9s
/s3/aws s3/normal/disk/environment credentials OK 1m 20s
/s3/aws s3/normal/disk/exports OK 42s 987ms
/s3/aws s3/normal/disk/generic url XFail 3ms
/s3/aws s3/normal/disk/imports OK 41s 19ms
/s3/aws s3/normal/disk/low cardinality offset XFail 42s 368ms
/s3/aws s3/normal/disk/max single part upload size syntax OK 43s 691ms
/s3/aws s3/normal/disk/mergetree OK 3m 2s
/s3/aws s3/normal/disk/mergetree collapsing OK 47s 49ms
/s3/aws s3/normal/disk/mergetree versionedcollapsing OK 44s 406ms
/s3/aws s3/normal/disk/metadata OK 42s 707ms
/s3/aws s3/normal/disk/min bytes for seek syntax OK 40s 936ms
/s3/aws s3/normal/disk/multiple storage OK 46s 144ms
/s3/aws s3/normal/disk/multiple storage query OK 48s 53ms
/s3/aws s3/normal/disk/no restart OK 16s 281ms
/s3/aws s3/normal/disk/perform ttl move on insert OK 1m 20s
/s3/aws s3/normal/disk/perform ttl move on insert default OK 44s 883ms
/s3/aws s3/normal/disk/performance ttl move OK 1m 0s
/s3/aws s3/normal/disk/remote host filter OK 1m 21s
/s3/aws s3/normal/disk/specific url OK 40s 880ms
/s3/aws s3/normal/disk/syntax OK 46s 310ms
/s3/aws s3/normal/disk/wide parts OK 40s 562ms
/s3/aws s3/normal/disk/ssec Skip 745us
/s3/aws s3/normal/sanity OK 49s 205ms
/s3/aws s3/normal/sanity/sanity OK 7s 604ms
/s3/aws s3/normal/invalid disk OK 1m 50s
/s3/aws s3/normal/invalid disk/access default OK 17s 593ms
/s3/aws s3/normal/invalid disk/access failed OK 18s 591ms
/s3/aws s3/normal/invalid disk/access failed skip check OK 36s 133ms
/s3/aws s3/normal/invalid disk/cache path conflict XFail 836us
/s3/aws s3/normal/invalid disk/empty endpoint OK 9s 528ms
/s3/aws s3/normal/invalid disk/invalid endpoint OK 9s 530ms
/s3/aws s3/normal/invalid disk/invalid type OK 19s 104ms
/s3/aws s3/normal/zero copy replication OK 17m 28s
/s3/aws s3/normal/zero copy replication/bucket should be empty before test begins XFail 3s 232ms
/s3/aws s3/normal/zero copy replication/add replica XFail 48s 309ms
/s3/aws s3/normal/zero copy replication/alter OK 48s 155ms
/s3/aws s3/normal/zero copy replication/alter repeat OK 1m 43s
/s3/aws s3/normal/zero copy replication/bad detached part XFail 39s 863ms
/s3/aws s3/normal/zero copy replication/check refcount after mutation OK 4s 189ms
/s3/aws s3/normal/zero copy replication/consistency during conflicting mutation OK 22s 20ms
/s3/aws s3/normal/zero copy replication/consistency during double mutation OK 21s 551ms
/s3/aws s3/normal/zero copy replication/default value OK 13s 340ms
/s3/aws s3/normal/zero copy replication/delete OK 52s 732ms
/s3/aws s3/normal/zero copy replication/delete all XFail 48s 444ms
/s3/aws s3/normal/zero copy replication/drop alter replica OK 1m 3s
/s3/aws s3/normal/zero copy replication/drop replica OK 1m 6s
/s3/aws s3/normal/zero copy replication/global setting OK 47s 946ms
/s3/aws s3/normal/zero copy replication/insert multiple replicas XFail 50s 363ms
/s3/aws s3/normal/zero copy replication/metadata OK 43s 646ms
/s3/aws s3/normal/zero copy replication/performance alter OK 1m 21s
/s3/aws s3/normal/zero copy replication/performance insert OK 1m 10s
/s3/aws s3/normal/zero copy replication/performance select OK 1m 17s
/s3/aws s3/normal/zero copy replication/ttl delete XFail 45s 939ms
/s3/aws s3/normal/zero copy replication/ttl move OK 52s 102ms
/s3/aws s3/normal/reconnect OK 2m 20s
/s3/aws s3/normal/reconnect/local and s3 disk OK 46s 614ms
/s3/aws s3/normal/reconnect/local and s3 volumes OK 46s 880ms
/s3/aws s3/normal/reconnect/s3 disk OK 47s 237ms
/s3/aws s3/normal/backup Skip 3ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232