Document status - Confidential

Copyright 2022, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateSep 16, 2022 22:00
Duration1h 0m
Framework TestFlows 1.9.220810.1192506

Artifacts

Test artifacts can be found at https://gitlab.com/altinity-qa/clickhouse/cicd/release/-/jobs/3041775529/artifacts/browse

Attributes

projectaltinity-qa/clickhouse/cicd/release
project.id22089296
user.nameAndrey Zvonov
user.emailzvonov.andrey@gmail.com
version22.3.10.24
repositoryhttps://github.com/Altinity/ClickHouse.git
job.id3041775529
job.urlhttps://gitlab.com/altinity-qa/clickhouse/cicd/release/-/jobs/3041775529
localTrue
clickhouse_versionNone
clickhouse_binary_path/usr/bin/clickhouse
stressFalse
storages['aws_s3']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriNone
gcs_key_idNone
gcs_key_secretNone

Tags

altinity-docker-runner

Summary

81.2%OK
15.6%Known

Statistics

Units Skip OK Fail XFail
Modules
1
1
Suites
11
3
8
Features
11
3
8
Scenarios
101
96
5
Examples
62
38
24
Steps
13766
13597
147
22

Known Fails

Test NameResultMessage
/s3/aws s3 disk/generic urlXFail 2ms
not yet supported
Generic URL is treated as invalid configuration, ClickHouse
             will not start if config is added
/s3/aws s3 zero copy replication/add replicaXFail 48s 56ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "regression/s3/regression.py", line 451, in 
    regression()
  File "regression/s3/regression.py", line 431, in regression
    aws_s3_regression(
  File "regression/s3/regression.py", line 325, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2145, in aws_s3
    outline()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2130, in outline
    scenario()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 382, in add_replica
    assert size_after + 1 == get_bucket_size(
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()

Assertion values
  assert size_after + 1 == get_bucket_size(
         ^ is 33663198
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
                    ^ is = 33663199
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
           ^ is 'altinity-qa-test'
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
             ^ is 'data/zero-copy-replication'
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
                    ^ is False
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is 
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is = 
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is 
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is = '[masked]:Secret(name='aws_s3_access_key')'
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is 
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is = 
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is 
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is = '[masked]:Secret(name='aws_s3_key_id')'
  ), error()
  assert size_after + 1 == get_bucket_size(
                           ^ is = 33663198
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
                        ^ is = False
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
  ^ is False
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()

Where
  File '/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py', line 388 in 'add_replica'

380\|                        than previously because of the additional replica"""
381\|              ):
382\|                  assert size_after + 1 == get_bucket_size(
383\|                      name=bucket_name,
384\|                      prefix=bucket_path,
385\|                      minio_enabled=minio_enabled,
386\|                      access_key=self.context.secret_access_key,
387\|                      key_id=self.context.access_key_id,
388\|>                 ), error()
389\|  
390\|              with And("I check simple queries on the first node"):
391\|                  check_query_node(
/s3/aws s3 zero copy replication/delete allXFail 48s 317ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "regression/s3/regression.py", line 451, in 
    regression()
  File "regression/s3/regression.py", line 431, in regression
    aws_s3_regression(
  File "regression/s3/regression.py", line 325, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2145, in aws_s3
    outline()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2130, in outline
    scenario()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 1383, in delete_all
    assert (
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()

Assertion values
  assert (
      get_bucket_size(
          name=bucket_name,
               ^ is 'altinity-qa-test'
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
                 ^ is 'data/zero-copy-replication'
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
                        ^ is False
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is 
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is = 
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is 
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is = '[masked]:Secret(name='aws_s3_access_key')'
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is 
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is = 
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is 
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is = '[masked]:Secret(name='aws_s3_key_id')'
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
      ^ is = 33663198
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
        ^ is 33663198
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
      ^ is = False
  ), error()
  assert (
  ^ is False
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()

Where
  File '/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py', line 1392 in 'delete_all'

1384\|                      get_bucket_size(
1385\|                          name=bucket_name,
1386\|                          prefix=bucket_path,
1387\|                          minio_enabled=minio_enabled,
1388\|                          access_key=self.context.secret_access_key,
1389\|                          key_id=self.context.access_key_id,
1390\|                      )
1391\|                      > size_before
1392\|>                 ), error()
1393\|  
1394\|          finally:
1395\|              with Finally("I drop the table on each node"):
/s3/aws s3 zero copy replication/insert multiple replicasXFail 48s 83ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "regression/s3/regression.py", line 451, in 
    regression()
  File "regression/s3/regression.py", line 431, in regression
    aws_s3_regression(
  File "regression/s3/regression.py", line 325, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2145, in aws_s3
    outline()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2130, in outline
    scenario()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 1193, in insert_multiple_replicas
    assert added_size >= expected * 0.99, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert added_size >= expected * 0.99, error()

Assertion values
  assert added_size >= expected * 0.99, error()
         ^ is 0
  assert added_size >= expected * 0.99, error()
                       ^ is 6306510
  assert added_size >= expected * 0.99, error()
                                ^ is = 6243444.9
  assert added_size >= expected * 0.99, error()
                    ^ is = False
  assert added_size >= expected * 0.99, error()
  ^ is False

Where
  File '/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py', line 1193 in 'insert_multiple_replicas'

1185\|                      name=bucket_name,
1186\|                      prefix=bucket_path,
1187\|                      minio_enabled=minio_enabled,
1188\|                      access_key=self.context.secret_access_key,
1189\|                      key_id=self.context.access_key_id,
1190\|                  )
1191\|                  added_size = current_size - size_before
1192\|  
1193\|>                 assert added_size >= expected * 0.99, error()
1194\|                  assert added_size <= expected * 1.01, error()
1195\|  
1196\|          finally:

Results

Test Name Result Duration
/s3 OK 1h 0m
/s3/aws s3 table function OK 3m 3s
/s3/aws s3 table function/auto OK 9s 558ms
/s3/aws s3 table function/compression OK 9s 359ms
/s3/aws s3 table function/credentials OK 1s 109ms
/s3/aws s3 table function/data format OK 7s 929ms
/s3/aws s3 table function/multipart OK 5s 241ms
/s3/aws s3 table function/multiple columns OK 1s 187ms
/s3/aws s3 table function/partition OK 1s 93ms
/s3/aws s3 table function/remote host filter OK 38s 537ms
/s3/aws s3 table function/syntax OK 986ms
/s3/aws s3 table function/wildcard OK 18s 42ms
/s3/aws s3 table function/ssec encryption check Skip 444us
/s3/aws s3 table function/ssec OK 1m 30s
/s3/aws s3 table function/ssec/auto OK 6s 795ms
/s3/aws s3 table function/ssec/compression OK 6s 732ms
/s3/aws s3 table function/ssec/credentials OK 2s 28ms
/s3/aws s3 table function/ssec/data format OK 9s 143ms
/s3/aws s3 table function/ssec/multipart OK 4s 95ms
/s3/aws s3 table function/ssec/multiple columns OK 2s 338ms
/s3/aws s3 table function/ssec/partition OK 4s 172ms
/s3/aws s3 table function/ssec/remote host filter OK 38s 629ms
/s3/aws s3 table function/ssec/syntax OK 2s 77ms
/s3/aws s3 table function/ssec/wildcard OK 3s 83ms
/s3/aws s3 invalid table function OK 1s 702ms
/s3/aws s3 invalid table function/empty path OK 222ms
/s3/aws s3 invalid table function/empty structure OK 172ms
/s3/aws s3 invalid table function/invalid compression OK 173ms
/s3/aws s3 invalid table function/invalid credentials OK 323ms
/s3/aws s3 invalid table function/invalid format OK 395ms
/s3/aws s3 invalid table function/invalid path OK 214ms
/s3/aws s3 invalid table function/invalid structure OK 190ms
/s3/aws s3 disk OK 32m 54s
/s3/aws s3 disk/access OK 39s 703ms
/s3/aws s3 disk/access skip check OK 39s 759ms
/s3/aws s3 disk/add storage OK 1m 25s
/s3/aws s3 disk/alter move OK 50s 777ms
/s3/aws s3 disk/alter on cluster modify ttl OK 1m 28s
/s3/aws s3 disk/cache OK 1m 28s
/s3/aws s3 disk/cache default OK 44s 223ms
/s3/aws s3 disk/cache path OK 44s 178ms
/s3/aws s3 disk/compact parts OK 42s 711ms
/s3/aws s3 disk/config over restart OK 1m 11s
/s3/aws s3 disk/default move factor OK 1m 2s
/s3/aws s3 disk/delete OK 2m 16s
/s3/aws s3 disk/download appropriate disk OK 1m 12s
/s3/aws s3 disk/environment credentials OK 1m 20s
/s3/aws s3 disk/exports OK 42s 893ms
/s3/aws s3 disk/generic url XFail 2ms
/s3/aws s3 disk/imports OK 45s 46ms
/s3/aws s3 disk/max single part upload size syntax OK 45s 795ms
/s3/aws s3 disk/mergetree OK 3m 15s
/s3/aws s3 disk/mergetree collapsing OK 45s 121ms
/s3/aws s3 disk/mergetree versionedcollapsing OK 46s 438ms
/s3/aws s3 disk/metadata OK 46s 706ms
/s3/aws s3 disk/min bytes for seek syntax OK 39s 182ms
/s3/aws s3 disk/multiple storage OK 50s 222ms
/s3/aws s3 disk/multiple storage query OK 47s 729ms
/s3/aws s3 disk/perform ttl move on insert OK 1m 23s
/s3/aws s3 disk/perform ttl move on insert default OK 48s 339ms
/s3/aws s3 disk/performance ttl move OK 58s 498ms
/s3/aws s3 disk/remote host filter OK 1m 25s
/s3/aws s3 disk/restart OK 16s 385ms
/s3/aws s3 disk/specific url OK 40s 111ms
/s3/aws s3 disk/syntax OK 47s 158ms
/s3/aws s3 disk/wide parts OK 43s 26ms
/s3/aws s3 disk/ssec Skip 680us
/s3/aws s3 sanity OK 50s 453ms
/s3/aws s3 sanity/sanity OK 8s 595ms
/s3/aws s3 invalid disk OK 2m 2s
/s3/aws s3 invalid disk/access default OK 14s 591ms
/s3/aws s3 invalid disk/access failed OK 15s 634ms
/s3/aws s3 invalid disk/access failed skip check OK 41s 623ms
/s3/aws s3 invalid disk/cache path conflict OK 12s 554ms
/s3/aws s3 invalid disk/empty endpoint OK 9s 572ms
/s3/aws s3 invalid disk/invalid endpoint OK 9s 482ms
/s3/aws s3 invalid disk/invalid type OK 19s 110ms
/s3/aws s3 zero copy replication OK 15m 19s
/s3/aws s3 zero copy replication/add replica XFail 48s 56ms
/s3/aws s3 zero copy replication/alter OK 48s 404ms
/s3/aws s3 zero copy replication/alter repeat OK 1m 31s
/s3/aws s3 zero copy replication/default value OK 12s 105ms
/s3/aws s3 zero copy replication/delete OK 47s 981ms
/s3/aws s3 zero copy replication/delete all XFail 48s 317ms
/s3/aws s3 zero copy replication/drop alter replica OK 59s 997ms
/s3/aws s3 zero copy replication/drop replica OK 1m 0s
/s3/aws s3 zero copy replication/global setting OK 46s 172ms
/s3/aws s3 zero copy replication/insert multiple replicas XFail 48s 83ms
/s3/aws s3 zero copy replication/lost data during mutation OK 3s 75ms
/s3/aws s3 zero copy replication/metadata OK 42s 417ms
/s3/aws s3 zero copy replication/performance alter OK 1m 12s
/s3/aws s3 zero copy replication/performance insert OK 1m 13s
/s3/aws s3 zero copy replication/performance select OK 1m 11s
/s3/aws s3 zero copy replication/ttl delete OK 49s 653ms
/s3/aws s3 zero copy replication/ttl move OK 52s 138ms
/s3/aws s3 reconnect OK 2m 41s
/s3/aws s3 reconnect/local and s3 disk OK 48s 455ms
/s3/aws s3 reconnect/local and s3 volumes OK 52s 118ms
/s3/aws s3 reconnect/s3 disk OK 1m 0s
/s3/aws s3 backup Skip 1ms

Generated by TestFlows Open-Source Test Framework v1.9.220810.1192506