Document status - Confidential

Copyright 2022, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateOct 04, 2022 18:27
Duration1h 1m
Framework TestFlows 1.9.220810.1192506

Artifacts

Test artifacts can be found at https://gitlab.com/altinity-qa/clickhouse/cicd/release/-/jobs/3125479629/artifacts/browse

Attributes

projectaltinity-qa/clickhouse/cicd/release
project.id22089296
user.nameAndrey Zvonov
user.emailzvonov.andrey@gmail.com
version22.3.12.20
repositoryhttps://github.com/Altinity/ClickHouse.git
job.id3125479629
job.urlhttps://gitlab.com/altinity-qa/clickhouse/cicd/release/-/jobs/3125479629
localTrue
clickhouse_versionNone
clickhouse_binary_path/usr/bin/clickhouse
stressFalse
storages['aws_s3']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriNone
gcs_key_idNone
gcs_key_secretNone

Tags

altinity-docker-runner

Summary

81.2%OK
15.6%Known

Statistics

Units Skip OK Fail XFail
Modules
1
1
Suites
11
3
8
Features
11
3
8
Scenarios
101
96
5
Examples
62
38
24
Steps
13754
13585
147
22

Known Fails

Test NameResultMessage
/s3/aws s3 disk/generic urlXFail 3ms
not yet supported
Generic URL is treated as invalid configuration, ClickHouse
             will not start if config is added
/s3/aws s3 zero copy replication/add replicaXFail 1m 52s
Under investigation
AssertionError
Traceback (most recent call last):
  File "regression/s3/regression.py", line 451, in 
    regression()
  File "regression/s3/regression.py", line 431, in regression
    aws_s3_regression(
  File "regression/s3/regression.py", line 325, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2145, in aws_s3
    outline()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2130, in outline
    scenario()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 382, in add_replica
    assert size_after + 1 == get_bucket_size(
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()

Assertion values
  assert size_after + 1 == get_bucket_size(
         ^ is 55775941
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
                    ^ is = 55775942
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
           ^ is 'altinity-qa-test'
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
             ^ is 'data/zero-copy-replication'
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
                    ^ is False
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is 
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is = 
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is 
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
                 ^ is = '[masked]:Secret(name='aws_s3_access_key')'
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is 
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is = 
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is 
  ), error()
  assert size_after + 1 == get_bucket_size(
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
             ^ is = '[masked]:Secret(name='aws_s3_key_id')'
  ), error()
  assert size_after + 1 == get_bucket_size(
                           ^ is = 55775941
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
                        ^ is = False
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()
  assert size_after + 1 == get_bucket_size(
  ^ is False
      name=bucket_name,
      prefix=bucket_path,
      minio_enabled=minio_enabled,
      access_key=self.context.secret_access_key,
      key_id=self.context.access_key_id,
  ), error()

Where
  File '/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py', line 388 in 'add_replica'

380\|                        than previously because of the additional replica"""
381\|              ):
382\|                  assert size_after + 1 == get_bucket_size(
383\|                      name=bucket_name,
384\|                      prefix=bucket_path,
385\|                      minio_enabled=minio_enabled,
386\|                      access_key=self.context.secret_access_key,
387\|                      key_id=self.context.access_key_id,
388\|>                 ), error()
389\|  
390\|              with And("I check simple queries on the first node"):
391\|                  check_query_node(
/s3/aws s3 zero copy replication/delete allXFail 51s 692ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "regression/s3/regression.py", line 451, in 
    regression()
  File "regression/s3/regression.py", line 431, in regression
    aws_s3_regression(
  File "regression/s3/regression.py", line 325, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2145, in aws_s3
    outline()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2130, in outline
    scenario()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 1383, in delete_all
    assert (
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()

Assertion values
  assert (
      get_bucket_size(
          name=bucket_name,
               ^ is 'altinity-qa-test'
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
                 ^ is 'data/zero-copy-replication'
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
                        ^ is False
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is 
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is = 
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is 
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
                     ^ is = '[masked]:Secret(name='aws_s3_access_key')'
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is 
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is = 
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is 
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
                 ^ is = '[masked]:Secret(name='aws_s3_key_id')'
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
      ^ is = 55775941
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
        ^ is 55775941
  ), error()
  assert (
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
      ^ is = False
  ), error()
  assert (
  ^ is False
      get_bucket_size(
          name=bucket_name,
          prefix=bucket_path,
          minio_enabled=minio_enabled,
          access_key=self.context.secret_access_key,
          key_id=self.context.access_key_id,
      )
      > size_before
  ), error()

Where
  File '/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py', line 1392 in 'delete_all'

1384\|                      get_bucket_size(
1385\|                          name=bucket_name,
1386\|                          prefix=bucket_path,
1387\|                          minio_enabled=minio_enabled,
1388\|                          access_key=self.context.secret_access_key,
1389\|                          key_id=self.context.access_key_id,
1390\|                      )
1391\|                      > size_before
1392\|>                 ), error()
1393\|  
1394\|          finally:
1395\|              with Finally("I drop the table on each node"):
/s3/aws s3 zero copy replication/insert multiple replicasXFail 50s 186ms
Under investigation
AssertionError
Traceback (most recent call last):
  File "regression/s3/regression.py", line 451, in 
    regression()
  File "regression/s3/regression.py", line 431, in regression
    aws_s3_regression(
  File "regression/s3/regression.py", line 325, in aws_s3_regression
    Feature(test=load("s3.tests.zero_copy_replication", "aws_s3"))(
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2145, in aws_s3
    outline()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 2130, in outline
    scenario()
  File "/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py", line 1193, in insert_multiple_replicas
    assert added_size >= expected * 0.99, error()
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert added_size >= expected * 0.99, error()

Assertion values
  assert added_size >= expected * 0.99, error()
         ^ is 0
  assert added_size >= expected * 0.99, error()
                       ^ is 6306510
  assert added_size >= expected * 0.99, error()
                                ^ is = 6243444.9
  assert added_size >= expected * 0.99, error()
                    ^ is = False
  assert added_size >= expected * 0.99, error()
  ^ is False

Where
  File '/builds/altinity-qa/clickhouse/cicd/release/regression/s3/../s3/tests/zero_copy_replication.py', line 1193 in 'insert_multiple_replicas'

1185\|                      name=bucket_name,
1186\|                      prefix=bucket_path,
1187\|                      minio_enabled=minio_enabled,
1188\|                      access_key=self.context.secret_access_key,
1189\|                      key_id=self.context.access_key_id,
1190\|                  )
1191\|                  added_size = current_size - size_before
1192\|  
1193\|>                 assert added_size >= expected * 0.99, error()
1194\|                  assert added_size <= expected * 1.01, error()
1195\|  
1196\|          finally:

Results

Test Name Result Duration
/s3 OK 1h 1m
/s3/aws s3 table function OK 3m 0s
/s3/aws s3 table function/auto OK 9s 991ms
/s3/aws s3 table function/compression OK 9s 824ms
/s3/aws s3 table function/credentials OK 985ms
/s3/aws s3 table function/data format OK 8s 93ms
/s3/aws s3 table function/multipart OK 4s 542ms
/s3/aws s3 table function/multiple columns OK 1s 289ms
/s3/aws s3 table function/partition OK 1s 68ms
/s3/aws s3 table function/remote host filter OK 36s 739ms
/s3/aws s3 table function/syntax OK 1s 49ms
/s3/aws s3 table function/wildcard OK 18s 776ms
/s3/aws s3 table function/ssec encryption check Skip 412us
/s3/aws s3 table function/ssec OK 1m 27s
/s3/aws s3 table function/ssec/auto OK 7s 504ms
/s3/aws s3 table function/ssec/compression OK 7s 711ms
/s3/aws s3 table function/ssec/credentials OK 2s 98ms
/s3/aws s3 table function/ssec/data format OK 6s 89ms
/s3/aws s3 table function/ssec/multipart OK 4s 157ms
/s3/aws s3 table function/ssec/multiple columns OK 2s 325ms
/s3/aws s3 table function/ssec/partition OK 4s 213ms
/s3/aws s3 table function/ssec/remote host filter OK 36s 373ms
/s3/aws s3 table function/ssec/syntax OK 2s 35ms
/s3/aws s3 table function/ssec/wildcard OK 3s 138ms
/s3/aws s3 invalid table function OK 2s 97ms
/s3/aws s3 invalid table function/empty path OK 254ms
/s3/aws s3 invalid table function/empty structure OK 261ms
/s3/aws s3 invalid table function/invalid compression OK 172ms
/s3/aws s3 invalid table function/invalid credentials OK 444ms
/s3/aws s3 invalid table function/invalid format OK 424ms
/s3/aws s3 invalid table function/invalid path OK 235ms
/s3/aws s3 invalid table function/invalid structure OK 291ms
/s3/aws s3 disk OK 32m 20s
/s3/aws s3 disk/access OK 41s 64ms
/s3/aws s3 disk/access skip check OK 37s 973ms
/s3/aws s3 disk/add storage OK 1m 26s
/s3/aws s3 disk/alter move OK 51s 23ms
/s3/aws s3 disk/alter on cluster modify ttl OK 1m 23s
/s3/aws s3 disk/cache OK 1m 24s
/s3/aws s3 disk/cache default OK 42s 756ms
/s3/aws s3 disk/cache path OK 44s 452ms
/s3/aws s3 disk/compact parts OK 38s 659ms
/s3/aws s3 disk/config over restart OK 1m 13s
/s3/aws s3 disk/default move factor OK 55s 377ms
/s3/aws s3 disk/delete OK 2m 17s
/s3/aws s3 disk/download appropriate disk OK 1m 9s
/s3/aws s3 disk/environment credentials OK 1m 20s
/s3/aws s3 disk/exports OK 41s 331ms
/s3/aws s3 disk/generic url XFail 3ms
/s3/aws s3 disk/imports OK 42s 994ms
/s3/aws s3 disk/max single part upload size syntax OK 45s 772ms
/s3/aws s3 disk/mergetree OK 3m 11s
/s3/aws s3 disk/mergetree collapsing OK 46s 193ms
/s3/aws s3 disk/mergetree versionedcollapsing OK 45s 481ms
/s3/aws s3 disk/metadata OK 44s 898ms
/s3/aws s3 disk/min bytes for seek syntax OK 41s 236ms
/s3/aws s3 disk/multiple storage OK 47s 169ms
/s3/aws s3 disk/multiple storage query OK 47s 779ms
/s3/aws s3 disk/perform ttl move on insert OK 1m 21s
/s3/aws s3 disk/perform ttl move on insert default OK 47s 243ms
/s3/aws s3 disk/performance ttl move OK 57s 141ms
/s3/aws s3 disk/remote host filter OK 1m 24s
/s3/aws s3 disk/restart OK 16s 548ms
/s3/aws s3 disk/specific url OK 42s 446ms
/s3/aws s3 disk/syntax OK 44s 649ms
/s3/aws s3 disk/wide parts OK 45s 497ms
/s3/aws s3 disk/ssec Skip 657us
/s3/aws s3 sanity OK 49s 76ms
/s3/aws s3 sanity/sanity OK 8s 189ms
/s3/aws s3 invalid disk OK 2m 1s
/s3/aws s3 invalid disk/access default OK 14s 576ms
/s3/aws s3 invalid disk/access failed OK 15s 553ms
/s3/aws s3 invalid disk/access failed skip check OK 41s 666ms
/s3/aws s3 invalid disk/cache path conflict OK 11s 543ms
/s3/aws s3 invalid disk/empty endpoint OK 9s 550ms
/s3/aws s3 invalid disk/invalid endpoint OK 9s 626ms
/s3/aws s3 invalid disk/invalid type OK 19s 262ms
/s3/aws s3 zero copy replication OK 16m 58s
/s3/aws s3 zero copy replication/add replica XFail 1m 52s
/s3/aws s3 zero copy replication/alter OK 49s 705ms
/s3/aws s3 zero copy replication/alter repeat OK 1m 36s
/s3/aws s3 zero copy replication/default value OK 12s 114ms
/s3/aws s3 zero copy replication/delete OK 49s 942ms
/s3/aws s3 zero copy replication/delete all XFail 51s 692ms
/s3/aws s3 zero copy replication/drop alter replica OK 59s 551ms
/s3/aws s3 zero copy replication/drop replica OK 1m 5s
/s3/aws s3 zero copy replication/global setting OK 51s 330ms
/s3/aws s3 zero copy replication/insert multiple replicas XFail 50s 186ms
/s3/aws s3 zero copy replication/lost data during mutation OK 3s 273ms
/s3/aws s3 zero copy replication/metadata OK 43s 6ms
/s3/aws s3 zero copy replication/performance alter OK 1m 15s
/s3/aws s3 zero copy replication/performance insert OK 1m 14s
/s3/aws s3 zero copy replication/performance select OK 1m 19s
/s3/aws s3 zero copy replication/ttl delete OK 50s 790ms
/s3/aws s3 zero copy replication/ttl move OK 51s 916ms
/s3/aws s3 reconnect OK 2m 42s
/s3/aws s3 reconnect/local and s3 disk OK 48s 643ms
/s3/aws s3 reconnect/local and s3 volumes OK 52s 381ms
/s3/aws s3 reconnect/s3 disk OK 1m 1s
/s3/aws s3 backup Skip 1ms

Generated by TestFlows Open-Source Test Framework v1.9.220810.1192506