Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateJul 22, 2024 9:23
Duration1h 53m
Framework TestFlows 2.0.240705.1133204

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/b6642564dbf296d6e55b907d7af6c8087470b672/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/24.3/b6642564dbf296d6e55b907d7af6c8087470b672/package_aarch64/clickhouse-common-static_24.3.5.47.altinitystable_arm64.deb
version24.3.5.47.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashc5e1513a2214ee33696c29717935e0a94989ac2a
job.id10037417827
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/10037417827
archaarch64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/24.3/b6642564dbf296d6e55b907d7af6c8087470b672/package_aarch64/clickhouse-common-static_24.3.5.47.altinitystable_arm64.deb
keeper_binary_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerFalse
reuse_envFalse
storages['gcs']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

94.1%OK
3.3%Known

Statistics

Units Skip OK Fail Error XFail
Modules
2
2
Features
10
2
8
Scenarios
98
4
86
8
Checks
15
15
Examples
59
59
Steps
18374
21
18327
6
10
10

Known Fails

Test NameResultMessage
/s3/gcs/disk/cacheXFail 3ms
Under development for 22.8 and newer.
None
/s3/gcs/disk/cache defaultXFail 844us
Under development for 22.8 and newer.
None
/s3/gcs/disk/cache pathXFail 1ms
Under development for 22.8 and newer.
None
/s3/gcs/disk/low cardinality offsetXFail 46s 696ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 472, in gcs_regression
    Feature(test=load("s3.tests.disk", "gcs"))(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2453, in gcs
    disk_tests(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2431, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2394, in low_cardinality_offset
    assert output == "23999\n", error()
           ^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2394 in 'low_cardinality_offset'

2386\|                      "1",
2387\|                  ),
2388\|                  (
2389\|                      "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2390\|                      "1",
2391\|                  ),
2392\|              ],
2393\|          ).output
2394\|>         assert output == "23999\n", error()
2395\|  
2396\|  
2397\|  @TestFeature
/s3/gcs/disk/no restartXFail 3ms
https://github.com/ClickHouse/ClickHouse/issues/58924
None
/s3/gcs/invalid disk/cache path conflictXFail 2ms
Under development for 22.8 and newer.
None
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=compactXFail 4m 12s
Unstable test
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 475, in gcs_regression
    Feature(test=load("s3.tests.combinatoric_table", "feature"))(uri=uri)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/combinatoric_table.py", line 179, in feature
    Scenario(title, test=check_table_combination)(**table_config)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/combinatoric_table.py", line 143, in check_table_combination
    retry(assert_row_count, timeout=60, delay=5)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 1531, in assert_row_count
    assert rows == actual_count, error()
           ^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert rows == actual_count, error()

Assertion values
  assert rows == actual_count, error()
         ^ is 10000
  assert rows == actual_count, error()
                 ^ is 0
  assert rows == actual_count, error()
              ^ is = False
  assert rows == actual_count, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 1531 in 'assert_row_count'

1523\|  
1524\|  @TestStep(Then)
1525\|  def assert_row_count(self, node, table_name: str, rows: int = 1000000):
1526\|      """Assert that the number of rows in a table is as expected."""
1527\|      if node is None:
1528\|          node = current().context.node
1529\|  
1530\|      actual_count = get_row_count(node=node, table_name=table_name)
1531\|>     assert rows == actual_count, error()
1532\|  
1533\|  
1534\|  @TestStep(Then)

Results

Test Name Result Duration
/s3 OK 1h 53m
/s3/gcs OK 1h 53m
/s3/gcs/table function OK 1m 51s
/s3/gcs/table function/auto OK 10s 24ms
/s3/gcs/table function/compression OK 9s 748ms
/s3/gcs/table function/credentials OK 1s 125ms
/s3/gcs/table function/data format OK 15s 795ms
/s3/gcs/table function/measure file size Skip 3ms
/s3/gcs/table function/multipart OK 8s 299ms
/s3/gcs/table function/multiple columns OK 1s 309ms
/s3/gcs/table function/partition OK 1s 557ms
/s3/gcs/table function/remote host filter OK 44s 41ms
/s3/gcs/table function/syntax OK 1s 764ms
/s3/gcs/table function/wildcard OK 18s 229ms
/s3/gcs/invalid table function OK 2s 231ms
/s3/gcs/invalid table function/empty path OK 244ms
/s3/gcs/invalid table function/empty structure OK 249ms
/s3/gcs/invalid table function/invalid compression OK 255ms
/s3/gcs/invalid table function/invalid credentials OK 458ms
/s3/gcs/invalid table function/invalid format OK 515ms
/s3/gcs/invalid table function/invalid path OK 241ms
/s3/gcs/invalid table function/invalid structure OK 250ms
/s3/gcs/disk OK 37m 57s
/s3/gcs/disk/access OK 47s 747ms
/s3/gcs/disk/access skip check OK 49s 730ms
/s3/gcs/disk/add storage OK 1m 39s
/s3/gcs/disk/alter move OK 56s 933ms
/s3/gcs/disk/alter on cluster modify ttl OK 1m 46s
/s3/gcs/disk/cache XFail 3ms
/s3/gcs/disk/cache default XFail 844us
/s3/gcs/disk/cache path XFail 1ms
/s3/gcs/disk/compact parts OK 43s 982ms
/s3/gcs/disk/config over restart OK 1m 16s
/s3/gcs/disk/default move factor OK 1m 20s
/s3/gcs/disk/delete OK 3m 34s
/s3/gcs/disk/download appropriate disk OK 1m 19s
/s3/gcs/disk/drop sync OK 54s 942ms
/s3/gcs/disk/environment credentials Skip 16ms
/s3/gcs/disk/exports OK 49s 63ms
/s3/gcs/disk/generic url Skip 2ms
/s3/gcs/disk/imports OK 48s 541ms
/s3/gcs/disk/log OK 2m 49s
/s3/gcs/disk/low cardinality offset XFail 46s 696ms
/s3/gcs/disk/max single part upload size syntax OK 46s 758ms
/s3/gcs/disk/mergetree OK 3m 53s
/s3/gcs/disk/mergetree collapsing OK 57s 896ms
/s3/gcs/disk/mergetree versionedcollapsing OK 58s 596ms
/s3/gcs/disk/metadata OK 52s 554ms
/s3/gcs/disk/min bytes for seek syntax OK 48s 510ms
/s3/gcs/disk/multiple storage OK 1m 1s
/s3/gcs/disk/multiple storage query OK 1m 0s
/s3/gcs/disk/no restart XFail 3ms
/s3/gcs/disk/perform ttl move on insert OK 1m 32s
/s3/gcs/disk/perform ttl move on insert default OK 57s 693ms
/s3/gcs/disk/performance ttl move OK 1m 14s
/s3/gcs/disk/remote host filter OK 1m 36s
/s3/gcs/disk/specific url Skip 2ms
/s3/gcs/disk/syntax OK 1m 0s
/s3/gcs/disk/wide parts OK 51s 14ms
/s3/gcs/invalid disk OK 3m 47s
/s3/gcs/invalid disk/access default OK 11s 696ms
/s3/gcs/invalid disk/access failed OK 11s 679ms
/s3/gcs/invalid disk/access failed skip check OK 48s 679ms
/s3/gcs/invalid disk/cache path conflict XFail 2ms
/s3/gcs/invalid disk/empty endpoint OK 8s 742ms
/s3/gcs/invalid disk/invalid endpoint OK 2m 9s
/s3/gcs/invalid disk/invalid type OK 17s 285ms
/s3/gcs/alter OK 4m 0s
/s3/gcs/alter/attach from OK 12s 906ms
/s3/gcs/alter/columns OK 12s 197ms
/s3/gcs/alter/detach OK 22s 668ms
/s3/gcs/alter/drop OK 45s 912ms
/s3/gcs/alter/fetch OK 30s 456ms
/s3/gcs/alter/freeze OK 8s 752ms
/s3/gcs/alter/index OK 10s 506ms
/s3/gcs/alter/move to table OK 17s 779ms
/s3/gcs/alter/order by OK 1s 875ms
/s3/gcs/alter/projection OK 5s 391ms
/s3/gcs/alter/replace OK 16s 971ms
/s3/gcs/alter/sample by OK 1s 724ms
/s3/gcs/alter/update delete OK 6s 639ms
/s3/gcs/combinatoric table OK 24m 31s
/s3/gcs/combinatoric table/engine=MergeTree,replicated=False,n_cols=500,n_tables=1,part_type=wide OK 5m 7s
/s3/gcs/combinatoric table/engine=ReplacingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=unspecified OK 43s 713ms
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=10,n_tables=1,part_type=wide OK 22s 72ms
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=False,n_cols=2000,n_tables=3,part_type=unspecified OK 25s 108ms
/s3/gcs/combinatoric table/engine=VersionedCollapsingMergeTree,replicated=False,n_cols=10,n_tables=3,part_type=compact OK 11s 772ms
/s3/gcs/combinatoric table/engine=AggregatingMergeTree,replicated=False,n_cols=500,n_tables=1,part_type=unspecified OK 7s 275ms
/s3/gcs/combinatoric table/engine=SummingMergeTree,replicated=False,n_cols=500,n_tables=1,part_type=compact OK 5m 8s
/s3/gcs/combinatoric table/engine=MergeTree,replicated=True,n_cols=10,n_tables=3,part_type=unspecified OK 32s 996ms
/s3/gcs/combinatoric table/engine=ReplacingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 1m 12s
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=compact XFail 4m 12s
/s3/gcs/combinatoric table/engine=VersionedCollapsingMergeTree,replicated=True,n_cols=500,n_tables=1,part_type=unspecified OK 3m 10s
/s3/gcs/combinatoric table/engine=AggregatingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 1m 12s
/s3/gcs/combinatoric table/engine=SummingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 1m 10s
/s3/gcs/combinatoric table/engine=SummingMergeTree,replicated=True,n_cols=10,n_tables=1,part_type=unspecified OK 9s 832ms
/s3/gcs/zero copy replication Skip 11ms
/s3/gcs/backup Skip 1ms
/s3/gcs/settings OK 27m 41s
/s3/gcs/settings/setting combinations OK 26m 57s
/s3/gcs/table function performance OK 10m 40s
/s3/gcs/table function performance/setup OK 724ms
/s3/gcs/table function performance/wildcard OK 10m 39s

Generated by TestFlows Open-Source Test Framework v2.0.240705.1133204