Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

S3 Test Run Report

DateNov 05, 2024 18:25
Duration1h 57m
Framework TestFlows 2.0.240705.1133204

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/5b67ba5a4febfa774d6373929d551c6a697d7d92/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/24.3/5b67ba5a4febfa774d6373929d551c6a697d7d92/package_aarch64/clickhouse-common-static_24.3.12.76.altinitystable_arm64.deb
version24.3.12.76.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashc5e1513a2214ee33696c29717935e0a94989ac2a
job.id11672696143
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/11672696143
archaarch64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/24.3/5b67ba5a4febfa774d6373929d551c6a697d7d92/package_aarch64/clickhouse-common-static_24.3.12.76.altinitystable_arm64.deb
keeper_binary_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerFalse
reuse_envFalse
storages['gcs']
minio_urihttp://minio1:9001
minio_root_userminio
minio_root_passwordminio123
aws_s3_bucketSecret(name='aws_s3_bucket')
aws_s3_regionSecret(name='aws_s3_region')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_access_keySecret(name='aws_s3_access_key')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

94.1%OK
3.3%Known

Statistics

Units Skip OK Fail Error XFail
Modules
2
2
Features
10
2
8
Scenarios
98
4
86
8
Checks
15
15
Examples
59
59
Steps
18417
21
18370
6
10
10

Known Fails

Test NameResultMessage
/s3/gcs/disk/cacheXFail 2ms
Under development for 22.8 and newer.
None
/s3/gcs/disk/cache defaultXFail 1ms
Under development for 22.8 and newer.
None
/s3/gcs/disk/cache pathXFail 1ms
Under development for 22.8 and newer.
None
/s3/gcs/disk/low cardinality offsetXFail 48s 130ms
https://github.com/ClickHouse/ClickHouse/pull/44875
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 472, in gcs_regression
    Feature(test=load("s3.tests.disk", "gcs"))(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2453, in gcs
    disk_tests(uri=uri, bucket_prefix=bucket_prefix)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2431, in disk_tests
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py", line 2394, in low_cardinality_offset
    assert output == "23999\n", error()
           ^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert output == "23999\n", error()

Assertion values
  assert output == "23999\n", error()
         ^ is '23999'
  assert output == "23999\n", error()
                ^ is = False

  assert output == "23999\n", error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/disk.py', line 2394 in 'low_cardinality_offset'

2386\|                      "1",
2387\|                  ),
2388\|                  (
2389\|                      "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem",
2390\|                      "1",
2391\|                  ),
2392\|              ],
2393\|          ).output
2394\|>         assert output == "23999\n", error()
2395\|  
2396\|  
2397\|  @TestFeature
/s3/gcs/disk/no restartXFail 1ms
https://github.com/ClickHouse/ClickHouse/issues/58924
None
/s3/gcs/invalid disk/cache path conflictXFail 2ms
Under development for 22.8 and newer.
None
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=compactXFail 4m 12s
Unstable test
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 553, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 545, in regression
    Module(test=storage_module)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/regression.py", line 475, in gcs_regression
    Feature(test=load("s3.tests.combinatoric_table", "feature"))(uri=uri)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/combinatoric_table.py", line 179, in feature
    Scenario(title, test=check_table_combination)(**table_config)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/combinatoric_table.py", line 143, in check_table_combination
    retry(assert_row_count, timeout=60, delay=5)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py", line 1531, in assert_row_count
    assert rows == actual_count, error()
           ^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert rows == actual_count, error()

Assertion values
  assert rows == actual_count, error()
         ^ is 10000
  assert rows == actual_count, error()
                 ^ is 0
  assert rows == actual_count, error()
              ^ is = False
  assert rows == actual_count, error()
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/s3/../s3/tests/common.py', line 1531 in 'assert_row_count'

1523\|  
1524\|  @TestStep(Then)
1525\|  def assert_row_count(self, node, table_name: str, rows: int = 1000000):
1526\|      """Assert that the number of rows in a table is as expected."""
1527\|      if node is None:
1528\|          node = current().context.node
1529\|  
1530\|      actual_count = get_row_count(node=node, table_name=table_name)
1531\|>     assert rows == actual_count, error()
1532\|  
1533\|  
1534\|  @TestStep(Then)

Results

Test Name Result Duration
/s3 OK 1h 57m
/s3/gcs OK 1h 57m
/s3/gcs/table function OK 1m 55s
/s3/gcs/table function/auto OK 9s 63ms
/s3/gcs/table function/compression OK 10s 85ms
/s3/gcs/table function/credentials OK 1s 167ms
/s3/gcs/table function/data format OK 17s 85ms
/s3/gcs/table function/measure file size Skip 1ms
/s3/gcs/table function/multipart OK 8s 339ms
/s3/gcs/table function/multiple columns OK 1s 320ms
/s3/gcs/table function/partition OK 1s 387ms
/s3/gcs/table function/remote host filter OK 47s 511ms
/s3/gcs/table function/syntax OK 1s 668ms
/s3/gcs/table function/wildcard OK 17s 936ms
/s3/gcs/invalid table function OK 2s 695ms
/s3/gcs/invalid table function/empty path OK 387ms
/s3/gcs/invalid table function/empty structure OK 288ms
/s3/gcs/invalid table function/invalid compression OK 292ms
/s3/gcs/invalid table function/invalid credentials OK 532ms
/s3/gcs/invalid table function/invalid format OK 587ms
/s3/gcs/invalid table function/invalid path OK 310ms
/s3/gcs/invalid table function/invalid structure OK 277ms
/s3/gcs/disk OK 38m 27s
/s3/gcs/disk/access OK 47s 628ms
/s3/gcs/disk/access skip check OK 47s 545ms
/s3/gcs/disk/add storage OK 1m 38s
/s3/gcs/disk/alter move OK 58s 898ms
/s3/gcs/disk/alter on cluster modify ttl OK 1m 52s
/s3/gcs/disk/cache XFail 2ms
/s3/gcs/disk/cache default XFail 1ms
/s3/gcs/disk/cache path XFail 1ms
/s3/gcs/disk/compact parts OK 46s 742ms
/s3/gcs/disk/config over restart OK 1m 18s
/s3/gcs/disk/default move factor OK 1m 23s
/s3/gcs/disk/delete OK 3m 31s
/s3/gcs/disk/download appropriate disk OK 1m 21s
/s3/gcs/disk/drop sync OK 55s 156ms
/s3/gcs/disk/environment credentials Skip 5ms
/s3/gcs/disk/exports OK 49s 114ms
/s3/gcs/disk/generic url Skip 2ms
/s3/gcs/disk/imports OK 52s 359ms
/s3/gcs/disk/log OK 2m 56s
/s3/gcs/disk/low cardinality offset XFail 48s 130ms
/s3/gcs/disk/max single part upload size syntax OK 50s 130ms
/s3/gcs/disk/mergetree OK 3m 56s
/s3/gcs/disk/mergetree collapsing OK 56s 492ms
/s3/gcs/disk/mergetree versionedcollapsing OK 57s 410ms
/s3/gcs/disk/metadata OK 58s 307ms
/s3/gcs/disk/min bytes for seek syntax OK 48s 452ms
/s3/gcs/disk/multiple storage OK 58s 74ms
/s3/gcs/disk/multiple storage query OK 1m 6s
/s3/gcs/disk/no restart XFail 1ms
/s3/gcs/disk/perform ttl move on insert OK 1m 33s
/s3/gcs/disk/perform ttl move on insert default OK 1m 0s
/s3/gcs/disk/performance ttl move OK 1m 18s
/s3/gcs/disk/remote host filter OK 1m 33s
/s3/gcs/disk/specific url Skip 1ms
/s3/gcs/disk/syntax OK 56s 211ms
/s3/gcs/disk/wide parts OK 46s 196ms
/s3/gcs/invalid disk OK 3m 53s
/s3/gcs/invalid disk/access default OK 11s 746ms
/s3/gcs/invalid disk/access failed OK 12s 859ms
/s3/gcs/invalid disk/access failed skip check OK 48s 437ms
/s3/gcs/invalid disk/cache path conflict XFail 2ms
/s3/gcs/invalid disk/empty endpoint OK 9s 734ms
/s3/gcs/invalid disk/invalid endpoint OK 2m 11s
/s3/gcs/invalid disk/invalid type OK 19s 488ms
/s3/gcs/alter OK 4m 54s
/s3/gcs/alter/attach from OK 19s 486ms
/s3/gcs/alter/columns OK 17s 893ms
/s3/gcs/alter/detach OK 27s 196ms
/s3/gcs/alter/drop OK 1m 1s
/s3/gcs/alter/fetch OK 37s 450ms
/s3/gcs/alter/freeze OK 11s 913ms
/s3/gcs/alter/index OK 11s 723ms
/s3/gcs/alter/move to table OK 22s 456ms
/s3/gcs/alter/order by OK 2s 387ms
/s3/gcs/alter/projection OK 5s 891ms
/s3/gcs/alter/replace OK 18s 360ms
/s3/gcs/alter/sample by OK 1s 928ms
/s3/gcs/alter/update delete OK 7s 582ms
/s3/gcs/combinatoric table OK 24m 59s
/s3/gcs/combinatoric table/engine=MergeTree,replicated=False,n_cols=500,n_tables=1,part_type=wide OK 5m 8s
/s3/gcs/combinatoric table/engine=ReplacingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=unspecified OK 58s 39ms
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=10,n_tables=1,part_type=wide OK 24s 748ms
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=False,n_cols=2000,n_tables=3,part_type=unspecified OK 26s 774ms
/s3/gcs/combinatoric table/engine=VersionedCollapsingMergeTree,replicated=False,n_cols=10,n_tables=3,part_type=compact OK 11s 932ms
/s3/gcs/combinatoric table/engine=AggregatingMergeTree,replicated=False,n_cols=500,n_tables=1,part_type=unspecified OK 8s 176ms
/s3/gcs/combinatoric table/engine=SummingMergeTree,replicated=False,n_cols=500,n_tables=1,part_type=compact OK 5m 8s
/s3/gcs/combinatoric table/engine=MergeTree,replicated=True,n_cols=10,n_tables=3,part_type=unspecified OK 43s 151ms
/s3/gcs/combinatoric table/engine=ReplacingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 1m 15s
/s3/gcs/combinatoric table/engine=CollapsingMergeTree,replicated=True,n_cols=500,n_tables=3,part_type=compact XFail 4m 12s
/s3/gcs/combinatoric table/engine=VersionedCollapsingMergeTree,replicated=True,n_cols=500,n_tables=1,part_type=unspecified OK 3m 1s
/s3/gcs/combinatoric table/engine=AggregatingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 1m 11s
/s3/gcs/combinatoric table/engine=SummingMergeTree,replicated=True,n_cols=10,n_tables=3,part_type=wide OK 1m 6s
/s3/gcs/combinatoric table/engine=SummingMergeTree,replicated=True,n_cols=10,n_tables=1,part_type=unspecified OK 10s 879ms
/s3/gcs/zero copy replication Skip 2ms
/s3/gcs/backup Skip 1ms
/s3/gcs/settings OK 30m 7s
/s3/gcs/settings/setting combinations OK 29m 19s
/s3/gcs/table function performance OK 10m 38s
/s3/gcs/table function performance/setup OK 779ms
/s3/gcs/table function performance/wildcard OK 10m 37s

Generated by TestFlows Open-Source Test Framework v2.0.240705.1133204