Document status - Public

Copyright 2024, Altinity Inc. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc.

QA Software Build Report

ClickHouse v24.3.5.48.altinityfips

(c) 2024 Altinity Inc. All Rights Reserved.

Approval

Status: Approved for release

Reviewed by: vzakaznikov@altinity.com

Date: Mon Sep 2 08:28:12 AM -05 2024

Table of Contents

1 Test Results
- 1.1 x86_64 Results
2 Results Analysis
- 2.1 x86_64
  - 2.1.1 Release
    - 2.1.1.1 Trivy
    - 2.1.1.2 Scout
    - 2.1.1.3 Keeper Trivy
    - 2.1.1.4 Keeper Scout
    - 2.1.1.5 Integration
    - 2.1.1.6 Stateful
    - 2.1.1.7 Stateless
    - 2.1.1.8 Regression
  - 2.1.2 Asan
    - 2.1.2.1 Integration
    - 2.1.2.2 Analyzer Integration
    - 2.1.2.3 Stateless
    - 2.1.2.4 Stateful
    - 2.1.2.5 Stress
  - 2.1.4 Tsan
    - 2.1.3.1 Integration
    - 2.1.3.2 Stateless
    - 2.1.3.3 Stateful
    - 2.1.3.4 Stress
  - 2.1.5 UBsan
    - 2.1.4.1 Stateless
    - 2.1.4.2 Stateful
    - 2.1.4.3 Stress
  - 2.1.6 Debug
    - 2.1.5.1 Stateless
    - 2.1.5.2 Stateful
    - 2.1.5.3 Stress

Test Results

x86_64 Results

Stage	Status	Note
Trivy	Pass	Total: 38 (UNKNOWN: 0, LOW: 32, MEDIUM: 6, HIGH: 0, CRITICAL: 0)
Scout	Pass	Total: 15 (LOW: 12, MEDIUM: 3, HIGH: 0, CRITICAL: 0)
Keeper Trivy	Pass	No vulnerabilities.
Keeper Scout	Pass	No vulnerabilities.
Integration	Pass	with known fails.
Stateful	Pass
Stateless	Pass	with known fails.
Aes Encryption	Pass
Aggregate Functions	Pass
Alter	Pass
Atomic Insert	Pass
Base58	Pass
Benchmark AWS	Pass
Benchmark Minio	Pass
Benchmark GCS	Pass
ClickHouse Keeper	Pass
ClickHouse Keeper SSL	Pass
Data Types	Pass
DateTime64 Extended Range	Pass
Disk Level Encryption	Pass
DNS	Pass
Example	Pass
Engines	Pass
Extended Precision Data Types	Pass
Kafka	Pass
Kerberos	Pass
Key Value	Pass
LDAP Authentication	Pass
LDAP External User Directory	Pass
LDAP Role Mapping	Pass
Lightweight Delete	Pass
Memory	Pass
Parquet	Pass
Parquet AWS	Pass
Parquet Minio	Pass
Part Moves Between Shards	Pass
RBAC	Pass
Selects	Pass
Session Timezone	Pass
SSL Server	Pass
S3 AWS	Pass
S3 Minio	Pass
S3 GCS	Pass
Tiered Storage	Pass
Tiered Storage AWS	Pass
Tiered Storage Minio	Pass
Tiered Storage GCS	Pass
Window Functions	Pass
Asan Integration	Pass	with known fails.
Asan Analyzer Integration	Pass	with known fails.
Asan Stateful	Pass
Asan Stateless	Pass	with known fails.
Asan Stress	Pass
Tsan Integration	Pass	with known fails.
Tsan Stateful	Pass
Tsan Stateless	Pass	with known fails.
Tsan Stress	Pass
UBsan Stateful	Pass
UBsan Stateless	Pass	with known fails.
UBsan Stress	Pass
Debug Stateful	Pass
Debug Stateless	Pass	with known fails.
Debug Stress	Pass

Results https://altinity-test-reports.s3.amazonaws.com/index.html#builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/
GitLab Pipeline https://gitlab.com/altinity-qa/clickhouse/cicd/release/-/pipelines/1434789825
GitHub Actions https://github.com/Altinity/ClickHouse/actions/runs/10644525464
GitHub Commit https://github.com/Altinity/ClickHouse/commit/f030c602050d371934c7b7609b7a8228fb1c7779
GitHub Branch https://github.com/Altinity/ClickHouse/tree/releases/24.3.5-fips

Results Analysis

x86_64

Release

Release Trivy

Results
https://altinity-test-reports.s3.amazonaws.com/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/staging-docker-trivy-ubuntu-server-amd64/results.html

Release Scout

Results
https://altinity-test-reports.s3.amazonaws.com/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/staging-docker-scout-ubuntu-server-amd64/results.html

Keeper Release Trivy

Results
https://altinity-test-reports.s3.amazonaws.com/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/staging-docker-trivy-ubuntu-keeper-amd64/results.html

Keeper Release Scout

Results
https://altinity-test-reports.s3.amazonaws.com/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/staging-docker-scout-ubuntu-keeper-amd64/results.html

Release Integration

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/release/integration_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/release/integration_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/release/integration_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/release/integration_results_4.html

Fails

Test: test_storage_s3_queue/test.py::test_upgrade
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/release/integration_results_4.html
Comment: Known fail.
Status: OK to fail.

Release Stateful

Results https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateful/amd64/stateful_results.html

Release Stateless

Results https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/amd64/stateless_results.html

Fails

Test: 02797_aggregator_huge_mem_usage_bug
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/amd64/stateless_results.html
Comment: Flaky.
Status: OK to fail.

Release Regression

Results
- Aes Encryption
- Aggregate Functions
- Alter Attach
- Alter Move
- Alter Replace
- Atomic Insert
- Base58
- Benchmark AWS
- Benchmark Minio
- Benchmark GCS
- ClickHouse Keeper
- ClickHouse Keeper SSL
- Data Types
- DateTime64 Extended Range
- Disk Level Encryption
- DNS
- Example
- Engines
- Extended Precision Data Types
- Kafka
- Kerberos
- Key Value
- LDAP Authentication
- LDAP External User Directory
- LDAP Role Mapping
- Lightweight Delete
- Memory
- Parquet
- Parquet AWS
- Parquet Minio
- Part Moves Between Shards
- RBAC
- Selects
- Session Timezone
- SSL Server
- S3 AWS
- S3 Minio
- S3 GCS
- Tiered Storage
- Tiered Storage AWS
- Tiered Storage Minio
- Tiered Storage GCS
- Window Functions

Asan

Asan Integration

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan/integration_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan/integration_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan/integration_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan/integration_results_4.html

Fails

Test: test_storage_s3_queue/test.py::test_upgrade
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan/integration_results_4.html
Comment: Known fail.
Status: OK to fail.

Asan Analyzer Integration

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_4.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_5.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_6.html

Fails

Test: test_storage_s3_queue/test.py::test_upgrade
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/asan_analyzer/integration_results_2.html
Comment: Known fail.
Status: OK to fail.

Asan Stateless

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/asan/stateless_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/asan/stateless_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/asan/stateless_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/asan/stateless_results_4.html

Fails

Test: 00997_set_index_array
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/asan/stateless_results_2.html
Comment: Known fail.
Status: Flaky.

Asan Stateful

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateful/asan/stateful_results.html

Asan Stress

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stress/asan/stress_results.html

Tsan

Tsan Integration

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_4.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_5.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_6.html

Fails

Test: test_storage_s3_queue/test.py::test_upgrade
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/integration/tsan/integration_results_2.html
Comment: Known fail.
Status: OK to fail.

Tsan Stateless

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_4.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_5.html

Fails

Test: 02154_parser_backtracking
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_4.html
Comment: Known fail.
Status: Flaky.

Tsan Stateful

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateful/tsan/stateful_results.html

Tsan Stress

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stress/tsan/stress_results.html

UBsan

UBsan Stateless

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/ubsan/stateless_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/ubsan/stateless_results_2.html

Fails

Test: 02497_remote_disk_fat_column
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/tsan/stateless_results_2.html
Comment: Known fail.
Status: Flaky.

UBsan Stateful

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateful/ubsan/stateful_results.html

UBsan Stress

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stress/ubsan/stress_results.html

Debug

Debug Stateless

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_1.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_2.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_3.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_4.html
https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_5.html

Fails

Test: 01414_mutations_and_errors_zookeeper
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_3.html
Comment: Known fail.
Status: Flaky.

Test: 01825_type_json_in_array
Log: https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateless/debug/stateless_results_5.html
Comment: Known fail.
Status: Flaky.

Debug Stateful

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stateful/debug/stateful_results.html

Debug Stress

Results

https://s3.amazonaws.com/altinity-test-reports/builds/stable/v24.3.5.48.altinityfips/2024-08-31T19-56-11.305/stress/debug/stress_results.html