Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

External User Directory Test Run Report

DateMar 01, 2024 20:53
Duration33m 11s
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/19a57e6f4a8ca8dd7737ac12f3ba472b37ac166c/package_aarch64/clickhouse-common-static_23.3.19.33.altinitystable_arm64.deb
version23.3.19.33.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash6da94b78dc53cb8965ab56c04a89ebf54ed04cbc
job.id8115174679
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8115174679
archaarch64
stressFalse
allow_vfsFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/19a57e6f4a8ca8dd7737ac12f3ba472b37ac166c/package_aarch64/clickhouse-common-static_23.3.19.33.altinitystable_arm64.deb
collect_service_logsTrue

Summary

93.7%OK
6.3%Known

Statistics

Units OK Fail XFail
Features
7
7
Scenarios
83
80
3
Examples
21
17
4
Steps
6456
6444
12

Known Fails

Test NameResultMessage
/external user directory/restart/parallel loginXFail 1ms
fails on 23.3
None
/external user directory/connection protocols/tls cipher suiteXFail 20s 466ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 124, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 118, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 266, in tls_cipher_suite
    login(servers, "openldap4", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/external user directory/connection protocols/tls require cert default demandXFail 19s 562ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 124, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 118, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 219, in tls_require_cert_default_demand
    login(servers, "openldap2", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|

Results

Test Name Result Duration
/external user directory OK 33m 11s
/external user directory/sanity OK 1s 433ms
/external user directory/simple OK 20s 540ms
/external user directory/restart OK 1m 31s
/external user directory/restart/dynamically added users OK 31s 940ms
/external user directory/restart/multiple external user directories OK 29s 146ms
/external user directory/restart/one external user directory OK 30s 214ms
/external user directory/restart/parallel login XFail 1ms
/external user directory/server config OK 3m 20s
/external user directory/server config/auth dn value OK 21s 391ms
/external user directory/server config/empty ca cert dir OK 21s 530ms
/external user directory/server config/empty ca cert file OK 21s 476ms
/external user directory/server config/empty host OK 2s 88ms
/external user directory/server config/empty server name OK 2s 97ms
/external user directory/server config/invalid auth dn prefix OK 21s 508ms
/external user directory/server config/invalid auth dn suffix OK 21s 511ms
/external user directory/server config/invalid enable tls value OK 2s 121ms
/external user directory/server config/invalid host OK 21s 436ms
/external user directory/server config/invalid port OK 20s 331ms
/external user directory/server config/invalid tls require cert value OK 21s 464ms
/external user directory/server config/invalid verification cooldown value OK 12s 560ms
/external user directory/server config/missing host OK 5s 696ms
/external user directory/server config/syntax OK 5s 665ms
/external user directory/external user directory config OK 3m 13s
/external user directory/external user directory config/defined twice roles OK 22s 522ms
/external user directory/external user directory config/defined twice server OK 21s 659ms
/external user directory/external user directory config/empty roles OK 21s 753ms
/external user directory/external user directory config/empty server OK 21s 529ms
/external user directory/external user directory config/invalid role in roles OK 21s 408ms
/external user directory/external user directory config/invalid server OK 20s 560ms
/external user directory/external user directory config/missing roles OK 21s 722ms
/external user directory/external user directory config/missing server OK 20s 365ms
/external user directory/external user directory config/more than one user directory OK 21s 778ms
/external user directory/connection protocols OK 7m 7s
/external user directory/connection protocols/plain text OK 21s 685ms
/external user directory/connection protocols/plain text with custom port OK 21s 652ms
/external user directory/connection protocols/starttls OK 1m 26s
/external user directory/connection protocols/starttls with custom port OK 21s 529ms
/external user directory/connection protocols/tls OK 1m 26s
/external user directory/connection protocols/tls cipher suite XFail 20s 466ms
/external user directory/connection protocols/tls enable tls default yes OK 21s 524ms
/external user directory/connection protocols/tls minimum protocol version OK 1m 47s
/external user directory/connection protocols/tls require cert default demand XFail 19s 562ms
/external user directory/connection protocols/tls with custom port OK 21s 546ms
/external user directory/user authentications OK 13m 45s
/external user directory/user authentications/empty username and empty password OK 93ms
/external user directory/user authentications/invalid long username and valid short password OK 177ms
/external user directory/user authentications/invalid username and valid password OK 179ms
/external user directory/user authentications/login after clickhouse server is restarted OK 12s 961ms
/external user directory/user authentications/login after ldap server is restarted OK 3s 230ms
/external user directory/user authentications/login after user cn changed in ldap OK 293ms
/external user directory/user authentications/login after user is added to ldap OK 154ms
/external user directory/user authentications/login after user is deleted from ldap OK 291ms
/external user directory/user authentications/login after user password changed in ldap OK 387ms
/external user directory/user authentications/parallel login OK 42s 920ms
/external user directory/user authentications/parallel login with rbac users OK 43s 348ms
/external user directory/user authentications/parallel login with the same user OK 42s 140ms
/external user directory/user authentications/user lookup priority OK 674ms
/external user directory/user authentications/valid ascii username and utf8 password OK 156ms
/external user directory/user authentications/valid long username and short password OK 164ms
/external user directory/user authentications/valid short username and invalid long password OK 179ms
/external user directory/user authentications/valid short username and long password OK 159ms
/external user directory/user authentications/valid username and invalid empty password OK 168ms
/external user directory/user authentications/valid username and invalid password OK 176ms
/external user directory/user authentications/valid username and password OK 156ms
/external user directory/user authentications/valid username and password invalid server OK 107ms
/external user directory/user authentications/valid username with valid empty password OK 165ms
/external user directory/user authentications/valid utf8 username and ascii password OK 153ms
/external user directory/user authentications/login after ldap external user directory is removed OK 15s 843ms
/external user directory/user authentications/parallel login with multiple servers OK 1m 43s
/external user directory/user authentications/parallel login with rbac and multiple servers OK 2m 22s
/external user directory/user authentications/parallel login with the same user multiple servers OK 1m 1s
/external user directory/user authentications/default verification cooldown value OK 21s 748ms
/external user directory/user authentications/verification cooldown reset when invalid password is provided OK 21s 893ms
/external user directory/user authentications/valid verification cooldown value cn change OK 23s 755ms
/external user directory/user authentications/valid verification cooldown value ldap unavailable OK 24s 809ms
/external user directory/user authentications/valid verification cooldown value password change OK 24s 812ms
/external user directory/user authentications/verification cooldown performance OK 1m 59s
/external user directory/user authentications/verification cooldown reset on server auth dn prefix parameter change OK 23s 853ms
/external user directory/user authentications/verification cooldown reset on server auth dn suffix parameter change OK 23s 936ms
/external user directory/user authentications/verification cooldown reset on server host parameter change OK 24s 160ms
/external user directory/user authentications/verification cooldown reset on server port parameter change OK 24s 52ms
/external user directory/roles OK 2m 1s
/external user directory/roles/add privilege OK 15s 910ms
/external user directory/roles/new role OK 15s 963ms
/external user directory/roles/not present role added OK 18s 747ms
/external user directory/roles/readd privilege by readding role OK 16s 727ms
/external user directory/roles/remove privilege OK 16s 270ms
/external user directory/roles/remove privilege by removing role OK 16s 45ms
/external user directory/roles/remove role OK 16s 70ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232