Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

External User Directory Test Run Report

DateApr 28, 2024 16:55
Duration21m 2s
Framework TestFlows 2.0.231215.1221232

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/false/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.8/8e9be16153250c8307730c9b88c8517181f494ce/package_release/clickhouse-common-static_23.8.11.29.altinitystable_amd64.deb
version23.8.11.29.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash17a81c07fc1f41fbee651e0ef0ca4b44e537e5b1
job.id8868426423
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/8868426423
archx86_64
stressFalse
allow_vfsFalse
allow_experimental_analyzerFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.8/8e9be16153250c8307730c9b88c8517181f494ce/package_release/clickhouse-common-static_23.8.11.29.altinitystable_amd64.deb
collect_service_logsTrue

Summary

92.8%OK
7.2%Known

Statistics

Units OK Fail XFail
Features
7
7
Scenarios
83
79
4
Examples
21
17
4
Steps
6376
6364
12

Known Fails

Test NameResultMessage
/external user directory/restart/parallel loginXFail 956us
fails on 23.3
None
/external user directory/connection protocols/tls cipher suiteXFail 15s 454ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 125, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 119, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 266, in tls_cipher_suite
    login(servers, "openldap4", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/external user directory/connection protocols/tls require cert default demandXFail 16s 478ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 125, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 119, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 219, in tls_require_cert_default_demand
    login(servers, "openldap2", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 844, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 844 in 'query'

836\|                  assert message in r.output, error(r.output)
837\|  
838\|          if not ignore_exception:
839\|              if message is None or "Exception:" not in message:
840\|                  with Then("check if output has exception") if steps else NullStep():
841\|                      if "Exception:" in r.output:
842\|                          if raise_on_exception:
843\|                              raise QueryRuntimeException(r.output)
844\|>                         assert False, error(r.output)
845\|  
846\|          return r
847\|
/external user directory/roles/not present role addedXFail 697us
roles are not being applied to active clickhouse-client connections in 23.8 
None

Results

Test Name Result Duration
/external user directory OK 21m 2s
/external user directory/sanity OK 227ms
/external user directory/simple OK 14s 437ms
/external user directory/restart OK 1m 3s
/external user directory/restart/dynamically added users OK 22s 394ms
/external user directory/restart/multiple external user directories OK 20s 471ms
/external user directory/restart/one external user directory OK 20s 437ms
/external user directory/restart/parallel login XFail 956us
/external user directory/server config OK 2m 29s
/external user directory/server config/auth dn value OK 15s 421ms
/external user directory/server config/empty ca cert dir OK 15s 423ms
/external user directory/server config/empty ca cert file OK 15s 371ms
/external user directory/server config/empty host OK 2s 62ms
/external user directory/server config/empty server name OK 2s 64ms
/external user directory/server config/invalid auth dn prefix OK 15s 423ms
/external user directory/server config/invalid auth dn suffix OK 15s 458ms
/external user directory/server config/invalid enable tls value OK 2s 83ms
/external user directory/server config/invalid host OK 15s 371ms
/external user directory/server config/invalid port OK 15s 407ms
/external user directory/server config/invalid tls require cert value OK 15s 411ms
/external user directory/server config/invalid verification cooldown value OK 12s 409ms
/external user directory/server config/missing host OK 4s 382ms
/external user directory/server config/syntax OK 3s 372ms
/external user directory/external user directory config OK 2m 17s
/external user directory/external user directory config/defined twice roles OK 15s 787ms
/external user directory/external user directory config/defined twice server OK 15s 437ms
/external user directory/external user directory config/empty roles OK 16s 419ms
/external user directory/external user directory config/empty server OK 13s 935ms
/external user directory/external user directory config/invalid role in roles OK 15s 299ms
/external user directory/external user directory config/invalid server OK 15s 420ms
/external user directory/external user directory config/missing roles OK 15s 377ms
/external user directory/external user directory config/missing server OK 13s 979ms
/external user directory/external user directory config/more than one user directory OK 15s 430ms
/external user directory/connection protocols OK 5m 18s
/external user directory/connection protocols/plain text OK 15s 513ms
/external user directory/connection protocols/plain text with custom port OK 15s 385ms
/external user directory/connection protocols/starttls OK 1m 1s
/external user directory/connection protocols/starttls with custom port OK 15s 399ms
/external user directory/connection protocols/tls OK 1m 3s
/external user directory/connection protocols/tls cipher suite XFail 15s 454ms
/external user directory/connection protocols/tls enable tls default yes OK 15s 491ms
/external user directory/connection protocols/tls minimum protocol version OK 1m 22s
/external user directory/connection protocols/tls require cert default demand XFail 16s 478ms
/external user directory/connection protocols/tls with custom port OK 16s 521ms
/external user directory/user authentications OK 7m 5s
/external user directory/user authentications/empty username and empty password OK 52ms
/external user directory/user authentications/invalid long username and valid short password OK 85ms
/external user directory/user authentications/invalid username and valid password OK 78ms
/external user directory/user authentications/login after clickhouse server is restarted OK 10s 149ms
/external user directory/user authentications/login after ldap server is restarted OK 804ms
/external user directory/user authentications/login after user cn changed in ldap OK 134ms
/external user directory/user authentications/login after user is added to ldap OK 92ms
/external user directory/user authentications/login after user is deleted from ldap OK 146ms
/external user directory/user authentications/login after user password changed in ldap OK 182ms
/external user directory/user authentications/parallel login OK 13s 145ms
/external user directory/user authentications/parallel login with rbac users OK 12s 970ms
/external user directory/user authentications/parallel login with the same user OK 13s 66ms
/external user directory/user authentications/user lookup priority OK 305ms
/external user directory/user authentications/valid ascii username and utf8 password OK 87ms
/external user directory/user authentications/valid long username and short password OK 105ms
/external user directory/user authentications/valid short username and invalid long password OK 84ms
/external user directory/user authentications/valid short username and long password OK 89ms
/external user directory/user authentications/valid username and invalid empty password OK 68ms
/external user directory/user authentications/valid username and invalid password OK 71ms
/external user directory/user authentications/valid username and password OK 90ms
/external user directory/user authentications/valid username and password invalid server OK 29ms
/external user directory/user authentications/valid username with valid empty password OK 75ms
/external user directory/user authentications/valid utf8 username and ascii password OK 109ms
/external user directory/user authentications/login after ldap external user directory is removed OK 10s 977ms
/external user directory/user authentications/parallel login with multiple servers OK 41s 112ms
/external user directory/user authentications/parallel login with rbac and multiple servers OK 53s 641ms
/external user directory/user authentications/parallel login with the same user multiple servers OK 25s 659ms
/external user directory/user authentications/default verification cooldown value OK 15s 542ms
/external user directory/user authentications/verification cooldown reset when invalid password is provided OK 15s 646ms
/external user directory/user authentications/valid verification cooldown value cn change OK 17s 573ms
/external user directory/user authentications/valid verification cooldown value ldap unavailable OK 16s 241ms
/external user directory/user authentications/valid verification cooldown value password change OK 17s 626ms
/external user directory/user authentications/verification cooldown performance OK 1m 12s
/external user directory/user authentications/verification cooldown reset on server auth dn prefix parameter change OK 16s 964ms
/external user directory/user authentications/verification cooldown reset on server auth dn suffix parameter change OK 18s 41ms
/external user directory/user authentications/verification cooldown reset on server host parameter change OK 17s 964ms
/external user directory/user authentications/verification cooldown reset on server port parameter change OK 18s 4ms
/external user directory/roles OK 1m 14s
/external user directory/roles/add privilege OK 12s 545ms
/external user directory/roles/new role OK 11s 564ms
/external user directory/roles/not present role added XFail 697us
/external user directory/roles/readd privilege by readding role OK 11s 756ms
/external user directory/roles/remove privilege OK 11s 471ms
/external user directory/roles/remove privilege by removing role OK 11s 482ms
/external user directory/roles/remove role OK 11s 443ms

Generated by TestFlows Open-Source Test Framework v2.0.231215.1221232