Copyright 2024, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

External User Directory Test Run Report

DateNov 04, 2024 21:58
Duration24m 7s
Framework TestFlows 2.0.240705.1133204

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/5b67ba5a4febfa774d6373929d551c6a697d7d92/regression/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/24.3/5b67ba5a4febfa774d6373929d551c6a697d7d92/package_release/clickhouse-common-static_24.3.12.76.altinitystable_amd64.deb
version24.3.12.76.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashc5e1513a2214ee33696c29717935e0a94989ac2a
job.id11672696143
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/11672696143
archx86_64
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/24.3/5b67ba5a4febfa774d6373929d551c6a697d7d92/package_release/clickhouse-common-static_24.3.12.76.altinitystable_amd64.deb
keeper_binary_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerFalse
reuse_envFalse

Summary

92.8%OK
7.2%Known

Statistics

Units OK Fail XFail
Features
7
7
Scenarios
83
79
4
Examples
21
17
4
Steps
6396
6375
12
9

Known Fails

Test NameResultMessage
/external user directory/restart/parallel loginXFail 2ms
fails on 23.3
None
/external user directory/connection protocols/tls cipher suiteXFail 17s 516ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 130, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 124, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 266, in tls_cipher_suite
    login(servers, "openldap4", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 1079, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 1079 in 'query'

1071\|                  assert message in r.output, error(r.output)
1072\|  
1073\|          if not ignore_exception:
1074\|              if message is None or "Exception:" not in message:
1075\|                  with Then("check if output has exception") if steps else NullStep():
1076\|                      if "Exception:" in r.output:
1077\|                          if raise_on_exception:
1078\|                              raise QueryRuntimeException(r.output)
1079\|>                         assert False, error(r.output)
1080\|  
1081\|          return r
1082\|
/external user directory/connection protocols/tls require cert default demandXFail 16s 502ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 130, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/argparser.py", line 145, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 124, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 219, in tls_require_cert_default_demand
    login(servers, "openldap2", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 1079, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 1079 in 'query'

1071\|                  assert message in r.output, error(r.output)
1072\|  
1073\|          if not ignore_exception:
1074\|              if message is None or "Exception:" not in message:
1075\|                  with Then("check if output has exception") if steps else NullStep():
1076\|                      if "Exception:" in r.output:
1077\|                          if raise_on_exception:
1078\|                              raise QueryRuntimeException(r.output)
1079\|>                         assert False, error(r.output)
1080\|  
1081\|          return r
1082\|
/external user directory/roles/not present role addedXFail 1ms
roles are not being applied to active clickhouse-client connections in 23.8 
None

Results

Test Name Result Duration
/external user directory OK 24m 7s
/external user directory/sanity OK 368ms
/external user directory/simple OK 16s 407ms
/external user directory/restart OK 1m 7s
/external user directory/restart/dynamically added users OK 24s 232ms
/external user directory/restart/multiple external user directories OK 21s 911ms
/external user directory/restart/one external user directory OK 21s 844ms
/external user directory/restart/parallel login XFail 2ms
/external user directory/server config OK 2m 37s
/external user directory/server config/auth dn value OK 16s 330ms
/external user directory/server config/empty ca cert dir OK 16s 363ms
/external user directory/server config/empty ca cert file OK 16s 357ms
/external user directory/server config/empty host OK 2s 83ms
/external user directory/server config/empty server name OK 2s 88ms
/external user directory/server config/invalid auth dn prefix OK 15s 381ms
/external user directory/server config/invalid auth dn suffix OK 16s 445ms
/external user directory/server config/invalid enable tls value OK 2s 84ms
/external user directory/server config/invalid host OK 16s 353ms
/external user directory/server config/invalid port OK 16s 407ms
/external user directory/server config/invalid tls require cert value OK 16s 449ms
/external user directory/server config/invalid verification cooldown value OK 12s 533ms
/external user directory/server config/missing host OK 3s 660ms
/external user directory/server config/syntax OK 4s 662ms
/external user directory/external user directory config OK 2m 23s
/external user directory/external user directory config/defined twice roles OK 15s 988ms
/external user directory/external user directory config/defined twice server OK 15s 564ms
/external user directory/external user directory config/empty roles OK 16s 537ms
/external user directory/external user directory config/empty server OK 14s 844ms
/external user directory/external user directory config/invalid role in roles OK 16s 224ms
/external user directory/external user directory config/invalid server OK 16s 332ms
/external user directory/external user directory config/missing roles OK 16s 322ms
/external user directory/external user directory config/missing server OK 14s 860ms
/external user directory/external user directory config/more than one user directory OK 16s 547ms
/external user directory/connection protocols OK 5m 32s
/external user directory/connection protocols/plain text OK 16s 374ms
/external user directory/connection protocols/plain text with custom port OK 16s 337ms
/external user directory/connection protocols/starttls OK 1m 5s
/external user directory/connection protocols/starttls with custom port OK 16s 442ms
/external user directory/connection protocols/tls OK 1m 6s
/external user directory/connection protocols/tls cipher suite XFail 17s 516ms
/external user directory/connection protocols/tls enable tls default yes OK 16s 575ms
/external user directory/connection protocols/tls minimum protocol version OK 1m 24s
/external user directory/connection protocols/tls require cert default demand XFail 16s 502ms
/external user directory/connection protocols/tls with custom port OK 16s 597ms
/external user directory/user authentications OK 9m 5s
/external user directory/user authentications/empty username and empty password OK 97ms
/external user directory/user authentications/invalid long username and valid short password OK 133ms
/external user directory/user authentications/invalid username and valid password OK 130ms
/external user directory/user authentications/login after clickhouse server is restarted OK 10s 821ms
/external user directory/user authentications/login after ldap server is restarted OK 1s 202ms
/external user directory/user authentications/login after user cn changed in ldap OK 235ms
/external user directory/user authentications/login after user is added to ldap OK 156ms
/external user directory/user authentications/login after user is deleted from ldap OK 218ms
/external user directory/user authentications/login after user password changed in ldap OK 313ms
/external user directory/user authentications/parallel login OK 21s 478ms
/external user directory/user authentications/parallel login with rbac users OK 21s 452ms
/external user directory/user authentications/parallel login with the same user OK 20s 519ms
/external user directory/user authentications/user lookup priority OK 465ms
/external user directory/user authentications/valid ascii username and utf8 password OK 153ms
/external user directory/user authentications/valid long username and short password OK 133ms
/external user directory/user authentications/valid short username and invalid long password OK 127ms
/external user directory/user authentications/valid short username and long password OK 145ms
/external user directory/user authentications/valid username and invalid empty password OK 121ms
/external user directory/user authentications/valid username and invalid password OK 112ms
/external user directory/user authentications/valid username and password OK 152ms
/external user directory/user authentications/valid username and password invalid server OK 43ms
/external user directory/user authentications/valid username with valid empty password OK 111ms
/external user directory/user authentications/valid utf8 username and ascii password OK 149ms
/external user directory/user authentications/login after ldap external user directory is removed OK 11s 555ms
/external user directory/user authentications/parallel login with multiple servers OK 58s 88ms
/external user directory/user authentications/parallel login with rbac and multiple servers OK 1m 21s
/external user directory/user authentications/parallel login with the same user multiple servers OK 36s 43ms
/external user directory/user authentications/default verification cooldown value OK 16s 607ms
/external user directory/user authentications/verification cooldown reset when invalid password is provided OK 16s 674ms
/external user directory/user authentications/valid verification cooldown value cn change OK 18s 705ms
/external user directory/user authentications/valid verification cooldown value ldap unavailable OK 17s 642ms
/external user directory/user authentications/valid verification cooldown value password change OK 17s 699ms
/external user directory/user authentications/verification cooldown performance OK 1m 38s
/external user directory/user authentications/verification cooldown reset on server auth dn prefix parameter change OK 19s 309ms
/external user directory/user authentications/verification cooldown reset on server auth dn suffix parameter change OK 19s 358ms
/external user directory/user authentications/verification cooldown reset on server host parameter change OK 19s 311ms
/external user directory/user authentications/verification cooldown reset on server port parameter change OK 19s 479ms
/external user directory/roles OK 1m 21s
/external user directory/roles/add privilege OK 13s 610ms
/external user directory/roles/new role OK 12s 732ms
/external user directory/roles/not present role added XFail 1ms
/external user directory/roles/readd privilege by readding role OK 12s 937ms
/external user directory/roles/remove privilege OK 12s 625ms
/external user directory/roles/remove privilege by removing role OK 12s 552ms
/external user directory/roles/remove role OK 12s 360ms

Generated by TestFlows Open-Source Test Framework v2.0.240705.1133204