Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

External User Directory Test Run Report

DateOct 05, 2023 14:02
Duration29m 16s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/23.3.13.7.altinitystable/6416627213/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/23.3/f040635d5f373c5e4be08e5fbec2622a9889adde/package_aarch64/clickhouse-common-static_23.3.13.7.altinitystable_arm64.deb
version23.3.13.7.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash19e8624c5e4ccc65b128d27b19836c0570e53991
job.id6416627213
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/6416627213
archaarch64
stressFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/23.3/f040635d5f373c5e4be08e5fbec2622a9889adde/package_aarch64/clickhouse-common-static_23.3.13.7.altinitystable_arm64.deb
collect_service_logsTrue

Summary

94.1%OK
5.9%Known

Statistics

Units OK Fail XFail
Suites
7
7
Features
7
7
Scenarios
83
80
3
Examples
21
17
4
Steps
6161
6149
12

Known Fails

Test NameResultMessage
/external user directory/restart/parallel loginXFail 1ms
fails on 23.3
None
/external user directory/connection protocols/tls cipher suiteXFail 16s 557ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 126, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 118, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 266, in tls_cipher_suite
    login(servers, "openldap4", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 780, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 780 in 'query'

772\|                  assert message in r.output, error(r.output)
773\|  
774\|          if not ignore_exception:
775\|              if message is None or "Exception:" not in message:
776\|                  with Then("check if output has exception") if steps else NullStep():
777\|                      if "Exception:" in r.output:
778\|                          if raise_on_exception:
779\|                              raise QueryRuntimeException(r.output)
780\|>                         assert False, error(r.output)
781\|  
782\|          return r
783\|
/external user directory/connection protocols/tls require cert default demandXFail 16s 590ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 126, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 118, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 219, in tls_require_cert_default_demand
    login(servers, "openldap2", *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 780, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 780 in 'query'

772\|                  assert message in r.output, error(r.output)
773\|  
774\|          if not ignore_exception:
775\|              if message is None or "Exception:" not in message:
776\|                  with Then("check if output has exception") if steps else NullStep():
777\|                      if "Exception:" in r.output:
778\|                          if raise_on_exception:
779\|                              raise QueryRuntimeException(r.output)
780\|>                         assert False, error(r.output)
781\|  
782\|          return r
783\|

Results

Test Name Result Duration
/external user directory OK 29m 16s
/external user directory/sanity OK 808ms
/external user directory/simple OK 18s 531ms
/external user directory/restart OK 1m 16s
/external user directory/restart/dynamically added users OK 26s 883ms
/external user directory/restart/multiple external user directories OK 25s 741ms
/external user directory/restart/one external user directory OK 24s 104ms
/external user directory/restart/parallel login XFail 1ms
/external user directory/server config OK 2m 47s
/external user directory/server config/auth dn value OK 17s 878ms
/external user directory/server config/empty ca cert dir OK 17s 944ms
/external user directory/server config/empty ca cert file OK 17s 763ms
/external user directory/server config/empty host OK 2s 98ms
/external user directory/server config/empty server name OK 2s 104ms
/external user directory/server config/invalid auth dn prefix OK 16s 867ms
/external user directory/server config/invalid auth dn suffix OK 18s 618ms
/external user directory/server config/invalid enable tls value OK 2s 123ms
/external user directory/server config/invalid host OK 18s 695ms
/external user directory/server config/invalid port OK 17s 557ms
/external user directory/server config/invalid tls require cert value OK 16s 571ms
/external user directory/server config/invalid verification cooldown value OK 11s 579ms
/external user directory/server config/missing host OK 4s 511ms
/external user directory/server config/syntax OK 3s 556ms
/external user directory/external user directory config OK 2m 36s
/external user directory/external user directory config/defined twice roles OK 19s 590ms
/external user directory/external user directory config/defined twice server OK 17s 641ms
/external user directory/external user directory config/empty roles OK 16s 725ms
/external user directory/external user directory config/empty server OK 17s 92ms
/external user directory/external user directory config/invalid role in roles OK 17s 606ms
/external user directory/external user directory config/invalid server OK 16s 471ms
/external user directory/external user directory config/missing roles OK 15s 458ms
/external user directory/external user directory config/missing server OK 17s 232ms
/external user directory/external user directory config/more than one user directory OK 18s 723ms
/external user directory/connection protocols OK 5m 45s
/external user directory/connection protocols/plain text OK 18s 434ms
/external user directory/connection protocols/plain text with custom port OK 17s 409ms
/external user directory/connection protocols/starttls OK 1m 7s
/external user directory/connection protocols/starttls with custom port OK 17s 604ms
/external user directory/connection protocols/tls OK 1m 10s
/external user directory/connection protocols/tls cipher suite XFail 16s 557ms
/external user directory/connection protocols/tls enable tls default yes OK 17s 662ms
/external user directory/connection protocols/tls minimum protocol version OK 1m 26s
/external user directory/connection protocols/tls require cert default demand XFail 16s 590ms
/external user directory/connection protocols/tls with custom port OK 16s 641ms
/external user directory/user authentications OK 13m 2s
/external user directory/user authentications/empty username and empty password OK 123ms
/external user directory/user authentications/invalid long username and valid short password OK 231ms
/external user directory/user authentications/invalid username and valid password OK 229ms
/external user directory/user authentications/login after clickhouse server is restarted OK 12s 39ms
/external user directory/user authentications/login after ldap server is restarted OK 2s 383ms
/external user directory/user authentications/login after user cn changed in ldap OK 312ms
/external user directory/user authentications/login after user is added to ldap OK 158ms
/external user directory/user authentications/login after user is deleted from ldap OK 277ms
/external user directory/user authentications/login after user password changed in ldap OK 406ms
/external user directory/user authentications/parallel login OK 43s 119ms
/external user directory/user authentications/parallel login with rbac users OK 43s 765ms
/external user directory/user authentications/parallel login with the same user OK 41s 360ms
/external user directory/user authentications/user lookup priority OK 683ms
/external user directory/user authentications/valid ascii username and utf8 password OK 163ms
/external user directory/user authentications/valid long username and short password OK 160ms
/external user directory/user authentications/valid short username and invalid long password OK 208ms
/external user directory/user authentications/valid short username and long password OK 187ms
/external user directory/user authentications/valid username and invalid empty password OK 200ms
/external user directory/user authentications/valid username and invalid password OK 188ms
/external user directory/user authentications/valid username and password OK 159ms
/external user directory/user authentications/valid username and password invalid server OK 113ms
/external user directory/user authentications/valid username with valid empty password OK 173ms
/external user directory/user authentications/valid utf8 username and ascii password OK 191ms
/external user directory/user authentications/login after ldap external user directory is removed OK 13s 927ms
/external user directory/user authentications/parallel login with multiple servers OK 1m 45s
/external user directory/user authentications/parallel login with rbac and multiple servers OK 2m 24s
/external user directory/user authentications/parallel login with the same user multiple servers OK 58s 630ms
/external user directory/user authentications/default verification cooldown value OK 18s 788ms
/external user directory/user authentications/verification cooldown reset when invalid password is provided OK 19s 971ms
/external user directory/user authentications/valid verification cooldown value cn change OK 21s 57ms
/external user directory/user authentications/valid verification cooldown value ldap unavailable OK 20s 332ms
/external user directory/user authentications/valid verification cooldown value password change OK 19s 935ms
/external user directory/user authentications/verification cooldown performance OK 1m 57s
/external user directory/user authentications/verification cooldown reset on server auth dn prefix parameter change OK 19s 69ms
/external user directory/user authentications/verification cooldown reset on server auth dn suffix parameter change OK 20s 3ms
/external user directory/user authentications/verification cooldown reset on server host parameter change OK 20s 11ms
/external user directory/user authentications/verification cooldown reset on server port parameter change OK 19s 110ms
/external user directory/roles OK 1m 57s
/external user directory/roles/add privilege OK 15s 96ms
/external user directory/roles/new role OK 16s 286ms
/external user directory/roles/not present role added OK 15s 788ms
/external user directory/roles/readd privilege by readding role OK 16s 564ms
/external user directory/roles/remove privilege OK 16s 13ms
/external user directory/roles/remove privilege by removing role OK 15s 945ms
/external user directory/roles/remove role OK 15s 809ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122