Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

External User Directory Test Run Report

DateMay 25, 2023 16:44
Duration28m 7s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/22.8.15.25.altinityfips/5080932159/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/22.8/ef5580902f444e6a5d617e7c7a6e8d7a1f5fc9fb/package_release/clickhouse-common-static_22.8.15.25.altinityfips_amd64.deb
version22.8.15.25.altinityfips
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash63a15b5dfc55badefcf4b869296e3ec99ca08141
job.id5080932159
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/5080932159
archx86_64
stressFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/22.8/ef5580902f444e6a5d617e7c7a6e8d7a1f5fc9fb/package_release/clickhouse-common-static_22.8.15.25.altinityfips_amd64.deb
collect_service_logsTrue

Summary

94.9%OK
5.1%Known

Statistics

Units OK Fail XFail
Suites
7
7
Features
7
7
Scenarios
83
81
2
Examples
21
17
4
Steps
6413
6401
12

Known Fails

Test NameResultMessage
/external user directory/connection protocols/tls cipher suiteXFail 16s 6ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 111, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 103, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 266, in tls_cipher_suite
    login(servers, "openldap4", *users)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 760, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect or there is no user with such name. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 760 in 'query'

752\|                  assert message in r.output, error(r.output)
753\|  
754\|          if not ignore_exception:
755\|              if message is None or "Exception:" not in message:
756\|                  with Then("check if output has exception") if steps else NullStep():
757\|                      if "Exception:" in r.output:
758\|                          if raise_on_exception:
759\|                              raise QueryRuntimeException(r.output)
760\|>                         assert False, error(r.output)
761\|  
762\|          return r
763\|
/external user directory/connection protocols/tls require cert default demandXFail 16s 19ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 111, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/regression.py", line 103, in regression
    Feature(run=load("ldap.external_user_directory.tests.connections", "feature"))
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 323, in feature
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/connections.py", line 219, in tls_require_cert_default_demand
    login(servers, "openldap2", *users)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../ldap/external_user_directory/tests/common.py", line 269, in login
    current().context.node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py", line 760, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/external_user_directory/../../helpers/cluster.py', line 760 in 'query'

752\|                  assert message in r.output, error(r.output)
753\|  
754\|          if not ignore_exception:
755\|              if message is None or "Exception:" not in message:
756\|                  with Then("check if output has exception") if steps else NullStep():
757\|                      if "Exception:" in r.output:
758\|                          if raise_on_exception:
759\|                              raise QueryRuntimeException(r.output)
760\|>                         assert False, error(r.output)
761\|  
762\|          return r
763\|

Results

Test Name Result Duration
/external user directory OK 28m 7s
/external user directory/sanity OK 826ms
/external user directory/simple OK 18s 113ms
/external user directory/restart OK 2m 31s
/external user directory/restart/dynamically added users OK 24s 379ms
/external user directory/restart/multiple external user directories OK 23s 167ms
/external user directory/restart/one external user directory OK 23s 47ms
/external user directory/restart/parallel login OK 1m 20s
/external user directory/server config OK 2m 34s
/external user directory/server config/auth dn value OK 15s 891ms
/external user directory/server config/empty ca cert dir OK 15s 968ms
/external user directory/server config/empty ca cert file OK 16s 9ms
/external user directory/server config/empty host OK 2s 77ms
/external user directory/server config/empty server name OK 2s 86ms
/external user directory/server config/invalid auth dn prefix OK 15s 906ms
/external user directory/server config/invalid auth dn suffix OK 15s 946ms
/external user directory/server config/invalid enable tls value OK 2s 84ms
/external user directory/server config/invalid host OK 16s 26ms
/external user directory/server config/invalid port OK 15s 907ms
/external user directory/server config/invalid tls require cert value OK 15s 998ms
/external user directory/server config/invalid verification cooldown value OK 11s 509ms
/external user directory/server config/missing host OK 4s 463ms
/external user directory/server config/syntax OK 4s 408ms
/external user directory/external user directory config OK 2m 27s
/external user directory/external user directory config/defined twice roles OK 16s 716ms
/external user directory/external user directory config/defined twice server OK 16s 49ms
/external user directory/external user directory config/empty roles OK 16s 14ms
/external user directory/external user directory config/empty server OK 16s 999ms
/external user directory/external user directory config/invalid role in roles OK 16s 914ms
/external user directory/external user directory config/invalid server OK 16s 28ms
/external user directory/external user directory config/missing roles OK 15s 900ms
/external user directory/external user directory config/missing server OK 16s 800ms
/external user directory/external user directory config/more than one user directory OK 16s 46ms
/external user directory/connection protocols OK 5m 20s
/external user directory/connection protocols/plain text OK 16s 13ms
/external user directory/connection protocols/plain text with custom port OK 15s 903ms
/external user directory/connection protocols/starttls OK 1m 4s
/external user directory/connection protocols/starttls with custom port OK 16s 43ms
/external user directory/connection protocols/tls OK 1m 3s
/external user directory/connection protocols/tls cipher suite XFail 16s 6ms
/external user directory/connection protocols/tls enable tls default yes OK 16s 2ms
/external user directory/connection protocols/tls minimum protocol version OK 1m 20s
/external user directory/connection protocols/tls require cert default demand XFail 16s 19ms
/external user directory/connection protocols/tls with custom port OK 15s 988ms
/external user directory/user authentications OK 9m 43s
/external user directory/user authentications/empty username and empty password OK 73ms
/external user directory/user authentications/invalid long username and valid short password OK 165ms
/external user directory/user authentications/invalid username and valid password OK 141ms
/external user directory/user authentications/login after clickhouse server is restarted OK 12s 195ms
/external user directory/user authentications/login after ldap server is restarted OK 2s 94ms
/external user directory/user authentications/login after user cn changed in ldap OK 232ms
/external user directory/user authentications/login after user is added to ldap OK 128ms
/external user directory/user authentications/login after user is deleted from ldap OK 215ms
/external user directory/user authentications/login after user password changed in ldap OK 287ms
/external user directory/user authentications/parallel login OK 26s 958ms
/external user directory/user authentications/parallel login with rbac users OK 27s 438ms
/external user directory/user authentications/parallel login with the same user OK 26s 602ms
/external user directory/user authentications/user lookup priority OK 478ms
/external user directory/user authentications/valid ascii username and utf8 password OK 130ms
/external user directory/user authentications/valid long username and short password OK 125ms
/external user directory/user authentications/valid short username and invalid long password OK 123ms
/external user directory/user authentications/valid short username and long password OK 128ms
/external user directory/user authentications/valid username and invalid empty password OK 124ms
/external user directory/user authentications/valid username and invalid password OK 135ms
/external user directory/user authentications/valid username and password OK 128ms
/external user directory/user authentications/valid username and password invalid server OK 60ms
/external user directory/user authentications/valid username with valid empty password OK 130ms
/external user directory/user authentications/valid utf8 username and ascii password OK 129ms
/external user directory/user authentications/login after ldap external user directory is removed OK 12s 521ms
/external user directory/user authentications/parallel login with multiple servers OK 1m 8s
/external user directory/user authentications/parallel login with rbac and multiple servers OK 1m 34s
/external user directory/user authentications/parallel login with the same user multiple servers OK 42s 178ms
/external user directory/user authentications/default verification cooldown value OK 16s 189ms
/external user directory/user authentications/verification cooldown reset when invalid password is provided OK 16s 306ms
/external user directory/user authentications/valid verification cooldown value cn change OK 18s 253ms
/external user directory/user authentications/valid verification cooldown value ldap unavailable OK 18s 580ms
/external user directory/user authentications/valid verification cooldown value password change OK 20s 324ms
/external user directory/user authentications/verification cooldown performance OK 1m 29s
/external user directory/user authentications/verification cooldown reset on server auth dn prefix parameter change OK 17s 546ms
/external user directory/user authentications/verification cooldown reset on server auth dn suffix parameter change OK 18s 463ms
/external user directory/user authentications/verification cooldown reset on server host parameter change OK 18s 520ms
/external user directory/user authentications/verification cooldown reset on server port parameter change OK 18s 646ms
/external user directory/roles OK 1m 41s
/external user directory/roles/add privilege OK 14s 364ms
/external user directory/roles/new role OK 12s 239ms
/external user directory/roles/not present role added OK 14s 416ms
/external user directory/roles/readd privilege by readding role OK 15s 655ms
/external user directory/roles/remove privilege OK 13s 306ms
/external user directory/roles/remove privilege by removing role OK 14s 355ms
/external user directory/roles/remove role OK 13s 87ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122