Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Authentication Test Run Report

DateApr 13, 2023 16:33
Duration26m 16s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/22.8.15.25.altinitystable/4688213790/testflows/

Attributes

projectAltinity/ClickHouse
project.id
packagehttps://s3.amazonaws.com/altinity-build-artifacts/22.8/a454ebb2761c2732ab39211459f19fa026dc8527/package_release/clickhouse-common-static_22.8.15.25.altinitystable_amd64.deb
version22.8.15.25.altinitystable
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash4023e0eafd6319edc79b82d79ab800c0c3cf20ec
job.id4688213790
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/4688213790
archx86_64
stressFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/22.8/a454ebb2761c2732ab39211459f19fa026dc8527/package_release/clickhouse-common-static_22.8.15.25.altinitystable_amd64.deb
collect_service_logsTrue

Summary

95.4%OK
4.6%Known

Statistics

Units OK Fail XFail
Suites
7
7
Features
7
7
Scenarios
92
90
2
Examples
25
21
4
Steps
6260
6248
12

Known Fails

Test NameResultMessage
/authentication/connection protocols/tls cipher suiteXFail 16s 778ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 89, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 82, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 242, in tls_cipher_suite
    login(servers, *users)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 432, in login
    current().context.node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 766, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect or there is no user with such name. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 766 in 'query'

758\|                  assert message in r.output, error(r.output)
759\|  
760\|          if not ignore_exception:
761\|              if message is None or "Exception:" not in message:
762\|                  with Then("check if output has exception") if steps else NullStep():
763\|                      if "Exception:" in r.output:
764\|                          if raise_on_exception:
765\|                              raise QueryRuntimeException(r.output)
766\|>                         assert False, error(r.output)
767\|  
768\|          return r
769\|
/authentication/connection protocols/tls require cert default demandXFail 16s 858ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 89, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 82, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 199, in tls_require_cert_default_demand
    login(servers, *users)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 432, in login
    current().context.node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 766, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 766 in 'query'

758\|                  assert message in r.output, error(r.output)
759\|  
760\|          if not ignore_exception:
761\|              if message is None or "Exception:" not in message:
762\|                  with Then("check if output has exception") if steps else NullStep():
763\|                      if "Exception:" in r.output:
764\|                          if raise_on_exception:
765\|                              raise QueryRuntimeException(r.output)
766\|>                         assert False, error(r.output)
767\|  
768\|          return r
769\|

Results

Test Name Result Duration
/authentication OK 26m 16s
/authentication/sanity OK 798ms
/authentication/multiple servers OK 15s 905ms
/authentication/connection protocols OK 5m 28s
/authentication/connection protocols/plain text OK 15s 797ms
/authentication/connection protocols/plain text with custom port OK 15s 876ms
/authentication/connection protocols/starttls OK 1m 5s
/authentication/connection protocols/starttls with custom port OK 16s 771ms
/authentication/connection protocols/tls OK 1m 6s
/authentication/connection protocols/tls cipher suite XFail 16s 778ms
/authentication/connection protocols/tls enable tls default yes OK 16s 808ms
/authentication/connection protocols/tls minimum protocol version OK 1m 21s
/authentication/connection protocols/tls require cert default demand XFail 16s 858ms
/authentication/connection protocols/tls with custom port OK 16s 833ms
/authentication/server config OK 3m 19s
/authentication/server config/auth dn value OK 16s 866ms
/authentication/server config/empty ca cert dir OK 15s 834ms
/authentication/server config/empty ca cert file OK 16s 890ms
/authentication/server config/empty host OK 16s 869ms
/authentication/server config/empty server name OK 2s 96ms
/authentication/server config/invalid auth dn prefix OK 15s 863ms
/authentication/server config/invalid auth dn suffix OK 16s 873ms
/authentication/server config/invalid enable tls value OK 15s 912ms
/authentication/server config/invalid host OK 16s 893ms
/authentication/server config/invalid port OK 15s 883ms
/authentication/server config/invalid tls require cert value OK 16s 866ms
/authentication/server config/invalid verification cooldown value OK 12s 509ms
/authentication/server config/missing host OK 15s 872ms
/authentication/server config/syntax OK 4s 411ms
/authentication/user config OK 1m 6s
/authentication/user config/empty server name OK 5s 508ms
/authentication/user config/empty server not defined OK 16s 926ms
/authentication/user config/empty user name OK 5s 513ms
/authentication/user config/ldap and password OK 6s 554ms
/authentication/user config/multiple servers OK 15s 926ms
/authentication/user config/valid user config OK 15s 882ms
/authentication/user authentications OK 14m 23s
/authentication/user authentications/rbac=False OK 9m 11s
/authentication/user authentications/rbac=False/add user to ldap and login OK 12s 624ms
/authentication/user authentications/rbac=False/empty username and empty password OK 72ms
/authentication/user authentications/rbac=False/invalid long username and valid short password OK 12s 591ms
/authentication/user authentications/rbac=False/invalid username and valid password OK 12s 587ms
/authentication/user authentications/rbac=False/login after clickhouse server is restarted OK 24s 919ms
/authentication/user authentications/rbac=False/login after ldap server is restarted OK 15s 296ms
/authentication/user authentications/rbac=False/login after user cn changed in ldap OK 13s 581ms
/authentication/user authentications/rbac=False/login after user is deleted from ldap OK 12s 614ms
/authentication/user authentications/rbac=False/login after user password changed in ldap OK 12s 637ms
/authentication/user authentications/rbac=False/parallel login OK 27s 357ms
/authentication/user authentications/rbac=False/valid ascii username and utf8 password OK 12s 502ms
/authentication/user authentications/rbac=False/valid long username and short password OK 12s 470ms
/authentication/user authentications/rbac=False/valid short username and invalid long password OK 12s 472ms
/authentication/user authentications/rbac=False/valid short username and long password OK 12s 536ms
/authentication/user authentications/rbac=False/valid username and invalid empty password OK 12s 448ms
/authentication/user authentications/rbac=False/valid username and invalid password OK 12s 463ms
/authentication/user authentications/rbac=False/valid username and password OK 12s 533ms
/authentication/user authentications/rbac=False/valid username and password invalid server OK 12s 398ms
/authentication/user authentications/rbac=False/valid username with valid empty password OK 12s 457ms
/authentication/user authentications/rbac=False/valid utf8 username and ascii password OK 12s 500ms
/authentication/user authentications/rbac=False/verification cooldown OK 4m 39s
/authentication/user authentications/rbac=False/verification cooldown/default verification cooldown value OK 15s 997ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is provided OK 16s 153ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value cn change OK 16s 15ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value ldap unavailable OK 18s 506ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value password change OK 18s 185ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown performance OK 1m 30s
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 25s 957ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 25s 962ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server host parameter change OK 25s 944ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server port parameter change OK 26s 53ms
/authentication/user authentications/rbac=True OK 5m 12s
/authentication/user authentications/rbac=True/add user to ldap and login OK 281ms
/authentication/user authentications/rbac=True/empty username and empty password OK 67ms
/authentication/user authentications/rbac=True/invalid long username and valid short password OK 258ms
/authentication/user authentications/rbac=True/invalid username and valid password OK 249ms
/authentication/user authentications/rbac=True/login after clickhouse server is restarted OK 12s 275ms
/authentication/user authentications/rbac=True/login after ldap server is restarted OK 2s 123ms
/authentication/user authentications/rbac=True/login after user cn changed in ldap OK 339ms
/authentication/user authentications/rbac=True/login after user is deleted from ldap OK 323ms
/authentication/user authentications/rbac=True/login after user password changed in ldap OK 395ms
/authentication/user authentications/rbac=True/parallel login OK 15s 701ms
/authentication/user authentications/rbac=True/valid ascii username and utf8 password OK 252ms
/authentication/user authentications/rbac=True/valid long username and short password OK 250ms
/authentication/user authentications/rbac=True/valid short username and invalid long password OK 250ms
/authentication/user authentications/rbac=True/valid short username and long password OK 245ms
/authentication/user authentications/rbac=True/valid username and invalid empty password OK 237ms
/authentication/user authentications/rbac=True/valid username and invalid password OK 236ms
/authentication/user authentications/rbac=True/valid username and password OK 235ms
/authentication/user authentications/rbac=True/valid username and password invalid server OK 175ms
/authentication/user authentications/rbac=True/valid username with valid empty password OK 240ms
/authentication/user authentications/rbac=True/valid utf8 username and ascii password OK 247ms
/authentication/user authentications/rbac=True/verification cooldown OK 4m 33s
/authentication/user authentications/rbac=True/verification cooldown/default verification cooldown value OK 15s 978ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is provided OK 16s 63ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value cn change OK 15s 944ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value ldap unavailable OK 18s 342ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value password change OK 18s 41ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown performance OK 1m 25s
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 25s 858ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 26s 7ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server host parameter change OK 25s 923ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server port parameter change OK 25s 892ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122