Copyright 2023, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Authentication Test Run Report

DateMay 25, 2023 16:12
Duration28m 3s
Framework TestFlows 1.9.230315.1003122

Artifacts

Test artifacts can be found at https://altinity-test-reports.s3.amazonaws.com/index.html#clickhouse/22.8.15.25.altinityfips/5080932159/testflows/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/22.8/ef5580902f444e6a5d617e7c7a6e8d7a1f5fc9fb/package_release/clickhouse-common-static_22.8.15.25.altinityfips_amd64.deb
version22.8.15.25.altinityfips
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash63a15b5dfc55badefcf4b869296e3ec99ca08141
job.id5080932159
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/5080932159
archx86_64
stressFalse
localTrue
clickhouse_versionNone
clickhouse_binary_pathhttps://s3.amazonaws.com/altinity-build-artifacts/22.8/ef5580902f444e6a5d617e7c7a6e8d7a1f5fc9fb/package_release/clickhouse-common-static_22.8.15.25.altinityfips_amd64.deb
collect_service_logsTrue

Summary

95.4%OK
4.6%Known

Statistics

Units OK Fail XFail
Suites
7
7
Features
7
7
Scenarios
92
90
2
Examples
25
21
4
Steps
6260
6248
12

Known Fails

Test NameResultMessage
/authentication/connection protocols/tls cipher suiteXFail 15s 849ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 89, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 82, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 242, in tls_cipher_suite
    login(servers, *users)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 432, in login
    current().context.node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 760, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect or there is no user with such name. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 760 in 'query'

752\|                  assert message in r.output, error(r.output)
753\|  
754\|          if not ignore_exception:
755\|              if message is None or "Exception:" not in message:
756\|                  with Then("check if output has exception") if steps else NullStep():
757\|                      if "Exception:" in r.output:
758\|                          if raise_on_exception:
759\|                              raise QueryRuntimeException(r.output)
760\|>                         assert False, error(r.output)
761\|  
762\|          return r
763\|
/authentication/connection protocols/tls require cert default demandXFail 15s 836ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 89, in 
    regression()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 82, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 199, in tls_require_cert_default_demand
    login(servers, *users)
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 432, in login
    current().context.node.query(
  File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 760, in query
    assert False, error(r.output)
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 760 in 'query'

752\|                  assert message in r.output, error(r.output)
753\|  
754\|          if not ignore_exception:
755\|              if message is None or "Exception:" not in message:
756\|                  with Then("check if output has exception") if steps else NullStep():
757\|                      if "Exception:" in r.output:
758\|                          if raise_on_exception:
759\|                              raise QueryRuntimeException(r.output)
760\|>                         assert False, error(r.output)
761\|  
762\|          return r
763\|

Results

Test Name Result Duration
/authentication OK 28m 3s
/authentication/sanity OK 844ms
/authentication/multiple servers OK 17s 926ms
/authentication/connection protocols OK 5m 26s
/authentication/connection protocols/plain text OK 15s 916ms
/authentication/connection protocols/plain text with custom port OK 15s 841ms
/authentication/connection protocols/starttls OK 1m 4s
/authentication/connection protocols/starttls with custom port OK 16s 784ms
/authentication/connection protocols/tls OK 1m 5s
/authentication/connection protocols/tls cipher suite XFail 15s 849ms
/authentication/connection protocols/tls enable tls default yes OK 15s 819ms
/authentication/connection protocols/tls minimum protocol version OK 1m 23s
/authentication/connection protocols/tls require cert default demand XFail 15s 836ms
/authentication/connection protocols/tls with custom port OK 16s 901ms
/authentication/server config OK 3m 17s
/authentication/server config/auth dn value OK 15s 864ms
/authentication/server config/empty ca cert dir OK 16s 945ms
/authentication/server config/empty ca cert file OK 15s 894ms
/authentication/server config/empty host OK 16s 843ms
/authentication/server config/empty server name OK 2s 89ms
/authentication/server config/invalid auth dn prefix OK 15s 941ms
/authentication/server config/invalid auth dn suffix OK 15s 930ms
/authentication/server config/invalid enable tls value OK 15s 917ms
/authentication/server config/invalid host OK 17s 28ms
/authentication/server config/invalid port OK 15s 908ms
/authentication/server config/invalid tls require cert value OK 15s 908ms
/authentication/server config/invalid verification cooldown value OK 12s 515ms
/authentication/server config/missing host OK 15s 845ms
/authentication/server config/syntax OK 4s 471ms
/authentication/user config OK 1m 6s
/authentication/user config/empty server name OK 5s 528ms
/authentication/user config/empty server not defined OK 16s 984ms
/authentication/user config/empty user name OK 5s 490ms
/authentication/user config/ldap and password OK 6s 519ms
/authentication/user config/multiple servers OK 16s 8ms
/authentication/user config/valid user config OK 15s 876ms
/authentication/user authentications OK 14m 31s
/authentication/user authentications/rbac=False OK 9m 15s
/authentication/user authentications/rbac=False/add user to ldap and login OK 12s 548ms
/authentication/user authentications/rbac=False/empty username and empty password OK 71ms
/authentication/user authentications/rbac=False/invalid long username and valid short password OK 12s 539ms
/authentication/user authentications/rbac=False/invalid username and valid password OK 12s 596ms
/authentication/user authentications/rbac=False/login after clickhouse server is restarted OK 24s 976ms
/authentication/user authentications/rbac=False/login after ldap server is restarted OK 15s 442ms
/authentication/user authentications/rbac=False/login after user cn changed in ldap OK 13s 566ms
/authentication/user authentications/rbac=False/login after user is deleted from ldap OK 12s 610ms
/authentication/user authentications/rbac=False/login after user password changed in ldap OK 12s 728ms
/authentication/user authentications/rbac=False/parallel login OK 29s 824ms
/authentication/user authentications/rbac=False/valid ascii username and utf8 password OK 12s 468ms
/authentication/user authentications/rbac=False/valid long username and short password OK 12s 542ms
/authentication/user authentications/rbac=False/valid short username and invalid long password OK 12s 513ms
/authentication/user authentications/rbac=False/valid short username and long password OK 12s 480ms
/authentication/user authentications/rbac=False/valid username and invalid empty password OK 12s 455ms
/authentication/user authentications/rbac=False/valid username and invalid password OK 12s 570ms
/authentication/user authentications/rbac=False/valid username and password OK 12s 578ms
/authentication/user authentications/rbac=False/valid username and password invalid server OK 12s 481ms
/authentication/user authentications/rbac=False/valid username with valid empty password OK 12s 552ms
/authentication/user authentications/rbac=False/valid utf8 username and ascii password OK 12s 568ms
/authentication/user authentications/rbac=False/verification cooldown OK 4m 40s
/authentication/user authentications/rbac=False/verification cooldown/default verification cooldown value OK 15s 996ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is provided OK 16s 107ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value cn change OK 15s 972ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value ldap unavailable OK 18s 489ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value password change OK 18s 33ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown performance OK 1m 31s
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 26s 37ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 26s 34ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server host parameter change OK 25s 983ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server port parameter change OK 26s 63ms
/authentication/user authentications/rbac=True OK 5m 15s
/authentication/user authentications/rbac=True/add user to ldap and login OK 280ms
/authentication/user authentications/rbac=True/empty username and empty password OK 75ms
/authentication/user authentications/rbac=True/invalid long username and valid short password OK 282ms
/authentication/user authentications/rbac=True/invalid username and valid password OK 267ms
/authentication/user authentications/rbac=True/login after clickhouse server is restarted OK 12s 309ms
/authentication/user authentications/rbac=True/login after ldap server is restarted OK 2s 219ms
/authentication/user authentications/rbac=True/login after user cn changed in ldap OK 339ms
/authentication/user authentications/rbac=True/login after user is deleted from ldap OK 341ms
/authentication/user authentications/rbac=True/login after user password changed in ldap OK 411ms
/authentication/user authentications/rbac=True/parallel login OK 16s 500ms
/authentication/user authentications/rbac=True/valid ascii username and utf8 password OK 266ms
/authentication/user authentications/rbac=True/valid long username and short password OK 257ms
/authentication/user authentications/rbac=True/valid short username and invalid long password OK 256ms
/authentication/user authentications/rbac=True/valid short username and long password OK 253ms
/authentication/user authentications/rbac=True/valid username and invalid empty password OK 249ms
/authentication/user authentications/rbac=True/valid username and invalid password OK 265ms
/authentication/user authentications/rbac=True/valid username and password OK 260ms
/authentication/user authentications/rbac=True/valid username and password invalid server OK 187ms
/authentication/user authentications/rbac=True/valid username with valid empty password OK 260ms
/authentication/user authentications/rbac=True/valid utf8 username and ascii password OK 259ms
/authentication/user authentications/rbac=True/verification cooldown OK 4m 36s
/authentication/user authentications/rbac=True/verification cooldown/default verification cooldown value OK 16s 162ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is provided OK 16s 144ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value cn change OK 16s 63ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value ldap unavailable OK 18s 339ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value password change OK 18s 4ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown performance OK 1m 27s
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 25s 909ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 25s 933ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server host parameter change OK 26s 33ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server port parameter change OK 25s 891ms

Generated by TestFlows Open-Source Test Framework v1.9.230315.1003122