✘ 9s 19ms [ Fail ] /version/altinity/error message Aug 25,2025 19:34:24 ⟥ Scenario error message, flags:TE Check contents of log messages when an error occurs. Aug 25,2025 19:34:24 ⟥ Given a ClickHouse instance, flags:MANDATORY|SETUP 615us ⟥⟤ OK a ClickHouse instance, /version/altinity/error message/a ClickHouse instance Aug 25,2025 19:34:24 ⟥ When the ClickHouse process is killed with a SEGV signal Aug 25,2025 19:34:24 ⟥ By executing command kill -SEGV $(pidof clickhouse) 3ms [clickhouse1] bash# kill -SEGV $(pidof clickhouse) 8ms [clickhouse1] bash# echo $? 8ms [clickhouse1] 0 9ms [clickhouse1] bash# 9ms ⟥⟤ OK executing command, /version/altinity/error message/the ClickHouse process is killed with a SEGV signal/executing command Aug 25,2025 19:34:24 ⟥ Then exitcode should be 0 599us ⟥⟤ OK exitcode should be 0, /version/altinity/error message/the ClickHouse process is killed with a SEGV signal/exitcode should be 0 10ms ⟥⟤ OK the ClickHouse process is killed with a SEGV signal, /version/altinity/error message/the ClickHouse process is killed with a SEGV signal Aug 25,2025 19:34:24 ⟥ Then the issues link is pointing to Altinity＇s issues page, if present Aug 25,2025 19:34:24 ⟥ By executing command grep --color=never -i 'ClickHouse/issues' /var/log/clickhouse-server/clickhouse-server.log 4ms [clickhouse1] bash# grep --color=never -i 'ClickHouse/issues' /var/log/clickhouse-server/clickhouse-server.log 8ms [clickhouse1] bash# echo $? 8ms [clickhouse1] 1 8ms [clickhouse1] bash# 8ms ⟥⟤ OK executing command, /version/altinity/error message/the issues link is pointing to Altinity＇s issues page, if present/executing command 11ms ⟥⟤ OK the issues link is pointing to Altinity＇s issues page, if present, /version/altinity/error message/the issues link is pointing to Altinity＇s issues page, if present Aug 25,2025 19:34:24 ⟥ Then the log does not contain unexpected message ＇github․com∕ClickHouse∕ClickHouse＇ Aug 25,2025 19:34:24 ⟥ By executing command grep --color=never -i 'github.com/ClickHouse/ClickHouse' /var/log/clickhouse-server/clickhouse-server.log 4ms [clickhouse1] bash# grep --color=never -i 'github.com/ClickHouse/ClickHouse' /var/log/clickhouse-server/clickhouse-server.log 8ms [clickhouse1] bash# echo $? 8ms [clickhouse1] 1 9ms [clickhouse1] bash# 9ms ⟥⟤ OK executing command, /version/altinity/error message/the log does not contain unexpected message ＇github․com∕ClickHouse∕ClickHouse＇/executing command 10ms ⟥⟤ OK the log does not contain unexpected message ＇github․com∕ClickHouse∕ClickHouse＇, /version/altinity/error message/the log does not contain unexpected message ＇github․com∕ClickHouse∕ClickHouse＇ Aug 25,2025 19:34:24 ⟥ Then the log does not contain unexpected message ＇not official＇ Aug 25,2025 19:34:24 ⟥ By executing command grep --color=never -i 'not official' /var/log/clickhouse-server/clickhouse-server.log 4ms [clickhouse1] bash# grep --color=never -i 'not official' /var/log/clickhouse-server/clickhouse-server.log 8ms [clickhouse1] bash# echo $? 9ms [clickhouse1] 1 9ms [clickhouse1] bash# 9ms ⟥⟤ OK executing command, /version/altinity/error message/the log does not contain unexpected message ＇not official＇/executing command 10ms ⟥⟤ OK the log does not contain unexpected message ＇not official＇, /version/altinity/error message/the log does not contain unexpected message ＇not official＇ Aug 25,2025 19:34:24 ⟥ Then the log does not contain unexpected message ＇（official build）＇ Aug 25,2025 19:34:24 ⟥ By executing command grep --color=never -i '(official build)' /var/log/clickhouse-server/clickhouse-server.log 4ms [clickhouse1] bash# grep --color=never -i '(official build)' /var/log/clickhouse-server/clickhouse-server.log 9ms [clickhouse1] bash# echo $? 11ms [clickhouse1] 1 12ms [clickhouse1] bash# 12ms ⟥⟤ OK executing command, /version/altinity/error message/the log does not contain unexpected message ＇（official build）＇/executing command 13ms ⟥⟤ OK the log does not contain unexpected message ＇（official build）＇, /version/altinity/error message/the log does not contain unexpected message ＇（official build）＇ Aug 25,2025 19:34:24 ⟥ Then the log contains the expected message ＇（altinity build）＇ Aug 25,2025 19:34:24 ⟥ By executing command grep --color=never -i '(altinity build)' /var/log/clickhouse-server/clickhouse-server.log 5ms [clickhouse1] bash# grep --color=never -i '(altinity build)' /var/log/clickhouse-server/clickhouse-server.log 8ms [clickhouse1] bash# echo $? 8ms [clickhouse1] 1 9ms [clickhouse1] bash# 9ms ⟥⟤ OK executing command, /version/altinity/error message/the log contains the expected message ＇（altinity build）＇/executing command 11ms ⟥ Exception: Traceback (most recent call last): File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/regression.py", line 98, in regression() File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../helpers/argparser.py", line 171, in capture_cluster_args return func(self, cluster_args=cluster_args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/regression.py", line 94, in regression Scenario(run=load("version.tests.altinity_version", "feature")) File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py", line 188, in feature Scenario(run=scenario, flags=TE) File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py", line 133, in error_message assert match, error(f"expected message '{message}' not found in log") ^^^^^ AssertionError: Oops! Assertion failed The following assertion was not satisfied assert match, error(f"expected message '{message}' not found in log") Description Expected message '(altinity build)' not found in log Assertion values assert match, error(f"expected message '{message}' not found in log") ^ is False Where File '/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py', line 133 in 'error_message' 125| ) 126| 127| expected_messages = [ 128| "(altinity build)", 129| ] 130| for message in expected_messages: 131| with Then(f"the log contains the expected message '{message}'"): 132| match = grep_in_log(message) 133|> assert match, error(f"expected message '{message}' not found in log") 134| 135| with Then("the log contains a version message"): 136| version_messages = grep_in_log("(version ") 11ms ⟥⟤ Fail the log contains the expected message ＇（altinity build）＇, /version/altinity/error message/the log contains the expected message ＇（altinity build）＇, AssertionError Aug 25,2025 19:34:24 ⟥ Finally the ClickHouse process is restarted, flags:TE|MANDATORY|CLEANUP 3ms [clickhouse1] bash# cat /tmp/clickhouse-server.pid 4ms [clickhouse1] 30 7ms [clickhouse1] bash# echo $? 7ms [clickhouse1] 0 7ms [clickhouse1] bash# Aug 25,2025 19:34:24 ⟥ By capturing pid 3ms [clickhouse1] bash# cat /tmp/clickhouse-server.pid 5ms [clickhouse1] 30 7ms [clickhouse1] bash# echo $? 7ms [clickhouse1] 0 8ms [clickhouse1] bash# 8ms ⟥⟤ OK capturing pid, /version/altinity/error message/the ClickHouse process is restarted/capturing pid Aug 25,2025 19:34:24 ⟥ And sending kill -TERM to ClickHouse server process on clickhouse1 3ms [clickhouse1] bash# kill -TERM 30 5ms [clickhouse1] bash# echo $? 5ms [clickhouse1] 0 6ms [clickhouse1] bash# 6ms ⟥⟤ OK sending kill -TERM to ClickHouse server process on clickhouse1, /version/altinity/error message/the ClickHouse process is restarted/sending kill -TERM to ClickHouse server process on clickhouse1 Aug 25,2025 19:34:24 ⟥ And checking pid does not exist Aug 25,2025 19:34:24 ⟥ Step try #0, flags:RETRY 3ms [clickhouse1] bash# ps 30 | grep -v grep | grep ' clickhouse.server ' 7ms [clickhouse1] 30 ? Ssl 0:00 clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --pidfile=/tmp/clickhouse-server.pid --daemon 10ms [clickhouse1] bash# echo $? 10ms [clickhouse1] 0 10ms [clickhouse1] bash# 11ms ⟥⟤ Fail try #0, /version/altinity/error message/the ClickHouse process is restarted/checking pid does not exist/try #0, pid still alive Aug 25,2025 19:34:27 ⟥ Step try #1, flags:RETRY 5ms [clickhouse1] bash# ps 30 | grep -v grep | grep ' clickhouse.server ' 9ms [clickhouse1] 30 ? Ssl 0:00 clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --pidfile=/tmp/clickhouse-server.pid --daemon 11ms [clickhouse1] bash# echo $? 11ms [clickhouse1] 0 12ms [clickhouse1] bash# 12ms ⟥⟤ Fail try #1, /version/altinity/error message/the ClickHouse process is restarted/checking pid does not exist/try #1, pid still alive Aug 25,2025 19:34:30 ⟥ Step try #2, flags:RETRY 4ms [clickhouse1] bash# ps 30 | grep -v grep | grep ' clickhouse.server ' 11ms [clickhouse1] bash# echo $? 11ms [clickhouse1] 1 12ms [clickhouse1] bash# 12ms ⟥⟤ OK try #2, /version/altinity/error message/the ClickHouse process is restarted/checking pid does not exist/try #2 6s 38ms ⟥⟤ OK checking pid does not exist, /version/altinity/error message/the ClickHouse process is restarted/checking pid does not exist Aug 25,2025 19:34:30 ⟥ By deleting ClickHouse server pid file 4ms [clickhouse1] bash# rm -rf /tmp/clickhouse-server.pid 8ms [clickhouse1] bash# echo $? 8ms [clickhouse1] 0 9ms [clickhouse1] bash# 9ms ⟥⟤ OK deleting ClickHouse server pid file, /version/altinity/error message/the ClickHouse process is restarted/deleting ClickHouse server pid file 28ms [clickhouse1] Aug 25,2025 19:34:30 ⟥ Step try #0, flags:RETRY 7ms [bash] bash# docker-compose --log-level ERROR --ansi never --project-directory "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/version_env" --file "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/version_env/docker-compose.yml" restart clickhouse1 151ms [bash] [?25l[+] Restarting 0/0 151ms [bash] ⠋ Container version_env-clickhouse1-1 Restarting0.1s  250ms [bash] [?25h[?25l[+] Restarting 0/1 251ms [bash] ⠙ Container version_env-clickhouse1-1 Restarting0.2s  351ms [bash] [?25h[?25l[+] Restarting 0/1 351ms [bash] ⠹ Container version_env-clickhouse1-1 Restarting0.3s  380ms [bash] [?25h[?25l[+] Restarting 1/1 380ms [bash] ✔ Container version_env-clickhouse1-1 Started0.3s  385ms [bash] [?25hbash# echo $? 385ms [bash] 0 385ms [bash] bash# 385ms ⟥⟤ OK try #0, /version/altinity/error message/the ClickHouse process is restarted/try #0 Aug 25,2025 19:34:30 ⟥ By attempt #0 6ms [bash] bash# docker-compose --log-level ERROR --ansi never --project-directory "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/version_env" --file "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/version_env/docker-compose.yml" ps -q clickhouse1 49ms [bash] daffc853c26b7f150e3de2db795b24e6ab34e3a834c52b6c0fff5e534575bad7 53ms [bash] bash# echo $? 53ms [bash] 0 53ms [bash] bash# 53ms ⟥⟤ OK attempt #0, /version/altinity/error message/the ClickHouse process is restarted/attempt #0 Aug 25,2025 19:34:30 ⟥ And attempt #0 74ms [clickhouse1] ]0;root@clickhouse1: /root@clickhouse1:/# unset PROMPT_COMMAND; export PS1="bash# " 78ms [clickhouse1] bash# echo 1 78ms [clickhouse1] 1 80ms [clickhouse1] bash# echo $? 80ms [clickhouse1] 0 81ms [clickhouse1] bash# 81ms ⟥⟤ OK attempt #0, /version/altinity/error message/the ClickHouse process is restarted/attempt #0 6s 615ms [clickhouse1] bash# cat /tmp/clickhouse-server.pid 6s 623ms [clickhouse1] cat: /tmp/clickhouse-server.pid: No such file or directory 6s 628ms [clickhouse1] bash# echo $? 6s 628ms [clickhouse1] 1 6s 629ms [clickhouse1] bash# Aug 25,2025 19:34:30 ⟥ Step try #0, flags:RETRY 2s 317ms ⟥⟤ OK try #0, /version/altinity/error message/the ClickHouse process is restarted/try #0 8s 947ms ⟥⟤ OK the ClickHouse process is restarted, /version/altinity/error message/the ClickHouse process is restarted 9s 19ms ⟥⟤ Fail error message, /version/altinity/error message, AssertionError AssertionError ✘ 66ms [ Fail ] /version/altinity/version format Aug 25,2025 19:34:33 ⟥ Scenario version format, flags:TE Check that the version is formatted as expected. Aug 25,2025 19:34:33 ⟥ Given a ClickHouse instance, flags:MANDATORY|SETUP 513us ⟥⟤ OK a ClickHouse instance, /version/altinity/version format/a ClickHouse instance Aug 25,2025 19:34:33 ⟥ When running a query to get the version Aug 25,2025 19:34:33 ⟥ By executing command echo -e "SELECT version()" | clickhouse client -n 2>&1 3ms [clickhouse1] bash# echo -e "SELECT version()" | clickhouse client -n 2>&1 42ms [clickhouse1] 25.6.5.1 60ms [clickhouse1] bash# echo $? 60ms [clickhouse1] 0 60ms [clickhouse1] bash# 60ms ⟥⟤ OK executing command, /version/altinity/version format/running a query to get the version/executing command Aug 25,2025 19:34:33 ⟥ Then check if output has exception 395us ⟥⟤ OK check if output has exception, /version/altinity/version format/running a query to get the version/check if output has exception 61ms ⟥⟤ OK running a query to get the version, /version/altinity/version format/running a query to get the version Aug 25,2025 19:34:33 ⟥ Then the version is formatted correctly 1ms ⟥ Exception: Traceback (most recent call last): File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/regression.py", line 98, in regression() File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../helpers/argparser.py", line 171, in capture_cluster_args return func(self, cluster_args=cluster_args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/regression.py", line 94, in regression Scenario(run=load("version.tests.altinity_version", "feature")) File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py", line 188, in feature Scenario(run=scenario, flags=TE) File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py", line 46, in version_format assert VERSION_REGEX_QUERY.search(result), error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: Oops! Assertion failed The following assertion was not satisfied @TestScenario def version_format(self): """Check that the version is formatted as expected.""" with Given("a ClickHouse instance"): node = self.context.cluster.node("clickhouse1") with When("running a query to get the version"): query = "SELECT version()" result = node.query(query).output with Then("the version is formatted correctly"): assert VERSION_REGEX_QUERY.search(result), error( Description Version on query not formatted correctly, expected match for: '(\d+\.\d+\.\d+.\d+.altinity[a-z]*)' but got: '25.6.5.1' Where File '/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py', line 46 in 'version_format' 38| with Given("a ClickHouse instance"): 39| node = self.context.cluster.node("clickhouse1") 40| 41| with When("running a query to get the version"): 42| query = "SELECT version()" 43| result = node.query(query).output 44| 45| with Then("the version is formatted correctly"): 46|> assert VERSION_REGEX_QUERY.search(result), error( 47| f"Version on query not formatted correctly, expected match for: '{VERSION_REGEX_QUERY.pattern}' but got: '{result}'" 48| ) 49| 2ms ⟥⟤ Fail the version is formatted correctly, /version/altinity/version format/the version is formatted correctly, AssertionError 66ms ⟥⟤ Fail version format, /version/altinity/version format, AssertionError AssertionError ✘ 9s 981ms [ Fail ] /version/altinity AssertionError ✘ 1m 23s [ Fail ] /version AssertionError Traceback (most recent call last): File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/regression.py", line 98, in regression() File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../helpers/argparser.py", line 171, in capture_cluster_args return func(self, cluster_args=cluster_args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/regression.py", line 94, in regression Scenario(run=load("version.tests.altinity_version", "feature")) File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py", line 188, in feature Scenario(run=scenario, flags=TE) File "/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py", line 133, in error_message assert match, error(f"expected message '{message}' not found in log") ^^^^^ AssertionError: Oops! Assertion failed The following assertion was not satisfied assert match, error(f"expected message '{message}' not found in log") Description Expected message '(altinity build)' not found in log Assertion values assert match, error(f"expected message '{message}' not found in log") ^ is False Where File '/home/ubuntu/_work/clickhouse-regression/clickhouse-regression/version/../version/tests/altinity_version.py', line 133 in 'error_message' 125| ) 126| 127| expected_messages = [ 128| "(altinity build)", 129| ] 130| for message in expected_messages: 131| with Then(f"the log contains the expected message '{message}'"): 132| match = grep_in_log(message) 133|> assert match, error(f"expected message '{message}' not found in log") 134| 135| with Then("the log contains a version message"): 136| version_messages = grep_in_log("(version ") Failing ✘ [ Fail ] '/version/altinity/error message' (9s 19ms) ✘ [ Fail ] '/version/altinity/version format' (66ms) ✘ [ Fail ] '/version/altinity' (9s 981ms) ✘ [ Fail ] '/version' (1m 23s) Debugging Rerun the first failing test by executing your test program with the '--only' option. --only '/version/altinity/error message/*' See first failed test messages using the 'tfs show messages' command. tfs show messages --log test.log '/version/altinity/error message' 1 feature (1 failed) 6 scenarios (3 ok, 3 failed) 13 retries (11 ok, 2 failed) 100 steps (98 ok, 2 failed) Total time 1m 23s Executed on Aug 25,2025 19:33 TestFlows.com Open-Source Software Testing Framework v2.0.250110.1002922