Apr 30,2022 16:45:52 ⟥ Module stress
20ms [bash] builder@altinity-builder:~/builds/altinity-qa/clickhouse/cicd/release$ export PS1="bash# "
26ms [bash] bash# ./tests/stress/run.sh
1s 927ms [bash] WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
3s 42ms [bash] + dpkg -i package_folder/clickhouse-common-static_21.8.15.15.altinitystable_amd64.deb
3s 56ms [bash] Selecting previously unselected package clickhouse-common-static.
3s 67ms [bash] (Reading database ... 23165 files and directories currently installed.)
3s 67ms [bash] Preparing to unpack .../clickhouse-common-static_21.8.15.15.altinitystable_amd64.deb ...
3s 69ms [bash] Unpacking clickhouse-common-static (21.8.15.15.altinitystable) ...
7s 45ms [bash] Setting up clickhouse-common-static (21.8.15.15.altinitystable) ...
7s 54ms [bash] + dpkg -i package_folder/clickhouse-common-static-dbg_21.8.15.15.altinitystable_amd64.deb
7s 66ms [bash] Selecting previously unselected package clickhouse-common-static-dbg.
7s 75ms [bash] (Reading database ... 23181 files and directories currently installed.)
7s 76ms [bash] Preparing to unpack .../clickhouse-common-static-dbg_21.8.15.15.altinitystable_amd64.deb ...
7s 77ms [bash] Unpacking clickhouse-common-static-dbg (21.8.15.15.altinitystable) ...
17s 66ms [bash] Setting up clickhouse-common-static-dbg (21.8.15.15.altinitystable) ...
17s 73ms [bash] + dpkg -i package_folder/clickhouse-server_21.8.15.15.altinitystable_all.deb
17s 83ms [bash] Selecting previously unselected package clickhouse-server.
17s 93ms [bash] (Reading database ... 23192 files and directories currently installed.)
17s 93ms [bash] Preparing to unpack .../clickhouse-server_21.8.15.15.altinitystable_all.deb ...
17s 94ms [bash] Unpacking clickhouse-server (21.8.15.15.altinitystable) ...
17s 105ms [bash] Setting up clickhouse-server (21.8.15.15.altinitystable) ...
17s 207ms [bash] ClickHouse binary is already located at /usr/bin/clickhouse
17s 207ms [bash] Symlink /usr/bin/clickhouse-server already exists but it points to /clickhouse. Will replace the old symlink to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-server to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-client to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-local to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-benchmark to /usr/bin/clickhouse.
17s 207ms [bash] Symlink /usr/bin/clickhouse-copier already exists but it points to /clickhouse. Will replace the old symlink to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-copier to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-obfuscator to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-git-import to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-compressor to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-format to /usr/bin/clickhouse.
17s 207ms [bash] Symlink /usr/bin/clickhouse-extract-from-config already exists but it points to /clickhouse. Will replace the old symlink to /usr/bin/clickhouse.
17s 207ms [bash] Creating symlink /usr/bin/clickhouse-extract-from-config to /usr/bin/clickhouse.
17s 207ms [bash] Creating clickhouse group if it does not exist.
17s 208ms [bash] groupadd -r clickhouse
17s 208ms [bash] Creating clickhouse user if it does not exist.
17s 208ms [bash] useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse clickhouse
17s 208ms [bash] Will set ulimits for clickhouse user in /etc/security/limits.d/clickhouse.conf.
17s 208ms [bash] Creating config directory /etc/clickhouse-server/config.d that is used for tweaks of main server configuration.
17s 208ms [bash] Creating config directory /etc/clickhouse-server/users.d that is used for tweaks of users configuration.
17s 208ms [bash] Config file /etc/clickhouse-server/config.xml already exists, will keep it and extract path info from it.
17s 208ms [bash] /etc/clickhouse-server/config.xml has /var/lib/clickhouse/ as data path.
17s 208ms [bash] /etc/clickhouse-server/config.xml has /var/log/clickhouse-server/ as log path.
17s 208ms [bash] Users config file /etc/clickhouse-server/users.xml already exists, will keep it and extract users info from it.
17s 208ms [bash] Creating log directory /var/log/clickhouse-server/.
17s 208ms [bash] Creating data directory /var/lib/clickhouse/.
17s 208ms [bash] Creating pid directory /var/run/clickhouse-server.
17s 208ms [bash] chown -R clickhouse:clickhouse '/var/log/clickhouse-server/'
17s 208ms [bash] chown -R clickhouse:clickhouse '/var/run/clickhouse-server'
17s 208ms [bash] chown clickhouse:clickhouse '/var/lib/clickhouse/'
17s 208ms [bash] groupadd -r clickhouse-bridge
17s 209ms [bash] useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse-bridge clickhouse-bridge
17s 209ms [bash] chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-odbc-bridge'
17s 209ms [bash] chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-library-bridge'
17s 209ms [bash] [1mPassword for default user is empty string. See /etc/clickhouse-server/users.xml and /etc/clickhouse-server/users.d to change it.[0m
17s 209ms [bash] Setting capabilities for clickhouse binary. This is optional.
17s 209ms [bash] chown -R clickhouse:clickhouse '/etc/clickhouse-server'
17s 209ms [bash]
17s 209ms [bash] ClickHouse has been successfully installed.
17s 209ms [bash]
17s 209ms [bash] Start clickhouse-server with:
17s 209ms [bash] sudo clickhouse start
17s 209ms [bash]
17s 209ms [bash] Start clickhouse-client with:
17s 209ms [bash] clickhouse-client
17s 209ms [bash]
17s 219ms [bash] Processing triggers for systemd (245.4-4ubuntu3.7) ...
17s 228ms [bash] + dpkg -i package_folder/clickhouse-client_21.8.15.15.altinitystable_all.deb
17s 238ms [bash] Selecting previously unselected package clickhouse-client.
17s 250ms [bash] (Reading database ... 23209 files and directories currently installed.)
17s 251ms [bash] Preparing to unpack .../clickhouse-client_21.8.15.15.altinitystable_all.deb ...
17s 253ms [bash] Unpacking clickhouse-client (21.8.15.15.altinitystable) ...
17s 262ms [bash] Setting up clickhouse-client (21.8.15.15.altinitystable) ...
17s 271ms [bash] + dpkg -i package_folder/clickhouse-test_21.8.15.15.altinitystable_all.deb
17s 289ms [bash] Selecting previously unselected package clickhouse-test.
17s 294ms [bash] (Reading database ... 23224 files and directories currently installed.)
17s 295ms [bash] Preparing to unpack .../clickhouse-test_21.8.15.15.altinitystable_all.deb ...
17s 296ms [bash] Unpacking clickhouse-test (21.8.15.15.altinitystable) ...
20s 361ms [bash] Setting up clickhouse-test (21.8.15.15.altinitystable) ...
20s 373ms [bash] + cat /etc/clickhouse-server/config.d/listen.xml
20s 373ms [bash] cat: /etc/clickhouse-server/config.d/listen.xml: No such file or directory
20s 373ms [bash] + configure
20s 373ms [bash] + /usr/share/clickhouse-test/config/install.sh
20s 374ms [bash] + DEST_SERVER_PATH=/etc/clickhouse-server
20s 374ms [bash] + DEST_CLIENT_PATH=/etc/clickhouse-client
20s 375ms [bash] +++ dirname /usr/share/clickhouse-test/config/install.sh
20s 375ms [bash] ++ cd /usr/share/clickhouse-test/config
20s 375ms [bash] ++ pwd -P
20s 375ms [bash] Going to install test configs from /usr/share/clickhouse-test/config into /etc/clickhouse-server
20s 375ms [bash] + SRC_PATH=/usr/share/clickhouse-test/config
20s 375ms [bash] + echo 'Going to install test configs from /usr/share/clickhouse-test/config into /etc/clickhouse-server'
20s 375ms [bash] + mkdir -p /etc/clickhouse-server/config.d/
20s 376ms [bash] + mkdir -p /etc/clickhouse-server/users.d/
20s 377ms [bash] + mkdir -p /etc/clickhouse-client
20s 378ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/zookeeper.xml /etc/clickhouse-server/config.d/
20s 378ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/listen.xml /etc/clickhouse-server/config.d/
20s 379ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/part_log.xml /etc/clickhouse-server/config.d/
20s 379ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/text_log.xml /etc/clickhouse-server/config.d/
20s 380ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/metric_log.xml /etc/clickhouse-server/config.d/
20s 381ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/custom_settings_prefixes.xml /etc/clickhouse-server/config.d/
20s 381ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/macros.xml /etc/clickhouse-server/config.d/
20s 382ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/disks.xml /etc/clickhouse-server/config.d/
20s 383ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/secure_ports.xml /etc/clickhouse-server/config.d/
20s 384ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/clusters.xml /etc/clickhouse-server/config.d/
20s 384ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/graphite.xml /etc/clickhouse-server/config.d/
20s 385ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/database_atomic.xml /etc/clickhouse-server/config.d/
20s 385ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/max_concurrent_queries.xml /etc/clickhouse-server/config.d/
20s 386ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/test_cluster_with_incorrect_pw.xml /etc/clickhouse-server/config.d/
20s 386ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/keeper_port.xml /etc/clickhouse-server/config.d/
20s 387ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/logging_no_rotate.xml /etc/clickhouse-server/config.d/
20s 388ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/tcp_with_proxy.xml /etc/clickhouse-server/config.d/
20s 388ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/top_level_domains_lists.xml /etc/clickhouse-server/config.d/
20s 389ms [bash] + ln -sf /usr/share/clickhouse-test/config/config.d/top_level_domains_path.xml /etc/clickhouse-server/config.d/
20s 390ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/log_queries.xml /etc/clickhouse-server/users.d/
20s 390ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/readonly.xml /etc/clickhouse-server/users.d/
20s 391ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/access_management.xml /etc/clickhouse-server/users.d/
20s 391ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/database_atomic_drop_detach_sync.xml /etc/clickhouse-server/users.d/
20s 392ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/opentelemetry.xml /etc/clickhouse-server/users.d/
20s 393ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/remote_queries.xml /etc/clickhouse-server/users.d/
20s 394ms [bash] + ln -sf /usr/share/clickhouse-test/config/users.d/timeouts.xml /etc/clickhouse-server/users.d/
20s 394ms [bash] + ln -sf /usr/share/clickhouse-test/config/ints_dictionary.xml /etc/clickhouse-server/
20s 395ms [bash] + ln -sf /usr/share/clickhouse-test/config/strings_dictionary.xml /etc/clickhouse-server/
20s 396ms [bash] + ln -sf /usr/share/clickhouse-test/config/decimals_dictionary.xml /etc/clickhouse-server/
20s 396ms [bash] + ln -sf /usr/share/clickhouse-test/config/executable_dictionary.xml /etc/clickhouse-server/
20s 397ms [bash] + ln -sf /usr/share/clickhouse-test/config/executable_pool_dictionary.xml /etc/clickhouse-server/
20s 398ms [bash] + ln -sf /usr/share/clickhouse-test/config/top_level_domains /etc/clickhouse-server/
20s 398ms [bash] + ln -sf /usr/share/clickhouse-test/config/server.key /etc/clickhouse-server/
20s 399ms [bash] + ln -sf /usr/share/clickhouse-test/config/server.crt /etc/clickhouse-server/
20s 399ms [bash] + ln -sf /usr/share/clickhouse-test/config/dhparam.pem /etc/clickhouse-server/
20s 400ms [bash] + ln -sf --backup=simple --suffix=_original.xml /usr/share/clickhouse-test/config/config.d/query_masking_rules.xml /etc/clickhouse-server/config.d/
20s 401ms [bash] + [[ -n '' ]]
20s 401ms [bash] + [[ -n '' ]]
20s 401ms [bash] + [[ -n '' ]]
20s 401ms [bash] + ln -sf /usr/share/clickhouse-test/config/client_config.xml /etc/clickhouse-client/config.xml
20s 402ms [bash] + echo 'ASAN_OPTIONS='\''malloc_context_size=10 verbosity=1 allocator_release_to_os_interval_ms=10000'\'''
20s 403ms [bash] + export 'ASAN_OPTIONS=malloc_context_size=10 allocator_release_to_os_interval_ms=10000'
20s 403ms [bash] + ASAN_OPTIONS='malloc_context_size=10 allocator_release_to_os_interval_ms=10000'
20s 403ms [bash] + sudo chown root: /var/lib/clickhouse
20s 407ms [bash] + echo '1'
20s 407ms [bash] + echo '0.5'
20s 407ms [bash] + echo 'Enable IPv4'
20s 407ms [bash] + sed -i s/::/0.0.0.0/g /etc/clickhouse-server/config.d/listen.xml
20s 407ms [bash] Enable IPv4
20s 408ms [bash] + start
20s 409ms [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log ']'
20s 409ms [bash] + counter=0
20s 409ms [bash] + clickhouse-client --query 'SELECT 1'
20s 433ms [bash] Code: 210. DB::NetException: Connection refused (localhost:9000)
20s 433ms [bash]
20s 437ms [bash] + '[' 0 -gt 120 ']'
20s 438ms [bash] + clickhouse start --user root
21s 456ms [bash] + sleep 0.5
21s 958ms [bash] + counter=1
21s 958ms [bash] + clickhouse-client --query 'SELECT 1'
21s 987ms [bash] 1
21s 994ms [bash] + echo '
21s 994ms [bash] set follow-fork-mode child
21s 994ms [bash] handle all noprint
21s 994ms [bash] handle SIGSEGV stop print
21s 995ms [bash] handle SIGBUS stop print
21s 995ms [bash] handle SIGABRT stop print
21s 995ms [bash] continue
21s 995ms [bash] thread apply all backtrace
21s 995ms [bash] detach
21s 995ms [bash] quit
21s 995ms [bash] '
21s 995ms [bash] + /s3downloader --dataset-names hits visits
21s 995ms [bash] ++ cat /var/run/clickhouse-server/clickhouse-server.pid
21s 995ms [bash] + gdb -batch -command script.gdb -p 171
22s 72ms [bash] INFO:root:Will fetch following datasets: hits, visits
22s 73ms [bash] INFO:root:Processing hits
22s 73ms [bash] INFO:root:Downloading from https://clickhouse-datasets.s3.yandex.net/hits/partitions/hits_v1.tar to temp path /tmp/gu72d4zc
22s 548ms [bash] INFO:root:Content length is 1271623680 bytes
45s 523ms [bash] INFO:root:Downloading finished
45s 523ms [bash] INFO:root:Will unpack data from temp path /tmp/gu72d4zc to clickhouse db /var/lib/clickhouse/
46s 656ms [bash] INFO:root:Unpack finished
46s 656ms [bash] INFO:root:Will remove dowloaded file /tmp/gu72d4zc from filesystem if it exists
46s 779ms [bash] INFO:root:Processing of hits finished
46s 779ms [bash] INFO:root:Processing visits
46s 779ms [bash] INFO:root:Downloading from https://clickhouse-datasets.s3.yandex.net/visits/partitions/visits_v1.tar to temp path /tmp/f5q8qdop
47s 122ms [bash] INFO:root:Content length is 563968000 bytes
52s 962ms [bash] INFO:root:Downloading finished
52s 962ms [bash] INFO:root:Will unpack data from temp path /tmp/f5q8qdop to clickhouse db /var/lib/clickhouse/
53s 518ms [bash] INFO:root:Unpack finished
53s 518ms [bash] INFO:root:Will remove dowloaded file /tmp/f5q8qdop from filesystem if it exists
53s 579ms [bash]
53s 579ms [bash]
53s 579ms [bash] INFO:root:Processing of visits finished
53s 579ms [bash] INFO:root:Fetch finished, enjoy your tables!
53s 588ms [bash] + chmod 777 -R /var/lib/clickhouse
53s 605ms [bash] + clickhouse-client --query 'ATTACH DATABASE IF NOT EXISTS datasets ENGINE = Ordinary'
1m 26s [bash] + clickhouse-client --query 'CREATE DATABASE IF NOT EXISTS test'
1m 26s [bash] + stop
1m 26s [bash] + clickhouse stop
1m 30s [bash] script.gdb:9: Error in sourced command file:
1m 30s [bash] The program is not being run.
1m 30s [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 171.
1m 30s [bash] The process with pid = 171 is running.
1m 30s [bash] Sent terminate signal to process with pid 171.
1m 30s [bash] Waiting for server to stop
1m 30s [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 171.
1m 30s [bash] The process with pid = 171 is running.
1m 30s [bash] Waiting for server to stop
1m 30s [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 171.
1m 30s [bash] The process with pid = 171 is running.
1m 30s [bash] Waiting for server to stop
1m 30s [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 171.
1m 30s [bash] The process with pid = 171 is running.
1m 30s [bash] Waiting for server to stop
1m 30s [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 171.
1m 30s [bash] The process with pid = 171 is running.
1m 30s [bash] Waiting for server to stop
1m 30s [bash] Now there is no clickhouse-server process.
1m 30s [bash] Server stopped
1m 30s [bash] + start
1m 30s [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log ']'
1m 30s [bash] + log_file_counter=1
1m 30s [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log.1 ']'
1m 30s [bash] + mv /var/log/clickhouse-server/clickhouse-server.log /var/log/clickhouse-server/clickhouse-server.log.1
1m 30s [bash] + counter=0
1m 30s [bash] + clickhouse-client --query 'SELECT 1'
1m 30s [bash] Code: 210. DB::NetException: Connection refused (localhost:9000)
1m 30s [bash]
1m 30s [bash] + '[' 0 -gt 120 ']'
1m 30s [bash] + clickhouse start --user root
1m 31s [bash] + sleep 0.5
1m 32s [bash] + counter=1
1m 32s [bash] + clickhouse-client --query 'SELECT 1'
1m 32s [bash] 1
1m 32s [bash] + echo '
1m 32s [bash] set follow-fork-mode child
1m 32s [bash] handle all noprint
1m 32s [bash] handle SIGSEGV stop print
1m 32s [bash] handle SIGBUS stop print
1m 32s [bash] handle SIGABRT stop print
1m 32s [bash] continue
1m 32s [bash] thread apply all backtrace
1m 32s [bash] detach
1m 32s [bash] quit
1m 32s [bash] '
1m 32s [bash] + clickhouse-client --query 'SHOW TABLES FROM datasets'
1m 32s [bash] ++ cat /var/run/clickhouse-server/clickhouse-server.pid
1m 32s [bash] + gdb -batch -command script.gdb -p 383
1m 32s [bash] hits_v1
1m 32s [bash] visits_v1
1m 32s [bash] + clickhouse-client --query 'SHOW TABLES FROM test'
2m 29s [bash] + clickhouse-client --query 'RENAME TABLE datasets.hits_v1 TO test.hits'
2m 29s [bash] + clickhouse-client --query 'RENAME TABLE datasets.visits_v1 TO test.visits'
2m 29s [bash] + clickhouse-client --query 'SHOW TABLES FROM test'
2m 29s [bash] hits
2m 29s [bash] visits
2m 29s [bash] + /stress --hung-check --drop-databases --output-folder /test_output --skip-func-tests ''
2m 29s [bash] 2022-04-30 19:48:22,643 Run func tests '/usr/bin/clickhouse-test --use-skip-list --client-option='join_use_nulls=1' --client-option="join_algorithm='partial_merge'" --client-option="join_algorithm='auto'" --client-option="max_rows_in_join=1000" --global_time_limit=3600 '
2m 30s [bash] 2022-04-30 19:48:23,148 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --db-engine=Ordinary --database=test_1 --global_time_limit=3600 '
2m 30s [bash] 2022-04-30 19:48:23,651 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --client-option='allow_experimental_database_replicated=1' --db-engine="Replicated('/test/db/test_2', 's1', 'r1')" --global_time_limit=3600 '
2m 31s [bash] 2022-04-30 19:48:24,152 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --database=test_3 --global_time_limit=3600 '
2m 31s [bash] 2022-04-30 19:48:24,654 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --db-engine=Ordinary --global_time_limit=3600 '
2m 32s [bash] 2022-04-30 19:48:25,158 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --client-option='allow_experimental_database_replicated=1' --db-engine="Replicated('/test/db/test_5', 's1', 'r1')" --database=test_5 --global_time_limit=3600 '
2m 32s [bash] 2022-04-30 19:48:25,673 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --global_time_limit=3600 '
2m 33s [bash] 2022-04-30 19:48:26,176 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --db-engine=Ordinary --database=test_7 --client-option='join_use_nulls=1' --global_time_limit=3600 '
2m 33s [bash] 2022-04-30 19:48:26,695 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --client-option='allow_experimental_database_replicated=1' --db-engine="Replicated('/test/db/test_8', 's1', 'r1')" --global_time_limit=3600 '
2m 34s [bash] 2022-04-30 19:48:27,232 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --database=test_9 --global_time_limit=3600 '
2m 34s [bash] 2022-04-30 19:48:27,781 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --db-engine=Ordinary --global_time_limit=3600 '
2m 35s [bash] 2022-04-30 19:48:28,294 Run func tests '/usr/bin/clickhouse-test --use-skip-list --order=random --client-option='allow_experimental_database_replicated=1' --db-engine="Replicated('/test/db/test_11', 's1', 'r1')" --database=test_11 --global_time_limit=3600 '
2m 35s [bash] 2022-04-30 19:48:28,835 Will wait functests to finish
2m 35s [bash] 2022-04-30 19:48:28,836 Finished 0 from 12 processes
2m 39s [bash] [Attaching after Thread 0x7f6a1eb49700 (LWP 1256) vfork to child process 2381]
2m 39s [bash] [Detaching vfork parent process 383 after child exec]
2m 40s [bash] 2022-04-30 19:48:33,838 Finished 0 from 12 processes
2m 44s [bash] [Attaching after process 2381 fork to child process 2972]
2m 44s [bash] [Detaching after fork from parent process 2381]
2m 45s [bash] script.gdb:9: Error in sourced command file:
2m 45s [bash] The program is not being run.
2m 45s [bash] 2022-04-30 19:48:38,842 Finished 0 from 12 processes
2m 50s [bash] 2022-04-30 19:48:43,846 Finished 0 from 12 processes
2m 55s [bash] 2022-04-30 19:48:48,851 Finished 0 from 12 processes
3m 0s [bash] 2022-04-30 19:48:53,859 Finished 0 from 12 processes
3m 5s [bash] 2022-04-30 19:48:58,863 Finished 1 from 12 processes
3m 10s [bash] 2022-04-30 19:49:03,866 Finished 2 from 12 processes
3m 15s [bash] 2022-04-30 19:49:08,872 Finished 2 from 12 processes
3m 20s [bash] 2022-04-30 19:49:13,874 Finished 2 from 12 processes
3m 25s [bash] 2022-04-30 19:49:18,878 Finished 2 from 12 processes
3m 30s [bash] 2022-04-30 19:49:23,891 Finished 2 from 12 processes
3m 35s [bash] 2022-04-30 19:49:28,895 Finished 2 from 12 processes
3m 40s [bash] 2022-04-30 19:49:33,899 Finished 2 from 12 processes
3m 45s [bash] 2022-04-30 19:49:38,907 Finished 2 from 12 processes
3m 50s [bash] 2022-04-30 19:49:43,911 Finished 2 from 12 processes
3m 55s [bash] 2022-04-30 19:49:48,915 Finished 2 from 12 processes
4m 0s [bash] 2022-04-30 19:49:53,920 Finished 2 from 12 processes
4m 5s [bash] 2022-04-30 19:49:58,922 Finished 2 from 12 processes
4m 10s [bash] 2022-04-30 19:50:03,926 Finished 2 from 12 processes
4m 15s [bash] 2022-04-30 19:50:08,930 Finished 2 from 12 processes
4m 20s [bash] 2022-04-30 19:50:13,935 Finished 2 from 12 processes
4m 25s [bash] 2022-04-30 19:50:18,938 Finished 2 from 12 processes
4m 30s [bash] 2022-04-30 19:50:23,942 Finished 2 from 12 processes
4m 35s [bash] 2022-04-30 19:50:28,946 Finished 2 from 12 processes
4m 40s [bash] 2022-04-30 19:50:33,951 Finished 3 from 12 processes
4m 45s [bash] 2022-04-30 19:50:38,955 Finished 3 from 12 processes
4m 50s [bash] 2022-04-30 19:50:43,960 Finished 3 from 12 processes
4m 55s [bash] 2022-04-30 19:50:48,962 Finished 3 from 12 processes
5m 0s [bash] 2022-04-30 19:50:53,968 Finished 3 from 12 processes
5m 6s [bash] 2022-04-30 19:50:58,971 Finished 3 from 12 processes
5m 11s [bash] 2022-04-30 19:51:03,974 Finished 3 from 12 processes
5m 16s [bash] 2022-04-30 19:51:08,979 Finished 3 from 12 processes
5m 21s [bash] 2022-04-30 19:51:13,987 Finished 3 from 12 processes
5m 26s [bash] 2022-04-30 19:51:18,990 Finished 3 from 12 processes
5m 31s [bash] 2022-04-30 19:51:23,995 Finished 3 from 12 processes
5m 36s [bash] 2022-04-30 19:51:28,998 Finished 3 from 12 processes
5m 41s [bash] 2022-04-30 19:51:34,002 Finished 3 from 12 processes
5m 46s [bash] 2022-04-30 19:51:39,006 Finished 3 from 12 processes
5m 51s [bash] 2022-04-30 19:51:44,010 Finished 3 from 12 processes
5m 56s [bash] 2022-04-30 19:51:49,014 Finished 3 from 12 processes
6m 1s [bash] 2022-04-30 19:51:54,018 Finished 3 from 12 processes
6m 6s [bash] 2022-04-30 19:51:59,023 Finished 3 from 12 processes
6m 11s [bash] 2022-04-30 19:52:04,026 Finished 3 from 12 processes
6m 16s [bash] 2022-04-30 19:52:09,030 Finished 3 from 12 processes
6m 21s [bash] 2022-04-30 19:52:14,034 Finished 3 from 12 processes
6m 26s [bash] 2022-04-30 19:52:19,038 Finished 3 from 12 processes
6m 31s [bash] 2022-04-30 19:52:24,043 Finished 3 from 12 processes
6m 36s [bash] 2022-04-30 19:52:29,046 Finished 3 from 12 processes
6m 41s [bash] 2022-04-30 19:52:34,050 Finished 3 from 12 processes
6m 46s [bash] 2022-04-30 19:52:39,055 Finished 3 from 12 processes
6m 51s [bash] 2022-04-30 19:52:44,056 Finished 3 from 12 processes
6m 56s [bash] 2022-04-30 19:52:49,059 Finished 3 from 12 processes
7m 1s [bash] 2022-04-30 19:52:54,062 Finished 3 from 12 processes
7m 6s [bash] 2022-04-30 19:52:59,066 Finished 3 from 12 processes
7m 11s [bash] 2022-04-30 19:53:04,070 Finished 3 from 12 processes
7m 16s [bash] 2022-04-30 19:53:09,075 Finished 3 from 12 processes
7m 21s [bash] 2022-04-30 19:53:14,079 Finished 3 from 12 processes
7m 26s [bash] 2022-04-30 19:53:19,083 Finished 3 from 12 processes
7m 31s [bash] 2022-04-30 19:53:24,090 Finished 3 from 12 processes
7m 36s [bash] 2022-04-30 19:53:29,094 Finished 3 from 12 processes
7m 41s [bash] 2022-04-30 19:53:34,099 Finished 3 from 12 processes
7m 46s [bash] 2022-04-30 19:53:39,102 Finished 3 from 12 processes
7m 51s [bash] 2022-04-30 19:53:44,107 Finished 3 from 12 processes
7m 56s [bash] 2022-04-30 19:53:49,110 Finished 3 from 12 processes
8m 1s [bash] 2022-04-30 19:53:54,114 Finished 3 from 12 processes
8m 6s [bash] 2022-04-30 19:53:59,118 Finished 3 from 12 processes
8m 11s [bash] 2022-04-30 19:54:04,122 Finished 3 from 12 processes
8m 16s [bash] 2022-04-30 19:54:09,126 Finished 3 from 12 processes
8m 21s [bash] 2022-04-30 19:54:14,130 Finished 3 from 12 processes
8m 26s [bash] 2022-04-30 19:54:19,134 Finished 3 from 12 processes
8m 31s [bash] 2022-04-30 19:54:24,140 Finished 3 from 12 processes
8m 36s [bash] 2022-04-30 19:54:29,145 Finished 3 from 12 processes
8m 41s [bash] 2022-04-30 19:54:34,151 Finished 3 from 12 processes
8m 46s [bash] 2022-04-30 19:54:39,154 Finished 3 from 12 processes
8m 51s [bash] 2022-04-30 19:54:44,158 Finished 3 from 12 processes
8m 56s [bash] 2022-04-30 19:54:49,162 Finished 3 from 12 processes
9m 1s [bash] 2022-04-30 19:54:54,166 Finished 3 from 12 processes
9m 6s [bash] 2022-04-30 19:54:59,175 Finished 3 from 12 processes
9m 11s [bash] 2022-04-30 19:55:04,178 Finished 3 from 12 processes
9m 16s [bash] 2022-04-30 19:55:09,183 Finished 3 from 12 processes
9m 21s [bash] 2022-04-30 19:55:14,186 Finished 3 from 12 processes
9m 26s [bash] 2022-04-30 19:55:19,190 Finished 3 from 12 processes
9m 31s [bash] 2022-04-30 19:55:24,195 Finished 3 from 12 processes
9m 36s [bash] 2022-04-30 19:55:29,198 Finished 3 from 12 processes
9m 41s [bash] 2022-04-30 19:55:34,203 Finished 3 from 12 processes
9m 46s [bash] 2022-04-30 19:55:39,206 Finished 3 from 12 processes
9m 51s [bash] 2022-04-30 19:55:44,211 Finished 3 from 12 processes
9m 56s [bash] 2022-04-30 19:55:49,217 Finished 3 from 12 processes
10m 1s [bash] 2022-04-30 19:55:54,218 Finished 3 from 12 processes
10m 6s [bash] 2022-04-30 19:55:59,222 Finished 3 from 12 processes
10m 11s [bash] 2022-04-30 19:56:04,227 Finished 3 from 12 processes
10m 16s [bash] 2022-04-30 19:56:09,228 Finished 3 from 12 processes
10m 21s [bash] 2022-04-30 19:56:14,234 Finished 3 from 12 processes
10m 26s [bash] 2022-04-30 19:56:19,239 Finished 3 from 12 processes
10m 31s [bash] 2022-04-30 19:56:24,244 Finished 3 from 12 processes
10m 36s [bash] 2022-04-30 19:56:29,247 Finished 3 from 12 processes
10m 41s [bash] 2022-04-30 19:56:34,253 Finished 3 from 12 processes
10m 46s [bash] 2022-04-30 19:56:39,258 Finished 3 from 12 processes
10m 51s [bash] 2022-04-30 19:56:44,262 Finished 3 from 12 processes
10m 56s [bash] 2022-04-30 19:56:49,267 Finished 3 from 12 processes
11m 1s [bash] 2022-04-30 19:56:54,272 Finished 3 from 12 processes
11m 6s [bash] 2022-04-30 19:56:59,274 Finished 3 from 12 processes
11m 11s [bash] 2022-04-30 19:57:04,278 Finished 3 from 12 processes
11m 16s [bash] 2022-04-30 19:57:09,283 Finished 3 from 12 processes
11m 21s [bash] 2022-04-30 19:57:14,287 Finished 3 from 12 processes
11m 26s [bash] 2022-04-30 19:57:19,290 Finished 3 from 12 processes
11m 31s [bash] 2022-04-30 19:57:24,294 Finished 3 from 12 processes
11m 36s [bash] 2022-04-30 19:57:29,298 Finished 3 from 12 processes
11m 41s [bash] 2022-04-30 19:57:34,302 Finished 3 from 12 processes
11m 46s [bash] 2022-04-30 19:57:39,308 Finished 3 from 12 processes
11m 51s [bash] 2022-04-30 19:57:44,313 Finished 3 from 12 processes
11m 56s [bash] 2022-04-30 19:57:49,318 Finished 3 from 12 processes
12m 1s [bash] 2022-04-30 19:57:54,322 Finished 3 from 12 processes
12m 6s [bash] 2022-04-30 19:57:59,328 Finished 3 from 12 processes
12m 11s [bash] 2022-04-30 19:58:04,331 Finished 3 from 12 processes
12m 16s [bash] 2022-04-30 19:58:09,334 Finished 3 from 12 processes
12m 21s [bash] 2022-04-30 19:58:14,338 Finished 3 from 12 processes
12m 26s [bash] 2022-04-30 19:58:19,342 Finished 3 from 12 processes
12m 31s [bash] 2022-04-30 19:58:24,346 Finished 3 from 12 processes
12m 36s [bash] 2022-04-30 19:58:29,354 Finished 3 from 12 processes
12m 41s [bash] 2022-04-30 19:58:34,360 Finished 3 from 12 processes
12m 46s [bash] 2022-04-30 19:58:39,366 Finished 3 from 12 processes
12m 51s [bash] 2022-04-30 19:58:44,370 Finished 3 from 12 processes
12m 56s [bash] 2022-04-30 19:58:49,374 Finished 3 from 12 processes
13m 1s [bash] 2022-04-30 19:58:54,380 Finished 3 from 12 processes
13m 6s [bash] 2022-04-30 19:58:59,386 Finished 3 from 12 processes
13m 11s [bash] 2022-04-30 19:59:04,390 Finished 3 from 12 processes
13m 16s [bash] 2022-04-30 19:59:09,394 Finished 3 from 12 processes
13m 21s [bash] 2022-04-30 19:59:14,398 Finished 3 from 12 processes
13m 26s [bash] 2022-04-30 19:59:19,402 Finished 3 from 12 processes
13m 31s [bash] 2022-04-30 19:59:24,407 Finished 3 from 12 processes
13m 36s [bash] 2022-04-30 19:59:29,410 Finished 3 from 12 processes
13m 41s [bash] 2022-04-30 19:59:34,414 Finished 3 from 12 processes
13m 46s [bash] 2022-04-30 19:59:39,418 Finished 3 from 12 processes
13m 51s [bash] 2022-04-30 19:59:44,424 Finished 3 from 12 processes
13m 56s [bash] 2022-04-30 19:59:49,427 Finished 3 from 12 processes
14m 1s [bash] 2022-04-30 19:59:54,430 Finished 3 from 12 processes
14m 6s [bash] 2022-04-30 19:59:59,431 Finished 3 from 12 processes
14m 11s [bash] 2022-04-30 20:00:04,434 Finished 3 from 12 processes
14m 16s [bash] 2022-04-30 20:00:09,438 Finished 3 from 12 processes
14m 21s [bash] 2022-04-30 20:00:14,443 Finished 3 from 12 processes
14m 26s [bash] 2022-04-30 20:00:19,446 Finished 3 from 12 processes
14m 31s [bash] 2022-04-30 20:00:24,451 Finished 3 from 12 processes
14m 36s [bash] 2022-04-30 20:00:29,454 Finished 3 from 12 processes
14m 41s [bash] 2022-04-30 20:00:34,461 Finished 3 from 12 processes
14m 46s [bash] 2022-04-30 20:00:39,466 Finished 3 from 12 processes
14m 51s [bash] 2022-04-30 20:00:44,470 Finished 3 from 12 processes
14m 56s [bash] 2022-04-30 20:00:49,474 Finished 3 from 12 processes
15m 1s [bash] 2022-04-30 20:00:54,478 Finished 3 from 12 processes
15m 6s [bash] 2022-04-30 20:00:59,484 Finished 3 from 12 processes
15m 11s [bash] 2022-04-30 20:01:04,487 Finished 3 from 12 processes
15m 16s [bash] 2022-04-30 20:01:09,495 Finished 3 from 12 processes
15m 21s [bash] 2022-04-30 20:01:14,509 Finished 3 from 12 processes
15m 26s [bash] 2022-04-30 20:01:19,512 Finished 3 from 12 processes
15m 31s [bash] 2022-04-30 20:01:24,522 Finished 3 from 12 processes
15m 36s [bash] 2022-04-30 20:01:29,527 Finished 3 from 12 processes
15m 41s [bash] 2022-04-30 20:01:34,531 Finished 3 from 12 processes
15m 46s [bash] 2022-04-30 20:01:39,534 Finished 3 from 12 processes
15m 51s [bash] 2022-04-30 20:01:44,539 Finished 3 from 12 processes
15m 56s [bash] 2022-04-30 20:01:49,542 Finished 3 from 12 processes
16m 1s [bash] 2022-04-30 20:01:54,547 Finished 3 from 12 processes
16m 6s [bash] 2022-04-30 20:01:59,552 Finished 3 from 12 processes
16m 11s [bash] 2022-04-30 20:02:04,555 Finished 3 from 12 processes
16m 16s [bash] 2022-04-30 20:02:09,558 Finished 3 from 12 processes
16m 21s [bash] 2022-04-30 20:02:14,563 Finished 3 from 12 processes
16m 26s [bash] 2022-04-30 20:02:19,568 Finished 3 from 12 processes
16m 31s [bash] 2022-04-30 20:02:24,575 Finished 3 from 12 processes
16m 36s [bash] 2022-04-30 20:02:29,578 Finished 3 from 12 processes
16m 41s [bash] 2022-04-30 20:02:34,582 Finished 3 from 12 processes
16m 46s [bash] 2022-04-30 20:02:39,586 Finished 3 from 12 processes
16m 51s [bash] 2022-04-30 20:02:44,590 Finished 3 from 12 processes
16m 56s [bash] 2022-04-30 20:02:49,594 Finished 3 from 12 processes
17m 1s [bash] 2022-04-30 20:02:54,599 Finished 3 from 12 processes
17m 6s [bash] 2022-04-30 20:02:59,602 Finished 3 from 12 processes
17m 11s [bash] 2022-04-30 20:03:04,606 Finished 3 from 12 processes
17m 16s [bash] 2022-04-30 20:03:09,610 Finished 3 from 12 processes
17m 21s [bash] 2022-04-30 20:03:14,616 Finished 3 from 12 processes
17m 26s [bash] 2022-04-30 20:03:19,618 Finished 3 from 12 processes
17m 31s [bash] 2022-04-30 20:03:24,623 Finished 3 from 12 processes
17m 36s [bash] 2022-04-30 20:03:29,627 Finished 3 from 12 processes
17m 41s [bash] 2022-04-30 20:03:34,631 Finished 3 from 12 processes
17m 46s [bash] 2022-04-30 20:03:39,634 Finished 3 from 12 processes
17m 51s [bash] 2022-04-30 20:03:44,638 Finished 3 from 12 processes
17m 56s [bash] 2022-04-30 20:03:49,642 Finished 3 from 12 processes
18m 1s [bash] 2022-04-30 20:03:54,646 Finished 3 from 12 processes
18m 6s [bash] 2022-04-30 20:03:59,651 Finished 3 from 12 processes
18m 11s [bash] 2022-04-30 20:04:04,655 Finished 3 from 12 processes
18m 16s [bash] 2022-04-30 20:04:09,658 Finished 3 from 12 processes
18m 21s [bash] 2022-04-30 20:04:14,662 Finished 3 from 12 processes
18m 26s [bash] 2022-04-30 20:04:19,666 Finished 3 from 12 processes
18m 31s [bash] 2022-04-30 20:04:24,670 Finished 3 from 12 processes
18m 36s [bash] 2022-04-30 20:04:29,675 Finished 3 from 12 processes
18m 41s [bash] 2022-04-30 20:04:34,678 Finished 3 from 12 processes
18m 46s [bash] 2022-04-30 20:04:39,682 Finished 3 from 12 processes
18m 51s [bash] 2022-04-30 20:04:44,687 Finished 3 from 12 processes
18m 56s [bash] 2022-04-30 20:04:49,691 Finished 3 from 12 processes
19m 1s [bash] 2022-04-30 20:04:54,694 Finished 3 from 12 processes
19m 6s [bash] 2022-04-30 20:04:59,698 Finished 3 from 12 processes
19m 11s [bash] 2022-04-30 20:05:04,704 Finished 3 from 12 processes
19m 16s [bash] 2022-04-30 20:05:09,707 Finished 3 from 12 processes
19m 21s [bash] 2022-04-30 20:05:14,710 Finished 3 from 12 processes
19m 26s [bash] 2022-04-30 20:05:19,715 Finished 3 from 12 processes
19m 31s [bash] 2022-04-30 20:05:24,719 Finished 3 from 12 processes
19m 36s [bash] 2022-04-30 20:05:29,724 Finished 3 from 12 processes
19m 41s [bash] 2022-04-30 20:05:34,726 Finished 3 from 12 processes
19m 46s [bash] 2022-04-30 20:05:39,730 Finished 3 from 12 processes
19m 51s [bash] 2022-04-30 20:05:44,735 Finished 3 from 12 processes
19m 56s [bash] 2022-04-30 20:05:49,739 Finished 3 from 12 processes
20m 1s [bash] 2022-04-30 20:05:54,742 Finished 3 from 12 processes
20m 6s [bash] 2022-04-30 20:05:59,746 Finished 3 from 12 processes
20m 11s [bash] 2022-04-30 20:06:04,750 Finished 3 from 12 processes
20m 16s [bash] 2022-04-30 20:06:09,755 Finished 3 from 12 processes
20m 21s [bash] 2022-04-30 20:06:14,760 Finished 3 from 12 processes
20m 26s [bash] 2022-04-30 20:06:19,767 Finished 3 from 12 processes
20m 31s [bash] 2022-04-30 20:06:24,770 Finished 3 from 12 processes
20m 36s [bash] 2022-04-30 20:06:29,777 Finished 3 from 12 processes
20m 41s [bash] 2022-04-30 20:06:34,782 Finished 3 from 12 processes
20m 46s [bash] 2022-04-30 20:06:39,786 Finished 3 from 12 processes
20m 51s [bash] 2022-04-30 20:06:44,790 Finished 3 from 12 processes
20m 56s [bash] 2022-04-30 20:06:49,794 Finished 3 from 12 processes
21m 1s [bash] 2022-04-30 20:06:54,798 Finished 3 from 12 processes
21m 6s [bash] 2022-04-30 20:06:59,804 Finished 3 from 12 processes
21m 11s [bash] 2022-04-30 20:07:04,807 Finished 3 from 12 processes
21m 16s [bash] 2022-04-30 20:07:09,810 Finished 3 from 12 processes
21m 21s [bash] 2022-04-30 20:07:14,814 Finished 3 from 12 processes
21m 26s [bash] 2022-04-30 20:07:19,819 Finished 3 from 12 processes
21m 31s [bash] 2022-04-30 20:07:24,824 Finished 3 from 12 processes
21m 36s [bash] 2022-04-30 20:07:29,826 Finished 3 from 12 processes
21m 41s [bash] 2022-04-30 20:07:34,830 Finished 3 from 12 processes
21m 46s [bash] 2022-04-30 20:07:39,834 Finished 3 from 12 processes
21m 51s [bash] 2022-04-30 20:07:44,840 Finished 3 from 12 processes
21m 56s [bash] 2022-04-30 20:07:49,844 Finished 3 from 12 processes
22m 1s [bash] 2022-04-30 20:07:54,846 Finished 3 from 12 processes
22m 6s [bash] 2022-04-30 20:07:59,851 Finished 3 from 12 processes
22m 11s [bash] 2022-04-30 20:08:04,854 Finished 3 from 12 processes
22m 16s [bash] 2022-04-30 20:08:09,858 Finished 3 from 12 processes
22m 21s [bash] 2022-04-30 20:08:14,862 Finished 3 from 12 processes
22m 26s [bash] 2022-04-30 20:08:19,867 Finished 3 from 12 processes
22m 31s [bash] 2022-04-30 20:08:24,871 Finished 3 from 12 processes
22m 36s [bash] 2022-04-30 20:08:29,875 Finished 3 from 12 processes
22m 41s [bash] 2022-04-30 20:08:34,878 Finished 3 from 12 processes
22m 46s [bash] 2022-04-30 20:08:39,882 Finished 3 from 12 processes
22m 51s [bash] 2022-04-30 20:08:44,891 Finished 3 from 12 processes
22m 56s [bash] 2022-04-30 20:08:49,894 Finished 3 from 12 processes
23m 1s [bash] 2022-04-30 20:08:54,899 Finished 3 from 12 processes
23m 6s [bash] 2022-04-30 20:08:59,909 Finished 4 from 12 processes
23m 11s [bash] 2022-04-30 20:09:04,915 Finished 4 from 12 processes
23m 16s [bash] 2022-04-30 20:09:09,918 Finished 4 from 12 processes
23m 21s [bash] 2022-04-30 20:09:14,923 Finished 4 from 12 processes
23m 26s [bash] 2022-04-30 20:09:19,926 Finished 4 from 12 processes
23m 31s [bash] 2022-04-30 20:09:24,931 Finished 4 from 12 processes
23m 36s [bash] 2022-04-30 20:09:29,943 Finished 4 from 12 processes
23m 41s [bash] 2022-04-30 20:09:34,947 Finished 4 from 12 processes
23m 46s [bash] 2022-04-30 20:09:39,950 Finished 4 from 12 processes
23m 51s [bash] 2022-04-30 20:09:44,955 Finished 4 from 12 processes
23m 56s [bash] 2022-04-30 20:09:49,958 Finished 4 from 12 processes
24m 2s [bash] 2022-04-30 20:09:54,962 Finished 4 from 12 processes
24m 7s [bash] 2022-04-30 20:09:59,966 Finished 4 from 12 processes
24m 12s [bash] 2022-04-30 20:10:04,971 Finished 4 from 12 processes
24m 17s [bash] 2022-04-30 20:10:09,974 Finished 4 from 12 processes
24m 22s [bash] 2022-04-30 20:10:14,979 Finished 4 from 12 processes
24m 27s [bash] 2022-04-30 20:10:19,983 Finished 4 from 12 processes
24m 32s [bash] 2022-04-30 20:10:24,986 Finished 4 from 12 processes
24m 37s [bash] 2022-04-30 20:10:29,991 Finished 4 from 12 processes
24m 42s [bash] 2022-04-30 20:10:34,995 Finished 4 from 12 processes
24m 47s [bash] 2022-04-30 20:10:39,998 Finished 4 from 12 processes
24m 52s [bash] 2022-04-30 20:10:45,003 Finished 4 from 12 processes
24m 57s [bash] 2022-04-30 20:10:50,006 Finished 4 from 12 processes
25m 2s [bash] 2022-04-30 20:10:55,010 Finished 4 from 12 processes
25m 7s [bash] 2022-04-30 20:11:00,015 Finished 4 from 12 processes
25m 12s [bash] 2022-04-30 20:11:05,018 Finished 4 from 12 processes
25m 17s [bash] 2022-04-30 20:11:10,022 Finished 4 from 12 processes
25m 22s [bash] 2022-04-30 20:11:15,026 Finished 4 from 12 processes
25m 27s [bash] 2022-04-30 20:11:20,031 Finished 4 from 12 processes
25m 32s [bash] 2022-04-30 20:11:25,037 Finished 4 from 12 processes
25m 37s [bash] 2022-04-30 20:11:30,043 Finished 4 from 12 processes
25m 42s [bash] 2022-04-30 20:11:35,046 Finished 4 from 12 processes
25m 47s [bash] 2022-04-30 20:11:40,049 Finished 4 from 12 processes
25m 52s [bash] 2022-04-30 20:11:45,054 Finished 4 from 12 processes
25m 57s [bash] 2022-04-30 20:11:50,060 Finished 4 from 12 processes
26m 2s [bash] 2022-04-30 20:11:55,063 Finished 4 from 12 processes
26m 7s [bash] 2022-04-30 20:12:00,067 Finished 4 from 12 processes
26m 12s [bash] 2022-04-30 20:12:05,072 Finished 4 from 12 processes
26m 17s [bash] 2022-04-30 20:12:10,077 Finished 4 from 12 processes
26m 22s [bash] 2022-04-30 20:12:15,082 Finished 4 from 12 processes
26m 27s [bash] 2022-04-30 20:12:20,086 Finished 4 from 12 processes
26m 32s [bash] 2022-04-30 20:12:25,090 Finished 4 from 12 processes
26m 37s [bash] 2022-04-30 20:12:30,094 Finished 4 from 12 processes
26m 42s [bash] 2022-04-30 20:12:35,098 Finished 4 from 12 processes
26m 47s [bash] 2022-04-30 20:12:40,103 Finished 4 from 12 processes
26m 52s [bash] 2022-04-30 20:12:45,107 Finished 4 from 12 processes
26m 57s [bash] 2022-04-30 20:12:50,110 Finished 4 from 12 processes
27m 2s [bash] 2022-04-30 20:12:55,115 Finished 4 from 12 processes
27m 7s [bash] 2022-04-30 20:13:00,119 Finished 4 from 12 processes
27m 12s [bash] 2022-04-30 20:13:05,122 Finished 4 from 12 processes
27m 17s [bash] 2022-04-30 20:13:10,126 Finished 4 from 12 processes
27m 22s [bash] 2022-04-30 20:13:15,131 Finished 4 from 12 processes
27m 27s [bash] 2022-04-30 20:13:20,135 Finished 4 from 12 processes
27m 32s [bash] 2022-04-30 20:13:25,143 Finished 4 from 12 processes
27m 37s [bash] 2022-04-30 20:13:30,146 Finished 4 from 12 processes
27m 42s [bash] 2022-04-30 20:13:35,151 Finished 4 from 12 processes
27m 47s [bash] 2022-04-30 20:13:40,154 Finished 4 from 12 processes
27m 52s [bash] 2022-04-30 20:13:45,158 Finished 4 from 12 processes
27m 57s [bash] 2022-04-30 20:13:50,164 Finished 4 from 12 processes
28m 2s [bash] 2022-04-30 20:13:55,176 Finished 4 from 12 processes
28m 7s [bash] 2022-04-30 20:14:00,178 Finished 4 from 12 processes
28m 12s [bash] 2022-04-30 20:14:05,183 Finished 4 from 12 processes
28m 17s [bash] 2022-04-30 20:14:10,187 Finished 4 from 12 processes
28m 22s [bash] 2022-04-30 20:14:15,190 Finished 4 from 12 processes
28m 27s [bash] 2022-04-30 20:14:20,195 Finished 4 from 12 processes
28m 32s [bash] 2022-04-30 20:14:25,200 Finished 4 from 12 processes
28m 37s [bash] 2022-04-30 20:14:30,203 Finished 4 from 12 processes
28m 42s [bash] 2022-04-30 20:14:35,208 Finished 4 from 12 processes
28m 47s [bash] 2022-04-30 20:14:40,210 Finished 4 from 12 processes
28m 52s [bash] 2022-04-30 20:14:45,215 Finished 4 from 12 processes
28m 57s [bash] 2022-04-30 20:14:50,221 Finished 4 from 12 processes
29m 2s [bash] 2022-04-30 20:14:55,224 Finished 4 from 12 processes
29m 7s [bash] 2022-04-30 20:15:00,226 Finished 4 from 12 processes
29m 12s [bash] 2022-04-30 20:15:05,231 Finished 4 from 12 processes
29m 17s [bash] 2022-04-30 20:15:10,235 Finished 4 from 12 processes
29m 22s [bash] 2022-04-30 20:15:15,238 Finished 4 from 12 processes
29m 27s [bash] 2022-04-30 20:15:20,242 Finished 4 from 12 processes
29m 32s [bash] 2022-04-30 20:15:25,246 Finished 4 from 12 processes
29m 37s [bash] 2022-04-30 20:15:30,251 Finished 4 from 12 processes
29m 42s [bash] 2022-04-30 20:15:35,254 Finished 4 from 12 processes
29m 47s [bash] 2022-04-30 20:15:40,259 Finished 4 from 12 processes
29m 52s [bash] 2022-04-30 20:15:45,262 Finished 4 from 12 processes
29m 57s [bash] 2022-04-30 20:15:50,266 Finished 4 from 12 processes
30m 2s [bash] 2022-04-30 20:15:55,270 Finished 4 from 12 processes
30m 7s [bash] 2022-04-30 20:16:00,274 Finished 4 from 12 processes
30m 12s [bash] 2022-04-30 20:16:05,279 Finished 4 from 12 processes
30m 17s [bash] 2022-04-30 20:16:10,282 Finished 4 from 12 processes
30m 22s [bash] 2022-04-30 20:16:15,286 Finished 4 from 12 processes
30m 27s [bash] 2022-04-30 20:16:20,290 Finished 4 from 12 processes
30m 32s [bash] 2022-04-30 20:16:25,294 Finished 4 from 12 processes
30m 37s [bash] 2022-04-30 20:16:30,299 Finished 4 from 12 processes
30m 42s [bash] 2022-04-30 20:16:35,304 Finished 4 from 12 processes
30m 47s [bash] 2022-04-30 20:16:40,308 Finished 4 from 12 processes
30m 52s [bash] 2022-04-30 20:16:45,310 Finished 4 from 12 processes
30m 57s [bash] 2022-04-30 20:16:50,315 Finished 4 from 12 processes
31m 2s [bash] 2022-04-30 20:16:55,318 Finished 4 from 12 processes
31m 7s [bash] 2022-04-30 20:17:00,331 Finished 4 from 12 processes
31m 12s [bash] 2022-04-30 20:17:05,335 Finished 4 from 12 processes
31m 17s [bash] 2022-04-30 20:17:10,338 Finished 4 from 12 processes
31m 22s [bash] 2022-04-30 20:17:15,342 Finished 4 from 12 processes
31m 27s [bash] 2022-04-30 20:17:20,346 Finished 4 from 12 processes
31m 32s [bash] 2022-04-30 20:17:25,350 Finished 4 from 12 processes
31m 37s [bash] 2022-04-30 20:17:30,355 Finished 4 from 12 processes
31m 42s [bash] 2022-04-30 20:17:35,358 Finished 4 from 12 processes
31m 47s [bash] 2022-04-30 20:17:40,362 Finished 4 from 12 processes
31m 52s [bash] 2022-04-30 20:17:45,366 Finished 4 from 12 processes
31m 57s [bash] 2022-04-30 20:17:50,370 Finished 4 from 12 processes
32m 2s [bash] 2022-04-30 20:17:55,374 Finished 4 from 12 processes
32m 7s [bash] 2022-04-30 20:18:00,378 Finished 4 from 12 processes
32m 12s [bash] 2022-04-30 20:18:05,383 Finished 4 from 12 processes
32m 17s [bash] 2022-04-30 20:18:10,386 Finished 4 from 12 processes
32m 22s [bash] 2022-04-30 20:18:15,390 Finished 4 from 12 processes
32m 27s [bash] 2022-04-30 20:18:20,394 Finished 4 from 12 processes
32m 32s [bash] 2022-04-30 20:18:25,398 Finished 4 from 12 processes
32m 37s [bash] 2022-04-30 20:18:30,403 Finished 4 from 12 processes
32m 42s [bash] 2022-04-30 20:18:35,406 Finished 4 from 12 processes
32m 47s [bash] 2022-04-30 20:18:40,411 Finished 4 from 12 processes
32m 52s [bash] 2022-04-30 20:18:45,414 Finished 4 from 12 processes
32m 57s [bash] 2022-04-30 20:18:50,419 Finished 4 from 12 processes
33m 2s [bash] 2022-04-30 20:18:55,422 Finished 4 from 12 processes
33m 7s [bash] 2022-04-30 20:19:00,426 Finished 4 from 12 processes
33m 12s [bash] 2022-04-30 20:19:05,432 Finished 4 from 12 processes
33m 17s [bash] 2022-04-30 20:19:10,434 Finished 4 from 12 processes
33m 22s [bash] 2022-04-30 20:19:15,438 Finished 4 from 12 processes
33m 27s [bash] 2022-04-30 20:19:20,442 Finished 4 from 12 processes
33m 32s [bash] 2022-04-30 20:19:25,449 Finished 4 from 12 processes
33m 37s [bash] 2022-04-30 20:19:30,461 Finished 4 from 12 processes
33m 42s [bash] 2022-04-30 20:19:35,464 Finished 4 from 12 processes
33m 47s [bash] 2022-04-30 20:19:40,469 Finished 4 from 12 processes
33m 52s [bash] 2022-04-30 20:19:45,475 Finished 4 from 12 processes
33m 57s [bash] 2022-04-30 20:19:50,481 Finished 4 from 12 processes
34m 2s [bash] 2022-04-30 20:19:55,486 Finished 4 from 12 processes
34m 7s [bash] 2022-04-30 20:20:00,490 Finished 4 from 12 processes
34m 12s [bash] 2022-04-30 20:20:05,495 Finished 4 from 12 processes
34m 17s [bash] 2022-04-30 20:20:10,498 Finished 4 from 12 processes
34m 22s [bash] 2022-04-30 20:20:15,504 Finished 4 from 12 processes
34m 27s [bash] 2022-04-30 20:20:20,506 Finished 4 from 12 processes
34m 32s [bash] 2022-04-30 20:20:25,511 Finished 4 from 12 processes
34m 37s [bash] 2022-04-30 20:20:30,514 Finished 4 from 12 processes
34m 42s [bash] 2022-04-30 20:20:35,520 Finished 4 from 12 processes
34m 47s [bash] 2022-04-30 20:20:40,522 Finished 4 from 12 processes
34m 52s [bash] 2022-04-30 20:20:45,526 Finished 4 from 12 processes
34m 57s [bash] 2022-04-30 20:20:50,530 Finished 4 from 12 processes
35m 2s [bash] 2022-04-30 20:20:55,534 Finished 4 from 12 processes
35m 7s [bash] 2022-04-30 20:21:00,538 Finished 4 from 12 processes
35m 12s [bash] 2022-04-30 20:21:05,542 Finished 4 from 12 processes
35m 17s [bash] 2022-04-30 20:21:10,547 Finished 4 from 12 processes
35m 22s [bash] 2022-04-30 20:21:15,550 Finished 4 from 12 processes
35m 27s [bash] 2022-04-30 20:21:20,554 Finished 4 from 12 processes
35m 32s [bash] 2022-04-30 20:21:25,558 Finished 4 from 12 processes
35m 37s [bash] 2022-04-30 20:21:30,562 Finished 4 from 12 processes
35m 42s [bash] 2022-04-30 20:21:35,568 Finished 4 from 12 processes
35m 47s [bash] 2022-04-30 20:21:40,570 Finished 4 from 12 processes
35m 52s [bash] 2022-04-30 20:21:45,574 Finished 4 from 12 processes
35m 57s [bash] 2022-04-30 20:21:50,578 Finished 4 from 12 processes
36m 2s [bash] 2022-04-30 20:21:55,583 Finished 4 from 12 processes
36m 7s [bash] 2022-04-30 20:22:00,586 Finished 4 from 12 processes
36m 12s [bash] 2022-04-30 20:22:05,591 Finished 4 from 12 processes
36m 17s [bash] 2022-04-30 20:22:10,596 Finished 4 from 12 processes
36m 22s [bash] 2022-04-30 20:22:15,598 Finished 4 from 12 processes
36m 27s [bash] 2022-04-30 20:22:20,604 Finished 4 from 12 processes
36m 32s [bash] 2022-04-30 20:22:25,606 Finished 4 from 12 processes
36m 37s [bash] 2022-04-30 20:22:30,610 Finished 4 from 12 processes
36m 42s [bash] 2022-04-30 20:22:35,614 Finished 4 from 12 processes
36m 47s [bash] 2022-04-30 20:22:40,618 Finished 4 from 12 processes
36m 52s [bash] 2022-04-30 20:22:45,622 Finished 4 from 12 processes
36m 57s [bash] 2022-04-30 20:22:50,627 Finished 4 from 12 processes
37m 2s [bash] 2022-04-30 20:22:55,630 Finished 4 from 12 processes
37m 7s [bash] 2022-04-30 20:23:00,634 Finished 4 from 12 processes
37m 12s [bash] 2022-04-30 20:23:05,639 Finished 4 from 12 processes
37m 17s [bash] 2022-04-30 20:23:10,642 Finished 4 from 12 processes
37m 22s [bash] 2022-04-30 20:23:15,646 Finished 4 from 12 processes
37m 27s [bash] 2022-04-30 20:23:20,651 Finished 4 from 12 processes
37m 32s [bash] 2022-04-30 20:23:25,655 Finished 4 from 12 processes
37m 37s [bash] 2022-04-30 20:23:30,659 Finished 4 from 12 processes
37m 42s [bash] 2022-04-30 20:23:35,663 Finished 4 from 12 processes
37m 47s [bash] 2022-04-30 20:23:40,666 Finished 4 from 12 processes
37m 52s [bash] 2022-04-30 20:23:45,670 Finished 4 from 12 processes
37m 57s [bash] 2022-04-30 20:23:50,675 Finished 4 from 12 processes
38m 2s [bash] 2022-04-30 20:23:55,679 Finished 4 from 12 processes
38m 7s [bash] 2022-04-30 20:24:00,682 Finished 4 from 12 processes
38m 12s [bash] 2022-04-30 20:24:05,687 Finished 4 from 12 processes
38m 17s [bash] 2022-04-30 20:24:10,690 Finished 4 from 12 processes
38m 22s [bash] 2022-04-30 20:24:15,694 Finished 4 from 12 processes
38m 27s [bash] 2022-04-30 20:24:20,702 Finished 4 from 12 processes
38m 32s [bash] 2022-04-30 20:24:25,706 Finished 4 from 12 processes
38m 37s [bash] 2022-04-30 20:24:30,710 Finished 4 from 12 processes
38m 42s [bash] 2022-04-30 20:24:35,714 Finished 4 from 12 processes
38m 47s [bash] 2022-04-30 20:24:40,718 Finished 4 from 12 processes
38m 52s [bash] 2022-04-30 20:24:45,723 Finished 4 from 12 processes
38m 57s [bash] 2022-04-30 20:24:50,727 Finished 4 from 12 processes
39m 2s [bash] 2022-04-30 20:24:55,730 Finished 4 from 12 processes
39m 7s [bash] 2022-04-30 20:25:00,734 Finished 4 from 12 processes
39m 12s [bash] 2022-04-30 20:25:05,738 Finished 4 from 12 processes
39m 17s [bash] 2022-04-30 20:25:10,743 Finished 4 from 12 processes
39m 22s [bash] 2022-04-30 20:25:15,747 Finished 4 from 12 processes
39m 27s [bash] 2022-04-30 20:25:20,759 Finished 4 from 12 processes
39m 32s [bash] 2022-04-30 20:25:25,765 Finished 4 from 12 processes
39m 37s [bash] 2022-04-30 20:25:30,770 Finished 4 from 12 processes
39m 42s [bash] 2022-04-30 20:25:35,774 Finished 4 from 12 processes
39m 47s [bash] 2022-04-30 20:25:40,778 Finished 4 from 12 processes
39m 52s [bash] 2022-04-30 20:25:45,783 Finished 4 from 12 processes
39m 57s [bash] 2022-04-30 20:25:50,787 Finished 4 from 12 processes
40m 2s [bash] 2022-04-30 20:25:55,798 Finished 4 from 12 processes
40m 7s [bash] 2022-04-30 20:26:00,803 Finished 4 from 12 processes
40m 12s [bash] 2022-04-30 20:26:05,806 Finished 4 from 12 processes
40m 17s [bash] 2022-04-30 20:26:10,810 Finished 4 from 12 processes
40m 22s [bash] 2022-04-30 20:26:15,814 Finished 4 from 12 processes
40m 27s [bash] 2022-04-30 20:26:20,822 Finished 4 from 12 processes
40m 32s [bash] 2022-04-30 20:26:25,826 Finished 4 from 12 processes
40m 37s [bash] 2022-04-30 20:26:30,831 Finished 4 from 12 processes
40m 42s [bash] 2022-04-30 20:26:35,835 Finished 4 from 12 processes
40m 47s [bash] 2022-04-30 20:26:40,838 Finished 4 from 12 processes
40m 52s [bash] 2022-04-30 20:26:45,843 Finished 4 from 12 processes
40m 57s [bash] 2022-04-30 20:26:50,846 Finished 4 from 12 processes
41m 2s [bash] 2022-04-30 20:26:55,850 Finished 4 from 12 processes
41m 7s [bash] 2022-04-30 20:27:00,854 Finished 4 from 12 processes
41m 12s [bash] 2022-04-30 20:27:05,858 Finished 4 from 12 processes
41m 17s [bash] 2022-04-30 20:27:10,862 Finished 4 from 12 processes
41m 22s [bash] 2022-04-30 20:27:15,867 Finished 4 from 12 processes
41m 27s [bash] 2022-04-30 20:27:20,871 Finished 4 from 12 processes
41m 32s [bash] 2022-04-30 20:27:25,874 Finished 4 from 12 processes
41m 37s [bash] 2022-04-30 20:27:30,878 Finished 4 from 12 processes
41m 42s [bash] 2022-04-30 20:27:35,882 Finished 4 from 12 processes
41m 47s [bash] 2022-04-30 20:27:40,886 Finished 4 from 12 processes
41m 52s [bash] 2022-04-30 20:27:45,890 Finished 4 from 12 processes
41m 57s [bash] 2022-04-30 20:27:50,895 Finished 4 from 12 processes
42m 2s [bash] 2022-04-30 20:27:55,901 Finished 4 from 12 processes
42m 7s [bash] 2022-04-30 20:28:00,904 Finished 4 from 12 processes
42m 12s [bash] 2022-04-30 20:28:05,907 Finished 4 from 12 processes
42m 17s [bash] 2022-04-30 20:28:10,910 Finished 4 from 12 processes
42m 22s [bash] 2022-04-30 20:28:15,914 Finished 4 from 12 processes
42m 27s [bash] 2022-04-30 20:28:20,918 Finished 4 from 12 processes
42m 32s [bash] 2022-04-30 20:28:25,922 Finished 4 from 12 processes
42m 37s [bash] 2022-04-30 20:28:30,927 Finished 4 from 12 processes
42m 42s [bash] 2022-04-30 20:28:35,930 Finished 4 from 12 processes
42m 47s [bash] 2022-04-30 20:28:40,934 Finished 4 from 12 processes
42m 52s [bash] 2022-04-30 20:28:45,938 Finished 4 from 12 processes
42m 57s [bash] 2022-04-30 20:28:50,942 Finished 4 from 12 processes
43m 2s [bash] 2022-04-30 20:28:55,947 Finished 4 from 12 processes
43m 7s [bash] 2022-04-30 20:29:00,950 Finished 4 from 12 processes
43m 12s [bash] 2022-04-30 20:29:05,954 Finished 4 from 12 processes
43m 17s [bash] 2022-04-30 20:29:10,958 Finished 4 from 12 processes
43m 22s [bash] 2022-04-30 20:29:15,963 Finished 4 from 12 processes
43m 27s [bash] 2022-04-30 20:29:20,967 Finished 4 from 12 processes
43m 33s [bash] 2022-04-30 20:29:25,970 Finished 4 from 12 processes
43m 38s [bash] 2022-04-30 20:29:30,974 Finished 4 from 12 processes
43m 43s [bash] 2022-04-30 20:29:35,978 Finished 4 from 12 processes
43m 48s [bash] 2022-04-30 20:29:40,982 Finished 4 from 12 processes
43m 53s [bash] 2022-04-30 20:29:45,986 Finished 4 from 12 processes
43m 58s [bash] 2022-04-30 20:29:50,992 Finished 4 from 12 processes
44m 3s [bash] 2022-04-30 20:29:55,994 Finished 4 from 12 processes
44m 8s [bash] 2022-04-30 20:30:00,999 Finished 4 from 12 processes
44m 13s [bash] 2022-04-30 20:30:06,006 Finished 4 from 12 processes
44m 18s [bash] 2022-04-30 20:30:11,010 Finished 4 from 12 processes
44m 23s [bash] 2022-04-30 20:30:16,014 Finished 4 from 12 processes
44m 28s [bash] 2022-04-30 20:30:21,019 Finished 4 from 12 processes
44m 33s [bash] 2022-04-30 20:30:26,022 Finished 4 from 12 processes
44m 38s [bash] 2022-04-30 20:30:31,027 Finished 4 from 12 processes
44m 43s [bash] 2022-04-30 20:30:36,030 Finished 4 from 12 processes
44m 48s [bash] 2022-04-30 20:30:41,034 Finished 4 from 12 processes
44m 53s [bash] 2022-04-30 20:30:46,040 Finished 4 from 12 processes
44m 58s [bash] 2022-04-30 20:30:51,042 Finished 4 from 12 processes
45m 3s [bash] 2022-04-30 20:30:56,046 Finished 4 from 12 processes
45m 8s [bash] 2022-04-30 20:31:01,050 Finished 4 from 12 processes
45m 13s [bash] 2022-04-30 20:31:06,055 Finished 4 from 12 processes
45m 18s [bash] 2022-04-30 20:31:11,058 Finished 4 from 12 processes
45m 23s [bash] 2022-04-30 20:31:16,062 Finished 4 from 12 processes
45m 28s [bash] 2022-04-30 20:31:21,067 Finished 4 from 12 processes
45m 33s [bash] 2022-04-30 20:31:26,070 Finished 4 from 12 processes
45m 38s [bash] 2022-04-30 20:31:31,075 Finished 4 from 12 processes
45m 43s [bash] 2022-04-30 20:31:36,078 Finished 4 from 12 processes
45m 48s [bash] 2022-04-30 20:31:41,084 Finished 4 from 12 processes
45m 53s [bash] 2022-04-30 20:31:46,086 Finished 4 from 12 processes
45m 58s [bash] 2022-04-30 20:31:51,090 Finished 4 from 12 processes
46m 3s [bash] 2022-04-30 20:31:56,094 Finished 4 from 12 processes
46m 8s [bash] 2022-04-30 20:32:01,099 Finished 4 from 12 processes
46m 13s [bash] 2022-04-30 20:32:06,102 Finished 4 from 12 processes
46m 18s [bash] 2022-04-30 20:32:11,107 Finished 4 from 12 processes
46m 23s [bash] 2022-04-30 20:32:16,110 Finished 4 from 12 processes
46m 28s [bash] 2022-04-30 20:32:21,116 Finished 4 from 12 processes
46m 33s [bash] 2022-04-30 20:32:26,119 Finished 4 from 12 processes
46m 38s [bash] 2022-04-30 20:32:31,122 Finished 4 from 12 processes
46m 43s [bash] 2022-04-30 20:32:36,127 Finished 4 from 12 processes
46m 48s [bash] 2022-04-30 20:32:41,130 Finished 4 from 12 processes
46m 53s [bash] 2022-04-30 20:32:46,135 Finished 4 from 12 processes
46m 58s [bash] 2022-04-30 20:32:51,138 Finished 4 from 12 processes
47m 3s [bash] 2022-04-30 20:32:56,142 Finished 4 from 12 processes
47m 8s [bash] 2022-04-30 20:33:01,146 Finished 4 from 12 processes
47m 13s [bash] 2022-04-30 20:33:06,150 Finished 4 from 12 processes
47m 18s [bash] 2022-04-30 20:33:11,154 Finished 4 from 12 processes
47m 23s [bash] 2022-04-30 20:33:16,194 Finished 4 from 12 processes
47m 28s [bash] 2022-04-30 20:33:21,200 Finished 4 from 12 processes
47m 33s [bash] 2022-04-30 20:33:26,207 Finished 4 from 12 processes
47m 38s [bash] 2022-04-30 20:33:31,210 Finished 4 from 12 processes
47m 43s [bash] 2022-04-30 20:33:36,214 Finished 4 from 12 processes
47m 48s [bash] 2022-04-30 20:33:41,219 Finished 4 from 12 processes
47m 53s [bash] 2022-04-30 20:33:46,222 Finished 4 from 12 processes
47m 58s [bash] 2022-04-30 20:33:51,226 Finished 4 from 12 processes
48m 3s [bash] 2022-04-30 20:33:56,231 Finished 4 from 12 processes
48m 8s [bash] 2022-04-30 20:34:01,234 Finished 4 from 12 processes
48m 13s [bash] 2022-04-30 20:34:06,238 Finished 4 from 12 processes
48m 18s [bash] 2022-04-30 20:34:11,242 Finished 4 from 12 processes
48m 23s [bash] 2022-04-30 20:34:16,246 Finished 4 from 12 processes
48m 28s [bash] 2022-04-30 20:34:21,250 Finished 4 from 12 processes
48m 33s [bash] 2022-04-30 20:34:26,255 Finished 4 from 12 processes
48m 38s [bash] 2022-04-30 20:34:31,258 Finished 4 from 12 processes
48m 43s [bash] 2022-04-30 20:34:36,262 Finished 4 from 12 processes
48m 48s [bash] 2022-04-30 20:34:41,266 Finished 4 from 12 processes
48m 53s [bash] 2022-04-30 20:34:46,270 Finished 4 from 12 processes
48m 58s [bash] 2022-04-30 20:34:51,275 Finished 4 from 12 processes
49m 3s [bash] 2022-04-30 20:34:56,281 Finished 4 from 12 processes
49m 8s [bash] 2022-04-30 20:35:01,282 Finished 4 from 12 processes
49m 13s [bash] 2022-04-30 20:35:06,287 Finished 4 from 12 processes
49m 18s [bash] 2022-04-30 20:35:11,291 Finished 4 from 12 processes
49m 23s [bash] 2022-04-30 20:35:16,294 Finished 4 from 12 processes
49m 28s [bash] 2022-04-30 20:35:21,302 Finished 4 from 12 processes
49m 33s [bash] 2022-04-30 20:35:26,307 Finished 4 from 12 processes
49m 38s [bash] 2022-04-30 20:35:31,310 Finished 4 from 12 processes
49m 43s [bash] 2022-04-30 20:35:36,318 Finished 4 from 12 processes
49m 48s [bash] 2022-04-30 20:35:41,322 Finished 4 from 12 processes
49m 53s [bash] 2022-04-30 20:35:46,326 Finished 4 from 12 processes
49m 58s [bash] 2022-04-30 20:35:51,331 Finished 4 from 12 processes
50m 3s [bash] 2022-04-30 20:35:56,335 Finished 4 from 12 processes
50m 8s [bash] 2022-04-30 20:36:01,338 Finished 4 from 12 processes
50m 13s [bash] 2022-04-30 20:36:06,343 Finished 4 from 12 processes
50m 18s [bash] 2022-04-30 20:36:11,346 Finished 4 from 12 processes
50m 23s [bash] 2022-04-30 20:36:16,352 Finished 4 from 12 processes
50m 28s [bash] 2022-04-30 20:36:21,357 Finished 4 from 12 processes
50m 33s [bash] 2022-04-30 20:36:26,359 Finished 4 from 12 processes
50m 38s [bash] 2022-04-30 20:36:31,363 Finished 4 from 12 processes
50m 43s [bash] 2022-04-30 20:36:36,366 Finished 4 from 12 processes
50m 48s [bash] 2022-04-30 20:36:41,370 Finished 4 from 12 processes
50m 53s [bash] 2022-04-30 20:36:46,374 Finished 4 from 12 processes
50m 58s [bash] 2022-04-30 20:36:51,379 Finished 4 from 12 processes
51m 3s [bash] 2022-04-30 20:36:56,382 Finished 4 from 12 processes
51m 8s [bash] 2022-04-30 20:37:01,387 Finished 4 from 12 processes
51m 13s [bash] 2022-04-30 20:37:06,390 Finished 4 from 12 processes
51m 18s [bash] 2022-04-30 20:37:11,395 Finished 4 from 12 processes
51m 23s [bash] 2022-04-30 20:37:16,398 Finished 4 from 12 processes
51m 28s [bash] 2022-04-30 20:37:21,400 Finished 4 from 12 processes
51m 33s [bash] 2022-04-30 20:37:26,404 Finished 4 from 12 processes
51m 38s [bash] 2022-04-30 20:37:31,406 Finished 4 from 12 processes
51m 43s [bash] 2022-04-30 20:37:36,411 Finished 4 from 12 processes
51m 48s [bash] 2022-04-30 20:37:41,414 Finished 4 from 12 processes
51m 53s [bash] 2022-04-30 20:37:46,420 Finished 4 from 12 processes
51m 58s [bash] 2022-04-30 20:37:51,422 Finished 4 from 12 processes
52m 3s [bash] 2022-04-30 20:37:56,426 Finished 4 from 12 processes
52m 8s [bash] 2022-04-30 20:38:01,430 Finished 4 from 12 processes
52m 13s [bash] 2022-04-30 20:38:06,435 Finished 4 from 12 processes
52m 18s [bash] 2022-04-30 20:38:11,440 Finished 4 from 12 processes
52m 23s [bash] 2022-04-30 20:38:16,442 Finished 4 from 12 processes
52m 28s [bash] 2022-04-30 20:38:21,447 Finished 4 from 12 processes
52m 33s [bash] 2022-04-30 20:38:26,451 Finished 4 from 12 processes
52m 38s [bash] 2022-04-30 20:38:31,455 Finished 4 from 12 processes
52m 43s [bash] 2022-04-30 20:38:36,460 Finished 4 from 12 processes
52m 48s [bash] 2022-04-30 20:38:41,462 Finished 4 from 12 processes
52m 53s [bash] 2022-04-30 20:38:46,466 Finished 4 from 12 processes
52m 58s [bash] 2022-04-30 20:38:51,470 Finished 4 from 12 processes
53m 3s [bash] 2022-04-30 20:38:56,474 Finished 4 from 12 processes
53m 8s [bash] 2022-04-30 20:39:01,478 Finished 4 from 12 processes
53m 13s [bash] 2022-04-30 20:39:06,482 Finished 4 from 12 processes
53m 18s [bash] 2022-04-30 20:39:11,486 Finished 4 from 12 processes
53m 23s [bash] 2022-04-30 20:39:16,491 Finished 4 from 12 processes
53m 28s [bash] 2022-04-30 20:39:21,496 Finished 4 from 12 processes
53m 33s [bash] 2022-04-30 20:39:26,499 Finished 4 from 12 processes
53m 38s [bash] 2022-04-30 20:39:31,502 Finished 4 from 12 processes
53m 43s [bash] 2022-04-30 20:39:36,512 Finished 4 from 12 processes
53m 48s [bash] 2022-04-30 20:39:41,518 Finished 4 from 12 processes
53m 53s [bash] 2022-04-30 20:39:46,522 Finished 4 from 12 processes
53m 58s [bash] 2022-04-30 20:39:51,527 Finished 4 from 12 processes
54m 3s [bash] 2022-04-30 20:39:56,532 Finished 4 from 12 processes
54m 8s [bash] 2022-04-30 20:40:01,534 Finished 4 from 12 processes
54m 13s [bash] 2022-04-30 20:40:06,539 Finished 4 from 12 processes
54m 18s [bash] 2022-04-30 20:40:11,543 Finished 4 from 12 processes
54m 23s [bash] 2022-04-30 20:40:16,546 Finished 4 from 12 processes
54m 28s [bash] 2022-04-30 20:40:21,557 Finished 4 from 12 processes
54m 33s [bash] 2022-04-30 20:40:26,560 Finished 4 from 12 processes
54m 38s [bash] 2022-04-30 20:40:31,562 Finished 4 from 12 processes
54m 43s [bash] 2022-04-30 20:40:36,568 Finished 4 from 12 processes
54m 48s [bash] 2022-04-30 20:40:41,571 Finished 4 from 12 processes
54m 53s [bash] 2022-04-30 20:40:46,574 Finished 4 from 12 processes
54m 58s [bash] 2022-04-30 20:40:51,578 Finished 4 from 12 processes
55m 3s [bash] 2022-04-30 20:40:56,585 Finished 4 from 12 processes
55m 8s [bash] 2022-04-30 20:41:01,591 Finished 4 from 12 processes
55m 13s [bash] 2022-04-30 20:41:06,595 Finished 4 from 12 processes
55m 18s [bash] 2022-04-30 20:41:11,598 Finished 4 from 12 processes
55m 23s [bash] 2022-04-30 20:41:16,602 Finished 4 from 12 processes
55m 28s [bash] 2022-04-30 20:41:21,606 Finished 4 from 12 processes
55m 33s [bash] 2022-04-30 20:41:26,612 Finished 4 from 12 processes
55m 38s [bash] 2022-04-30 20:41:31,614 Finished 4 from 12 processes
55m 43s [bash] 2022-04-30 20:41:36,619 Finished 4 from 12 processes
55m 48s [bash] 2022-04-30 20:41:41,624 Finished 4 from 12 processes
55m 53s [bash] 2022-04-30 20:41:46,626 Finished 4 from 12 processes
55m 58s [bash] 2022-04-30 20:41:51,631 Finished 4 from 12 processes
56m 3s [bash] 2022-04-30 20:41:56,635 Finished 4 from 12 processes
56m 8s [bash] 2022-04-30 20:42:01,654 Finished 4 from 12 processes
56m 13s [bash] 2022-04-30 20:42:06,658 Finished 4 from 12 processes
56m 18s [bash] 2022-04-30 20:42:11,662 Finished 4 from 12 processes
56m 23s [bash] 2022-04-30 20:42:16,666 Finished 4 from 12 processes
56m 28s [bash] 2022-04-30 20:42:21,670 Finished 4 from 12 processes
56m 33s [bash] 2022-04-30 20:42:26,674 Finished 4 from 12 processes
56m 38s [bash] 2022-04-30 20:42:31,690 Finished 4 from 12 processes
56m 43s [bash] 2022-04-30 20:42:36,694 Finished 4 from 12 processes
56m 48s [bash] 2022-04-30 20:42:41,698 Finished 4 from 12 processes
56m 53s [bash] 2022-04-30 20:42:46,704 Finished 4 from 12 processes
56m 58s [bash] 2022-04-30 20:42:51,707 Finished 4 from 12 processes
57m 3s [bash] 2022-04-30 20:42:56,711 Finished 4 from 12 processes
57m 8s [bash] 2022-04-30 20:43:01,719 Finished 4 from 12 processes
57m 13s [bash] 2022-04-30 20:43:06,723 Finished 4 from 12 processes
57m 18s [bash] 2022-04-30 20:43:11,726 Finished 4 from 12 processes
57m 23s [bash] 2022-04-30 20:43:16,730 Finished 4 from 12 processes
57m 28s [bash] 2022-04-30 20:43:21,734 Finished 4 from 12 processes
57m 33s [bash] 2022-04-30 20:43:26,739 Finished 4 from 12 processes
57m 38s [bash] 2022-04-30 20:43:31,742 Finished 4 from 12 processes
57m 43s [bash] 2022-04-30 20:43:36,747 Finished 4 from 12 processes
57m 48s [bash] 2022-04-30 20:43:41,753 Finished 4 from 12 processes
57m 53s [bash] 2022-04-30 20:43:46,758 Finished 4 from 12 processes
57m 58s [bash] 2022-04-30 20:43:51,762 Finished 4 from 12 processes
58m 3s [bash] 2022-04-30 20:43:56,766 Finished 4 from 12 processes
58m 8s [bash] 2022-04-30 20:44:01,770 Finished 4 from 12 processes
58m 13s [bash] 2022-04-30 20:44:06,775 Finished 4 from 12 processes
58m 18s [bash] 2022-04-30 20:44:11,778 Finished 4 from 12 processes
58m 24s [bash] 2022-04-30 20:44:16,784 Finished 4 from 12 processes
58m 28s [bash] 2022-04-30 20:44:21,789 Finished 4 from 12 processes
58m 33s [bash] 2022-04-30 20:44:26,806 Finished 4 from 12 processes
58m 38s [bash] 2022-04-30 20:44:31,815 Finished 4 from 12 processes
58m 43s [bash] 2022-04-30 20:44:36,818 Finished 4 from 12 processes
58m 48s [bash] 2022-04-30 20:44:41,822 Finished 4 from 12 processes
58m 53s [bash] 2022-04-30 20:44:46,826 Finished 4 from 12 processes
58m 58s [bash] 2022-04-30 20:44:51,830 Finished 4 from 12 processes
59m 3s [bash] 2022-04-30 20:44:56,834 Finished 4 from 12 processes
59m 8s [bash] 2022-04-30 20:45:01,839 Finished 4 from 12 processes
59m 13s [bash] 2022-04-30 20:45:06,842 Finished 4 from 12 processes
59m 18s [bash] 2022-04-30 20:45:11,846 Finished 4 from 12 processes
59m 23s [bash] 2022-04-30 20:45:16,850 Finished 4 from 12 processes
59m 28s [bash] 2022-04-30 20:45:21,855 Finished 4 from 12 processes
59m 34s [bash] 2022-04-30 20:45:26,858 Finished 4 from 12 processes
59m 39s [bash] 2022-04-30 20:45:31,859 Finished 4 from 12 processes
59m 44s [bash] 2022-04-30 20:45:36,866 Finished 4 from 12 processes
59m 49s [bash] 2022-04-30 20:45:41,874 Finished 4 from 12 processes
59m 53s [bash] 2022-04-30 20:45:46,878 Finished 4 from 12 processes
59m 58s [bash] 2022-04-30 20:45:51,883 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:45:56,890 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:01,894 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:06,898 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:11,902 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:16,906 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:21,911 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:26,917 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:31,918 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:36,923 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:41,927 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:46,930 Finished 4 from 12 processes
1h 0m [bash] 2022-04-30 20:46:51,934 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:46:56,938 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:01,940 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:06,942 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:11,948 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:16,951 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:21,960 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:26,963 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:31,968 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:36,978 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:41,983 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:46,987 Finished 4 from 12 processes
1h 1m [bash] 2022-04-30 20:47:51,991 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:47:56,994 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:01,998 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:07,004 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:12,006 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:17,010 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:22,019 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:27,022 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:32,026 Finished 4 from 12 processes
1h 2m [bash] 2022-04-30 20:48:37,032 Finished 6 from 12 processes
1h 2m [bash] 2022-04-30 20:48:42,038 Finished 6 from 12 processes
1h 2m [bash] 2022-04-30 20:48:47,042 Finished 7 from 12 processes
1h 2m [bash] 2022-04-30 20:48:52,046 Finished 7 from 12 processes
1h 3m [bash] 2022-04-30 20:48:57,051 Finished 8 from 12 processes
1h 3m [bash] 2022-04-30 20:49:02,055 Finished 8 from 12 processes
1h 3m [bash] 2022-04-30 20:49:07,058 Finished 8 from 12 processes
1h 3m [bash] 2022-04-30 20:49:12,063 Finished 8 from 12 processes
1h 3m [bash] 2022-04-30 20:49:17,066 Finished 8 from 12 processes
1h 3m [bash] 2022-04-30 20:49:22,070 Finished 8 from 12 processes
1h 3m [bash] 2022-04-30 20:49:27,074 Finished 9 from 12 processes
1h 3m [bash] 2022-04-30 20:49:32,080 Finished 9 from 12 processes
1h 3m [bash] 2022-04-30 20:49:37,082 Finished 9 from 12 processes
1h 3m [bash] 2022-04-30 20:49:42,088 All processes finished
1h 3m [bash] 2022-04-30 20:49:42,088 Compressing stress logs
1h 3m [bash] 2022-04-30 20:49:43,974 Logs compressed
1h 3m [bash] 2022-04-30 20:49:43,975 Will terminate gdb (if any)
1h 3m [bash] /bin/sh: 1: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
1h 3m [bash] kill -l [exitstatus]
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test1601_detach_permanently_lazy/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01014_concurrent_test_1/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01014_concurrent_test_7/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01294_concurrent_test_3/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01294_concurrent_test_hiu17h/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01014_concurrent_test_9/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01294_concurrent_test_1/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01014_concurrent_test_wdgrq2/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_lazy_01294_concurrent_test_mav0x1/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 3m [bash] Received exception from server (version 21.8.15):
1h 3m [bash] Code: 219. DB::Exception: Received from localhost:9000. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty [/var/lib/clickhouse/data/test_7/]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting.
1h 8m [bash] 2022-04-30 20:54:49,146 Checking if some queries hung
1h 8m [bash] Using queries from '/usr/share/clickhouse-test/queries' directory
1h 8m [bash] Using clickhouse-client as client program (expecting split build)
1h 8m [bash] Connecting to ClickHouse server... OK
1h 8m [bash]
1h 8m [bash] Running 1 stateless tests (MainProcess).
1h 8m [bash]
1h 8m [bash] 00001_select_1: [ OK ]
1h 8m [bash]
1h 8m [bash] 1 tests passed. 0 tests skipped. 0.14 s elapsed (MainProcess).
1h 8m [bash] Won't run stateful tests because test data wasn't loaded.
1h 8m [bash]
1h 8m [bash] No queries hung.
1h 8m [bash] All tests have finished.
1h 8m [bash] 2022-04-30 20:54:49,883 Stress test finished
1h 8m [bash] + echo -e 'Test script exit code\tOK'
1h 8m [bash] + stop
1h 8m [bash] + clickhouse stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Sent terminate signal to process with pid 383.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] /var/run/clickhouse-server/clickhouse-server.pid file exists and contains pid = 383.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] Found pid = 383 in the list of running processes.
1h 9m [bash] The process with pid = 383 is running.
1h 9m [bash] Waiting for server to stop
1h 9m [bash] Now there is no clickhouse-server process.
1h 9m [bash] Server stopped
1h 9m [bash] + start
1h 9m [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log ']'
1h 9m [bash] + log_file_counter=1
1h 9m [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log.1 ']'
1h 9m [bash] + log_file_counter=2
1h 9m [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log.2 ']'
1h 9m [bash] + mv /var/log/clickhouse-server/clickhouse-server.log /var/log/clickhouse-server/clickhouse-server.log.2
1h 9m [bash] + counter=0
1h 9m [bash] + clickhouse-client --query 'SELECT 1'
1h 9m [bash] Code: 210. DB::NetException: Connection refused (localhost:9000)
1h 9m [bash]
1h 9m [bash] + '[' 0 -gt 120 ']'
1h 9m [bash] + clickhouse start --user root
1h 9m [bash] + sleep 0.5
1h 9m [bash] + counter=1
1h 9m [bash] + clickhouse-client --query 'SELECT 1'
1h 9m [bash] Code: 210. DB::NetException: Connection refused (localhost:9000)
1h 9m [bash]
1h 9m [bash] + '[' 1 -gt 120 ']'
1h 9m [bash] + clickhouse start --user root
1h 9m [bash] + sleep 0.5
1h 9m [bash] + counter=2
1h 9m [bash] + clickhouse-client --query 'SELECT 1'
1h 9m [bash] Code: 210. DB::NetException: Connection refused (localhost:9000)
1h 9m [bash]
1h 9m [bash] + '[' 2 -gt 120 ']'
1h 9m [bash] + clickhouse start --user root
1h 9m [bash] + sleep 0.5
1h 9m [bash] + counter=3
1h 9m [bash] + clickhouse-client --query 'SELECT 1'
1h 9m [bash] 1
1h 9m [bash] + echo '
1h 9m [bash] set follow-fork-mode child
1h 9m [bash] handle all noprint
1h 9m [bash] handle SIGSEGV stop print
1h 9m [bash] handle SIGBUS stop print
1h 9m [bash] handle SIGABRT stop print
1h 9m [bash] continue
1h 9m [bash] thread apply all backtrace
1h 9m [bash] detach
1h 9m [bash] quit
1h 9m [bash] '
1h 9m [bash] + clickhouse-client --query 'SELECT '\''Server successfully started'\'', '\''OK'\'''
1h 9m [bash] ++ cat /var/run/clickhouse-server/clickhouse-server.pid
1h 9m [bash] + gdb -batch -command script.gdb -p 766627
1h 10m [bash] + '[' -f /var/log/clickhouse-server/clickhouse-server.log ']'
1h 10m [bash] + '[' -f /var/log/clickhouse-server/stderr.log ']'
1h 10m [bash] + zgrep -Fa ' ' /var/log/clickhouse-server/clickhouse-server.log
1h 10m [bash] + zgrep -Fa ================== /var/log/clickhouse-server/stderr.log
1h 10m [bash] + zgrep -Fa WARNING /var/log/clickhouse-server/stderr.log
1h 10m [bash] + zgrep -Fav 'ASan doesn'\''t fully support makecontext/swapcontext functions'
1h 10m [bash] + echo -e 'No sanitizer asserts\tOK'
1h 10m [bash] + rm -f /test_output/tmp
1h 10m [bash] + zgrep -Fa ' Application: Child process was terminated by signal 9' /var/log/clickhouse-server/clickhouse-server.log
1h 10m [bash] + echo -e 'No OOM messages in clickhouse-server.log\tOK'
1h 10m [bash] + zgrep -Fa 'Code: 49, e.displayText() = DB::Exception:' /var/log/clickhouse-server/clickhouse-server.log
1h 10m [bash] + echo -e 'No logical errors\tOK'
1h 10m [bash] + zgrep -Fa '########################################' /var/log/clickhouse-server/clickhouse-server.log
1h 10m [bash] + echo -e 'Not crashed\tOK'
1h 10m [bash] + zgrep -Fa ' ' /var/log/clickhouse-server/clickhouse-server.log
1h 10m [bash] + echo -e 'No fatal messages in clickhouse-server.log\tOK'
1h 10m [bash] + zgrep -Fa '########################################' /test_output/gdb.log /test_output/stress_run_logs.tar.gz /test_output/test_results.tsv
1h 10m [bash] + for log_file in /var/log/clickhouse-server/clickhouse-server.log*
1h 10m [bash] + pigz
1h 10m [bash] ++ basename /var/log/clickhouse-server/clickhouse-server.log
1h 10m [bash] + for log_file in /var/log/clickhouse-server/clickhouse-server.log*
1h 10m [bash] + pigz
1h 10m [bash] ++ basename /var/log/clickhouse-server/clickhouse-server.log.1
1h 10m [bash] + for log_file in /var/log/clickhouse-server/clickhouse-server.log*
1h 10m [bash] + pigz
1h 10m [bash] ++ basename /var/log/clickhouse-server/clickhouse-server.log.2
1h 11m [bash] + tar -chf /test_output/coordination.tar /var/lib/clickhouse/coordination
1h 11m [bash] tar: Removing leading `/' from member names
1h 11m [bash] tar: Removing leading `/' from hard link targets
1h 11m [bash] + mv /var/log/clickhouse-server/stderr.log /test_output/
1h 11m [bash] + tar -chf /test_output/query_log_dump.tar /var/lib/clickhouse/data/system/query_log
1h 11m [bash] tar: Removing leading `/' from member names
1h 11m [bash] tar: Removing leading `/' from hard link targets
1h 11m [bash] + tar -chf /test_output/trace_log_dump.tar /var/lib/clickhouse/data/system/trace_log
1h 11m [bash] tar: Removing leading `/' from member names
1h 11m [bash] tar: Removing leading `/' from hard link targets
1h 11m [bash] + clickhouse-local --structure 'test String, res String' -q 'SELECT '\''failure'\'', test FROM table WHERE res != '\''OK'\'' order by (lower(test) like '\''%hung%'\'') LIMIT 1'
1h 11m [bash] + '[' -s /test_output/check_status.tsv ']'
1h 11m [bash] + echo -e 'success\tNo errors found'
1h 11m [bash]
1h 11m [bash] bash# echo $?
1h 11m [bash] 0
1h 11m [bash] bash#
Apr 30,2022 17:56:59 ⟥ Test Stress results
592us Processing file: /home/builder/builds/altinity-qa/clickhouse/cicd/release/tests/stress/test_output/test_results.tsv
787us ⟥⟤ OK Stress results, /stress/Stress results
1h 11m ⟥⟤ OK stress, /stress
Passing
✔ [ OK ] /stress/Stress results
✔ [ OK ] /stress
1 module (1 ok)
1 test (1 ok)
Total time 1h 11m
Executed on Apr 30,2022 16:45
TestFlows.com Open-Source Software Testing Framework v1.8.220312.1230154