Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis sessions, when set with the persistent flag and echo_check_liveness, generate an enormous number of echo calls #2415

Open
2 tasks done
bitactive opened this issue Nov 3, 2023 · 0 comments

Comments

@bitactive
Copy link
Contributor

bitactive commented Nov 3, 2023

Expected behaviour

When phpredis used as session handler it should ECHO similar amount of times as GET/SETEX on each server.

Actual behaviour

One of 12 redis instances got 10x more ECHO commands than others. Disabling echo_check_liveness fixes the problem.

Steps to reproduce, backtrace or example script

I have redis session handler to multiple redis instances using following string:

session.save_path = "tcp://10.0.1.11:7010?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.11:7011?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.11:7012?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.11:7013?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.11:7014?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.11:7015?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.12:7010?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.12:7011?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.12:7012?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.12:7013?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.12:7014?weight=1&timeout=0.2&persistent=1&read_timeout=0.5,tcp://10.0.1.12:7015?weight=1&timeout=0.2&persistent=1&read_timeout=0.5"

But one instance (10.0.1.11:7013) is getting ~5x higher load then other instances. I have run redis monitor on each instance for 5 seconds and check amount of GET/SETEX/ECHO commands.

Each instance has similar amount of GET/SETEX/ECHO:

# redis-cli -h 10.0.1.11 -p 7012 monitor > redismon
# cat redismon | grep GET | wc
  19371   98177 3416606
# cat redismon | grep SETEX | wc
  19340  203899 83526191
# cat redismon | grep ECHO | wc
  19327   96635 1526833

Except one: 10.0.1.11:7013

# redis-cli -h 10.0.1.11 -p 7013 monitor > redismon
# cat redismon | grep SETEX | wc
  16345  175717 65224193
# cat redismon | grep GET | wc
  16356   81820 1646265
# cat redismon | grep ECHO | wc
  187111  935555 14781769

for some unknown reason, this instance is getting 10x more ECHO commands than SETEX/GET resulting in 5x higher total load.
Setting
redis.pconnect.echo_check_liveness = 0
fixes the problem, so the issue is for sure in persistent liveness check. But why only one of 12 instances got 10x more ECHO commands?

I'm seeing this behaviour on

  • OS: RHEL9
  • Redis: 6.2.7-1.el9
  • PHP: 8.2.12-1.el9.remi
  • phpredis: php-pecl-redis5-5.3.7-2.el9.remi

Steps to reproduce, backtrace or example script

I've checked

  • There is no similar issue from other users
  • Issue isn't fixed in develop branch
@bitactive bitactive changed the title Redis sessions, when set with the persistent flag and 'echo_check_liveness,' generate an enormous number of echo calls Redis sessions, when set with the persistent flag and 'echo_check_liveness', generate an enormous number of echo calls Nov 3, 2023
@bitactive bitactive changed the title Redis sessions, when set with the persistent flag and 'echo_check_liveness', generate an enormous number of echo calls Redis sessions, when set with the persistent flag and echo_check_liveness, generate an enormous number of echo calls Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant