RE: [squid-users] squid 2.5s5, all ntlm_auth helpers stuck in reserved state

Hi,

I have already encountered it a couple of times. In my case, it appears during some period
of heavy traffic (a lot of ntlm authentication) coupled with a slow down of the DC answer
speed (the helper avg service time grows). The concerned helpers remain in state AR during
some time and then go in ARS. I'm using 200 helpers (I have nearly 1000 users on this proxy).
As you, I remarked that it as nothing to do with the number of requests handled, as it
usually at approximatively the same period, independently of the number of requests handled
till then.

        Pierre-Emmanuel

-----Message d'origine-----
De : James Wilkinson [mailto:jwilkinson@stbernard.com]
Envoyé : jeudi 20 mai 2004 20:29
À : squid-users@squid-cache.org
Objet : [squid-users] squid 2.5s5, all ntlm_auth helpers stuck in
reserved state

Hello Squidlist,
We are running squid 2.5s5 with the Samba 3.0.2 provided
ntlm_auth authenticator, no challenge reuse. We are seeing some
odd behavior, after some random number of requests, cachemgr shows all
helpers stuck in the "A R" reserved state, and they never recover.

Performing a 'squid -k reconfigure' launches new helpers, and flags
the current jammed helpers with the S shutdown flag, but the since
the jammed helpers are already reserved it doesn't really reap them,
leading to cachemgr outputs like this one after about 500k NTLM auth
requests:
-----------------------------------------------------
# FD PID # Requests Flags Time Offset Request
1 12 38234 424340 A RS 11090.098 0 (none)
2 13 38238 33772 A RS 83745.981 0 (none)
3 14 38239 10583 A RS 11090.096 0 (none)
4 15 38241 3583 A RS 11090.102 0 (none)
1 16 11567 4486 A 0.558 0 (none)
2 17 11570 965 A 0.958 0 (none)
3 18 11573 360 A 124.876 0 (none)
4 19 11576 154 A 124.869 0 (none)
5 20 11578 98 A 2073.044 0 (none)

In this case, we are running five helpers, and when the reconfigure
was called 4 were jammed, so the 5th was reaped properly and 5 new
helpers were spawned.

Has anyone else seen this sort of behavior? We suspect it might
be a user-agent that is mangling the NTLM exchange and dropping off
after getting to the challenge phase, but cannot reproduce it easily.
We have seen it happen as soon as after about 50k requests, or as
many as nearly 1,000K requests.

We are running on FreeBSD, and figure the problem must be occuring
right after the challenge phase when the helper is still reserved but
waiting on client ntlm AUTH response, or after the client AUTH response
when squid should release the reserved helper.

So has anyone else seen anything like this? The system is running very
smoothly and as expected otherwise.

Thanks in advance,

-James Wilkinson
jwilkinson@stbernard.com
Received on Fri May 21 2004 - 01:19:13 MDT