Re: [squid-users] externalAclLookup: 'memberof' queue overload (ch=0x7f7d873b8358)

On 26/08/2013 10:49 a.m., glenn.groves wrote:
> Hi All,
>
> Does anyone have suggestion to fix the error permanently? (I restarted squid which has fixed it right now - but this is not a permanently fix)
>
> This morning we had a problem with our squid proxy, it would not accept logons from users, the error in the cache.log was:
>
> externalAclLookup: 'memberof' queue overload (ch=0x7f7d873b8358)
>
>
> The above message would repeat about 10 times, then eventually is would seem to authenticate (but still continue to prompt users for a logon – and should not be prompting at all as the users are using Kerberos):

Two things to note here, the authenticate part is done by your
auth_param helper(s). The external ACL helper which is overloading is
"just" a authorization check based on the already authenticated credentials.

Secondly you have up to 6 lookups sent to the helper per-request. That
will fill the helepr queue quite fast. If you can reduce that at all it
will help. Otherwise you need to allocate enough helpers that the queue
does not overflow.

> 2013/08/26 07:13:48| externalAclLookup: 'memberof' queue overload (ch=0x7f7d873b8358)
> 2013/08/26 07:13:48| squid_kerb_auth: DEBUG: ←lots of code→== user_at_DOMAIN.COM.AU
> 2013/08/26 07:13:48| squid_kerb_auth: INFO: User user_at_DOMAIN.COM.AU authenticated
>
> We only have about 10 users on squid right now and was about to rollout further, I had previously increased the negotiate children to 50 to handle our future 500 users, we are running squid 3.1.10 on centos 6.4, below is the squid.conf:

Maye you should bump the external_acl_type "memberof" helper children up
as well to match.

>
> ### /etc/squid/squid.conf Configuration File ####
>
> ### cache manager
> cache_mgr helpdesk_at_domain.com.au
>
> ### negotiate kerberos and ntlm authentication
> auth_param negotiate program /usr/local/bin/negotiate_wrapper -d --ntlm /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=DOMAIN --kerberos /usr/lib64/squid/squid_kerb_auth -i -d -s HTTP/proxy.domain.com.au
> auth_param negotiate children 50
> auth_param negotiate keep_alive off
>
> ### pure ntlm authentication
> auth_param ntlm program /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=DOMAIN
> auth_param ntlm children 200
> auth_param ntlm keep_alive off
>
> ### provide basic authentication via ldap for clients not authenticated via kerberos/ntlm
> auth_param basic program /usr/lib64/squid/squid_ldap_auth -R -b "dc=domain,dc=com,dc=au" -D squid_at_domain.com.au -W /etc/squid/ldappass.txt -f sAMAccountName=%s -h dc1.domain.com.au
> auth_param basic children 100
> auth_param basic realm Internet Proxy
> auth_param basic credentialsttl 1 minute
>
> ### ldap authorisation
> external_acl_type memberof %LOGIN /usr/lib64/squid/squid_ldap_group -R -K -b "dc=domain,dc=com,dc=au" -D squid_at_domain.com.au -W /etc/squid/ldappass.txt -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=%g, ou=Internet,ou=Domain Groups,ou=Domain,dc=domain,dc=com,dc=au))" -h dc1.domain.com.au
>
> ### Squid Cache Manager
> cachemgr_passwd none info
>
> cache_dir aufs /var/spool/squid 30000 16 256
> minimum_object_size 2 KB
> maximum_object_size 10 MB
> cache_swap_low 95
> cache_swap_high 97
>
> # aclname acltype typename activedirectorygroup
> acl SSL method CONNECT
> acl allowedsites dstdomain "/etc/squid/allowedsites.txt"
> acl auth proxy_auth REQUIRED
> acl BlockedAccess external memberof "/etc/squid/blocked_access.txt"
> acl RestrictedAccess external memberof "/etc/squid/restricted_access.txt"
> acl StandardAccess external memberof "/etc/squid/standard_access.txt"
> acl ExceptionAccess external memberof "/etc/squid/exception_access.txt"
> acl FullAccess external memberof "/etc/squid/full_access.txt"
> acl AnonymousAccess external memberof "/etc/squid/anonymous_access.txt"
> acl blockedsites dstdomain "/etc/squid/blockedsites.txt"
> acl exceptedsites dstdomain "/etc/squid/exceptedsites.txt"
> acl prioritysites dstdomain "/etc/squid/prioritysites.txt"
>
> ### squid defaults
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32 ::1
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> acl cacheadmin src 192.168.11.221 192.168.8.175
>
>
> ### http_access rules
> http_access allow manager localhost
> http_access allow manager cacheadmin
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost
> http_access allow prioritysites
> http_access deny BlockedAccess all
> http_access allow allowedsites
> http_access deny RestrictedAccess all
> http_access allow AnonymousAccess auth
> http_access allow FullAccess auth
> http_access allow ExceptionAccess exceptedsites auth

To reduce at least one helepr lookup per request swap this line around
to be:
   http_access allow exceptedsites ExceptionAccess auth

> http_access deny blockedsites
> http_access allow StandardAccess auth
> http_access allow auth
> http_access deny !auth

This "deny !auth" line should be above the "allow AnonymousAccess auth"
... although why you would call it anonymous access while requiring them
to also login at the same time (auth ACL test) is beyond me.

> http_access deny all
>
> ### logging
> access_log /var/log/squid/access.log squid
>
> ### Set memory manually, to allow it to use more of the system
> cache_mem 1024 MB
>
> ### squid defaults
> http_port 8080
> hierarchy_stoplist cgi-bin ?
> coredump_dir /var/spool/squid
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
>
> Thanks,
>
> Glenn

Amos
Received on Mon Aug 26 2013 - 11:05:36 MDT