Re: [squid-users] Ubuntu Server 13.10. Squid 3.3.8. WARNING: external ACL 'memberof' queue overload

On 2013-11-13 09:19, Andrey ‪ wrote:
> On 11/12/2013 06:33 PM, Andrey ‪ wrote:
>> Hi everyone
>>
>> During configuration of LDAP basic and group authentication methods by
>> Squid, a came across this error (/var/log/squid3/cache.log):
>>
>>
>>
>> Code:
>> WARNING: external ACL 'memberof' queue overload. Request rejected
>> 'administrator InternetAccess'.For basic authentication I use
>> following
>> piece of code:
>>

What is going on is exactly what the warning states. Your external ACL
helper is being overloaded with traffic.

>> Code:
>> auth_param basic program /usr/lib/squid3/basic_ldap_auth -P -R -u cn
>> -b "cn=Users,dc=dot,dc=lan" ubuntu.dot.lan
>> auth_param basic realm ubuntu.dot.lanThe test shows:
>>
>> Administrator Pa77w0rd
>>
>> OK.
>>
>> For LDAP groups I use this:
>>
>>
>>
>> Code:
>> external_acl_type memberof %LOGIN /usr/lib/squid3/ext_ldap_group_acl
>> -P -R -K -b "dc=dot,dc=lan" -f
>> "(&(cn=%v)(memberOf=cn=%a,cn=Users,dc=dot,dc=lan))" -D
>> nslcd-service_at_dot.lan -w "Pa77w0rd" -h ubuntu.dot.lan
>> The test shows:
>>
>> Administrator InternetAccess
>>
>> OK
>>
>>
>> My ACL list has following rules:
>>
>>
>> Code:
<snip>
>> acl LDAP_Auth proxy_auth REQUIRED
>> acl ClientNet src 192.168.1.135
>> acl Block_site url_regex -i fb vk youtube
>> acl InetAccess external memberof InternetAccess
>>
<snip>
>> http_access allow InetAccess
>> http_access deny !LDAP_Auth
>> http_access allow ClientNet
>> http_access deny all
>>
>> Where is the problem? How to solve it?
>>

The big visible problem here is that group is being checked before
authentication. If the user is not already authenticated external ACL
has to trigger that authentication and wait for it before even starting
the group lookup. The request is queued the entire time that waiting is
happening - and yoru queue is overflowing.

You can re-order the lines so that group check is done after login
authentication.

    http_access deny !LDAP_Auth
    http_access allow InetAccess
    http_access allow ClientNet
    http_access deny all

... after which it becomes clear that you can speed up performance even
further for some user(s) by allowing the ClientNet through before
checking the group type (since theya re allowed through even if their
group is not InetAccess).

    http_access deny !LDAP_Auth
    http_access allow ClientNet
    http_access allow InetAccess
    http_access deny all

This should halve the load on the external ACL helper, and greatly
reduce the time each request spends in the queue.

If you still get these warnings, or if they shift to happening on the
authenticator you can increase the children parameter of the helper with
queue overload. That runs more sub-processes for handling the traffic
load.

Amos
Received on Tue Nov 12 2013 - 23:03:22 MST