Re: [squid-users] Ubuntu Server 13.10. Squid 3.3.8. WARNING: external ACL 'memberof' queue overload

Notes iside,

On 11/13/2013 11:42 PM, Andrey ‪ wrote:
> As far as I know the version 3.2 squid uses the IPv6 by default, except
> if you do by building –disable-ipv6.
By squid... but the OS should now support ipv6 by default to allow the
basic option of progress but most distributions give you the option to
blacklist this module at startup while a removal of this module can
leave you with you mouth like this
<0-0>
   ^
  0-0
O O
  0-0

And with a "kernel-panic" and lots of text.
(this is what it was back then but now it should not get into that)

> Before 3.2 you need do a trick like it is described here:
> http://www.squid-cache.org/Doc/config/tcp_outgoing_address/
> To enable IPv6 support.
Not that I know of.
>
> I use 3.3.8 from ubuntu official repository:
> http://packages.ubuntu.com/saucy/squid3
> where IPv6 is enabled by default.
Great!
> The helper which I use is delivered by default with the package (v2.17):
> http://manpages.ubuntu.com/manpages/saucy/man8/ext_ldap_group_acl.8.html
> As I understood from google there is 2 options IPv4/IPv6 in helper. By
> default it uses the IPv6 and than IPv4, but it is stuck on IPv6 so it
> does not going further.
> Therefore IPv4 flag must be defined.
Why would it get stuck on IPV4 or IPV6? if the helper supports them both
he will live throw it but I haven't seen even one line of logs that got
to the point which the program is actually started.
Also a FD :: should not be related to any of squid helpers but to
another parts of squid service levels.
Try to disable the external_acl and see the results in the cache.log.

>
> Some people trying to solve the FD line log with such lines in config
> (gogole is your best friend:) ):
> tcp_outgoing_address 192.168.1.130
> or
> dns_v4_first on
> or
> http_port 127.0.0.1:3128

We are no not even solving the FD line we are just merely testing the
cause of it.
each and every one of the mentioned "solutions" is not related in any
way directly but merely trying to tell squid to use ipv4 favored" and
explicit settings.
so just a small question.
What would you be doing to get to this site?
http://[2001:470:505a::101]/

Try to ping it or browse to it...

>
> I tried both of those possibilities.
> But I still have those lines in log(default debug options):
> 2013/11/13 22:25:18| commBind: Cannot bind socket FD 25 to [::1]: (99)
> Cannot assign requested address
> 2013/11/13 22:25:18| commBind: Cannot bind socket FD 26 to [::1]: (99)
> Cannot assign requested address
> 2013/11/13 22:25:18| ERROR: Failed to create helper child read FD: UDP[::1]
>
> According http://proxy.ccu.edu.tw/squidfaq/FAQ.pdf%e2%80%8e p115 (It is for
> IPv4, but logical it leads to IPv6. There should be same principles):
Since the mentioned page there was a change of logic in squid internals
from version 2.X to 3.X(1,2,3,4,TRUNK\HEAD).
In your specific case it might be possible.
again it's ubuntu 13.10 + 3.3.8 ?..
Are you using stock settings?
Are you using do you have ipv6 + ipv4 real or local addresses?
try to run
"ip addr" and also use the default squid.conf to make sure that squid
runs without any auth parameters.
Then step by step we can find the line that adds all the trouble and the
exact reason for that.
In order to make sure that there is a problem but not related to squid
directly I have a small bash script for you at:
https://github.com/elico/squid-helpers/blob/master/squid_helpers/proxy_hb_check/proxystatcheck.sh

The file will always return the test as OK unless the file
"/tmp/proxy1.err" will be created.

try to use the external_acl using the script instead of the
ext_ldap_group(whatever the name was).
in a case you want to debug it add after the "do" line the next:
echo $url >> /tmp/ext_acl.log

Which will write a log of the external_acl is properly configured.
try with a basic not %LOGIN but rather %SRC...
Just to make sure what happens.

Step by step build it to make sure everything is right and what exact
step you are adding to squid.conf adding the complication of warnings.
Please restart the system in order to avoid all ipv6 playtime results.

Regards,
Elizer

>
> "11.37 commBind: Cannot bind socket FD 5 to 127.0.0.1:0: (49) Can't
> assign requested address
> This likely means that your system does not have a loopback network
> device, or that device is not properly con¯gured. All Unix systems
> should have a network device named lo0, and it should be con¯gured with
> the address 127.0.0.1. If not, you may get the above error message. To
> check your system, run:
> % ifconfig lo0
> The result should look something like:
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 inet 127.0.0.1
> netmask 0xff000000"
>
> And If I do ifconfig lo0, I get (remember I disabled IPv6):
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> UP LOOPBACK RUNNING MTU:65536 Metric:1
> RX packets:1217 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1217 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:133681 (133.6 KB) TX bytes:133681 (133.6 KB)
>
> Therefore, I have conclusion that even with disabled IPv6 in networking,
> it is not possible to disable IPv6 request by Ubuntu's Squid 3.3.8 via
> conf file. Except I download the squid sources and rebuilt it with
> –disable-ipv6 (what I do not want to do - lazy :) ). For small
> environment it is ok, but for 50+ users addition requests are not good
> for traffic.
> It would be grate if there will appear a conf solution, but till now I
> don't see any.
>
> P.S.: a bit off topic on first post, but it still related to it. :)
>
>
> -----Oorspronkelijk bericht----- From: Eliezer Croitoru
> Sent: Wednesday, November 13, 2013 9:43 PM
> To: squid-users_at_squid-cache.org
> Cc: Andrey ‪
> Subject: Re: [squid-users] Ubuntu Server 13.10. Squid 3.3.8. WARNING:
> external ACL 'memberof' queue overload
>
> OK got it.
> The basic issue is that the helper is trying to use ip?
> I am trying to understand something about the docs and about the situation.
> external_acl_type memberof %LOGIN /usr/lib/squid3/ext_ldap_group_acl
> -P -R -K -b "dc=dot,dc=lan" -f
> "(&(cn=%v)(memberOf=cn=%a,cn=Users,dc=dot,dc=lan))" -D
> nslcd-service_at_dot.lan -w "Pa77w0rd" -h ubuntu.dot.lan
>
> Which the ext_ldap_group_acl that is provided by ? squid?
> It is a helper which suppose to communicate with squid via STDIN\STDOUT
> and errors STDERR.
>
> Correct me about the above if I am wrong.
> This will might help also me to understand the meaning of ipv4\ipv6 in
> the docs about external_acl:
> http://wiki.squid-cache.org/Features/IPv6#How_do_I_make_squid_use_IPv6_to_its_helpers.3F
>
>
> I am still unsure what it tries to do but:
> In linux "everything is a file" even the hardware suppose to be a file.
> it is not always brought to the hands of the Admin to spare the
> replacement of a very pricey devices which should be left alone with a
> tested piece of internal firmware!
> Else then that Linux OS uses for example 3 channels of communication
> between the user terminal\screen to communicate with the user\admin.
> The whole communication channel is suppose to be "one" FD and I maybe
> wrong but STDIN\STDOUT\STDERR is a communication channel between a user
> and the computer in a command-line interaction.
> There are many other ways to do that but leave it at that.
>
> So a FD is a way for the kernel and other sources to communicate.
> It can be a FILE on disk which has read\write channels or a TCP socket
> that has a read\write channel or a UDP socket which is a bit more
> complex to understand how it's a communication channel since it's a
> "datagram" channel.
>
> There is also the "unix" socket which is called a *pipe* which I do not
> remember right now how it works since it cannot be used by a read+write
> channels in the same sec If I do remember right.
>
> Squid as a server emulates for the software like the external_acl helper
> a communication channel as it(squid) was a terminal user that is now
> interacting the software\script.
> So squid has STDIN\STDOUT\STDERR on a "screen" (virtual inside the
> software) and then when a client sends a request squid by the ACLs rules
> "consults" the helper using STDIN(for the software while STDOUT for
> squid) and then consider the "offer" that correspond to the request in
> STDOUT(of the software while beaning squid STDIN) and any STDERR
> messages are logged into the cache.log.
>
> So the external_acl helper is like an interviewer for each request basic
> "looks" such as src-IP and\or request-url and\or other parameters
> available to squid.
>
> So as Amos suggested there is might be a miss configuration in squid
> ACLs order that forces the mentioned symptoms.
>
> The logs can help determine the state of each request and the status of
> each ACL and while doing so You can see in the logs that the problem is
> still there..
> "2013/11/13 20:29:13.689| WARNING: Cannot run
> '/usr/lib/squid3/ext_ldap_group_acl' process."
>
> The line you see in the logs:
> "2013/11/13 20:29:13.689| ERROR: Failed to create helper child read FD:
> TCP [::1]"
> Is a general line That you will see when the OS trying to bind some
> socket what ever it is using the TCP ipv6 protocol.
>
> Disabling the ipv6 sockets from the Linux OS\kernel is not really
> possible since once it is enabled it is there unless in the next reboot
> you will not load it.
> (I am wrong in a case there was some changes in Linux kernel and ipv6
> modules.)
>
> There might be a chance of converting the STD channel from one channel
> to a TCP channel but I am not sure that the kernel developers will apply
> it so soon.
>
> Try to force squid to bind the ipv4 of squid in http_port like:
> http_port 127.0.0.1:3128
>
> Which will might cause the comm bind error to be gone from the logs.
>
> Here if you need me,
> Eliezer
>
> On 11/13/2013 09:36 PM, Andrey ‪ wrote:
>> I think helper tries to access the IPv6 of the server (I'am not sure!),
>> but IPv6 is disabled:
>> /etc/sysctl.conf
>>
>> # Disable IPv6
>> net.ipv6.conf.all.disable_ipv6 = 1
>> net.ipv6.conf.default.disable_ipv6 = 1
>> net.ipv6.conf.lo.disable_ipv6 = 1
>>
>> #Enable IPv4 forward
>> net.ipv4.ip_forward = 1
>> net.ipv4.conf_all.rp_filter=1
>>
>> Here is the log without ipv4, well debug_options:82,9 84,9, I do not
>> know what is meaning of FD socket (No info on inet):
>
Received on Wed Nov 13 2013 - 22:29:23 MST