Re: [squid-users] Squid->DG->Squid from Andrew Rogers on 2011-07-28 (squid-users)

From: Andrew Rogers <andy.rogers_at_andyscomp.f9.co.uk>
Date: Thu, 28 Jul 2011 08:27:17 +0100

Hi Amos

Thanks for the detailed reply, I will study your comments accordingly
and will make changes to my squid.conf where necessary thankyou.

On 27 July 2011 07:15, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>
>
> This is the sandwich configuration looping on itself. You haev several
> choices:
>
> * configure two instances of Squid
> * configure client->DG->Squid
> * configure client->Squid->DG
> * disable "via off" and cross your fingers there is never any actual
> infinite loop. You will need a Squid built with HTTP violations enabled to
> do that. Loop protection is REQUIRED by the HTTP standards.

I have decided to with a Squid1->DG->Squid2 approach now, and have
already been changing my config files etc accordingly.

> This only solves the loop issue though. Cache MISS is separate...

After some initial testing the loop issue has disappeared, and iam
getting quite a few more what looks like cache hits now of
"TCP_REFRESH_UNMODIFIED" not which are showing like:-

1311836509.795 162 localhost TCP_REFRESH_UNMODIFIED/304 553 GET
http://i2.cdnds.net/11/30/P/gaming_sidpirates1.jpg -
DIRECT/213.244.185.38 -
1311836509.795 163 mycomp.tg.local TCP_MISS/304 691 GET
http://i2.cdnds.net/11/30/P/gaming_sidpirates1.jpg me@MY.LOCAL
FIRST_UP_PARENT/127.0.0.1

Which iam assumeing it has had a successfull cache hit from Squid2?
With you saying Cache MISS is seperate, will using the 2 seperate
Squid instances automatically have a better hit rate by the looks
already from here?

> On the whole, since you have requests that skip DG, and some get cached you
> are not in a position to only have caching on the second loop. Things
> entering the first loop WILL be found in the cache even if you wanted them
> to go through DG first.

Why do some request's skip DG and go direct from the first Squid
instance? I know I have asked whitelistsites to always go direct
which is not a problem, but it is other sites whcih initially go
through Squid->DG but them pages etc which the sites load up seem to
go direct in some circumstances, why is this? and can I configure so
all traffic is forced through DG then to Squid2 instance? If so how
can I achieve this, and what are the knock on effects if any?

Would something along the line of work:-

http_reply_access allow localhost
http_reply_access deny blockedsites
http_reply_access allow all
always_direct allow localhost
always_direct allow whitelistsites
always_direct deny all

I would want all traffic to go from Squid1->DG and then all traffic
from DG->Squid2 which would then pass all the traffic out direct as
expected?
What can of config along this line would I need in squid2.conf?

so far this is my changed squid1.conf file:-

####### /etc/squid3/squid.conf Configuration File #######
####### cache manager
cache_mgr squid_at_mydomain.co.uk
####### kerberos authentication
auth_param negotiate program /usr/lib/squid3/squid_kerb_auth -d -s
HTTP/squid.my.local
auth_param negotiate children 10
auth_param negotiate keep_alive on
####### provide access via ldap for clients not authenticated via kerberos
auth_param basic program /usr/lib/squid3/squid_ldap_auth -R \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f sAMAccountName=%s \
       -h dc.my.local
auth_param basic children 10
auth_param basic realm Internet Proxy
auth_param basic credentialsttl 1 minute
####### ldap authorizations
# restricted proxy access logged
external_acl_type internet_users %LOGIN /usr/lib/squid3/squid_ldap_group -R -K \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=Internet
Users,ou=Internet Groups,dc=my,dc=local))" \
       -h dc.my.local
# full proxy access no logging
external_acl_type internet_users_full_nolog %LOGIN
/usr/lib/squid3/squid_ldap_group -R -K \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=Internet
Users Full NoLog,ou=Internet Groups,dc=my,dc=local))" \
       -h dc.my.local
# full proxy access logged
external_acl_type internet_users_full_log %LOGIN
/usr/lib/squid3/squid_ldap_group -R -K \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=Internet
Users Full Log,ou=Internet Groups,dc=my,dc=local))" \
       -h dc.my.local
####### acl for proxy auth and ldap authorizations
acl auth proxy_auth REQUIRED
# format "acl, aclname, acltype, acltypename, activedirectorygroup"
acl RestrictedAccessLog external internet_users Internet\ Users
acl FullAccessNoLog external internet_users_full_nolog Internet\
Users\ Full\ NoLog
acl FullAccessLog external internet_users_full_log Internet\ Users\ Full\ Log
acl whitelistsites url_regex -i "/etc/squid3/whitelistsites.txt"
acl blockedsites url_regex -i "/etc/squid3/blockedsites.txt"

##Public DMZ Filtering, do not need to authenticate create acl type
acl DmzPublicWifiSubnet src 10.0.0.0/24

####### squid defaults
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl SSL_ports port 443
acl Safe_ports port 80-81 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
####### enforce auth: order of rules is important for authorization levels
no_cache deny whitelistsites
http_access allow whitelistsites

## Allow DMZ Public Wifi Access out without authorisation
http_access allow DmzPublicWifiSubnet

http_access allow FullAccessNoLog auth
http_access allow FullAccessLog auth
http_access deny blockedsites
http_access allow RestrictedAccessLog auth

####### logging
# don't log FullAccessNoLog
access_log /var/log/squid3/access-auth.log squid !FullAccessNoLog

####### squid defaults
http_access deny all

#Log Connecting Client DNS Names instead on IP Names.
log_fqdn on

http_port 8080
##Push Traffic Through DansGuradian for Content Filtering
cache_peer 127.0.0.1 parent 8081 0 no-query proxy-only no-delay
no-netdb-exchange no-digest connect-timeout=15 login=PASS
cache_peer_Access 127.0.0.1 deny localhost
##Push Public DMZ Wifi Subnet through DG
cache_peer_access 127.0.0.1 allow DmzPublicWifiSubnet

cache_peer_access 127.0.0.1 allow RestrictedAccessLog
cache_peer_access 127.0.0.1 deny all

pid_filename /var/run/squid3-auth.pid

visible_hostname mysquid-auth.my.local

hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid3
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

squid2.conf:-
####### /etc/squid3/squid.conf Configuration File #######
####### cache manager
cache_mgr squid_at_mycache.co.uk
####### kerberos authentication
auth_param basic program /usr/lib/squid3/squid_ldap_auth -R \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f sAMAccountName=%s \
       -h dc.my.local
auth_param basic children 10
auth_param basic realm Internet Proxy
auth_param basic credentialsttl 1 minute
####### ldap authorizations
# restricted proxy access logged
external_acl_type internet_users %LOGIN /usr/lib/squid3/squid_ldap_group -R -K \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=Internet
Users,ou=Internet Groups,dc=my,dc=local))" \
       -h dc.my.local
# full proxy access no logging
external_acl_type internet_users_full_nolog %LOGIN
/usr/lib/squid3/squid_ldap_group -R -K \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=Internet
Users Full NoLog,ou=Internet Groups,dc=my,dc=local))" \
       -h dc.my.local
# full proxy access logged
external_acl_type internet_users_full_log %LOGIN
/usr/lib/squid3/squid_ldap_group -R -K \
       -b "dc=my,dc=local" \
       -D squid_at_my.local \
       -w "password" \
       -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=Internet
Users Full Log,ou=Internet Groups,dc=my,dc=local))" \
       -h dc.my.local
####### acl for proxy auth and ldap authorizations
acl auth proxy_auth REQUIRED
# format "acl, aclname, acltype, acltypename, activedirectorygroup"
acl RestrictedAccessLog external internet_users Internet\ Users
acl FullAccessNoLog external internet_users_full_nolog Internet\
Users\ Full\ NoLog
acl FullAccessLog external internet_users_full_log Internet\ Users\ Full\ Log
acl whitelistsites url_regex -i "/etc/squid3/whitelistsites.txt"
acl blockedsites url_regex -i "/etc/squid3/blockedsites.txt"

##Public DMZ Filtering, do not need to authenticate create acl type
acl DmzPublicWifiSubnet src 10.0.0.0/24

## Allow DMZ Public Wifi Access out without authorisation
http_access allow DmzPublicWifiSubnet

http_access allow FullAccessNoLog auth
http_access allow FullAccessLog auth
http_access deny blockedsites
http_access allow RestrictedAccessLog auth

####### logging
# don't log FullAccessNoLog
access_log /var/log/squid3/access-proxy.log squid !FullAccessNoLog

####### squid defaults
http_access deny all

#Log Connecting Client DNS Names instead on IP Names.
log_fqdn on

http_port 127.0.0.1:3128

pid_filename /var/run/squid3-proxy.pid

visible_hostname mysquid-proxy.my.local

End of squid2.conf...

I know I have got duplicate acl's which don't need to be in
squid2.conf, i will remove these shortly, but just need to make sure
iam removing the correct lines which are no longer needed.
How can I make squid1 not cache anything, as I want all the cacheing
to be done at squid2?

Would you be able to re-look at the about squid1/2.conf and let me
know if iam missing anything that may cause problems?

Many thanks

Andy
Received on Thu Jul 28 2011 - 07:27:44 MDT

This archive was generated by hypermail 2.2.0 : Thu Jul 28 2011 - 12:00:03 MDT