First of all - thanks for your help.
> Problem #1: Please upgrade your Squid.
>
> Squid-2.6 has been 3 years since the last security update, nearly 5 years
> since your particular version was superceded.
ok, I will update to the latest version
> On 24/04/2013 12:15 a.m., Alex Domoradov wrote:
>>
>> Hello all, I encountered the problem with configuration 2 squids. I
>> have the following scheme -
>>
>> http://i.piccy.info/i7/0ecd5cb8276b78975a791c0e5f55ae60/4-57-1543/57409208/squids_schema.jpg
> Problem #2: Please read the section on how RAID0 interacts with Squid ...
> http://wiki.squid-cache.org/SquidFaq/RAID
>
> Also, since youa re using SSD, see #1. The older Squid like 2.6 push
> *everything* through disk which reduces your SSD lifetime a lot. Please
> upgrade to a current release (3.2 or 3.3 today) which try to avoid disk a
> lot more in general and offer cache types like rock for even better I/O
> savings on small responses.
ok. The main reason why I choose raid0 is to get necessary disk space ~400 Gb.
>> The main idea is to download all files from rackspace and amazon
>> through parent squid and store all files in his cache.
>
>
> Sure. Nothing abnormal there.
glad to hear it
>> On the main_squid server was configured PBR (iptables + ip route). All
>> packets go through the same channels through which the requests were
>> received
>>
>> # ip ru sh
>> 0: from all lookup local
>> 1000: from all fwmark 0x3e8 lookup ISP1
>> 2000: from all fwmark 0x7d0 lookup ISP2
>> 3011: from all fwmark 0xbc3 lookup ISP3
>> 32762: from xxx.xxx.xxx.62 lookup ISP1
>> 32763: from yyy.yyy.yyy.239 lookup ISP2
>> 32764: from zzz.zzz.zzz.10 lookup ISP3
>> 32766: from all lookup main
>> 32767: from all lookup default
>>
>> *** main_squid squid.conf ***
>> http_port 192.168.210.1:3128 transparent
>>
>> cache_peer 192.168.220.2 sibling 3128 3130
>> dead_peer_timeout 5 seconds
>>
>> acl AMAZON dstdom_regex -i (.*)s3\.amazonaws\.com
>> cache_peer_access 192.168.220.2 allow AMAZON
>>
>> acl RACKSPACE dstdom_regex -i (.*)rackcdn\.com
>> cache_peer_access 192.168.220.2 allow RACKSPACE
>
>
> FYI: these dstdom_regex look like they can be far more efficiently replaced
> by dstdomain ACLs and even combined into one ACL name.
Did you mean something like
acl RACKSPACE dstdomain .rackcdn.com
cache_peer_access 192.168.220.2 allow RACKSPACE
Would be any REALLY speed improvements? I know that regexp is more "heavy"
>> url_rewrite_program /usr/bin/squidguard
>> url_rewrite_children 32
>>
>> cache_dir null /tmp
>> cache_store_log none
>> cache deny all
>>
>> acl local_net src 192.168.0.0/16
>> http_access allow local_net
>>
>> *** parent_squid squid.conf ***
>>
>> http_port 192.168.220.2:3128
>> acl main_squid src 192.168.220.1
>>
>> http_access allow main_squid
>> http_access allow manager localhost
>> http_access allow manager main_squid
>>
>> icp_access allow main_squid
>>
>> cache_mem 30 GB
>> maximum_object_size_in_memory 128 MB
>> cache_dir aufs /squid 400000 16 256
>> minimum_object_size 16384 KB
>> maximum_object_size 1024 MB
>> cache_swap_low 93
>> cache_swap_high 98
>
>
> The numbers here look a little strange. Why the high minimum object size?
I have made some investigation and got the following information
The number of projects for 2013
# svn export http://svn.example.net/2013/03/ ./
# find . -maxdepth 1 -type d -name "*" | wc -l
1481
The number of psd files
# find . -type f -name "*.psd" | wc -l
8680
Total amount of all psd files
# find . -type f -name '*.psd' -exec ls -l {} \; | awk '{ print $5}' |
awk '{s+=$0} END {print s/1073741824" Gb"}'
116.6 Gb
The number of psd files less than 10 Mb
# find . -type f -name '*.psd' -size +10000k -size -100000k | wc -l
915
The number of psd files between 10-50 Mb
# find . -type f -name '*.psd' -size +10000k -size -50000k | wc -l
5799
The number of psd files between 50-100 Mb
# find . -type f -name '*.psd' -size +50000k -size -100000k | wc -l
1799
The number of psd files larger than 100 Mb
# find . -type f -name '*.psd' -size +50000k -size -100000k | wc -l
167
So as you can see ~87% of all psd files 10-100 Mb
>> acl PSD urlpath_regex -i \.psd$
>> cache allow PSD
>>
>> acl ZIP urlpath_regex -i \.zip$
>> cache allow ZIP
>>
>> acl OTHER url_regex -i ^http://*
>> cache deny OTHER
>
>
> At the very least that OTHER definition can be replaced by the much faster:
> acl OTHER proto HTTP
> However, the feeder mechanism to this hierarchy is NAT interception. Meaning
> *all* traffic is HTTP so you can instead use:
> cache_deny all
ok, I will replace it
>> refresh_pattern \.psd$ 2592000 100 2592000 override-lastmod
>> override-expire ignore-reload ignore-no-cache
>> refresh_pattern \.zip$ 2592000 100 2592000 override-lastmod
>> override-expire ignore-reload ignore-no-cache
>>
>> All work fine, until I uncomment on main_squid the following line
>>
>> tcp_outgoing_address yyy.yyy.yyy.239
>>
>> When I try to download any zip file from amazon I see the following
>> message in cache.log
>>
>> 2013/04/22 01:00:41| TCP connection to 192.168.220.2/3128 failed
>>
>> If I run tcpdump on yyy.yyy.yyy.239 I see that main_squid trying to
>> connect to parent via external interface without success.
>>
>> So my question. How may I configure main_squid that it could connect
>> to the parent even with configured
>> tcp_outgoing_address option?
>
>
> #3 The failure is in TCP. Probably your firewall settings forbidding
> yyy.yyy.yyy.239 from talking to 192.168.220.2.
No, there are no forbidding from firewall. As I describe above, all
packets with src ip yyy.yyy.yyy.239 go through table ISP2 that looks
like
# ip ro sh table ISP2
yyy.yyy.yyy.0/24 dev bond1.2000 scope link src yyy.yyy.yyy.239
default via yyy.yyy.yyy.254 dev bond1.2000
and that's a problem. I see the following packets on my external interface
# tcpdump -nnpi bond1.2000 port 3128
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bond1.3013, link-type EN10MB (Ethernet), capture size 65535 bytes
13:19:43.808422 IP yyy.yyy.yyy.239.36541 > 192.168.220.2.3128: Flags
[S], seq 794807972, win 14600, options [mss 1460,sackOK,TS val
3376000672 ecr 0,nop,wscale 7], length 0
13:19:44.807904 IP yyy.yyy.yyy.239.36541 > 192.168.220.2.3128: Flags
[S], seq 794807972, win 14600, options [mss 1460,sackOK,TS val
3376001672 ecr 0,nop,wscale 7], length 0
13:19:46.807904 IP yyy.yyy.yyy.239.36541 > 192.168.220.2.3128: Flags
[S], seq 794807972, win 14600, options [mss 1460,sackOK,TS val
3376003672 ecr 0,nop,wscale 7], length 0
So as I understand connection to my parent go through table ISP2
(because tcp_outgoing_address set src ip for the packets to the
yyy.yyy.yyy.239) and external interface bond1.2000 when I expected
that it would be established via internal interface bond0.
Received on Fri Apr 26 2013 - 08:37:29 MDT
This archive was generated by hypermail 2.2.0 : Fri Apr 26 2013 - 12:00:04 MDT
