Re: [squid-users] squid deployment in 6Gbit network with tproxy as L2 bridge.

W dniu 2013-08-23 12:12, Amos Jeffries pisze:
> On 22/08/2013 8:47 p.m., Pawel Mojski wrote:
>> Hi Guys;
>>
>> I have some intresting deployment scenario. I have to install squid
>> box(es) as L2 bridge in 10Gbit network with 6Gbit amonunt of traffic in
>> peak.
>> Squid is used to forward traffic to our ecap adapter. Ofcourse it's
>> impossible to handle that traffic amount on one box.
>> So, how to deploy it? I imagine such scenario. The first will be
>> "balancer", it will be a linux box with 2 10gigs cards. Then, the
>> "balancer" have to somehow redirect traffic to a squid boxes.
>> My first thought was to use wccpv2 protocol, but I have figured that
>> wccp router mode is very weakly supported on linux. I've found only two
>> projects, one writen in C from 2002 and one in python from 2011.
>> So, do you have any suggestions how to forward traffic to squid boxes?
>> The main thing is to provide source-ip spoofing functionality and have
>> only one bridge in 10gig network.
>> Squid boxes will be connected to the balancer seperate interfaces over
>> separate switch.
>>
>> Thanks in advance for further ideas.
>
> Yes WCCP may let you down here. There are quite a few limits to Squid
> support for it as well, not least of which is weak support for
> multiple caches per router.
>
> Doing this with only 1 bridge is probaly going to bite you as well, it
> will need one massive beast of a machine. Each Squid process will
> easily handle 100-150Mbps or so of traffic so you are looking at an
> estimate of around 45-60 Squid with 1 CPU core each.
>
> If you can find a box with enough CPU to split the traffic that way SM
> should serve you okay although it may have a lower bps capacity than
> standalone Squid due to the accept() races and UDS traffic the workers
> have to manage. Otherwise I suggest looking in the direction of policy
> routing the traffic using the iptables model for splitting traffic
> over ports
> (http://wiki.squid-cache.org/ConfigExamples/ExtremeCarpFrontend#Frontend_Balancer_Alternative_1:_iptables)
> but possibly splitting the traffic over routes to several cache boxes.
> Each of which doing regualar TPROXY on a smaller segment of the
> traffic before being aggregated back into the line upstream by another
> splitter/joiner.
>
> Amos
Thanks a lot Amos.

I've already solved the problem yesterday.
I made a "balancer" machine which split the traffic over 8 squid boxes
(via ip rule) and then join it into one stream.
Disadvantage of the solution is require 16 (2 per each scanning box) on
the balancer, but I can deal with it.
Works like a charm.

Regards;
Pawel Mojski
Received on Fri Aug 23 2013 - 12:08:46 MDT