Re: [squid-users] squid deployment in 6Gbit network with tproxy as L2 bridge.

W dniu 2013-08-23 14:08, Pawel Mojski pisze:
> W dniu 2013-08-23 12:12, Amos Jeffries pisze:
>> On 22/08/2013 8:47 p.m., Pawel Mojski wrote:
>>> Hi Guys;
>>>
>>> I have some intresting deployment scenario. I have to install squid
>>> box(es) as L2 bridge in 10Gbit network with 6Gbit amonunt of traffic in
>>> peak.
>>> Squid is used to forward traffic to our ecap adapter. Ofcourse it's
>>> impossible to handle that traffic amount on one box.
>>> So, how to deploy it? I imagine such scenario. The first will be
>>> "balancer", it will be a linux box with 2 10gigs cards. Then, the
>>> "balancer" have to somehow redirect traffic to a squid boxes.
>>> My first thought was to use wccpv2 protocol, but I have figured that
>>> wccp router mode is very weakly supported on linux. I've found only two
>>> projects, one writen in C from 2002 and one in python from 2011.
>>> So, do you have any suggestions how to forward traffic to squid boxes?
>>> The main thing is to provide source-ip spoofing functionality and have
>>> only one bridge in 10gig network.
>>> Squid boxes will be connected to the balancer seperate interfaces over
>>> separate switch.
>>>
>>> Thanks in advance for further ideas.
>> Yes WCCP may let you down here. There are quite a few limits to Squid
>> support for it as well, not least of which is weak support for
>> multiple caches per router.
>>
>> Doing this with only 1 bridge is probaly going to bite you as well, it
>> will need one massive beast of a machine. Each Squid process will
>> easily handle 100-150Mbps or so of traffic so you are looking at an
>> estimate of around 45-60 Squid with 1 CPU core each.
>>
>> If you can find a box with enough CPU to split the traffic that way SM
>> should serve you okay although it may have a lower bps capacity than
>> standalone Squid due to the accept() races and UDS traffic the workers
>> have to manage. Otherwise I suggest looking in the direction of policy
>> routing the traffic using the iptables model for splitting traffic
>> over ports
>> (http://wiki.squid-cache.org/ConfigExamples/ExtremeCarpFrontend#Frontend_Balancer_Alternative_1:_iptables)
>> but possibly splitting the traffic over routes to several cache boxes.
>> Each of which doing regualar TPROXY on a smaller segment of the
>> traffic before being aggregated back into the line upstream by another
>> splitter/joiner.
>>
>> Amos
> Thanks a lot Amos.
>
> I've already solved the problem yesterday.
> I made a "balancer" machine which split the traffic over 8 squid boxes
> (via ip rule) and then join it into one stream.
> Disadvantage of the solution is require 16 (2 per each scanning box) on
> the balancer, but I can deal with it.
> Works like a charm.
>
> Regards;
> Pawel Mojski
require 16 additional vlan interfaces ofcourse, sorry for convinient.

Regards;
Pawel Mojski
Received on Fri Aug 23 2013 - 12:11:02 MDT