Re: squid-smp

On Tue, Oct 27, 2009 at 8:09 AM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> Alex Rousskov wrote:
>>
>> Hello,
>>
>>    I will be working on SMP support in the coming months. I caught up
>> on this and the previous SMP-related squid-dev thread and it looks like
>>  the approach I currently favor has been discussed (again) and did not
>> cause any violent objections, although some ideas were much more
>> ambitious. I am not sure how we can reach consensus on this topic, but I
>> will through in some specifics in hope to better identify competing
>> approaches.
>
> Okay. Before you get buried in this work might I request that you look at
> the stale-while-revalidate / stale-if-error code that was nearly finished?
> http://bugs.squid-cache.org/show_bug.cgi?id=2255
>
> Yahoo! named those as two of their requirements before it would be possible
> for them to assist with performance testing Squid-3. They might not be
> adverse to helping test SMP support if the other requirements are available.
>
> Collapsed forwarding is also a requirement, but I suspect it is too close to
> the request handling and needs a re-designed code architecture to fit with
> whatever SMP threading model is taken anyways.
>
>>
>> My short-term focus would be on the following three areas:
>>
>> A) Identifying a few "large", "rarely-interacting" threads that would
>> work reasonably well on an 8-core 2-CPU machine with 8 http_ports. This
>> should take the lessons learned from existing SMP designs into account,
>> with Squid specifics in mind. Henrik, Amos, and Adrian started
>> discussing this already.
>>
>> B) Making commonly used primitives thread-safe (mostly not in terms of
>> locking their shared state but in terms of not using static/shared data
>> that needs locking). Many posts on this subject, starting with Roberts
>> advice to desynchronize.
>>
>> C) Posting performance benchmarking results for single- and
>> multi-instance Squids on mutli-core systems as a baseline.
>>
>>
>> My mid-term focus will probably be on sharing http_port, memory cache,
>> disk cache and possibly loggin/stats among a "few large threads".
>>
>>
>> My overall goal is to at least approach the performance of a
>> multi-instance caching Squid on 8-core hardware.
>>
>>
>> I am not excited by the "one thread per message", "one thread per
>> AsyncJob", or similar "many tiny threads" designs because, IMO, they
>> would require too much rewriting to be implemented properly. This may
>> need to be re-evaluated as the world moves towards 1000-core systems,
>> but a lot of improvements necessary for the "few large threads" design
>> will not be wasted anyway.
>>
>> I hope that by focusing on a "few large threads" design and fixing
>> primitives we can gain "enough" SMP benefits in a few months of active
>> development. If you think there is a better way to get SMP benefits in
>> the foreseeable future, please post.
>>
>> Thank you,
>>
>> Alex.
>
>
> --
> Please be using
>  Current Stable Squid 2.7.STABLE7 or 3.0.STABLE19
>  Current Beta Squid 3.1.0.14
>

Thats very good Alex, I am also working on the same, have not explored
everything yet, But with all you guys hopefully we will complete this
project as soon as possible, Issues in my mind will be discussed soon.

-- 
Mr. S. H. Malave
Computer Science & Engineering Department,
Walchand College of Engineering,Sangli.
sachinmalave_at_wce.org.in