Re: squid-smp

On Tue, Oct 13, 2009 at 8:12 AM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> Sachin Malave wrote:
>>
>>
>> On Mon, Oct 12, 2009 at 8:33 PM, Amos Jeffries <squid3_at_treenet.co.nz
>> <mailto:squid3_at_treenet.co.nz>> wrote:
>>
>>    On Tue, 13 Oct 2009 00:29:56 +0200, Henrik Nordstrom
>>    <henrik_at_henriknordstrom.net <mailto:henrik_at_henriknordstrom.net>> wrote:
>>     > fre 2009-10-09 klockan 01:50 -0400 skrev Sachin Malave:
>>     >
>>     >> I think it is possible to have a thread , which will be watching
>>     >> AsyncCallQueue, if it finds an entry there then it will execute the
>>     >> dial() function.
>>     >
>>     > Except that none of the dialed AsyncCall handlers is currently
>> thread
>>     > safe.. all expect to be running in the main thread all alone.
>>
>>    Which raises the issue of whether to add a second main queue loop for
>>    thread-safe calls.  Then schedule calls which have been audited and
>>    found
>>    safe to that queue instead of the current main queue. Usage would be
>>    low to
>>    start with but would allow ongoing incremental SMP improvements by
>>    gradually migrating chunks of code to be thread-safe.
>>
>>    An alternate would be thread-safe the queue and add a flag to say
>>    particular calls are thread-safe. That would mean walking the queue
>>    repeatedly looking for them. Which is perhaps less desirable at the
>>    start
>>    of conversion when few calls are threaded. But gains in utility
>>    relative to
>>    the thread-safety progress.
>>
>>    This involves a small amount of extra code in schedule() to flag which
>>    queue the calls is sent to, and a chunk of extra memory for
>>    duplicate queue
>>    management objects.
>>
>>
>> ---------------------------------------------------------------------------------------------------------------
>> Ok if that is possible then would like to make those changes,  Either of
>> them will be tried...
>>
>> One more thing...
>>
>> Are you thinking about spawning multiple threads or single thread
>> separated from main is sufficient for handling all scheduled calls.
>> Here multiple threads means, we could have threads all trying to dial
>> entries in AsyncCallQueue simultaneously.....
>
> That would be up to you.
>
> I had not thought more than one dialer thread per CPU necessary at this
> stage. Though with both verified thread-safe calls and a thread-safe queue,
> multiple dialer threads should not be an issue. Doing more than necessary
> would merely be a waste of resources.

Yeah !!!!! I am also thinking the same...

>
> Squid would essentially segment into multiple 'main' threads / dialers
> running one to a CPU and sharing minimal amounts of state. Slightly more
> efficient and far easier to configure than current setups of multiple
> interlinked Squid instances.

Ok, This could be done... will give good performance results.. But
want to know more about this, please come again with more precise
definition.
Are you talking about multi-instance squid ?
http://wiki.squid-cache.org/MultipleInstances.

>
> Without knowing too much, I'm assuming the Job ID can be used to identify
> calls a particular thread/job runs.
>
> Amos
>

>>
>> -------------------------------------------------------------------------------------------------------------------------------
>>
>>     >
>>     >> can we separate dispatchCalls() in EventLoop.cc for that purpose?
>> We
>>     >> can have a thread executing distatchCalls() continuously
>>
>>    This is an end-goal. Jumping straight there for everything is usually a
>>    mistake. But good to re-state it anyway.
>>
>>     >> and if error
>>     >> condition occurs it is written  in "error" shared variable.....
>>    which
>>     >> is then read by main thread executing mainLoop....... in the
>>    same way
>>     >> returned dispatchedSome can also be passed to main thread...
>>
>>    I think I follow. You mean something like the way errno works in the
>> OS?
>>    Doing that would be a major crutch in Squid. I'd rather have an error
>>    object per-job (field in the job descriptor object) which the job
>>    handlers
>>    can use according to the job needs.
>>    Some will result in data sent back to the client, some in a completely
>>    altered handling pathway.
>>
>>    Amos
>>
>>
>>
>>
>> --
>> Mr. S. H. Malave
>> Computer Science & Engineering Department,
>> Walchand College of Engineering,Sangli.
>> sachinmalave_at_wce.org.in <mailto:sachinmalave_at_wce.org.in>
>
>
> --
> Please be using
>  Current Stable Squid 2.7.STABLE7 or 3.0.STABLE19
>  Current Beta Squid 3.1.0.14
>

-- 
Mr. S. H. Malave
Computer Science & Engineering Department,
Walchand College of Engineering,Sangli.
sachinmalave_at_wce.org.in