Re: [squid-users] Squid Performance Issues - reproduced

Andres Kroonmaa wrote:

> Cond_signal is documented to unblock "at least one thread".
> There is good reason for the "at least" part. Cond variable is nothing
> more than just plain boolean flag. Signalling is nothing more than
> setting it to true. Now, when kernel scheduler is called, it goes
> through all runnable threads and gives them time quanta by their prio.
> If there is 1 cpu, only 1 thread at a time can be run, and cond can
> only unblock exactly one thread. But if there is SMP and many cpus,
> then many threads are run concurrently, and they all are put to
> "compete" for the same mutex. They spin on mutex, and eventually only
> one will unblock. IF, mutex is used at all. cond_signal does not
> require mutex, in this case, potentially several threads are unblocked
> in MP system. But whether 1 or more depends on many things, including
> microtiming, therefore they say "at least one thread".

The GLIBC documentation is strictly "exacly 1", and multiple calls of
cond_signal will indead signal exacly as many threads as there is calls
to cond_signal.

I do not have a SMP system, but from what I can tell all this takes
place in the calling thread so there should be no difference on SMP
systems there.

> So if you think that mutex_unlock after signal always causes
> thread switch, then you are wrong. If you think that it never does
> thread switch, you are wrong. If you think that its likely to make
> switch, you are wrong, if you think its unlikely, you are wrong.

I don't, not anymore at least.

> All of the above is completely irrelevant for coding, unless you
> are stuck with some wrong assumptions. The only assumption I've
> found to hold is that _anything_ that CAN happen, WILL happen.
> About 100 times sooner than you'd expect. for eg. take 2 threads
> changing (read-modify-write) single atomic int once per request
> without a mutex lock. Likelyhood that they'll hit it concurrently
> is neglible, especially on an UP system. But you can bet money that
> this prog will crash in less than a day. This means that thread
> switch happens at moment when one thread has read the var, modified
> it, but not yet written out. imagine. nanosec timing.

This I always assume.

> > > Ah. Well, what about:
> > > pthread_mutex_lock(&queuemutex)
> > > do {
> > > schedule_io_request()
> > > pthread_cond_signal()
> > > } for <count> requests
> > > pthread_mutex_unlock(&queuemutex)
> > > pthread_mutex_lock(&queuemutex)
> > > pthread_mutex_unlock(&queuemutex)
>
> this will loose all signals but last one. Only 1 thread will run.

Not in my experiments on Linux glibc pthreads.

However, the mutex unlock/lock is mostly a noop here, except that it
wastes a few CPU cycles..

> yes, when schedule per request. But if we'd append to queue and after
> comm run append whole queue to requestqueue, we'd want thread run.
>
> Creating reliable thread switch for specific job is quite a brain
> exercise. It should be possible with 2 mutexes alone, but try, knowing
> that mutex unlock may but also might not cause a thread switch...
>
> I thought we might only send 1 signal to worker threads if there is
> work to do and there are idle threads. Then let threads themselves
> make sure they awake up enough workers. We make only sure that first
> worker gets reliably chance to run, before we proceed with poll().

Threads keep running until the queue is empty so it is not that big deal
if to few or too many threads are awakened.

However, it is desireable if on SMP the worker theads gets going while
the main thread is working on other network I/O, this to have I/O events
completed while the network I/O is beeing processed.

For now I select to ignore the issue about having worker threads
started. I have no indication this is actually needed in real life
workloads.

Regards
Henrik
Received on Fri Jan 03 2003 - 17:17:28 MST