Re: [squid-users] Squid Performance Issues - reproduced

On 2 Jan 2003, at 19:45, Henrik Nordstrom <hno@squid-cache.org> wrote:

> The likelyhood for the I/O thread to get scheduled "immediately" is
> rather high on a UP machine. Same thing on a SMP machine, but on a SMP

 hmm. likely - why?

> > If I recall right, cond_wait is boolean, and, if many threads are
> > blocked on cond, and cond_signal does not make thread switch, and
> > another cond_signal is sent, only 1 thread would be unblocked
> > eventually.
>
> > I assume that mutex is put into to-be-locked state upon
> > first cond_signal (owner unspecified, but one of threads on wait),
> > and second attempt to signal would cause solid thread switch to
> > consume first cond (because mutex is "locked").
>
> Documentation says mutex should be locked prior to cond signal to avoid
> races.

 er, my fault. badly expressed myself. my path on thinking was:
  m_lock(mutex) - mutex locked
  signal(mutex,var) - mutex locked, marked to-be-relocked, var set true
  m_unlock(mutex) - mutex unlocked, but marked to-be-relocked
    ...
  m_lock(mutex)
    __if (mutex & to-be-relocked) then force_call scheduler

 I'm highly likely wrong here. But I can't get rid of belief that
 thread switch after unlock is inefficient. And if it doesn't happen,
 and main thread reaches m_lock for second time, there is bad case:
 threads blocked on cond mutex haven't had a chance to run and second
 signal is about to be raised. If its allowed to happen, then previous
 signal is lost, but man pages do not document signal loss, therefore
 such case must be detected and resolved. One way is to forcibly make
 thread switch on every single m_unlock which is cpu-buster, or somehow
 mark mutex into intermediate state, denying new locks until scheduler
 has been run.

> The signal implementation I had was shielded for load conditions by only
> signal is the main thread was in select/poll (save for races). The
> signalling does not need to be foolproof as things automatically recover
> by timeout (10ms). As long as it gets it mostly right then it will be
> fine.

 Signal is bad only because it causes all threads to be unblocked and
 forced to respin on mutexes. This has quite some background overhead.

> Attached is a small patch which introduces a completetion "signal" pipe.
> Comments?

 does it work? ;) I'm only worried about pipe buffering. does the
 1-byte write into the pipe immediately propagate to the other end?
 Or does it poll ready only when 8Kb of data is in the pipe?

> What to do about '1' I do not know, but I am not sure it is actually
> needed either. In such case the main thread whill get rather low
> priority due to repeatedly eating up it's quantum so most schedulers
> should give the I/O threads priority I think..

 I'm not sure either. aufs is not good choice for singleuser squid
 anyway.
 As to priority, iothreads have explicit priority set to higher than
 main thread. But some scheduler might assume that thread eating up
 its quanta might be doing something useful, so on contrary up its
 priority. But that all is rather unimportant i guess. Worst case
 is 10ms latency when poll() spins like nuts, ie under high loads.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Data AS
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn
 11317 Estonia
Received on Fri Jan 03 2003 - 08:47:26 MST