Re: (Fwd) Re: --enable-time-hack

On Mon, Sep 11, 2000, Andres Kroonmaa wrote:
> On 8 Sep 2000, at 0:27, Adrian Chadd <adrian@creative.net.au> wrote:
>
> > > I am quite worried about CPU usage of squid. My P3-800 box is doing about
> > > 100-150 reqs/sec and its CPU is very loaded. I can't see or find the
> > > source of CPU burning, gprof or other profiling tools are of little help,
> > > they all have only 10ms precision. I wonder if it makes sense to design
> > > a macro that could measure any func with very small overhead and very high
> > > precision. Solaris has got gethrtime() which returns nanosec time with
> > > CPU clocktick precision. It consults on-CPU counter. Most popular platforms
> > > do have something similar. It would be very useful to have cachemgr return
> > > stats table that shows where most of the CPU is burned. Then we could
> > > much better understand what needs more optimisation.
> >
> > There are many, MANY places where CPU burn can be fixed. Try profiling
> > a kernel running squid - you'll see most of its time spent handling poll().
>
> Hey, but what about moving poll itself into a separate thread?
> What about moving HTTP, ICP, DNS socket polling into a separate thread,
> and calling aiocheckcallbacks to get notifications back to main thread?
>

I thought about shifting stuff around, but then you end up having to go
once through the IO loop to get your HTTP/ICP/DNS socket data rather than
periodically during the normal IO loop.

Also, there are WAY too many context switches with the pthreads-based aio
code. On a linux box I have here I'm seeing something upward of 30,000/sec
on a very idle box. Thats a lot of time to be spending context switching.
I'm guessing there is mutex/cond black magic dating back quite a while,
but I haven't got the time to figure it out yet.

You could move some of the functionality out to the seperate thread (eg
you could accept() and pre-parse bits and pieces before handing it back
to the main thread), but I'm currently attempting to keep things 'clean'
until my next patchset comes along, after which I'm going to move to
threading squid.

(My next patchset will include commloops with and event driven IO support,
and I'd rather not 'mess' with things for the sake of poll speedups just
yet .. :-)

<PLUG TYPE="DOC">
I'll commit your patch tomorrow, since there were no objections. How about
you knock your description of the dns/http/icp incoming interval up into
some basic docbook for the programmers guide? :-P
</PLUG>

Adrian

-- 
Adrian Chadd			"If a butterfly flaps its wings in China,
<adrian@creative.net.au>	    will a women get naked in Amsterdam?"
				      -- Ashley Penney on Chaos Theory