Re: Deferred reads

On Wed, 31 May 2000, Adrian Chadd wrote:

>
> The next step in my nefarious plan with the comm code is to make
> squid work well with event driven systems such as FreeBSD's kqueue
> system. In this light, I'd like to kill deferred reads. The

I'll argue that deferred reads are (however ugly) a very important
part of squid's operation. As much as you may dislike them, they
work.

> problem with the curent deferred reads setup is that you can
> only really throw away the read handler call after you have the
> read event, and if you have lots of deferred reads your event
> queue spends most of its time returning fds that will be
> thrown away anyway.

Can you be more specific? What does it mean to "throw away the read
handler call?" Do you mean to defer it?

Do you have any data from real caches? Is it ever the case
that "you have lots of deferred reads?"

>
> There are five functions passed to commSetDefer:
>
> httpAcceptDefer - defers accepting HTTP connections if we are out of fd's .
> clientReadDefer - haven't figured this ne out yet

clientReadDefer is used for half-closed connections, and also
for pipelined requests.

> sslDeferServerRead - used only if DELAY_POOLS is enabled
> pumpReadDefer - used so PUMP_MAXBUFFER isn't overflowed
> fwdCheckDeferRead - don't read too far ahead from server if clients are lagged
>
> Now, from what I can see these can be replaced with either periodic events
> or properly scheduled reads.

I don't like the sound of "periodic events." How do you choose the
period? If the period is too long then the request is delayed
unnecessarily. If its too short, and you still don't want to read,
then you have almost the same problem you're trying to fix -- a lot of
cycles spent not reading.

Also, can you explain what a properly scheduled read is?
How is that supposed to work?

> Since this is part of the commloops development, I'm going to
> start removing these and replacing them with suitable interfaces.
> Can anyone think of a reason to keep deferred reads?

To me, deferring reads are tightly coupled with the use of select and
poll. I'm far from convinced that you can/should "kill them" because
kqueue works differently. Admittedly, I don't know enough about the
kqueue implementation, so maybe you can educate me further. If a
kqueue event (or whatever) says that data is read, but you want to
ignore it (or, the application isn't ready to accept the data), how do
you handle that?

Whatever you do, please do not commit any of these changes until you
have run the code for a while on live caches.
Received on Wed May 31 2000 - 11:10:34 MDT