Re: memory-mapped files in Squid

> Thus, FIFO may work for most of reasonably configured caches,
> especially if augmented with a small buffer for popular objects (say,
> those referenced at least twice).

        Like you say below, most replacement policies will do about as
well as LRU (plus or minus 10%). The question is whether FIFO will be
much more efficient -- will it reduce the disk bottleneck? The web
cache workload is heavily write intensive, like 1:3 read:write. This is
in contrast to traditional (OLTP or database) workloads which are more
like 4:1 read:write. Systems/disks tuned for OLTP will not work well as
web caches. You bring up an interesting notion with the small buffer,
see below.

> > 1) we want to have our disks as full as possible. 90% at least.
> > 2) we want that these disks are full of _useful_ stuff, not waste.
>
> Sure. However, none of the traditional policies caches useful data! All
> traditional policies that I know of fill cache with garbage, on average.
> Provided your cache is of a reasonable size, of course.

        And this is not likely to change, due to the nature of the web
proxy workload. We and others have found in studies of long-term traces
that something like 60% of objects are never re-referenced. So most of
the objects that pass through the cache are garbage (i.e., have no value
since they are never re-referenced. There was some interesting work at
the last Web Cache Workshop from a group in Poland on how to identify
these objects (which Martin Arlitt in our group here referred to as
"one-timers"). By doing this they were able to reduce disk write rate
significantly without reducing much the disk read rate (i.e., they did
not suffer much loss in byte hit rate).

> We may need an intelligent _caching_ (or cache admission) policy, but
> I am afraid we are not ready for that yet.

        Why are we not ready for that yet? I believe there is room for
improvement of replacement policies (and can say more about this at the
San Diego workshop, I hope!), and believe there are other opportunities
as well. Please help me understand if there are fundamental limitations
to being able to address these, though... Thanks!

                             -- jad --
                          John Dilley <jad@hpl.hp.com>
Received on Tue Jul 29 2003 - 13:15:56 MDT