Re: [squid-users] Re: Squid 1.2 formats and other Q's.

Chris Tilbury wrote:

> You still don't address write performance doing this, though.
> You'll still have synchronous writes taking place (metadata
> updates, etc), which will cause a performance degradation.
> Using a transaction log (or NVRAM if you are very rich) _will_
> help this.

A high end 1.2 Squid uses asyncronous I/O operations. As long as
the OS filesystem can handle large number of parallell operations on
different files Squid is not slowed down much. I/O performance close to
linear to the number of disks.

Yes, there will be excra writes (and reads) done by the filesystem, and
this causes some performance degration, and you may need to have more
disks to get the same performance as log based filesystems.

The goal is to make Squid scale a lot better on both disk and network
performance, without the need of RAID0 or log based filesystems. You are
ofcourse allowed to use log based filesystems or RAID0 to improve I/O
performance even further on the same hardware.

> Sure, you can often turn many of these off by disabling synchronous
> writes of metadata, but that can have adverse side effects too.

Not a good idea if the cache is valuable and there is a moretately high
risk that the machine hard-reboots. If it isn't then you can change the
filesystem mount procedure to run a newfs instead of fsck on unclean
cache filesystems, provided that the cache is on a separate filesystem
of course.

> > The same applies to if a machine crashes & reboots. It does
> > not matter if you loose some of the cached files.

I should have said that unfortunately there is no known filesystems that
can take advantage of this (except perhaps the custom filesystem used in
NetCache Appliance, which is of very limited use to Squid people...)

> Perhaps some of the work done with INN2 on CNFS could be of value
> here, as news seems to at least superficially similar in its use
> of filesystems for object storage (large numbers of directories,
> with clusters of generally small files).

Unfortunately Squid caches and News has very different properties. CNFS
does not apply well to Squid I am told. Reason: CNFS assumes a ordered
FIFO type replacement, Squid uses a more random like replacement
procedure.

---
Henrik Nordström
Sparetime Squid Hacker