Re: [squid-users] Ramdisks

On Monday 26 November 2001 10.45, Andres Kroonmaa wrote:

> Maybe to you guys. To me it seemed pretty radical when I met it first.

WAFL is basically what you get if you take a traditional filesystem like FFS,
adopt it to the requirements of log structured and then drop the log to
optimize space efficiency of snapshots and overwritten files.

I.e. middleground between "traditional" and "log strucured", trying to
combine the best of both for the specific application.

> But I actually quoted it in regards to some of your previous comments.
> It manages metadata, it doesn't require fixed predefined structure of
> FS layout, it gives you freedom to write the stuff the way you like,
> either in chunks or filling the gaps, or at current head position, it
> scales well to stripes and RAID5 disks, it has fast checkpointing and
> thus fast recovery after a crash, allows resizing the FS on the fly.
> It is very write-optimised FS, for both small and large objects. In
> many ways it seems like perfect FS for squid.

Except that it still has many of the negative sides as well:

* Removal of data is still a relatively heavy operation requiring writes to
delete (or truncate) a file.

* The structure is likely to degrade with time as the files moves around,
requiring the use of a defragmenter to keep the "tree of blocks" nice..

* Free space will most certainly degrade and fragment over time unless there
is a defragmenter cleaning things up.

* Poor packing of small objects (not tiny, but smaller than a couple of
blocks), wasting a significant amount of cache memory and I/O bandwidth on
small obejcts relative to their size.

* The structure cannot easily be optimized for the specific application of
caching.

> There are lots of bits there to make it failsafe for NFS which squid
> wouldn't need, like NVRAM caching, or ability to restore prev versions
> of files.

Sure.

> There are few key ideas that could provide source of inspiration
> when designing your own cyclic store for squid. Not to be ignored.

True. The report covers much of the same challenging issues as many of the
log structured filesystem papers, but for Squid it is still a bit to centric
about the "normal filesystem requirements" which a web cache does not have..

It is a quite good description of the effects of wandering meta data on a
traditional FFS filesystem design, and presents these problems and how to
solve them in a way that is easy to understand to most people.

Regards
Henrik
Received on Mon Nov 26 2001 - 08:18:34 MST