Re: [squid-users] Cache Partition Problem

 On Wed, 2 Mar 2011 07:10:48 -0500, wrote:
> Hi,
> I have Four (4) partitions for my cache directory and occasionally
> one of
> the cache directory will get full and low disk error is displayed and
> squid stop responding to request. After some times or days squid
> rebuilds
> and said cache directory is free of space, why?

 Sounds like the storage index journal (swap.state) is filling up the
 whole spare space in the cache. It gets erased at the start of a slow
 "DIRTY" disk scan which appears as disk emptying. The scan also purges
 obsolete objects, which frees even more space.

> Why is squid keeps rebuilding most of the time?

 When squid runs out of disk space it crashes. The daemon manager
 restarts it, and the startup process begins a slow "(DIRTY)" scan of the
 entire cache area file-by-file to rebuild the index and swap journal
 gets reset.

> Is there any configuration option to adjust the cache dir
> re(building)
> operations?

 Log rotation (squid -k rotate) performs all the log, journal and index
 cleanups Squid requires.
 Normally it is sufficient to do this daily, but it depends on your
 available disk space and traffic loads. Some networks go for weeks, some
 need to do it hourly. Look at how long your squid is staying up for and
 rotate at some period less than this time.

> Below is extract from my cache log.
> Regards,
> Yomi.
>
>
> 2011/02/16 09:25:37| Starting Squid Cache version 2.7.STABLE8 for
> i686-pc-winnt...
 ...
> 2011/02/16 09:25:37| Swap maxSize 89128960 + 1048576 KB, estimated
> 6936733
> objects
 ...
> 2011/02/16 09:25:37| Rebuilding storage in c:/squid/var/cache (DIRTY)
> 2011/02/16 09:25:37| Rebuilding storage in d:/squid/var/cache (DIRTY)
> 2011/02/16 09:25:37| Rebuilding storage in e:/squid/var/cache (DIRTY)
> 2011/02/16 09:25:37| Rebuilding storage in f:/squid/var/cache (DIRTY)
 ...

 Yep, crash recovery.

 ...
> 2011/02/16 09:43:07| Finished rebuilding storage from disk.
> 2011/02/16 09:43:07| 44358346 Entries scanned
> 2011/02/16 09:43:07| 0 Invalid entries.
> 2011/02/16 09:43:07| 0 With invalid flags.
> 2011/02/16 09:43:07| 37246661 Objects loaded.
> 2011/02/16 09:43:07| 0 Objects expired.
> 2011/02/16 09:43:07| 37242428 Objects cancelled.

 I'm not completely certain, I think each of these is a disk object
 erased.

> 2011/02/16 09:43:07| 842589 Duplicate URLs purged.

 Each of these duplicates is a disk object erased.

> 2011/02/16 09:43:07| 6269159 Swapfile clashes avoided.
> 2011/02/16 09:43:07| Took 1049.9 seconds (35476.7 objects/sec).
> 2011/02/16 09:43:07| Beginning Validation Procedure
> 2011/02/16 09:43:07| Completed Validation Procedure
> 2011/02/16 09:43:07| Validated 4170 Entries
> 2011/02/16 09:43:07| store_swap_size = 62576k

 After re-scanning and purging old data, your cache index apparently
 takes 63 MB to describe the contents of the cache. From this point until
 next crash it will grow at one entry per client request.

 Amos
Received on Wed Mar 02 2011 - 22:48:24 MST