Re: Curious distribution of objects per directory, HASH is buggy or non uniform ???

Pedro Ribeiro wrote:
>
> While analyzing the use of the cache disk of my proxy
> (squid1.2beta21+patch1+patch2) running in a Alpha433au under Digital Unix
> 4.0d and i'v noticed something very strange:
>
> issuing the command:
>
> du -k -s cache/*
>
> the results were surprising:
>
> 869047 cache/00
> 919019 cache/01
> 322627 cache/02
> 260 cache/03
...

This is intended.

> the same results are obtained for the second level dirs. i.e. only
> the three first dirs are used.

This is NOT intended, and not how my test Squid behaves. The usage in L2
directories are more orless evenly distributed in each used L1 directory
(32 files in each used L2 directory).

Squid tries to allocate files in as few directories as possible, while
keeping each L2 directory small enought to limit search times.

> i think that this could impact very negativly in the squid
> performance ...

On the contrary. It helps to minimize the memory used for directory
lookups. Of course based on the assumtion that enought L2 directories
get used to keep each directoryfile in a reasonable size.

> I'm i missing something ?, the hash function used should have
> a uniform distribution no ??

The files are not allocated using a hash. The allocation key is simply a
unique number starting at 0 ending at "max swap files". Files are stored
using the lowest available swap file number in each cache_dir.

---
Henrik Nordström
Sparetime Squid Hacker