Re: [SQU] URLs vs. MD5 key?

----- Original Message -----
From: "Liapis, Sam (Melb, Aust)" <Sam.Liapis@EastmanSoftware.com.au>
To: <squid-users@ircache.net>
Cc: <hno@hem.passagen.se>
Sent: Saturday, February 03, 2001 6:02 PM
Subject: [SQU] URLs vs. MD5 key?

>
> Hello fellow squid-sters,
>
> (Running squid2.3STABLE4)
>
> I'm running a significantly large squid cache whose
> content is static. I'd like to be able to restore a lost
> cache_dir (disk/partition) with the original content.
> So I'm keeping a daily backup of the swap.state files
> thinking they're much like a manifest of each cache_dir.

I presume you mean that you'll use the list list of urls with something like wget to hit the origin servers again?

> NOT so I discovered! Yes they can tell you IF they
> they have something in cache not WHAT they have
> in cache, there's a difference. The reason for this
> is because the incoming URL I understand is hashed
> through the MD5 algorithm which is only one-way.
> That means unlike the old squid 1.x swap.state file I
> can't map URLs to swap-filenames i.e. the object.

Correct.

> The alternatives I've realised is to store my "store.log"
> files which contains both URL and swap filename.
> Problem is this equates to a ludicrous amount of bytes
> in disk space since I'm looking many millions of objects.
> Similarly the other alternative to scan the header info
> of the objects in cache daily is just as painful for like
> reasons.

But less than backing up the swap dir itself. You could parse the access.log with grep and de-dup the entries and strip the unneeded
fields to create a custom record..that could run in a cron job easily enough.

> Is my understanding of the above correct and if so can
> anybody suggest anything to help me out?

Rob

--
To unsubscribe, see http://www.squid-cache.org/mailing-lists.html