Re: [squid-users] Caching large amounts of images...

Forrest Aldrich wrote:

> We are developing an infrastructure that will serve up images directly
> from a database. The numbers will vary into the thousands and higher.
>
> In front of this process, I would like to place a cache of the most
> frequently accessed images, to reduce the load on the database server
> itself.
>
> The images are small (under 200k).
>
> My question is what is the most effective way to accomplish this
> (without a commercial cache solution like content distribution).
> Would Squid be appropriate. There are other approaches such as
> Apache's mod_cache_disk et al.

Squid would be appropriate. If your need is not too urgent, I would
suggest deploying your test on Squid2.6 (which is still in the Release
Candidate stage), as it apparently has improvements to the reverse proxy
setup. Apache's mod_proxy or mod_cache_disk would certainly be another
option. I'm not familiar enough with them to provide an opinion on
which would be better.

>
> Looking through the Squid book, there are a lot of tunables...
> including consideration of the underlying filesystem (the OS is open
> here, we can use Linux or FreeBSD (preferred)).

While the page at
http://www.squid-cache.org/Doc/Users-Guide/opt/performance.html is old,
the bullet points are still valid. The question of underlying file
system is posed on occasion to the list. I've been subscribed to the
list for close to two years now, and I have yet to see any compelling
evidence of one over the other. Use what you are familiar with. Tune
it to the best of your ability (at the very least follow the advice in
the bullets from the above link. Squid is actively supported on (at
least) Linux, FreeBSD and Windows. Pick your platform and go to town.

>
> COSS sounds interesting, but it appears to be dead - or, at least, not
> ready for production. The web page I found hasn't been updated in a
> long time and indicated development being on Windows.

There has been some recent activity on COSS (check the mailing list
archive), but it's not ready for production.

>
> Any pointers would be appreciated (I'm new to caching), thank you.
>
In no particular order:
*Stay away from regex ACLs.
*Throw as much RAM as you can afford in to the caching servers.
*Use lots of disks.
* Don't RAID your cache disks.
*Stay away from the ufs store type (aufs seems to be the more stable
than diskd, diskd might be a bit quicker).
*Keep your cache_mem setting fairly low (it's only used for objects
fetched from the network), let your system's disk caching work to your
advantage.

Chris
Received on Wed Jun 28 2006 - 12:53:22 MDT