Re: [squid-users] Pinning objects in Squid Cache

On 03/13/2014 09:52 AM, aditya agarwal wrote:

> We had already thought of the second option to fetch the objects at
> regular intervals so that they are always at the head of the queue in
> cache, but it doesn't seem to be very scalable as we can have 100s of
> videos which the client might want to pin to cache.

Please also keep in mind that not all cache_dir types support LRU. For
example, frequently requesting URLs in Rock storage would not help much.

> I wanted to know if there is any modification that can be done in
> squid to support pinning of objects.

Yes, it would be possible to add such support. I can think of several
options:

* A cache manager interface to pin and unpin individual cached objects.
It will not be simple if you want pinning to last across Squid restarts
or if you want to pin using regular expressions and such.

* An extension HTTP request method for the same purpose, but cache
manager may be an overall better approach, especially from access
control point of view.

* It is also possible to add an eCAP/ICAP (or even a new helper)
interface to mark misses for pinning. Adaptation makes pinning using
regular expressions easy, but it will add performance overheads unless
you are already using an adaptation service.

* Finally, one could make a pinning set configurable via squid.conf
ACLs. For mostly static sets that can be stored in a few MB or RAM
(thousands of URLs, not millions), this is probably the most efficient
and simple option.

If you decide to work on any of this, please consider discussing
specifics on squid-dev first. There are caveats related to each option
and the choice of the best option is not obvious IMO.

Cheers,

Alex.

> On Thursday, 13 March 2014 3:05 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On 13/03/2014 9:22 p.m., aditya agarwal wrote:
>
>> Hi,
>>
>> I wanted to know if there is a way to PIN certain objects in Squid's
>> cache, so that they are not removed or subjected to eviction because
>> of the LRU policy running in squid.
>>
>> Thanks, Aditya
>>
>
> That depends on what the objects are ... so "what exactly are you trying
> to achieve?"
>
> Meanwhile ... objects locally served up by Squid using the
> /squid-internal-static/ well-known URL path prefix have it. Such things
> as icons for the error pages and FTP directory listings.
> See the mime.conf file installed with your Squid on how to configure
> those URL objects.
>
>
> However, if you are wanting this for arbitrary objects served up elsewhere:
>
> * the best way is not to bother.
> Cache is a _temporary_ storage area (a type of buffer) not a long term
> archive. Correctly following HTTPP protocol ensures up to date reliable
> content at all times.
>
> * the second-best way is to simply poll your proxy with a request for
> it before the replacement policy removes it. This works on the same
> principle as prefetching and has all the same problems with generating
> correct client headers.
>
> Amos
>
Received on Thu Mar 13 2014 - 17:43:02 MDT