Re: [squid-users] RE: Memory leak in 3.2.5

On 27/12/2012 4:52 a.m., carteriii wrote:
> I came back to these discussions because I have 3.2.3 and see some excessive
> memory usage and found this discussion. I don't know enough to know if my
> situation is the same, but it feels like a memory leak and seems relevant to
> this discussion. From the cachemgr.cgi "info" page, I found the "space in
> area" vs. "total in use" numbers particularly confusing:
>
> Memory usage for squid via mallinfo():
> Total space in arena: 1360760 KB
> Ordinary blocks: 1360469 KB 25 blks
> Small blocks: 0 KB 0 blks
> Holding blocks: 25360 KB 6 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 291 KB
> Total in use: 291 KB 0%
> Total free: 291 KB 0%
> Total size: 1386120 KB
> Memory accounted for:
> Total accounted: 1305182 KB 94%
> memPool accounted: 1305182 KB 94%
> memPool unaccounted: 80938 KB 6%
> memPoolAlloc calls: 7170387
> memPoolFree calls: 7170578
>
> If I'm not mistaken, this says squid is consuming over 1GB of memory but
> only 291 KB are being used. memPool says that 94% is "accounted", so
> wouldn't that imply this is not a memory leak? Or is it a memory leak and
> squid knows about it?

There is 1.3 GB allocated to Squid this is agreed by mallinfo and
memPools. ~50MB of that is being tracked by MemPools so is technically
'free' but only for Squid usage.
Not sure what mallinfo is going on about that 291KB for, it is less than
the amount mallinfo is tracking, if this is a 64-bit system I would put
it down to the well-known 32-bit wrap bugs in mallinfo.

It is memory use and Squid knows about it (both agree on the total size,
leak is usually disagreement with mallinfo larger). There is no
identification from the report whether it is leaked internally to Squid
or actually used by some transaction or data cache.

> Am I reading this correctly? This is from a system
> with light usage that has only been running about a day. Prior to getting
> those statistics, I did set these two options in my configuration file to
> try to keep the usage down, but it didn't seem to help:
>
> cache_mem 128 MB

This is total size of the RAM object storage, equivalent to a cachr_dir
in memory. Squid uses a lot of additional memory, so this is more like a
minimum than a maximum of Squid memory usage.

> maximum_object_size_in_memory 128 KB
>
> The "mem" report shows me the following top-4 lines
>
> Pool Allocated
> In Use
> (#) (KB)
> (#) (KB)
> 32K Buffer 20955 670560 20955
> 670560
> 16K Buffer 21380 342080 21378
> 342048
> 8K Buffer 11547 92376 11546
> 92368
> Short Strings 1851173 65081 1851173
> 65081
>
> What are the generic 32/16/8K buffers and what is the next thing I can
> investigate to figure it out? I'm in way over my head and not sure where to
> go next.

See the cache manager "memory" report to get details of what allocation
types the memPools are tracking. Read this thread from the beginning to
see how Mike debugged the ident usage and compare it to your repliation
of what he did.

Amos
Received on Thu Dec 27 2012 - 01:39:48 MST