Re: [squid-users] 100% CPU Load problem with squid 3.3.8

On 09/14/2013 03:44 PM, Eliezer Croitoru wrote:
> SORRY typo:
> http://www.linuxtopia.org/online_books/linux_administrators_security_guide/16_Linux_Limiting_and_Monitoring_Users.html#PAM
>
> the above can clarify more about the ulimit stuff.
>
> The basic solution is to define the soft limit in the init script.

I don't think that this is a good solution. It could work as a temporary
hook. See my thoughts below.
> I would go to make sure that the hard and the soft limits are the problem..
> like
> ulimit -Sa >>/tmp/ulimit_test
> ulimit -Ha >>/tmp/ulimit_test
>
> this will make sure that the limits problem are in the soft and hard.
>
> It's a basic linux issue which is not related to squid but more to the
> distribution and how you define ulimits
>
> I assume the limit is on the bash level rather then on the OS level.
> http://www.linuxtopia.org/online_books/linux_administrators_security_guide/16_Linux_Limiting_and_Monitoring_Users.html#Bash
>
> hope it helps clarify the issue.
>
> There could be an option that will be added to the init.d script to
> specify the ulimit soft and hard by a config file or variable.
>
> I hope to post a new script for centos in the comming weeks.
>
> Eliezer
>
> On 09/14/2013 03:33 PM, Eliezer Croitoru wrote:
>> as stated before the mentioned solution was to insert the ulimit into
>> the init script to make sure the limit is absolute!
>>
>> there might be a chance for this to solve or help solve and find the issue:
>> On 09/14/2013 12:05 PM, Mohsen Dehghani wrote:
>>> Oh , no...it is 1024
>>> thanks for the help
>>> Now I added 'ulimit -n 65000' in squid init file and the problem is
>>> resolved. But some questions:
>>>
>>> 1-why is it 1024 While I've set 65535 FD at compile time and squid user
>>> which is "proxy" has this much limitation in limit.conf file?
This is an interesting question. I guess we need someone like
package/distribution maintainer here, because I don't know why limits.d
doesn't work.
>>> 2-is it ok to increase FD limit in this way?
No, that's not a good idea. You will have the problem each time you will
try to update your squid using package manager (as you .
You have to set limits in /etc/security/sysctl.d/squid.conf file (or
file with another name). Of course, you have to find out why this
doesn't work at the moment.
>>> 3-Apearantly according to "# cat /proc/sys/fs/file-max" my os FD limit is
>>> 400577. Can I increase squid FD to it
Not really good idea to, as thus you allow squid to use all the
available system-wide file descriptors. This value doesn't seem to be
too high though so you can increase both system file descriptors
(sys.fs.file-max) and squid's one.
>>> 4-What is best FD limit for about 150Mbps bandwidth and 18000 RPM
18000 rpm means you need 18000 descriptors available. I guess it will
not be hard to find out appropriate value watching squid log and
increasing nofile system limit each time you encounter warning in
squid's log.
>>>
>>> This could be ugly troubleshooting practice, but you can try to modify your
>>> init script (or upstart job, not sure how exactly squid is being started in
>>> ubuntu). The idea is to add 'ulimit -n > /tmp/squid.descriptors' and see if
>>> the number is really 65k.
>>>
>>> On 09/14/2013 09:41 AM, Mohsen Dehghani wrote:
>>>>> I don't see any logic here. Are you sure your squid is started not by
>>> root?
>>>>> Is replacing 'root' by 'squid' or '*' solves issue as well?
>>>> When I manually start service by root, there is no file descriptor
>>>> warning and squid works as normal.
>>>> But when the system boots up and starts the service automatically,
>>>> squid runs out of FD.
>>>>
>>>> I've tested different the following settings without any luck. Every
>>>> time that the box reboots, I have to login and restart service manually.
>>>>
>>>> root soft nofile 65000
>>>> root hard nofile 65000
>>>> proxy soft nofile 65000
>>>> proxy hard nofile 65000
>>>> squid soft nofile 65000
>>>> squid hard nofile 65000
>>>> * soft nofile 65000
>>>> * hard nofile 65000

The values seems fine. What exactly the name of file you put them into?
>>>>
>>>> It seems these settings only works if the user logins to system.
>>>> My squid user is "proxy"(I configured it at the time of compile).
>>>>
>>>> Maybe some useful info:
>>>> OS:Ubuntu 12.04
>>>>
>>>> # ulimit -n
>>>> 65000
>>>>
>>>> # squidclient mgr:info | grep 'file descri'
>>>> Maximum number of file descriptors: 65536
>>>> Available number of file descriptors: 65527
>>>> Reserved number of file descriptors: 100
>>>>
>>>>
>>>>
>>>>
>>>
>>>
Received on Sat Sep 14 2013 - 15:07:19 MDT