php5-fpm high cpu usage and 111 connection refused

Hi, I recently have a problem on nginx caching. My daily traffic has dropped 20%, so please help!

I’m on:

Linode 1Gb VPS, Ubuntu 12.04 32bit LTS, Nginx 1.4.1, Php 5.3.10, Mysql 5.5.34.

Problem:

  1. random Nginx error: "connect failed() (111 connection refused) while connecting to upstream, client: IP, server: DOMAIN, request:“GET url”, upstream: “fastcgi 127.0.0.1:9000”

  2. php5-fpm usage (4% ~ 70%, even more):

Cpu(s): 9.3 %us, 0.5% sy, 59.8%id …
PID USER PR NI VIRT RES SHR %cpu %mem command
xxx xxxx xx xx 146m 25m 15m 48 2.6 php5-fpm
xxx xxxx xx xx 146m 26m 19m 40 2.7 php5-fpm
xxx xxxx xx xx 146m 27m 17m 38 2.8 php5-fpm

  1. out of command: ll /proc/15919/fd/

./
…/
0 -> socket:[33033]
1 -> /dev/null
10-> /tmp/.xcache.0.16.1649438898.lock (deleted)
11-> /tmp/.xcache.0.17.xxxxxxxx.lock (deleted)
16-> anon_inode:[eventpoll]
2 -> /dev/null
3 -> /tmp/.xcache.0.0.xxxxxxx.lock (deleted)

  1. PHP5-fpm log:

random edit-comment.php execution time out (72.xxxx sec), terminating.
random wp-cron.php.php execution time out (72.xxxx sec), terminating.

My config files:

Nginx.conf:

user www-data;  
worker_processes 6;  
pid /var/run/nginx.pid;  

events {  
	worker_connections 768;  
	 multi_accept on;  
}  

http {  

	##  
	# Basic Settings  
	##  
        add_header rt-Fastcgi-Cache $upstream_cache_status;  
	sendfile on;  
	tcp_nopush on;  
	tcp_nodelay on;  
	keepalive_timeout 15;  
	types_hash_max_size 2048;  

	include /etc/nginx/mime.types;  
	default_type application/octet-stream;  

	##  
	# Logging Settings  
	##  

	#access_log /var/log/nginx/access.log;  
        access_log off;  
	error_log /var/log/nginx/error.log;  

	##  
	# Gzip Settings  
	##  

	gzip on;  
	gzip_disable "msie6";  

	gzip_vary on;  
	gzip_proxied any;  
	gzip_comp_level 3;  
	gzip_buffers 16 8k;  
	gzip_http_version 1.1;  
	gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;  

        large_client_header_buffers 4 16k;  
	include /etc/nginx/conf.d/*.conf;  
	include /etc/nginx/sites-enabled/*;  
}

Site config under site-available folder:

fastcgi_cache_path /var/log/nginx-cache levels=1:2 keys_zone=WORDPRESS:15m max_size=60m inactive=60m;  
fastcgi_cache_key "$scheme$request_method$host$request_uri";  
fastcgi_cache_use_stale updating error timeout invalid_header http_500 http_503;  

server {  
	listen   80; ## listen for ipv4; this line is default and implied  
	listen   [::]:80 default ipv6only=on; ## listen for ipv6  

	root /usr/share/nginx/www;  
	index index.php;  

	# Make site accessible from http://localhost/  
	server_name MYDOMIAN_NAME;  
         
        set $no_cache 0;  
         
        if ($request_method = POST) {  
                set $no_cache 1;  
        }  
         
        if ($query_string != "") {  
                set $no_cache 1;  
        }  
         
        if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|/wp-login.php|/wp-.*.php|/sitemap.xml)") {  
                set $no_cache 1;  
        }  
         
        if ($http_cookie = "PHPSESSID") {  
                set $no_cache 1;  
        }  

	location / {  
		# First attempt to serve request as file, then  
		# as directory, then fall back to index.html  
                
		try_files $uri $uri/ /index.php?$args;  
	}  

	location /doc/ {  
		alias /usr/share/doc/;  
		autoindex on;  
		allow 127.0.0.1;  
		deny all;  
	}  

       location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {  
                 root /usr/share/nginx/www;  
                 access_log off;  
                 log_not_found off;  
                 expires 180d;  
                 valid_referers none blocked ubuntuhandbook.org www.ubuntuhandbook.org ~\.google\. ~\.yahoo\. ~\.bing\. ~\.facebook\. ~\.fbcdn\.;  
                 if ($invalid_referer) {  
                      return 403;  
                  }               
        }  
	error_page 404 /404.html;  

	# redirect server error pages to the static page /50x.html  
	#  
	error_page 500 502 503 504 /50x.html;  
	location = /50x.html {  
		root /usr/share/nginx/www;  
	}  

	# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000  
	#  
	location ~ \.php$ {  
        	#fastcgi_split_path_info ^(.+\.php)(/.+)$;  
		# NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini  
	  
		# With php5-cgi alone:  
		fastcgi_pass 127.0.0.1:9000;  
        	# With php5-fpm:  
		#fastcgi_pass unix:/var/run/php5-fpm.sock;  
		#fastcgi_index index.php;  
                try_files $uri /index.php;  
		include fastcgi_params;  
                
                fastcgi_cache_bypass $no_cache;  
                fastcgi_no_cache $no_cache;  
                 
                fastcgi_cache WORDPRESS;  
                fastcgi_cache_valid 60m;  
	}  

	# deny access to .htaccess files, if Apache's document root  
	# concurs with nginx's one  
	#  
        location ~ /purge(/.*) {  
            fastcgi_cache_purge WORDPRESS "$scheme$request_method$host$1";  
        }         

	location ~ /\.ht {  
		deny all;  
	}  
}

I’m new to all this stuff, soooooo much thanks for any help!

If CPU load spikes are temporary AND you are using nginx-helper, first make sure archive purging is disabled. It can create huge number of internal PHP processes if site has thousands of posts.

Also, did you verify if cache is working properly by following checklist? Link - http://rtcamp.com/wordpress-nginx/tutorials/checklist/

Cache is working (rt-Fastcgi-Cache:Hit in http://web-sniffer.net/ result) and php5-fpm cpu usage seems good now. What I did is disable Purge Archives, purge post when comment deleted, purge homepage when post is trashed, and reduce (a little) value of pm.max_children

Thanks for you help, Rahul Bansal

There are still a few questions, thanks for your time!

  1. Is it good to switch fastcgi_cache_path from /var/log to /var/run? I changed it, but nginx said “failed to open dir /var/run/nginx-cache”. It seems that www-data doesn’t have enough privilege, is it good to change user & group in Nginx.conf and www.conf from www-data to root?

  2. Nginx-helper said following when I deleted spam comments:

2014-01-01 09:48:46 | INFO | * * * * *
2014-01-01 09:48:46 | INFO | * Blog :: xxxxx(1).
2014-01-01 09:48:46 | INFO | * Post :: xxxxxxxx (1564).
2014-01-01 09:48:46 | INFO | * Comment :: 84047.
2014-01-01 09:48:46 | INFO | * Status Changed from to spam

Does this mean the plugin purge the post cache when I deleting spams?? I have disable “purge post when a comment is unapproved/deleted.”

  1. I just heard ngx_pagespeed (mod_pagespeed for nginx), can I use it together with fastcgi_cache.

Much thanks for your help.

Answers for your questions:

  1. Don’t bother much about cache path. Any location will work. OS are smart enough to keep frequently accessed files in RAM.

  2. Looks like a bug. If a comment is spam then deleting it shouldn’t trigger purge. We will look into this.

  3. pagespeed is good. I manage to got it working with static sites but for wordpress sites, it wasn’t working nicely so dropped it. I might give it a try again in future. If it works for you, feel free to use it.

OK, thanks again!

Glad to know I could help. Marking this issue as solved.

Feel free to open new support request if you need any help.