SOS! High load after migrating to DO using easyengine


#1

Easy engine made my life easy. Setting up a new server and migration was like cakewalk and I was super happy with easy engine until I started seeing failures due to the high load on the server. I’m using hhvm but when there is a high load, I see several php5-fpm processes in the output of top. They seem to be hogging all the resources. I have to reboot everytime this happens. I looked into various options configured by easyengine but don’t know if they are the right ones. Any help would be highly appreciated.

Here is some info but do lemme know if you need me to provide more logs.

jonathanivedb@jonathanive:~$ sudo ee info
[sudo] password for jonathanivedb:

NGINX (1.8.0):

user                             www-data
worker_processes                 auto
worker_connections               4096
keepalive_timeout                30
fastcgi_read_timeout             300
client_max_body_size             100m
allow                            127.0.0.1

PHP (5.6.14-1):

user
expose_php                       Off
memory_limit                     128M
post_max_size                    100M
upload_max_filesize              100M
max_execution_time               300

Information about www.conf
ping.path                        /ping
pm.status_path                   /status
process_manager                  ondemand
pm.max_requests                  500
pm.max_children                  100
pm.start_servers                 20
pm.min_spare_servers             10
pm.max_spare_servers             30
request_terminate_timeout        300
xdebug.profiler_enable_trigger   off
listen                           127.0.0.1:9000

Information about debug.conf
ping.path                        /ping
pm.status_path                   /status
process_manager                  ondemand
pm.max_requests                  500
pm.max_children                  100
pm.start_servers                 20
pm.min_spare_servers             10
pm.max_spare_servers             30
request_terminate_timeout        300
xdebug.profiler_enable_trigger   on
listen                           127.0.0.1:9001

MySQL (10.0.21-MariaDB) on localhost:

port                             3306
wait_timeout                     30
interactive_timeout              60
max_used_connections             3
datadir                          /var/lib/mysql/
socket                           /var/run/mysqld/mysqld.sock
jonathanivedb@jonathanive:~$
jonathanivedb@jonathanive:~$ sudo mysqltuner
[sudo] password for jonathanivedb:
[OK] Logged in using credentials from debian maintenance account.
 >>  MySQLTuner 1.6.0 - Major Hayden <major@mhtx.net>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 10.0.21-MariaDB-1~trusty
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +ARCHIVE +Aria +BLACKHOLE +CSV +FEDERATED +InnoDB +MRG_MyISAM
[--] Data in MyISAM tables: 165M (Tables: 234)
[--] Data in InnoDB tables: 784K (Tables: 16)
[!!] Total fragmented tables: 4

-------- Security Recommendations  -------------------------------------------
[OK] There is no anonymous account in all database users
[OK] All database users have passwords assigned
[!!] There is not basic password file list !

-------- Performance Metrics -------------------------------------------------
[--] Up for: 7h 38m 51s (1M q [37.372 qps], 22K conn, TX: 3B, RX: 257M)
[--] Reads / Writes: 99% / 1%
[--] Binary logging is disabled
[--] Total buffers: 404.0M global + 2.7M per thread (151 max threads)
[OK] Maximum reached memory usage: 412.1M (41.45% of installed RAM)
[OK] Maximum possible memory usage: 809.8M (81.47% of installed RAM)
[OK] Slow queries: 0% (0/1M)
[OK] Highest usage of available connections: 1% (3/151)
[OK] Aborted connections: 0.03%  (6/22854)
[OK] Query cache efficiency: 46.4% (816K cached / 1M selects)
[!!] Query cache prunes per day: 47165
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 17K sorts)
[OK] Temporary tables created on disk: 7% (2K on disk / 31K total)
[OK] Thread cache hit rate: 99% (3 created / 22K connections)
[OK] Table cache hit rate: 121% (357 open / 294 opened)
[OK] Open file limit used: 53% (552/1K)
[OK] Table locks acquired immediately: 99% (46K immediate / 46K locks)

-------- MyISAM Metrics -----------------------------------------------------
[!!] Key buffer used: 21.9% (22M used / 104M cache)
[OK] Key buffer size / total MyISAM indexes: 100.0M/42.5M
[OK] Read Key buffer hit rate: 100.0% (6M cached / 3K reads)
[!!] Write Key buffer hit rate: 0.5% (1K cached / 1K writes)

-------- InnoDB Metrics -----------------------------------------------------
[--] InnoDB is enabled.
[OK] InnoDB buffer pool / data size: 128.0M/784.0K
[OK] InnoDB buffer pool instances: 1
[!!] InnoDB Used buffer: 5.47% (448 used/ 8191 total)
[OK] InnoDB Read buffer efficiency: 95.24% (8969 hits/ 9417 total)
[!!] InnoDB Write buffer efficiency: 0.00% (0 hits/ 1 total)
[OK] InnoDB log waits: 0.00% (0 waits / 1 writes)

-------- AriaDB Metrics -----------------------------------------------------
[--] AriaDB is disabled.

-------- Replication Metrics -------------------------------------------------
[--] No replication slave(s) for this server.
[--] This is a standalone server..

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
Variables to adjust:
    query_cache_size (> 16M)
jonathanivedb@jonathanive:~$
jonathanivedb@jonathanive:~$ free -m
             total       used       free     shared    buffers     cached
Mem:           994        308        685          2          1         88
-/+ buffers/cache:        218        775
Swap:         2047        615       1432
jonathanivedb@jonathanive:~$
jonathanivedb@jonathanive:~$ nproc
1
jonathanivedb@jonathanive:~$
jonathanivedb@jonathanive:~$ sudo ee site show jonathanive.com
Display NGINX configuration for jonathanive.com

server {


    server_name jonathanive.com   www.jonathanive.com;


    access_log /var/log/nginx/jonathanive.com.access.log rt_cache;
    error_log /var/log/nginx/jonathanive.com.error.log;


    root /var/www/jonathanive.com/htdocs;



    index index.php index.html index.htm;


    include common/php-hhvm.conf;

    include common/wpcommon.conf;
    include common/locations.conf;
    include /var/www/jonathanive.com/conf/nginx/*.conf;
}

jonathanivedb@jonathanive:~$
jonathanivedb@jonathanive:/var/log$ cd nginx
-bash: cd: nginx: Permission denied
jonathanivedb@jonathanive:/var/log$ sudo cd nginx
sudo: cd: command not found
jonathanivedb@jonathanive:/var/log$ su -
Password:
root@jonathanive:~# cd /var/log/nginx
root@jonathanive:/var/log/nginx# awk '{print $3}' jonathanive.com.access.log  | sort | uniq -c | sort -r
  31993 -
      1 9.353
      1 87.313
      1 84.720
      1 73.544
     17 0.000
      1 65.096
      1 61.529
      1 301.003
      1 25.091
      1 13.383
root@jonathanive:/var/log/nginx#
root@jonathanive:/var/log/nginx# awk '($3 ~ /MISS/)'  jonathanive.com.access.log | awk '{print $7}' | sort | uniq -c | sort -r
root@jonathanive:/var/log/nginx# awk '($3 ~ /BYPASS/)'  jonathanive.com.access.log | awk '{print $7}' | sort | uniq -c | sort -r
root@jonathanive:/var/log/nginx#

#2