Site down after updating to v4.0.12

Shit man, really sorry about this. Sorry I cannot be of more help at this point in time! But good luck and I will follow this thread for further issues and fixes.

Here is my quick and dirty V4 checklist for problems.

  1. Are the dockers all running?
docker ps

For a single site, there should be 7 running dockers.

Multiple sites will have additional site-specific dockers.

  1. Are all the docker networks running?
docker network ls

There should be 5.
image

  1. Are the docker networks set up correctly?
docker network inspect <network name>

What you are looking for here is the correct containers assigned to the various networks. Below is a network diagram of the ee setup. It shows the containers and the docker network. Make sure each container is setup correctly in each network.

Once you have validated this info, then you have to start checking in each container for the flow of traffic.

1 Like

This is my output from running

docker ps

CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS                          PORTS               NAMES
28bd738086fa        easyengine/postfix:v4.0.0       "postfix start-fg"       10 minutes ago      Up 10 minutes                   25/tcp              devbrightraycom_postfix_1
1b4090158073        easyengine/nginx:v4.0.0         "/usr/bin/openresty …"   10 minutes ago      Up 10 minutes                   80/tcp              devbrightraycom_nginx_1
d3e236452b6a        easyengine/php:v4.0.2           "docker-entrypoint.s…"   10 minutes ago      Up 10 minutes                   9000/tcp            devbrightraycom_php_1
1247db8ddd7e        easyengine/nginx-proxy:v4.0.2   "/app/docker-entrypo…"   17 hours ago        Restarting (0) 21 seconds ago                       services_global-nginx-proxy_1
89164495b3d1        easyengine/cron:v4.0.0          "/usr/bin/ofelia dae…"   4 weeks ago         Up About an hour                                    ee-cron-scheduler
2d294b8f854c        easyengine/mariadb:v4.0.0       "docker-entrypoint.s…"   4 weeks ago         Up About an hour                3306/tcp            services_global-db_1
978426dcaf17        easyengine/redis:v4.0.0         "docker-entrypoint.s…"   4 weeks ago         Up About an hour                6379/tcp            services_global-redis_1

My nginx-proxy seems to be restarting outside of my force restarts and it seems to lack any port information. I am assuming that could be the problem. Now to figure out how to troubleshoot nginx-proxy.

This is the results from

docker network ls

NETWORK ID          NAME                         DRIVER              SCOPE
29c594ba9ff0        bridge                       bridge              local
36d6b4abac34        dev.brightray.com            bridge              local
2d271239c2a1        ee-global-backend-network    bridge              local
bd02fc5c1efc        ee-global-frontend-network   bridge              local
0b0fe6efc4c5        host                         host                local
dc2e556e4518        none                         null                local

I am not sure what I am looking for when I try to inspect the different networks nor how to check the different containers for a flow of traffic.

Maybe stop all dockers. Then start only the nginx proxy and bring the rest up. Maybe there is a specific order in which to do things. It is quite possible.

Anyone know how I might see the reasons why the easyengine/nginx-proxy:v4.0.2 container keeps restarting?

You will need to run the following command from the shell

docker logs --tail=10 -f services_global-nginx-proxy_1

This will show the logs associated with the nginx proxy. Change the tail number to increase the number of lines shown. This will help you debug why the nginx-proxy is restarting. most likely it is a bad config file.

This will keep showing you the log files for the service until you enter a Ctrl+c.

The ee log show --global does not show any of the service related docker containers. I have entered an issue for this. https://github.com/EasyEngine/log-command/issues/5

Thank you for the info. Below is what I got from the log.

forego     | sending SIGTERM to nginx.1
forego     | sending SIGTERM to dockergen.1
Custom dhparam.pem file found, generation skipped
forego     | starting dockergen.1 on port 5000
forego     | starting nginx.1 on port 5100
nginx.1    | 2019/04/12 04:20:55 [emerg] 22#22: PEM_read_bio_DHparams("/etc/nginx/dhparam/dhparam.pem") failed (SSL: error:0906D06C:PEM routines:PEM_read_bio:no start line:Expecting: DH PARAMETERS)
nginx.1    | nginx: [emerg] PEM_read_bio_DHparams("/etc/nginx/dhparam/dhparam.pem") failed (SSL: error:0906D06C:PEM routines:PEM_read_bio:no start line:Expecting: DH PARAMETERS)
forego     | starting nginx.1 on port 5200
forego     | sending SIGTERM to dockergen.1
forego     | sending SIGTERM to nginx.1

I was looking online for the error I posted above and came across an issue posted to the
nginx-proxy repository. Issue with recent container update and SSL #1226 It looks like the issue still isn’t resolved and I am not sure how to get around it. I recommend avoiding EE v4.0.12 until they resolve the issue.

To do a temp fix, you will need to locate the following file:

dockerfiles/nginx-proxy/Dockerfile

The first line reads:

FROM jwilder/nginx-proxy:latest

This needs to be replaced with:

FROM jwilder/nginx-proxy:0.5.0

This will revert to the older proxy version and remove the error. I do not know where this file is located within the easy engine folder structure. I have looked for it, but can not find it. any help would be appreciated. Here is the source code from git hub:

Anyone have any idea where this file might be located in the EE structure?

For me, this worked:

wget -qO ee rt.cx/ee4 && sudo bash ee
sudo ee site restart example.com
sudo ee service restart nginx-proxy

Why did you run the install command again?

As it wasnt working fine, I tried to reinstall it, hopping things would go fine

Unfortunately, I wasn’t able to wait any longer for a response so I ended up having to setup a new server and move everything over. Just sucks that it happened on the “stable” release.

1 Like

hey, thanks for the solution guys!! i was also looking for the same and it was helpful.
best regards!!

That nginx_proxy container has always given problems, not just in version 4.0.12.
For me it worked to stop the container with docker stop CONTAINER-ID and another docker start CONTAINER-ID . Hope this will help anyone else having this issue.

I had the same problem after restarting server, I get the 502 error on all of my Websites.
Multiple Restarts Did not fix the Problem.

I was able to Fix this by:

ee site disable <site_name>
ee site enable <site_name>

I had to do this for each and every website.

After this, In case you get “Error establishing a database connection” message.
Try:
ee service enable db

I Hope this Helps:smiley:

I had also all these problems and some more. I used first easyengine v3 which I really liked and after switching to easyengine v4, I had problems all the time. Sites were not working, because I could not renew SSL certifcates, my HDD filled completely up with log files. Getting SFTP working was an impossible task and I had others problems which I do not remember anymore.

So I looked at Wordops a easyengine v3 fork, which looked interesting and it does have a more active community, but in the end it would have been a step back to an older system.

In end I found an article about spinupwp.com . So I got a new server. I did some testing with sites that I had running on easyengine. I moved them to spinupwp and I got the same speed.

But I got also a lot of service and an easy user interface.

I do have now right from the start for each new site a SFTP and SSH access.
It comes with an integrated backup system.
The certificates are getting renewed automatically.
The update from PHP 7.3 to 7.4 were just 2 mouse clicks.
I do get always an email when my server does need to get an update, which is again just 1 mouse click.

So I will never go back to easyengine, because spinupwp does save you a lot of time and when you do have a problem or a question you do always get the help of a very competent team.

This is also a new system and the team told me that they will implement more things. They told me today that they are planning to implement also webp images, which would help also a lot with load times, which we will properly never see here on easyengine.

This service is not gratis, but it just cost 9 dollar per month per server. So if you have some sites on just one server this the best system what you can get.

so everyone that can spare 9 dollar a month, should go in my opinion with spinupwp. It will save you a lot of time and hassle.

1 Like

I have been running EE 4.0.17 for a while now without issues until this issue happened to me after a server reboot. I initially noticed when all of the HTTPS sites went down but the sites without a certificate were still up. Firefox and Chrome were returning Connection interrupted or Empty responses.

Running

docker logs --tail=10 -f services_global-nginx-proxy_1

didn’t provide any helpful information such as what @Mr_Anonymous had here . Nginx-proxy was working.

Speaking of nginx-proxy, docker ps | grep proxy was showing me that it was running version 4.0.3 which is not available on Dockerhub anymore.

Stopping and restarting the containers, disabling the sites, rebooting the server, updating to nightly, etc. didn’t help. I had the gut feeling it was related to nginx-proxy issue 1226 referrenced here.

How to fix: Update Change the nginx-proxy version

in /opt/easyengine/services there is a docker-compose.yml where I was able to change the nginx-proxy version from 4.0.3 to 4.1.2. . Bring the EE services down, rebuild the containers (fetch the prebuilt images), bring the containers up and you should be good.

cd /opt/easyengine/services
docker-compose stop
vim docker-compose.yml #change the nginx-proxy version number to 4.1.2
docker-compose build
docker-compose up
# Still wasn't working for me so I restarted the server in case there was a magic order for EE to launch
reboot
#Sites and services were back up but not nginx-proxy, force start
ee service restart nginx-proxy

I am not sure why these images are not updated automatically and there is no information from the EasyEngine team on how to force it. I was hesitant not knowing how intereconnected EE made their containers and if anything would break. In this case, I was lucky enough to get it to work. Your mileage may vary.

I hope this can help someone save quite a few hours of troubleshooting.