System and Site File Structure


#1

Hello,

I am trying to set up just a standard WordPress website with EasyEngine 4 installation on a Ubuntu 16 or Ubuntu 18 (I have tried both) and I am pretty lost on the filesystem structure of EE4. I have read the handbook couple of times now and the filesystem structure still makes very little sense to me.

When I go to /opt/easyengine/sites/example.com I see the /app directory but this directory has a new path to /var/lib/docker/volumes/examplecom_htdocs/_data. Even though I set WordPress up with a sudo user and put this user in a specific group, the /app directory is still using root:root. Once you then go to the /app directory you see the /htdocs directory with WP files.

Can someone help me better understand the EE4 filesystem structure as a whole?

Thank you


#2

Hey there,

I’ll try my best. Basically you should read any article that gives a quick introduction to Docker, like on Docker’s website for example. It’ll help you to get a better grasp of how EE4 works.

EE4 has started to use Docker containers for all services, instead of actually writing files to your system. This is a brilliant approach and I commend rtCamp for going this route as it’s used more and more in production systems for performance and stability.

Below is my understanding of EE4. If any EE4 admins/devs are reading this feel free to pitch in and correct me, I’m just trying to get the idea over as I’ve noticed since EE4’s release a lot of folks have been quite critical of the usage of Docker and the resulting file system structure change.

Basically, EE4 sets up lightweight, containerized services for each of the main components of EE – namely NGiNX, MariaDB, and Redis (PHP is run within the NGiNX container).

What this means is that no native packages are installed whatsoever. Consider the fact that when you install EE3 for example on a fresh server, there may be slight differences in the image used (such as Ubuntu 18.04 x64) across DigitalOcean, Linode, a random VPS provider, etc. There is also the possibility of services failing, package updates breaking dependencies, and singular services going down in EE3.

With Docker, these issues are mitigated since the container images for NGiNX/MariaDB/Redis are identical across every EE4 installation for everyone, everywhere. Docker Engine is the underlying orchestration platform that manages the containers, creates them, ensures they’re running, performs health checks, etc. Basically, each service is run as a Docker container without needing to modify the system.

To access your website files, EE4 creates a symbolic link that points to the persistent volume storage of the NGiNX Docker container, and places it at /opt/easyengine/sites/yoursite.com/app/htdocs.

In the /opt/easyengine/sites/yoursite.com directory you’ll also see all the configuration files available to you for editing (NGiNX, PHP, etc). While those are located there for ease of use, they’re basically linked to the persistent storage and services that run within the Docker container.

You can see a list of all the Docker volumes by running the command ls /var/lib/docker/volumes. All of those volumes represent the isolated filesystem volumes for each of your sites, databases, cache, etc. – very neat! Your website’s files are located in /var/lib/docker/volumes/yoursitecom_htdocs/_data.

Could EE4 have symlinked /var/www instead? Perhaps, and some people have mentioned it, but rtCamp is correct in following best practices since "/opt is a directory for installing unbundled packages (i.e. packages not part of the OS distribution, but provided by an independent source), each one in its own subdirectory." [0]

So, in EE3 your website would have been located at /var/www/mysite.com/htdocs. In EE4, your website is located at /opt/easyengine/sites/mysite.com/app/htdocs. In EE3, you used system packages for NGiNX, PHP, MySQL/MariaDB etc. In EE4, everything is in its own Docker container.

EE4 literally lets you use the power of Docker to run your infrastructure (which doing correctly manually is difficult, esp where networking is concerned. How EE automated and solved that is amazing).

The MariaDB container and Redis container is shared globally in EE4. This is no different in theory than EE3 installing MariaDB and Redis on your machine and having each website use it for their database and caching. In practice however, it’s different because those services are now securely containerized in EE4 and run using Docker. Docker Engine as I mentioned before, is responsible for allowing NGiNX, MariaDB, and Redis containers to actually network with each other.

Each website you create is basically a Docker NGiNX container that runs NGiNX, PHP, and serves up your website files. It communicates with the MariaDB and Redis server containers. Unlike EE3 where NGiNX was a webserver by definition on port 80/443 with virtual hosts simply in /var/www subdirectories, in EE4 there is an intermediate layer that uses NGiNX as a reverse proxy to correctly route 80/443 traffic to each backend container instead.

The benefit of this overall approach filesystem-wise is that all the dependencies and binaries are stored within the Docker containers; nothing on your system will interfere with your webserver, database, or cache because these services are all located securely within containers powered by Docker Engine.

This provides flexibility; for example EE4’s ee site create --with-local-redis option allows your NGiNX container to speak to its own little Redis container and not the global one, so one website’s cache is not shared with the other.

I hope this helps. Sorry for the long answer but trust me, EE4’s roadmap is quite on target and once you get used to it you’ll be amazed at the baseline performance increase.

Also, keep in mind that you should SSH into your server and run all your commands as root user; same goes for SFTP. This assumes you use SSH keys only, which is a fundamental security posture.

EE4 will handle the permissions correctly when creating sites and if you browse to your site, you’ll notice that the htdocs folder has correct permissions (755 + www-data). You can verify this by running stat /opt/easyengine/sites/yoursite.com/app/htdocs/.

[0] https://unix.stackexchange.com/questions/11544/what-is-the-difference-between-opt-and-usr-local

Overview of the File System tree in EE4: https://easyengine.io/handbook/global-filesystem-structure/site-filesystem-structure/

Overview of how NGiNX works on a per-site basis in EE4: https://easyengine.io/handbook/internal/nginx-proxy/

Symlinked location of Redis and MariaDB data in EE4: https://easyengine.io/handbook/global-filesystem-structure/services-filesystem-structure/


#3

It seems the app directory is a symbolic link inside the docker installation path. Hi, I’m new too. The file structure shouldn’t matter as long as the web server can read the files.


#5

@paramdeo first of all, superb writeup :heart: . You seem to have explored EasyEngine v4 in detail.

We setup Akismet and seems Akismet is having some difficulty finding actual spam as it marked your post as spam. :confused: Sorry about it and I’ve restored it now.

As for a few parts of your post.

This line is incorrect.

We have separate containers for each service. By default, the Nginx-Proxy, database and redis containers are run as global containers(one single instance on a host) which you can run per site by using --with-db and --with-local-redis flags. The site specific Nginx and PHP containers run per site.

As for running commands specific to the application, you can run your commands inside the container using the ee shell command. It gives you the ability to run the commands as www-data user. You can run it both interactively and non-interactively.


#6

Hey @mriyam.tamuli

Thanks for unflagging the post. And I appreciate the corrections!

I didn’t immediately realize the extent of the Docker volumes, thanks for clearing that up. I haven’t used the ee-shell command a lot either so wasn’t aware it ran with those specific permissions, that’s very useful for everyone to know.

Thanks again :slightly_smiling_face:


#7

Hello,

So the beginning of the year has been pretty busy so I am just now revisiting as I am planning on doing some much needed server and website updates and maintenance.

@paramdeo, thank you very much. Very awesome breakdown. I have been reading a bit more on container engines (Docker and Kubernetes) and have bookmarked a few resources to revisit. @mriyam.tamuli, thank you for the followup to the paramdeo’s breakdown.

@mriyam.tamuli, I know I am late to the party I started but I always set everything up through SSH keys as I try to follow best practices when it comes to security. So for us to run any commands as sudo users we would need to run ee shell? What about SFTP? Would we need to SFTP as root? That just doesn’t sound right to me.

Thank you


#8

I am not finding the folder of my wordpress installation in the EE4 can someone help me with this?