Brotli + EE


#1

Did anyone install and run properly Google Brotli optimization?


Recompile nginx with secure link module
#2

AFAIK, it is not possible to configure Brotli over EasyEngine so far.


#3

Yes I managed to install it and also I upgrade to the latest nginx. First you need to install ee then install stack. Then replace nginx with latest one but you need to compile from source.


#4

Here’s how I did it:

sudo ee stack install

apt-get install -y --force-yes build-essential libtool automake autoconf zlib1g-dev libpcre3-dev libssl-dev libxslt1-dev libxml2-dev libgd2-xpm-dev libgeoip-dev libgoogle-perftools-dev libperl-dev

wget http://nginx.org/download/nginx-1.13.4.tar.gz
tar -xzvf nginx-1.13.4.tar.gz
git clone https://github.com/FRiCKLE/ngx_cache_purge.git
git clone https://github.com/openresty/memc-nginx-module.git
git clone https://github.com/simpl/ngx_devel_kit.git
git clone https://github.com/openresty/headers-more-nginx-module.git
git clone https://github.com/openresty/echo-nginx-module.git
git clone https://github.com/yaoweibin/ngx_http_substitutions_filter_module.git
git clone https://github.com/openresty/redis2-nginx-module.git
git clone https://github.com/openresty/srcache-nginx-module.git
git clone https://github.com/openresty/set-misc-nginx-module.git
wget https://people.freebsd.org/~osa/ngx_http_redis-0.3.8.tar.gz
tar -zxf ngx_http_redis-0.3.8.tar.gz
mv ngx_http_redis-0.3.8 ngx_http_redis
git clone https://github.com/google/ngx_brotli.git
cd ngx_brotli
git submodule update --init --recursive

sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
sudo apt remove -y nginx nginx-common nginx-full nginx-core

cd /usr/local/src/nginx-1.13.4
./configure --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-threads --add-module=/usr/local/src/ngx_cache_purge --add-module=/usr/local/src/memc-nginx-module --add-module=/usr/local/src/ngx_devel_kit --add-module=/usr/local/src/headers-more-nginx-module --add-module=/usr/local/src/echo-nginx-module --add-module=/usr/local/src/ngx_http_substitutions_filter_module --add-module=/usr/local/src/redis2-nginx-module --add-module=/usr/local/src/srcache-nginx-module --add-module=/usr/local/src/set-misc-nginx-module --add-module=/usr/local/src/ngx_http_redis --add-module=/usr/local/src/ngx_brotli --sbin-path=/usr/sbin/nginx

make
make install
sudo systemctl unmask nginx
sudo systemctl start nginx
sudo systemctl enable nginx

#Workaround NGINX bug >> https://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1581864
mkdir /etc/systemd/system/nginx.service.d
printf "[Service]\nExecStartPost=/bin/sleep 0.1\n" > /etc/systemd/system/nginx.service.d/override.conf
systemctl daemon-reload
service nginx restart

The reason ee should install the stack first is to allow ee to configure NGINX. After the upgrade, the configuration should still be intact. I tried to replicate the current modules already being used by ee’s nginx plus the brotli module. I tested this in Debian 8 but I believe it should also work with Ubuntu 16.04.

I’m not an expert, just got all these through a lot of searching and trial and error. Let me know if it works. P.S. my current nginx setup also integrates modsecurity, so far it’s working without issues.


#5

Excellent!

I’ll use your tips to install a new server.

I wonder if an apt-get update won’t overwrite the custom Nginx.


#6

There are some minor adjustments to your code. :slight_smile:

First, you should start your code with:

cd /usr/local/src

Otherwise there will be inconsistencies later.

You should include libpam0g-dev to your apt-get install line. It is necessary to include Auth PAM module, for EE compatibility.

Also, you should git clone https://github.com/sto/ngx_http_auth_pam_module.git and include. The ./configure line should have --add-module=/usr/local/src/ngx_http_auth_pam_module.

Finally, it is necessary to add a directive brotly on to the vhost configuration file, or to the main nginx configuration file.


#7

Could you rewrite working tutorial?


#8

Yes, right we need to point to the correct directory initially. I actually copied this from my full bash script (which also upgrades to PHP 7.1) and I missed that line :slight_smile:!

I enabled Brotli by adding under the http block in nginx.conf:

brotli on;
brotli_static on;
brotli_types *;

Not sure about including

ngx_http_auth_pam_module

because the original EE NGINX compile doesn’t use it. It does appear to use:

https://github.com/protonet/nginx-debian/tree/master/modules/nginx-auth-pam

However I had problems in the past including it. So I just removed it. Basic auth still works for me without it, probably because it already includes

--with-http_auth_request_module

#9

If the nginx compilation work, ee is broken after that. You can try

ee site create yoursite.tld --wp

It will not work because ee will try to reinstall the package nginx-ee

I’m working on the new nginx-ee build, but I haven’t found the time to follow the process during the past few week. I will try to finish it before the end of august. If you want to help EE to build the new nginx packages, the tutorial is available here : https://github.com/EasyEngine/nginx-build/wiki You can join us the on channel packages if you have any question : https://easyengine.slack.com


#10

I actually had it working. I’ve created several sites using ‘ee site create’ after the upgrade and it still works. The nginx update survived and brotli is working for me. Doing nginx -v after I create the site confirms this and the headers indicate brotli compression is active.

I’ll look at the links you’ve sent in the evening.


#11

I have done a bash script to automate the process :

bash <(wget --no-check-certificate -O - https://git.virtubox.net/virtubox/ee/raw/master/nginx.sh)

Configuration : Nginx 1.13.4 OpenSSL 1.1.1-dev TLS 1.3 support Brotli

Currently tested on Ubuntu 16.04 LTS, I will try debian this morning Edit : work fine on Debian 8 jessie too. Just install sudo before launching the script :

apt install sudo

#12

Good one. Thanks for the OpenSSL tip + TLS 1.3 support.


#13

Actually, your script didn’t work for me, @virtubox. :frowning:

Nginx wouldn’t start after all the process, lots of SSL errors.

Actually, @newton’s tutorial didn’t work fine, also. I had to revert back to original EE stack: I couldn’t add new sites anymore.

Despite how much I desire Nginx with Brotli and Pagespeed, I’ll stand by any official package for EE.


#14

Worked for me. I executed @virtubox’s code after installing EE stack and removing the default nginx. His code is the same as mine, except it has support for TLS 1.3 draft.

sudo apt-get install sudo git unzip
sudo ee stack install
sudo wget -qO ee rt.cx/ee && sudo bash ee
sudo ee stack install
sudo apt remove -y nginx nginx-common nginx-full
bash <(wget --no-check-certificate -O - https://git.virtubox.net/virtubox/ee/raw/master/nginx.sh)

You should be able to do ee site create example.com --wp #or whatever directive you want without errors after running above code.


#15

Perhaps my problem was the existing vhosts in the server.

I’ll try again later with a brand new server.


#16

hey can someone provide us with a full tutorial on this. as i messed up my server with newtons tutorials and need some help so waiting for all steps to get brotli working again


#17

Hello @portofacil, I also have some issues on debian 8 (in fact, I have moved on Ubuntu 16.04 LTS on the most part of my servers). But I will try to see what are the issues on debian and I will publish another script.

I’m also working on redis-server, to use the latest release 4.0 instead of the packages available on the official repository.


#18

HI, @virtubox.

I don’t use Debian anymore. I have a dozen legacy Ubuntu 14.04 servers, and the newest boxes are already 16.04.

I really tried to compile my own Nignx with Brotli and PageSpeed, but, man, it is very hard to get everything working!


#19

I haven’t tried with Pagespeed, it seems really useless for me. I have only compiled Nginx with all modules used by EE and added TLS 1.3 support (already deployed on the virtubox) and Brotli. Anyway, I will be more available on the next few weeks, so I will try to build new nginx-ee packages with this configuration and the last Nginx stable release. If that work, I will probalby try to see if I can do the same for redis.


#20