NGinx Helper Is Caching My Forum... Disasters Ensue


#1

Hey Guys,

I have nginx helper to resolve some issues I was having with Wishlist Member and it's redirects not working.

Unfortunately, it's also caching all of my pages. It's not a big deal except it could be messing up some advanced forum voting plugins and stuff on my forum here:

http://howtofightnow.com/fighting-forum

I'm not sure that it's causing the errors on the page, BUT, simple press won't even address my support issues as long as theres something in the source code saying "Page cached by nginx helper".

I really need to figure out how to add some type of exclusion to any subpages of /fighting-forum/

Please let me know if you can help.


#2

First, Nginx-Helper do NOT create any caching issue.

Caching is done on Nginx's end only if you are using fastcgi-cache. Sample config - http://rtcamp.com/wordpress-nginx/tutorials/single-site/fastcgi-cache-with-purging/

I assume your Nginx config have a line line below...

if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {  
        set $skip_cache 1;  
}

You can add extra URL patterns to that line which you do not want Nginx to cache.

We have created another guide for cache exception here - http://rtcamp.com/wordpress-nginx/tutorials/wordpress/woocommerce/

It is mainly for Woocommerce but can be adopted for any other plugin's need.

As a last resort, share your Nginx config in case you cannot find your way through it.

I will surely have a look at it. :-)


#3

Thank you for the reply!

I don't know how to access my nginx config unfortunately... a developer set this up for me, but if you tell me where to look, maybe I can guess my way in.

That exclusion string looks like it will be perfect, the only thing that I need to ensure is that all subpages of /fighting-forum/ are not cached either. Would adding the following string to nginx config work?

($request_uri ~* "(/fighting-forum.*") { set $skip_cache 1; }

Also, you're probably right about the forum poll... however, they won't help me figure out what's wrong with it until the caching is definitely off!


#4

@Mrwizard64

You need to add that sting to correct config in correct way.

Can't be sure what will work unless I see it?

Better ask your developer to get involved in this thread.


#5

I think he's on vacation :-(

Is there any way you could point me to the nginx config and I can see if I can guess my own login details?


#6

Sorry. Its not possible to provide free support beyond this point. It will take too much of my time!

If your developer is following our guidelines, you should be able to find your config files by following http://rtcamp.com/wordpress-nginx/tutorials/conventions/


#7

@rahul bansal thank you for all of your help so far. He has been able to fix the caching issue of the forum… now I need to stop nginx from caching anything within my wp-admin. I’m getting weird caching messages any time I try to create a new forum within simple press forum console. As soon as I create a new forum, I see this in the forum slug:

new-forum-post<–cached-by-nginx-helper-at-230pm and a bunch of other stuff.

I don’t know why it’s doing this, but it’s making the creation of new forums really difficult.

My developer told me how to access the console, I guess I just need to update the caching exception command that he previously entered. Please let me know if you have any insight on this issue and thank you for your time.


#8

If timestamp in “cached-by-nginx-helper” line keep changing as you reload page, it is not cached.

I know this could get confusing but please reload page 2-3 times and check if timestamp is changing or not.

If timestamp is not changing, you can add URL pattern or something else to cache-exception list.


#9

@rahul.bansal is there any way that I can just pay you to fix the caching of my forum? My developer seems to be unable to do it, and it’s killing me inside slowly.

When I reload the page, it stops updating that forum slug.

I don’t know what’s going on, but I’ve had 33,000 visitors to my site today, and my poll is theoretically not working due to caching :frowning:


#10

@Mrwizard64 We provide Nginx services. I will email you with more details.


#11

@rahul bansal I will definitely consider hiring you guys to do that, for right now I’m trying to determine if nginx helper is even the problem. I deactivated the plugin, but after even a day, my source code reports that the page in question is STILL being cached by nginx helper. I need to get rid of this for now… it’s persistance is frustrating!


#12

Even after deleting the plugin entirely, I am still seeing that this page is cached by nginx helper. Please view the source code of the page found here.

I cant get support on my poll until that’s out of the source code… and it’s really holding my progress on this poll back.


#13

Hey Rahul, I wanted to add some details here so we can get to the bottom of this quickly.

The issue in a nutshell: Nginx Helper, despite being deleted, is still showing an HTML comment on forum pages and the timestamp is not changing.

This HTML comment has persisted through full server reboots, nginx restarts and reloads, and a move to different physical hardware. It’s almost as if the HTML itself has actually been modified.

Nginx does have the fastcgi_cache module enabled, but we’re not doing anything with it — W3TC is handling caching here.

Here’s the Nginx config: https://gist.github.com/williejackson/defdbaba9c71572abc6e

Please advise and/or email me directly, and thanks very much for your help in advance.


#14

@Wille

Nginx reboots do not clear nginx’s fastcgi_cache. This was a surprise for me also when I was digging into it last year.

You can clean existing nginx cache by running rm -rf on folder where cache is stored.

I checked nginx config you have sent. It contains only server {} block. Outside it, somewhere, you will find a line like:

fastcgi_cache_path /var/run/nginx-cache levels=1:2 keys_zone=WORDPRESS:500m inactive=60m;

/var/run/nginx-cache is location where Nginx stores cache. As name implies nginx-helper only helps in caching & optimising. Nginx handles cache on its own using fastcgi_cache.

By the way, nginx-helper has a purge button like W3TC. That purge button can clean entire cache if nginx is complied with nginx-cache-purge module. Instructions for this are here - http://rtcamp.com/wordpress-nginx/tutorials/single-site/fastcgi-cache-with-purging/

Since you are already going with W3TC, try my config http://rtcamp.com/wordpress-nginx/tutorials/single-site/w3-total-cache/ here. It has some tweaks which most tutorials about W3TC do not cover.

The way my tutorial runs try_files is also appreciated on hackernews - https://news.ycombinator.com/item?id=4578856 :wink:

Apart from that some more tweaks…

For Line - https://gist.github.com/williejackson/defdbaba9c71572abc6e#file-gistfile1-txt-L41 , add log_not_found off; access_log off; in addition to expires max;. That will reduce disk I/O for static file requests logging.

Surround sitemap related rewrite rules in location ~ ([^/])sitemap(.).xml$ { } block. It will improve nginx’s response time by few miliseconds for every request.

For poll functionality, if its ajax, then you can still cache page but allow voting. I can’t explain more without analysing poll functionality, but since its a high traffic site, it will be good to add some extra rules. I did similar thing for a woocommerce based high traffic store and posted some magic rules here - http://rtcamp.com/wordpress-nginx/tutorials/wordpress/woocommerce/

Basically, we cached store pages also till a visitor clicks add-to-cart button for the first time. Cache store pages loaded fast and increased sales by few percentage! I think you can do some conditional caching for polls.

Since you are dealing with technical part, feel free to discuss. Nginx can do wonders and I will be happy to share my knowledge :slight_smile:


#15

Rahul, I have no idea what you just said but thank you for the detailed response!


#16

My pleasure. :slight_smile:

I am sure your developer know what I am saying so no need to worry about it.


#17

@Rahul: when you say:

Surround sitemap related rewrite rules in location ~ ([^/]*)sitemap(.*)\.xml$ { } block. It will improve nginx’s response time by few miliseconds for every request.

what exactly do you mean? YOAST’S Wordpress SEO pöugin suggests to add these lines for sitemaps:

rewrite ^/sitemap_index\.xml$ /index.php?sitemap=1 last; rewrite ^/([^/]+?)-sitemap([0-9]+)?\.xml$ /index.php?sitemap=$1&sitemap_n=$2 last;

How should that look if I follow your advice?


#18

I will post details (why part) later in tutorial series.

Below is code snippet we use on rtcamp.com

location ~ ([^/]*)sitemap(.*)\.xml$ {  
    rewrite ^/sitemap\.xml$ /sitemap_index.xml permanent;  
    rewrite ^/sitemap_index\.xml$ /index.php?sitemap=1 last;  
    rewrite ^/([^/]+?)-sitemap([0-9]+)?\.xml$ /index.php?sitemap=$1&sitemap_n=$2 last;  
    access_log   off;  
}

#19