111 replies to this topic

[James Israel]

Sad to say, even the main pages are slow now (though not as slow as the pagination pages). It's like nothings getting cached, even api. But the folders are populated with files. I don't get it.

[eric]

Definitely frustrating! FWIW the main gallery pages don't seem slow on my end, but the pagination is definitely not caching as it should.

 

I've just created a Pingdom check to monitor one of the pages of Gallery 1. Every 15 minutes it will request the page and monitor how long the response takes. After a few hours we should have some performance data to get an idea of when the cache fails. Maybe that will clue is in to what's going on.

 

Would you be able to zip up the entire html-cache directory and send it to us? I'd like to examine it to see if I can find anything that looks wrong.

 

Any chance you are willing and able to provide me with SSH access to the server? I'd be glad to login and investigate. If not (I understand!) here's the experiment that I would run:

1. Empty the entire HTML cache directory (rm -rf html-cache/*)
    
2. Visit one of your galleries with your web browser, then ensure that the HTML cache directory has new contents from the previous step.
    
3. Wait 2 minutes.
    
4. In your web browser, click on one of the page numbers for the gallery from Step 2.
    
5. Run find html-cache/ -type f -cmin -1 to find the cache file that was created from Step 4. It should be a PHP file.
    
6. Open up this PHP file to view its contents. Near the top, there should be an "expiration" number (e.g. 1442295002), and below that there should be a "data" entry that contains HTML.
    
7. Enter the expiration into this converter to see the human-readable expiration time for the page. It should be just under one day from the present.

If you can perform and verify all of those steps, then we can confidently say that the cache files are getting written correctly and that something is modifying the cache down the road.

 

Thoughts? I feel terrible that we're dragging you through this! Very sorry for all the trouble.

[James Israel]

Okay, sent you that zipped file, and converted the expiration date, which came out as Sat, 19 Sep 2015 15:56:08 GMT. I converted that to pacific time (our time) and it was 8:56 AM Saturday, Pacific Time (PT). I did this at 12:30 PM on Friday, so it would appear the expiration time is three plus hours short of 24 hours, for some reason.

 

I also sent that php file.

 

No problem for 'all the trouble,' however, that said, any chance you'd give me a lifetime subscription?

[eric]

Thanks - I received the files you sent and was able to confirm that they all look normal and the expiration times are (generally) accurate. I'm not sure why in your experiment it was only showing an expiration of 21 hours (instead of the expected just under 24), but let's put that aside for now as we're not getting anywhere close to 21 hours of a working cache. I feel confident in saying that the cache files are getting written correctly, and it seems to me that *something* is interfering with the cache files.

 

I propose a simple solution: run the cron job every 3 hours. It seems that the cache works for at least 3 hours, so this should keep it speedy all day. The script will run behind the scenes and will incur a completely negligible performance hit on your server. Yes, we are glossing over the unanswered questions here and I don't feel great about that, but at least this way we could acheive the end goal and (if you'd like) continue to investigate the underlying issue.

 

Thoughts?

 

No problem for 'all the trouble,' however, that said, any chance you'd give me a lifetime subscription?

 

We don't have lifetime subscriptions (right now at least) but how about a big ol' coupon instead? I'll send you the code in a PM. You can use it for any of our add-ons (the YouTube Black Bars Remover is very popular) or to pay for your annual TubePress Pro membership.

[James Israel]

Are you sure it's a negligible hit on server resources? Also, how do you know the cache is working for three hours? Lastly, yes, I hope you'll keep investigating, I'd like to get to the bottom of this.

[eric]

Are you sure it's a negligible hit on server resources?

 

Every web server is different, so I'm not 100% sure. But only a really low-end server will not be able to satisfy the requests from the cron job simultaneously with requests from your visitors. And since you have W3TC running, most of your visitors will be viewing cached HTML which is served really fast no matter what. It's your call, of course, but I wouldn't worry about it.

 

Also, how do you know the cache is working for three hours?

 

It's just a total guess based on my testing from the past two weeks or so. It just *seems* like the cache works for a few hours.

 

Lastly, yes, I hope you'll keep investigating, I'd like to get to the bottom of this.

 

Great! I too hate unsolved mysteries so am happy to persist. I probably sound like a broken record but is there any chance you could provide me with SSH access? I'd like to get in and do some sleuthing. I'd like to get a detailed look at the filesystems on the server (i.e. anything network mounted? any weird permissions or ownerships?), which processes are running, and do some live experiments and observation with the cache.

 

If that's not feasible, no problem - please just let me know and I'll try to put together some PHP code that can help us run some tests.

 

Thanks!

[James Israel]

Okay, just got around to changing it to every 3 hours.

[James Israel]

Even the every 3 hours setting is not working. Just checked a couple pagination pages, very slow. This is weird.

 

How can I send you the SSH login info?

[James Israel]

* BUMP! *

[James Israel]

I had a thought: Could it be possible that the cache expiration setting for the api is overriding the setting for the html cache? The api cache directory setting was affecting the html cache directory before, remember? I had to make separate directories for each. So, maybe something in the code is allowing the api cache expiration setting to affect the html's. Just a thought.

[James Israel]

On second thought, since I've changed the cron job to every three hours, that shouldn't matter, as the api cache expires in six hours. But perhaps something in the code for the html cache is preventing the expiration setting from working.

[James Israel]

* BUMP! *

 

Eric, I'd really like to get you the ssh info, so you could take a peek and see what you can find, but how can I get it to you securely?