111 replies to this topic

[eric]

Aha! That tmp directory 1) looks suspiciously empty, like it's emptied often and 2) TubePress's sytem cache was created sometime within the past day. I really think that changing your HTML cache directory will be a win for us.

 

Should I change all of them, do you think?

 

I would, yes. The HTML cache is first in the processing order, so a hit there will skip all the other caches. But for consistency and sanity's sake I would get everything into a directory under wp-content/tubepress-content.

[James Israel]

All the pagination pages are loading in about 3 seconds today, so it looks like we've solved the problem. However, I've seen them load faster before. Is there any chance putting the cache directory under the regular site folder tree is the reason for the slight delay? I could put it outside that. Anyway, huge improvement!

 

Will change all the cache directories to that for now, though, for consistency.

[eric]

Hmm, I'm not sure I'm convinced that it's fixed. After I read your reply I went to test and the pages took a long time to load. Let me do some more testing today..

 

Is there any chance putting the cache directory under the regular site folder tree is the reason for the slight delay?

 

It's possible, but very unlikely. Since the rest of your wp-content directory doesn't exhibit slow performance, I see no reason that any of its subdirectories (including the cache directory that you created).

 

I'll report back after some tests. Thanks!

[James Israel]

You're right, doing the same for me now.

[eric]

What's the output of ls -alh for your cache directory? I'd like to see if it's somehow getting regenerated. You have disabled cache cleaning so it should be working..

[James Israel]

The directory was empty. I forgot to check the ownership, it was different than the others, so I changed it to www-data:www-data like the others. For some reason, the way the server's set up, and the fact that I'm not root, everything I create has a different ownership, which causes problems.

 

I think it'll write okay now, let's see.

 

P.S. I ran the script manually, but nothing appears in the cache directory. Does the script need something in it telling it where to save the cache, maybe?

[eric]

I loathe managing filesystem permissions for web servers!

 

P.S. I ran the script manually, but nothing appears in the cache directory. Does the script need something in it telling it where to save the cache, maybe?

 

The script should be good to go as it is - TubePress should use the settings you gave it from WP Admin > Settings > TubePress > Cache and store the cache in the right place.

 

I just checked your site now and the pages still seem slow. I'm assuming that the cache directory is still empty? I would suggest the following:

1. Enable debugging (at least temporarily) from WP Admin > Settings > TubePress > Advanced
    
2. Put your site into debug mode and see if there are any errors in there related to the cache directory. TubePress is pretty chatty about this. Feel free to send me the debug output if you don't feel like peering through the reams of debug messages!

[James Israel]

I did that, but don't see anything. Shouldn't it show up at the end of the page source? I even turned off Cloudflare and W3T Cache, but didn't see anything.

[eric]

I was able to view the debug output (screenshot). Relevant messages from there:
 

[10:16.366110 - INFO] Starting to build HTML cache driver. User candidate is "http://www.humortime...nt/html-cache/"
[10:16.366175 - INFO] "http://www.humortime...nt/html-cache/"is not a directory. Let's try to create it...
[10:16.366227 - INFO] We don't have a directory that we can use for the HTML cache. Giving up and falling back to system directory.
[10:16.366273 - INFO] Final HTML cache directory is "/tmp/tubepress-system-cache-682ae65c6391c22089b1aa778f61a274/cache-html"

 

This is actually good to see, as it explains why things aren't working. For the cache directory, you need to give TubePress a filesystem path (i.e. /home/humortimes/public_html/wp-content/tubepress-content/html-cache) but instead you gave it a full URL. TubePress realized that this wasn't a filesystem path and instead falls back to the system temp directory.

 

So the solution is simple: Go to WP Admin > Settings > TubePress > Cache and update the HTML cache directory to /home/humortimes/public_html/wp-content/tubepress-content/html-cache. You should then see that directory start to fill up with the cache data.

 

Give that a try?

[James Israel]

Okay, done, and it started to fill up right away. But then, after I opened a couple pages, the cache info disappeared. Also, even though I click on a pagination page, then go to another one and then come back, it is slow loading. Usually, it'll hold the cache at least for a bit so that would load fast the second time. Weird.

 

Also, even after flushing the cache and turning off Cloudflare, the debug info continues to say:

46:12.753407 - INFO] 46:12.728972 No readable settings file at /home/humortimes/public_html/wp-content/tubepress-content/config/settings.php
[46:12.753451 - INFO] 46:12.729075 Add-on blacklist: []
[46:12.753491 - INFO] 46:12.729098 Class loader enabled? yes
[46:12.753531 - INFO] 46:12.729119 Cache directory: /tmp/tubepress-system-cache-682ae65c6391c22089b1aa778f61a274

 

So it's still looking for the old cache. And why does it say no readable settings at /home/humortimes/public_html/wp-content/tubepress-content/config/settings.php??? And why is there not even a folder at /home/humortimes/public_html/wp-content/tubepress-content/config???

[James Israel]

Okay, I created a config folder there, in case it's needed. Now, here's something VERY strange: something keeps changing the permissions for the html-cache folder! When I checked it just now, it was set at chmod 750, and I know I had set it at 775 when I created it. So, I changed it again, and it definitely took, but within a couple minutes it was back to 750! Also tried 755, and that didn't stay either. Never seen that happen before, do you know what could be causing it? That's probably why it's not getting written to.

 

Also, while keeping an eye on the html-cache folder, stuff keeps appearing, then disappearing quickly. I'm leaving the debug on for now.

 

Also, I'm creating different directories for the different caches, like html-template. Thought maybe they were interfering with each other somehow.

 

Once I did that, the html-cache directory seemed to stabilize, and stopped erasing itself.

[eric]

Also, even after flushing the cache and turning off Cloudflare, the debug info continues to say:

46:12.753407 - INFO] 46:12.728972 No readable settings file at /home/humortimes/public_html/wp-content/tubepress-content/config/settings.php
[46:12.753451 - INFO] 46:12.729075 Add-on blacklist: []
[46:12.753491 - INFO] 46:12.729098 Class loader enabled? yes
[46:12.753531 - INFO] 46:12.729119 Cache directory: /tmp/tubepress-system-cache-682ae65c6391c22089b1aa778f61a274

 

So it's still looking for the old cache. And why does it say no readable settings at /home/humortimes/public_html/wp-content/tubepress-content/config/settings.php??? And why is there not even a folder at /home/humortimes/public_html/wp-content/tubepress-content/config???

 

You can ignore those messages. They're referring to settings.php, which is primarily used by folks that use TubePress in standalone PHP environments. Your debug output shows that TubePress has accepted the cache directory that you have supplied:

 

 

 

[13:55.236298 - INFO] Starting to build HTML cache driver. User candidate is "/home/humortimes/public_html/wp-content/tubepress-content/html-cache"

 

There's a similar message for the API cache directory, so those settings are looking great now. This explains why you see the directory filling up, so now we just need to figure out why you're seeing files disappearing.

 

Also, I'm creating different directories for the different caches, like html-template. Thought maybe they were interfering with each other somehow.

 

Once I did that, the html-cache directory seemed to stabilize, and stopped erasing itself.

 

Aha! I see that you have the API cache cleaning factor set to 20, which means that every 5% of requests, the entire cache will be cleared. Since the API cache and HTML cache were sharing the same directory, I'll bet that the API cache was inadvertently clearing the HTML cache. Is your head spinning yet? Anyway, I'm glad you separated those directories and I'll bet that will solve the disappearing files issue.

 

something keeps changing the permissions for the html-cache folder! When I checked it just now, it was set at chmod 750, and I know I had set it at 775 when I created it. So, I changed it again, and it definitely took, but within a couple minutes it was back to 750! Also tried 755, and that didn't stay either. Never seen that happen before, do you know what could be causing it? That's probably why it's not getting written to.

 

Hmm. Is the top level directory (/home/humortimes/public_html/wp-content/tubepress-content/html-cache) showing 750? Or is it just the subdirectories? What is the ownership of that directory and its subdirectories? Feel free to post the output of

ls -alh /home/humortimes/public_html/wp-content/tubepress-content/html-cache/

and I'll be glad to examine the ownerships and permissions. You basically just need to make sure that your PHP process has read & write access to that directory. So 750 would be fine as long as the ownership is www-data (or whichever process runs PHP). TubePress shouldn't be changing any ownerships or permissions...

[James Israel]

Yeah, I think that must've been the problem all along, the api cache clearing setting was erasing everything, since they were all using the same folder. You might need to patch the program to write those different caches to different subfolders automatically, yes?

 

Question: Since the api cache is set to expire every six hours anyway, what does it need a clearing setting for? Should I just set that to 0? I think 20 was the default, so I just left it there.

 

Also: Whenever someone goes to a page, if there's no cache for it already, their visit triggers a cache write, correct? Then the next person gets the cached info, and so on until the cache expires, right? So, if we're getting enough visitors, maybe I don't even need the caching script. If that's the case, I may experiment with removing the cron job, now that we've solved the disappearing cache problem.

 

Yeah, 750 seems to be fine, since it's able to write to it. However, all the cache directories I created (html-cache, api-cache, html-template) changed to 750 by themselves - I think your plugin must be doing it, don't you? Don't know what else would cause it, it doesn't happen to any other directories.

 

One other issue: The main video shown (the big one at the top) for each page in pagination is the same as the first pagination page. Shouldn't it change to show the first video on each page?

[eric]

Yeah, I think that must've been the problem all along, the api cache clearing setting was erasing everything, since they were all using the same folder. You might need to patch the program to write those different caches to different subfolders automatically, yes?

 

Yes! That's a bug and I'll fix it for the next release.

 

Question: Since the api cache is set to expire every six hours anyway, what does it need a clearing setting for? Should I just set that to 0? I think 20 was the default, so I just left it there.

 

When items in the cache expire, they aren't actually deleted (i.e. they still take up space on the filesystem). So TubePress gives you a way to clear the entire cache every now and then, just in case it grows really large with expired items. I would suggest that you set the API cache cleaning factor to 1 (rare cleaning) and the HTML cache to 0 (no cleaning), but of course this is up to you.

 

Also: Whenever someone goes to a page, if there's no cache for it already, their visit triggers a cache write, correct? Then the next person gets the cached info, and so on until the cache expires, right? So, if we're getting enough visitors, maybe I don't even need the caching script. If that's the case, I may experiment with removing the cron job, now that we've solved the disappearing cache problem.

 

You are correct. Any requests to TubePress will look in the cache and, if nothing is there, a write will be triggered. Depending on your traffic levels, you might be able to get away without a cron job.

 

However, all the cache directories I created (html-cache, api-cache, html-template) changed to 750 by themselves - I think your plugin must be doing it, don't you? Don't know what else would cause it, it doesn't happen to any other directories.

 

Had I not written the code, I too would think that TubePress was changing permissions on the directories. If the top-level cache directory doesn't exist, TubePress tries to create it with chmod 755. Any internal directories should be created with chmod 770. So 750 is very strange and I'm frankly not sure why that's happening. I'll investigate.

 

One other issue: The main video shown (the big one at the top) for each page in pagination is the same as the first pagination page. Shouldn't it change to show the first video on each page?

 

In case the user is currently watching that video, we don't want to yank it away from them so we leave it there. I think that perhaps we should add an option to allow the behavior that you described. I've filed an internal ticket. Unfortunately it's not a trivial change so I can't suggest a hotfix :/

 

Let's continue to monitor the performance of your gallery to make sure that the cache holds up.

[James Israel]

Okay, thanks for everything. I think we've got it now, everything is loading very fast! Hurray!!!

[James Israel]

Well, sheesh! Looks like I spoke too soon. They're all loading slow again, even the initial pages! The cache directories are not empty.

[eric]

I noticed the same. Send another screenshot of WP Admin > Settings > TubePress > Cache ?

 

We're really close!

[James Israel]

That's weird, I don't see how to attach a file, I forget how I did it before. The My Media button doesn't allow for uploading anything new...

[James Israel]

Okay, I see it now, it's the "More Reply Options" button ... here's the screen shot.

 

I must say, though, that the 48 kb limit is quite small. Hope you can read this screenshot.

Attached Files

•  corel934.jpg   46.68KB   0 downloads

[eric]

Those settings look good. I just ran a copy of the priming script and all your galleries are super fast right now. I'll check again later today and over the weekend. Fingers crossed that they stay fast..