111 replies to this topic

[James Israel]

If you're able to run it, the ownership shouldn't be a problem, right? Is there a simple way to see if the cron job is not allowed to run it? Also, can you check the script for the funny001 page and see if you can catch any kind of format error or anything? It seems to be the only gallery not getting cached.

[James Israel]

Also, what is

&tubepress_options[adjustedResultsPerPage]=4

... that you've added that to each one?

 

The only thing I have like that on the actual pages is

resultsPerPage="1"

to keep the results for each playlist or user to one each.

[James Israel]

Aug 18 18:10:01 content /USR/SBIN/CRON[31723]: (root) CMD (wget --delete-after http://www.humortimes.com/wp-cron.php2>&1 /dev/null)
Aug 18 18:10:01 content /USR/SBIN/CRON[31725]: (humortimes) CMD (/etc/prime-tubepress-funny4.sh /dev/null)

Okay, so I did a little test with a reduced script to try and see if it was running. At first, I left the ownership at humortimes, and it was set to run every 10 minutes. I kept an eye on the syslog, and it never showed, while the regular wp-cron.php job we have set up to run every 5 minutes did show up. Then I changed it to root, and it did run (see above log). So, I guess it does need to be root owned.

 

Meanwhile, I replaced the regular script with your new one. It's set to run late at night, so I'll check the pages again tomorrow.

[eric]

The ownership of the script shouldn't really matter, that's correct. You can set the ownership to anyone you like - I just suggested humortimes since that's the crontab owner and I prefer to avoid root whenever possible.
 

Is there a simple way to see if the cron job is not allowed to run it?

 
You can just copy the command (from crontab -l) and paste it into the command line - that will emulate what cron does. Does that make sense?

 

Also, can you check the script for the funny001 page and see if you can catch any kind of format error or anything? It seems to be the only gallery not getting cached.

 

I double-checked the arguments, cleared your HTML cache, re-ran the script from my machine, and confirmed that each page in funny001 was cached. So my impression is that the script is OK and we just need to get cron to behave.

 

Also, what is

... you've added that to each one, with a different number. The only thing I have like that on the actual pages is

to keep the results for each playlist or user to one each.

 

You can ignore the adjustedResultsPerPage parameter; it's used internally by TubePress. Instead you can focus on the resultsPerPage parameter, which you'll note is set to 1 as you've requested.

[James Israel]

Checked today, and no pagination caches were working, even most of the main pages were loading very slowly!

 

For example, funny002 (http://www.humortime...funny-videos-2/) loaded very slowly, as did all the pagination pages on all of them.

 

Of course, a lot of them are primed now, because I visited them.

[eric]

I visited your site this morning and noticed the same. Could you let me know what you have for your HTML cache settings at WP Admin > Settings > TubePress > Cache? I want to make sure that you have the HTML cache lifetime set to at least 86400 (one day). I'm starting to think that the cache is getting primed but then cleared... maybe.

[James Israel]

Yes, it's 86400, cleaning factor 0. I'm switching back to the original script for tonight (not the updated one you gave me), to see if that will work, cuz it seems like it did to some degree, at least, the first time.

[James Israel]

Still not working today.  

[eric]

I suppose it's possible that the cron job is running just before the cache expires, in which case the script would do nothing since it would be hitting the cache. You could try adjusting the cache lifetime to something like 23 hours and 55 minutes (86100 seconds). That way we can be sure that the script will hit an unprimed cache.

 

To further testing, I've just installed the following cron job on one of our servers:

3 */3 * * * /path/to/prime.sh >/dev/null 2>&1

So this will run the priming script every three hours. Over the next day, let's monitor your galleries to ensure that they're staying primed. Once we verify that, we can start backing it down to run less. After we ensure it's still working, we can turn off my cron job and just ensure that it's set up properly on your server. I think this path will help us get to a working solution quickly, but I'm open to ideas. Please let me know what you think. Thanks!

[James Israel]

After some testing with the crontab and scripts, I think ownership was the issue. I finally got a test script to work setting the ownership to humortimes:www-data (was root, tried www-data:www-data too). It wouldn't work before that. So, I've got the prime script set to that, let's see how it works. I've also changed the cache lifetime as you said.

 

Please turn your cron job off and flush the cache (but don't flush it until close to 2am PST, if possible), so we can see if this will do it. Let me know at what time (PST) you did it.

 

Thanks.

[James Israel]

What's weird is that now the original pages are loading very slowly, while the pagination pages are loading fast. That is actually worse...

 

You won't be able to see it, as I've loaded them all manually, so now they're cached. Maybe I should just change the other cache setting to 86100 seconds also? But those original pages should be added to the script too, I guess...

[eric]

Just now reading your message and I've just turned off my cron job (it's 12:25 pm PST on Friday). I haven't flushed any caches. Glad to hear that you were able to figure out the ownership issues with the script!

 

What's weird is that now the original pages are loading very slowly, while the pagination pages are loading fast. That is actually worse...

 

Hmm, I can't imagine that the cron script could affect the original page load. Hopefully it was just a spurious issue, but if it continues we can investigate.

 

I'll keep an eye on your galleries over the weekend and fingers crossed they'll continue to load quickly!

[James Israel]

Sad to say, the cache still isn't working. The original pages are loading fine, so that's good. But the pagination pages are as slow as ever...

 

Is it possible the cache expiration setting isn't working?

[eric]

I would guess that the cron script not executing properly. Just ran a local copy of the script to fully prime the cache and have confirmed that the pages are nice and snappy. It's 2:16 p.m. PDT on Sunday right now, so if you get this message within 24 hours please check to ensure that the pages are loading quickly. That will help verify that the cache expiration is working properly.

 

If you are willing and able to share credentials to your server with me (via a ticket), I'd be happy to login and take a look at the cron job and script. Not sure how feasible that is.

 

Remind me who your hosting provider is? Any chance they could take a look at the cron job to see if they can find anything wrong with it? I still feel we're really close to figuring it out!

[James Israel]

Checked it at 12:13pm Monday, about 10 hours later, and the pagination pages are loading very slowly (about 25 seconds on the funny001 page). So, there does seem to be a problem with the cache expiration. Could my W3T Cache plugin be interfering somehow?

[eric]

Hmm, well it sounds like the cron job is working and perhaps the HTML cache expiration is to blame. I also checked your site when I woke up the next morning and noticed that the pages were slow.

 

Could you send me a screenshot of your settings at WP Admin > Settings > TubePress > Cache? Feel free to either post it here or send it in via a ticket. I just want to confirm those settings before we dig too much further. In particular, I'm wondering if we are storing the cache files in your web server's tmp directory, and that directory is perhaps getting cleared overnight as part of usual maintenance. Just a hunch.

 

I doubt that W3TC is interfering as clicking on TubePress's page numbers should bypass WP caching plugins entirely. And I can't think of how those plugins would even be aware of TubePress's cache.

[James Israel]

Sure, here you go, in two parts, to get it all.

 

How can I check if the directory is getting cleared?

Attached Files

•  corel931.jpg   92.77KB   0 downloads
•  corel932.jpg   83.81KB   0 downloads

[eric]

Thanks! Those settings look good to me, though I'll suggest one change later in this post.

 

How can I check if the directory is getting cleared?

 

Two ways:

1. Ask your hosting provider if the server's tmp directory is cleared daily.
    
2. If you have SSH access to your server, cd to the tmp directory then run "ls -alh". This will show you the fire and directory creation times. So you could find TubePress's cache and see if it was created within the past day.

Just to further rule out any interference from the tmp directory, I would suggest that you update the "Cache directory" setting for TubePress's HTML cache. Set it to /path/to/your/wordpress/wp-content/tubepress-content/html-cache, or any other directory under your control. Make sense?

 

I'm running the script again right now and have set an alarm for myself to check your site in about 9 hours, just to gather another data point.

 

We will get this issue resolved!

[eric]

I'm running the script again right now and have set an alarm for myself to check your site in about 9 hours, just to gather another data point.

 

It's now about 9 hours later and the pagination is slow again, so it seems that the cache was cleared again :/

[James Israel]

Here is the result of the ls -alh command:

humortimes@content:/tmp$ ls -alh
total 755K
drwxrwxrwt  6 root     root     735K Sep  6 20:20 .
drwxr-xr-x 24 root     root     1.0K Mar  9 16:38 ..
drwxrwxrwt  2 root     root     1.0K Jul 28 22:21 .ICE-unix
drwx------  2 root     root      12K Apr  2  2012 lost+found
drwxr-x---  5 www-data www-data 1.0K Sep  6 19:39 tubepress-system-cache-682ae65c6391c22089b1aa778f61a274
drwxrwxrwt  2 root     root     1.0K Jul 28 22:21 .X11-unix

I set the html cache directory to http://www.humortime...ent/html-cache/, but left the others blank. Should I change all of them, do you think?