0

I have a scheduled import task that runs nightly on our CM instance. The task is huge - it pulls tens of thousands of products from an API and updates, creates, or deletes all of the corresponding product items in the Sitecore database. When I trigger the task manually, it takes about an hour to complete.

The task runs nightly but often, we find that certain products haven't synced, or they've updated but didn't get published. Based on the logs, it appears that the process just stops. We have a custom logger that writes to Custom.log for this import. It writes a message that it is starting the import; that it is making each API call (for each product type); each product ID that is successfully imported or that fails; a message that it has completed importing; and some logging during the publishing.

The thing is that the logs just end. I never get to this line:

_Logger.Info("ProductsSync: Finished; publishing products");

I'll have thousands of product completion lines logged, and then the log just ends. There's no error; if there was, I would get this line in the logs, but there's no errors in Custom.log, or in log.txt

_Logger.Error(String.Format("Error on ExecuteProductSync ") + ex.Message);

Below is a picture showing how the logs just stop abruptly

enter image description here

So I expect the issue is that the app is recycling or going to sleep or something, and this is aborting the scheduled task. I do have keepalive in my CM instance:

<agent type="Sitecore.Tasks.UrlAgent" method="Run" interval="00:15:00">
    <param desc="url">/sitecore/service/keepalive.aspx</param>
    <LogActivity>true</LogActivity>
</agent>

so why would it be stopping/restarting, and how can I prevent it from doing so while the task is running?

6
  • This sounds like your App Pool is recycling. Check your recycle intervals on your IIS App Pool.
    – Chris Auer
    Mar 3 at 15:34
  • The interval is the default, 1740 minutes Mar 3 at 15:46
  • 1
    I changed the interval to 0, set scheduled time of 09:00:00, and changed the Idle Timeout to 120 Mar 3 at 15:49
  • If you look in your windows event logs you can find the times it was resetting, but I bet this is your issue. We too had some overnight long running processes what we had to move our resets around for.
    – Chris Auer
    Mar 3 at 19:20
  • So, I already set it to 9:00 yesterday, and disabled the time interval. The task started at 3:15. I got log entries until 5:07, then they abruptly stopped again. I checked the Windows event logs and see the restart at 9:00, no restarts before then Mar 4 at 16:11

1 Answer 1

2

You app pool is probably recycling in the early morning when it dies. The default is 1740 minutes. If you move this to a time when the long process is complete, it should complete without issue.

Right click on your site app pool and select recycling from the list.

enter image description here

enter image description here

2
  • 1
    A step in the right direction would be to set the recycle to a specific time, but even better would be to disable it entirely. If the app is well written, scheduled recycles shouldn't be necessary. Mar 3 at 20:51
  • 1
    So, I already set it to 9:00 yesterday, and disabled the time interval. The task started at 3:15. I got log entries until 5:07, then they abruptly stopped again. I checked the Windows event logs and see the restart at 9:00, no restarts before then Mar 4 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.