The Mystery of Unexplained Errors

As a web application developer, you sometimes encounter unexpected issues that seem to arise out of nowhere. I like to think of these as Gremlins in the machine—issues caused by bugs, hardware errors, hidden features, or other factors that can lead to unpredictable and sometimes disastrous outcomes. The challenge? These issues can be nearly impossible to recreate. Below is one such experience where we uncovered and resolved a puzzling problem.

Why Long-Running Pages Matter

There are times when a dynamic web page needs to run for more than 60 seconds before it times out—perhaps for generating a daily report or handling a large database query. The last thing you want is for that page to execute a second time while it's still processing the first instance.

For scheduled reports, one simple solution is to run the process as a scheduled job and make the report available as a file on the web server. However, if you're using a J2EE-based site hosted on Microsoft's IIS web server, an often-overlooked IIS feature can cause unexpected behavior that isn’t immediately obvious.

Understanding IIS App Pool Recycling

IIS isolates applications using application pools (app pools), a mechanism that ensures security and stability. However, what many developers don’t realize is that IIS automatically recycles the app pool every 29 hours by default.

This behavior originated in IIS 6.0 (released with Windows Server 2003 and Windows XP Pro x64). According to Scott Forsyth, the reason for the 29-hour cycle is simple: 29 is the first prime number greater than 24, ensuring that recycling happens at different times each day rather than at a fixed hour.

How This Affects Web Applications

Because of this odd scheduling, the app pool doesn't recycle at a consistent time. For example:

• If the first recycle happens at midnight on January 1st, the next will occur at 5:00:01 AM on January 2nd (due to the extra second added from the recycling process).
• The third recycle will be at 10:00:02 AM on January 3rd.
• By the sixth cycle, it has shifted to 1:00:05 AM on January 7th.
• Every 24 weeks, the app pool will eventually recycle at the same hour but will have shifted by about 2 minutes due to the accumulation of small time variations.

The Real Problem: Duplicate Requests

Why does this matter? If a long-running J2EE page is still processing when the app pool recycles, IIS will automatically trigger a second execution of that page.

What This Means for Scheduled Jobs

Imagine a web application that imports data from a third-party source into a database.

1. A scheduled job reads data from a flat file, processes it, and loads it into the database.
2. The IIS app pool recycles mid-process.
3. The page executes again, causing duplicate records or incomplete data imports.

Even worse, if the job clears a temporary table before inserting new data, the app pool recycle could wipe out existing data before the import completes—resulting in partial records with no error messages because, technically, the job “ran successfully.”

The Impact on User Experience

Now, let’s consider another scenario.

• A user initiates a large database query before going to lunch.
• Five different customer support agents run similar queries at the same time.
• Each query takes 5-6 minutes to generate results.
• The IIS app pool recycles while the queries are still processing.

What happens next?

• IIS triggers duplicate requests for all five users.
• The original five queries continue running in the background, but IIS no longer expects their responses.
• The duplicate requests consume server resources, potentially causing delays or failures.
• Users return from lunch to blank pages, unaware of what happened.

How to Prevent IIS App Pool Issues

Now that we know the problem, what can we do about it? Here are three potential solutions:

1. Adjust IIS Recycling Settings: 

The best approach is to manually configure the app pool recycle time to occur at a predictable hour with low site activity and no scheduled jobs. This minimizes disruption and ensures that critical jobs aren’t interrupted.

2. Consider Alternative Web Servers: 

If IIS app pool recycling is causing too many issues, switching to a different web server—such as Apache or Nginx—may be a viable option. These alternatives do not use application pools, eliminating the risk of unexpected recycling.

3. Avoid Long-Running IIS-Hosted Pages: 

If you must run scheduled jobs, avoid using J2EE web pages hosted in IIS. Instead, schedule these processes through a background job system or a dedicated application server to prevent unwanted duplicate executions.

Diagnosing IIS Recycling Issues

If your site sometimes experiences unexplained spikes in traffic or server load, check your Windows event logs to see if IIS recently recycled the app pool. If so, that could be the hidden culprit.

Need Help Taming the Gremlins in your System?

Whether you're dealing with unexpected app behavior or just want to keep things running smoothly, our team is here to help.

Written by Reuben Brown, Senior ColdFusion Developer at WRIS Web Services
With years of experience troubleshooting and optimizing ColdFusion applications, Reuben brings deep technical insight and practical solutions to complex web challenges.