In the process of implementing the Fenomen platform (www.fenomen.pro), I decided to configure IIS so that hosted applications on my server would not bring the machine down because of high and uncontrolled memory usage. It happened to me indeed that one my applications started to consume memory up to no limit, quickly using all the memory available and rendering the server unusable...
To that purpose I did set the Private Memory Limit and Virtual Memory of my AppPools to 500M0 to make sure a flawed application won't bring the machine down. Indeed if either the Private Memory Limit or Virtual Memory would go higher than 500Mo then the AppPool would automatically be recycled, the flawed process killed, and memory freed.
However, following this action, I had the following problem: my AppPools did recycle too often, almost at every single request, even though the memory usage was low!
After some research I found the cause of my trouble:
If you are on a 64bit platform (and I am), ASP.NET app's aggressively reserve virtual memory. It's also important to understand that on a 64bit platform, reserving large portions of virtual memory has zero performance impact. Therefore, the Virtual Memory Limit is reached very quickly, even thought the process is making low usage of private memory! This renders the AppPool Virtual Memory Limit useless on 64bit machines...
Therefore solution is:
Do not set any AppPool Virtual Memory Limit, just set the AppPool Physical Memory Limit.
Hope this can help others!