Sounds pretty odd..
Like Stephen mentioned it should recompile once and be back to it's normal response times. Large amounts of loose App_Code files would make that process somewhat slow. During that time the site won't respond at all.
I'm not sure what's causing the slowness. Is the application using caching mechanisms? It could take a whole to refill that cache depending on the application. I admit it's a bit of a long shot, but I can't think of anything else that might cause this right now.