Oh wow, there are so many things that could be happening. It's next to impossible for someone to really help you in a forum like this since your application is a black box to us and all we know is that it's experiencing a performance problem and you have no idea as to why...
I think you need to start by narrowing your possibilities. Closing a web browser and opening it again, if you close every instance of that browser will ditch your authentication ticket, and as a result your session, and create a new one for you when you log in again...Restarting IIS dumps all sessions, and restarts all application pools I think...logging in as another user would give you a new session as well. So if I had to start digging into this, I would be looking to make sure the application pool memory is not being interfered with by any other applications, so as to make sure the sessions are not being corrupted somehow. Try running the app in it's very own application pool and see if there is any difference. Thats suggestion #1.
The other question is, are you coming up short in performance somewhere, and thats causing strang behavior...this is also possible. If the problem is intermitent, then perhaps it is only happening when the server is under too high of a load. Try load testing the application to see if that replicates the problem.
The fact that it happens on your dev box as well as your production box might point to this being a flaw in the design. Perhaps there is some poor performing part of the code. Case in point: one of my coworkers spent all of last week trying to figure out why his page load times were exceeding 11 seconds when he was only load testing with 20 users. Turns out, that a huge data binding function operating on a gridview was running many more times than he thought. He thought it ran once per column (4 times total per page load), but we started stepping through the code and discovered it was running columns * rows number of times, which was far, far greater. He rewrote the code using lighter weight objects, and now the same load testing profile yeilds less than 1 sec page load times, a significant improvement. You might have to step through code and try to figure out where your performance might be falling short if it comes down to the application being the source of the problem.