You must be creating your threads programmatically in the main thread. In your main thread, don't create new threads once your maximum is hit. Instead, pool them up and wait for an existing thread to become available.
I don't think you need anywhere close to 100 threads, though. Servlet servers like Tomcat by default only create 20 threads for handling requests. So are you sure you're even using existing threads at all? What you want to build is a "thread pool", which the main thread keeps track of. As requests come in, the main thread should pass them off to existing threads that aren't processing current requests.
Jon Emerson
http://www.jonemerson.net/