IIS site - slow to deliver Certificate - then normal speed.

I have a new IIS 5.0 site that is configured for SSL. I have it
configured to be it's own CA because it is only a numbered site and
Verisign requires it to be named through DNS. Anyway, it is working,
but when you hit it, there is a ~30 second pause before the Security
Warning about an untrusted certificate comes up. Once you accept it,
the actual site runs fast. Is there anyway to speed up this initial
pause? While I could add this as a trust to the internal clients,
people would still have to wait when connecting from home. Thanks for
your help.