Memory leak in session caching?

I've come across an apparent bug that I'm surprised no-one come across
before: Mod_ssl's SSL-session cache handling, both the shmht and shmcb
variants, leaks memory. Not directly (there's no alloc calls in shmcb),
but memory is definitely leaked.

Is this a known bug?

Here are the details on how to reproduce it, and a bit about my guesses
about what's causing this bug:

Try the following: make https requests to an Apache1+mod_ssl server (use
a single-process httpd -X for easier monitoring). The server's memory use
will grow and grow, apparently without stopping, over time. The growth
is relatively slow, but definitely noticible. The growth doesn't happen
when session caching is switched off in the configuration file. The growth
becomes alarmingly quick when the requests are made with client-
authentication (in which case the SSL session objects are much larger).

In fact, in one test, a server with 128MB RAM and configured to use a 5 MB
shared memory (shmht) session cache, was made to exhaust all its memory
and start thrashing in a little less than a minute of stressing it with
client-authenticated https requests; With the same server, and the same
load, but with session caching disabled, the server withstood admiringly
well and an abundance of memory was left.

Monitoring the memory use during the aforementioned test, it was easy to
see how each of Apache's processes (about 50-100 were used during that
high load) was growing and growing, until in about a minute all memory was
finished and intensive paging activity started. This is a good sign of a
memory leak :(

The memory growth cannot be attributed to the shared-memory cache itself
getting gradually filled, because it was only 5 MB large. It had to be
something else. What?

My guess, and I hoped that one of the more experienced people on this
list can help me verify/disprove it, and perhaps even fix this apparent
bug, is that OpenSSL is wasting all that memory. My guess is that though
mod_ssl has "callback functions" implementing an external cache (dbm, shmht,
shmct, and recently distcache) it still keeps its own per-process cache.

This per-process cache is obviously useless for Apache (there's a tiny chance
that a second connection will reach the same process again), but by default
(see SSL_CTX_sess_set_cache_size(3)) as many as 20,000 sessions are kept
in that cache. Even if Apache's configuration is to kill a process after as
little as, say, 1000 connections, this can mean that each process may be
wasting this number of session objects, of sizes 100-1000 bytes. Multiply
100 processes by 1000 sessions by 100-1000 bytes, and you get a total memory
waste of 10MB-100MB...

Even if this guess is wrong, something is definitely leaking memory. Maybe
it's a plain-old-memory-leak inside OpenSSL, but I wasn't able to prove
that (I tried valgrind, which claimed that CRYPTO_malloc was the one that
allocated the leaked memory, but that wasn't much help in finding the
real problem inside OpenSSL. Valgrind also showed me that the growing
memory use was not leaked per se - something was still pointing to it.
But there is no plausible excuse why OpenSSL should be keeping around a
quickly growing state in each process).

If this is something that some mod_ssl developer is planning to look into,
I'd be glad to help in any way I can, and can tell you of more tests and
guesses I've made.

Thanks,
Nadav.


--
Nadav Har'El | Friday, Oct 18 2002, 12 Heshvan 5763
nyh@math.technion.ac.il |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |This signature was intentionally left
http://nadav.harel.org.il |boring.
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Memory leak in session caching?

Hi there,

On Thursday 17 Oct 2002 8:41 pm, Nadav Har'El wrote:
> I've come across an apparent bug that I'm surprised no-one come
> across before: Mod_ssl's SSL-session cache handling, both the shmht
> and shmcb variants, leaks memory. Not directly (there's no alloc
> calls in shmcb), but memory is definitely leaked.
>
> Is this a known bug?

I saw your related email on the openssl lists recently but have not had
the time to reply (and search out the necessary links). Anyway, this
may not do it justice, but w.r.t. turning of process-local
openssl-internal cachine, see the following;
http://marc.theaimsgroup.com/?l=apache-modssl&m=997175851064 20&w=2

The issue isn't just memory footprint (though you're right, that can
also become an unecessary issue) but in fact is security as well. If a
session needs to be deleted or marked non-resumable, it's too late if
one of the other processes has cached it locally - so when plugging in
external caching hooks to openssl, mod_ssl should also turn off the
process-local caching. End of story.

This has apparently been fixed in Apache 2 but hasn't (IIRC) in mod_ssl.
I mentioned it more than once, so I've given up.

Cheers,
Geoff

--
Geoff Thorpe
geoff@geoffthorpe.net
http://www.geoffthorpe.net/

____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Memory leak in session caching?

On Thu, Oct 17, 2002, Geoff Thorpe wrote about "Re: Memory leak in session caching?":
> On Thursday 17 Oct 2002 8:41 pm, Nadav Har'El wrote:
> > I've come across an apparent bug that I'm surprised no-one come
> > across before: Mod_ssl's SSL-session cache handling, both the shmht
> > and shmcb variants, leaks memory. Not directly (there's no alloc
> > calls in shmcb), but memory is definitely leaked.
> >
> > Is this a known bug?
>
> I saw your related email on the openssl lists recently but have not had
> the time to reply (and search out the necessary links). Anyway, this
> may not do it justice, but w.r.t. turning of process-local
> openssl-internal cachine, see the following;
> http://marc.theaimsgroup.com/?l=apache-modssl&m=997175851064 20&w=2
>
> The issue isn't just memory footprint (though you're right, that can
> also become an unecessary issue) but in fact is security as well. If a
>...
> This has apparently been fixed in Apache 2 but hasn't (IIRC) in mod_ssl.
> I mentioned it more than once, so I've given up.

Thanks. I do think your security concern is correct and mod_ssl should be
changed like you recommended. But security is obviously secondary to the
program actually working, and in the setup I'm trying the memory waste is
too big to ignore, unfortunately. (by the way, as I said, I may be barking
at the wrong tree: I wasn't able to prove yet that the entire "memory leak"
is indeed the internal session cache).

If I understand correctly, both your suggestion and the Apache 2 code
simply uses SSL_SESS_CACHE_NO_INTERNAL_LOOKUP in
SSL_CTX_set_session_cache_mode(), so that the per-process ("internal")
cache does not get looked into.
Too bad we can't tell OpenSSL also not to *put* stuff in that cache, or at
least I couldn't figure out how to do that. A question I sent about that to
the openssl list remained unanswered.
Frankly, I don't understand why that OpenSSL option exists at all. Why would
anyone want to populate a cache that (s)he will never use?


--
Nadav Har'El | Sunday, Oct 20 2002, 14 Heshvan 5763
nyh@math.technion.ac.il |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |How long a minute depends on what side of
http://nadav.harel.org.il |the bathroom door you're on.
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Memory leak in session caching?

Hi,

On Sunday 20 Oct 2002 6:45 am, Nadav Har'El wrote:
> Thanks. I do think your security concern is correct and mod_ssl should
> be changed like you recommended. But security is obviously secondary
> to the program actually working, and in the setup I'm trying the
> memory waste is too big to ignore, unfortunately. (by the way, as I
> said, I may be barking at the wrong tree: I wasn't able to prove yet
> that the entire "memory leak" is indeed the internal session cache).

Actually security is paramount, otherwise mod_ssl has no purpose at
all. Apache can actually cope with memory leaks pretty well so that it's
stability is not dependant on the perfect functioning of all its
modules. You just kill off child processes after they've served some
maximum number of requests (just set the "MaxRequestsPerChild" or
whatever it's called). An administrator can not, on the other hand,
cope with a security breach that could have been prevented had the
software done what it was supposed to do - which in this case is, after
all, to provide security!

> If I understand correctly, both your suggestion and the Apache 2 code
> simply uses SSL_SESS_CACHE_NO_INTERNAL_LOOKUP in
> SSL_CTX_set_session_cache_mode(), so that the per-process ("internal")
> cache does not get looked into.

Yup.

> Too bad we can't tell OpenSSL also not to *put* stuff in that cache,

Are you sure that it is storing sessions in an internal cache when that
flag is set?

> or at least I couldn't figure out how to do that. A question I sent
> about that to the openssl list remained unanswered.

I do hacking inside the "ssl/" tree as/when I absolutely have to and
otherwise keep myself more happily occupied in the "crypto/" tree - I
don't pretend to have the whole SSL/TLS implementation groked nor do I
pretend that the code in its current form can all be justified, however
it works pretty well in general and has undergone more bashing (in both
source and executable senses) than pretty much any other implementation.
So we mess with it when we must rather than when we think we can. :-)

> Frankly, I don't understand why that OpenSSL option exists at all. Why
> would anyone want to populate a cache that (s)he will never use?

As I say, are you sure that sessions are being stored in the SSL_CTX
cache when the SSL_SESS_CACHE_NO_INTERNAL_LOOKUP flag is set? If I find
time, I'll try and dig back through the openssl list mail and take a
closer look at your post - but I guarantee nothing right now. If you
have evidence that bad (or stupid) things are happening, please take the
analysis as far as you can and point us to a concise summary of the
problem - and if you're feeling very kind, a "diff -u" patch to fix the
problem wouldn't go amiss either. If you already did any/all of that, my
apologies - I'll see what I can do.

Cheers,
Geoff

--
Geoff Thorpe
geoff@geoffthorpe.net
http://www.geoffthorpe.net/


____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Memory leak in session caching?

On Sun, Oct 20, 2002, Geoff Thorpe wrote about "Re: Memory leak in session caching?":
> Actually security is paramount, otherwise mod_ssl has no purpose at
> all. Apache can actually cope with memory leaks pretty well so that it's
> stability is not dependant on the perfect functioning of all its
> modules. You just kill off child processes after they've served some
> maximum number of requests (just set the "MaxRequestsPerChild" or

You're absolutely right. The limit I had set (1000) was too high for my
case, but setting a lower limit would have worked (at some small cost of
performance because of all that extra forking/killing).

> > Too bad we can't tell OpenSSL also not to *put* stuff in that cache,
>
> Are you sure that it is storing sessions in an internal cache when that
> flag is set?

Well, the SSL_CTX_set_session_cache_mode(3) manual says about
SSL_SESS_CACHE_NO_INTERNAL_LOOKUP that:

"By setting this flag sessions are cached in the internal storage but
they are not looked up automatically. ..."

Which is why I assumed that it still caches sessions, even if it doesn't
look them up.

To make sure that the internal cache is indeed being filled up, I tried
a simple experiment: In the NewSessionCacheEntry callback I added code that
printed the current size of the internal cache. I printed:
lh_num_items(ssl->ctx->sessions)
and a second method (counting the linked list instead of the hash table)
int n1 = 0;
SSL_SESSION *s;
for (s=ssl->ctx->session_cache_head; s; s=s->next)
n1++;
and to check that I didn't forget setting SSL_SESS_CACHE_NO_INTERNAL_LOOKUP
I also printed
ssl->ctx->session_cache_mode&SSL_SESS_CACHE_NO_INTERNAL_LOOK UP

I than ran several requests (using curl, so sessions were never resumed),
and sure enough, after 7 requests (to only one server process) I saw a
message like

items in ssl->ctx->sessions: 7
second algorithm: 8
ssl->ctx->session_cache_mode&NO_INTERNAL_LOOKUP: 100

(I don't know where that 8th session came from...).

I did check, by the way, that SSL_SESS_CACHE_NO_INTERNAL_LOOKUP prevented
the internal lookup. At least that :) (so this is indeed a solution to
the security problem you raised)

> > Frankly, I don't understand why that OpenSSL option exists at all. Why
> > would anyone want to populate a cache that (s)he will never use?
>
> As I say, are you sure that sessions are being stored in the SSL_CTX
> cache when the SSL_SESS_CACHE_NO_INTERNAL_LOOKUP flag is set? If I find
> time, I'll try and dig back through the openssl list mail and take a
> closer look at your post - but I guarantee nothing right now. If you

I did that digging earlier today.
Here is what I found. Note that I'm a very small OpenSSL expert (read:
not at all), so take my analysis with a pound (not just a grain :)) of salt.

In short, I found that SSL_SESS_CACHE_NO_INTERNAL_LOOKUP doesn't always
prevent the internal session cached from being filled.
In long,

The relevant functions in OpenSSL are: (I looked at openssl-engine-0.9.6g)
SSL_CTX_add_session (ssl/ssl_sess.c)
ssl_update_cache() (ssl/ssl_lib.c)
ssl_get_prev_session in (ssl/ssl_sess.c)

SSL_CTX_add_session() (ssl_sess.c) is the function that actually adds a
session to the internal cache. It always does it's job - it doesn't
check session_cache_mode flags before doing it.

So the question is whether this function is called when the flag
SSL_SESS_CACHE_NO_INTERNAL_LOOKUP is set.

SSL_CTX_add_session() is only called in two places in OpenSSL: in
ssl_update_cache() (ssl_lib.c) and in ssl_get_prev_session() (ssl_sess.c).
So now we need to check if those functions check the
SSL_SESS_CACHE_NO_INTERNAL_LOOKUP flag or not.

ssl_update_cache() (called when OpenSSL creates a new, not resumed, session)
indeed *appears* not to call SSL_CTX_add_session() when
SSL_SESS_CACHE_NO_INTERNAL_LOOKUP is on. This fits your guess, contradicts
what the manual page says, and also seems to contradict with the experiment
I outlined above - I can't explain it...

The second function, ssl_get_prev_session() (called to fetch a session
with an existing id), however, calls SSL_CTX_add_session WITHOUT first
checking the NO_INTERNAL_LOOKUP flag! So when an answer
is fetched from the external cache, it is also cached in the internal
cache regardless of the NO_INTERNAL_LOOKUP flag.

So to sum it up, I belive NO_INTERNAL_LOOKUP does not consistently
prevent sessions from being entered into the cache; According to my
experiment (which is not backed by firm understanding the source code,
unfortunately) even in cases (new sessions) when sessions should apparently
not be cached in the internal cache, they do end up being cached.

I'm baffled, I have to admit. It will probably take a bigger OpenSSL expert
than me to fully understand this.

> have evidence that bad (or stupid) things are happening, please take the
> analysis as far as you can and point us to a concise summary of the
> problem - and if you're feeling very kind, a "diff -u" patch to fix the
> problem wouldn't go amiss either. If you already did any/all of that, my
> apologies - I'll see what I can do.

I will continue researching this a bit, and will post here patches if
I finally find a working one. The next thing I'm thinking of trying is
to make the SSL_CTX_add_session() function itself (the function that
physically puts the session in the internal cache) check the NO_INTERNAL_LOOKUP
flag, not doing anything if that flag is on. Of course, that would
contradict with the manual - perhaps a new flag is needed... But that
is getting deep OpenSSL territory - I'll probably need to suggest such a
patch to the OpenSSL people.

Thanks,
Nadav.

--
Nadav Har'El | Monday, Oct 21 2002, 15 Heshvan 5763
nyh@math.technion.ac.il |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |Lottery: A tax on people who are bad at
http://nadav.harel.org.il |math.
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Memory leak in session caching?

On Mon, Oct 21, 2002 at 01:52:26AM +0200, Nadav Har'El wrote:
> On Sun, Oct 20, 2002, Geoff Thorpe wrote about "Re: Memory leak in session caching?":
> > > Too bad we can't tell OpenSSL also not to *put* stuff in that cache,
> >
> > Are you sure that it is storing sessions in an internal cache when that
> > flag is set?
>
> Well, the SSL_CTX_set_session_cache_mode(3) manual says about
> SSL_SESS_CACHE_NO_INTERNAL_LOOKUP that:
>
> "By setting this flag sessions are cached in the internal storage but
> they are not looked up automatically. ..."

See below.

> The relevant functions in OpenSSL are: (I looked at openssl-engine-0.9.6g)
> SSL_CTX_add_session (ssl/ssl_sess.c)
> ssl_update_cache() (ssl/ssl_lib.c)
> ssl_get_prev_session in (ssl/ssl_sess.c)
>
> SSL_CTX_add_session() (ssl_sess.c) is the function that actually adds a
> session to the internal cache. It always does it's job - it doesn't
> check session_cache_mode flags before doing it.
>
> So the question is whether this function is called when the flag
> SSL_SESS_CACHE_NO_INTERNAL_LOOKUP is set.
>
> SSL_CTX_add_session() is only called in two places in OpenSSL: in
> ssl_update_cache() (ssl_lib.c) and in ssl_get_prev_session() (ssl_sess.c).
> So now we need to check if those functions check the
> SSL_SESS_CACHE_NO_INTERNAL_LOOKUP flag or not.
>
> ssl_update_cache() (called when OpenSSL creates a new, not resumed, session)
> indeed *appears* not to call SSL_CTX_add_session() when
> SSL_SESS_CACHE_NO_INTERNAL_LOOKUP is on. This fits your guess, contradicts
> what the manual page says, and also seems to contradict with the experiment
> I outlined above - I can't explain it...
>
> The second function, ssl_get_prev_session() (called to fetch a session
> with an existing id), however, calls SSL_CTX_add_session WITHOUT first
> checking the NO_INTERNAL_LOOKUP flag! So when an answer
> is fetched from the external cache, it is also cached in the internal
> cache regardless of the NO_INTERNAL_LOOKUP flag.
>
> So to sum it up, I belive NO_INTERNAL_LOOKUP does not consistently
> prevent sessions from being entered into the cache; According to my
> experiment (which is not backed by firm understanding the source code,
> unfortunately) even in cases (new sessions) when sessions should apparently
> not be cached in the internal cache, they do end up being cached.

I will cross check your investigation.

For 0.9.6d (see CHANGES file), I commited an according change:
*) Do not store session data into the internal session cache, if it
is never intended to be looked up (SSL_SESS_CACHE_NO_INTERNAL_LOOKUP
flag is set). Proposed by Aslam .
[Lutz Jaenicke]

It however seems, that I did not only forget to update the manual page.

Best regards,
Lutz
--
Lutz Jaenicke Lutz.Jaenicke@aet.TU-Cottbus.DE
http://www.aet.TU-Cottbus.DE/personen/jaenicke/
BTU Cottbus, Allgemeine Elektrotechnik
Universitaetsplatz 3-4, D-03044 Cottbus
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Memory leak in session caching?

On Mon, Oct 21, 2002, Nadav Har'El wrote about "Re: Memory leak in session caching?":
> I than ran several requests (using curl, so sessions were never resumed),
> and sure enough, after 7 requests (to only one server process) I saw a
> message like
>
> items in ssl->ctx->sessions: 7
>....
> indeed *appears* not to call SSL_CTX_add_session() when
> SSL_SESS_CACHE_NO_INTERNAL_LOOKUP is on. This fits your guess, contradicts
> what the manual page says, and also seems to contradict with the experiment
> I outlined above - I can't explain it...
>...
> I'm baffled, I have to admit. It will probably take a bigger OpenSSL expert
> than me to fully understand this.

I found why my reading of the source code contradicted with my above
experiment: like an idiot, I compiled my test with Redhat's variant of
OpenSSL 0.9.6b, rather than with the OpenSSL 0.9.6g I was intending to use
and whose source code I was reading!

I ran the test again with the correct library, and OpenSSL 0.9.6g indeed
does not cache new sessions in the internal cache (when NO_INTERNAL_LOOKUP),
like I already noticed in the source code.

But it does cache resumed sessions in the internal cache, again like I
noticed in the source code (ssl_get_prev_session()) - so I think this is
the bug.

Thanks,
Nadav

--
Nadav Har'El | Monday, Oct 21 2002, 15 Heshvan 5763
nyh@math.technion.ac.il |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |As far as we know, our computer has never
http://nadav.harel.org.il |had an undetected error.
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org