DevHeads.net

Apache HTTPD not responding after running for several days

Hi all,

I'm using Apache HTTPD 2.4.29 and 2.4.33 on Solaris 10 and 11. These HTTPDs
were compiled from source with the latest APR, APR-util and PCRE.
Upon starting, the HTTPDs initially run fine. After running for several
days to 2 weeks, HTTPD would stop responding. It wouldn't accept any new
connections although the processes are running. In the whole period the
incoming traffic volumn is low.
When in the "not responding" situation, stopping and then starting the
HTTPD would workaround the issue temporarily.

The symptoms are:
1. About 5 HTTPD processes are still running (checked by the "ps -ef"
command)
2. The TCP port is still being listened (checked by the "netstat -an | grep
LISTEN" command)
3. Any new incoming request would result in a connection in SYN_SENT state
(checked by "netstat -an") which usually disappears 1 minute later.

I have several Redhat 6.7 and Solaris 10 and 11 machines, and I find that
this problem only occurs on Solaris 10 and 11.

I'd appreciate your help. Thanks in advance.

Regards,
Eric

Comments

Re: Apache HTTPD not responding after running for

By Yann Ylavic at 08/07/2018 - 04:15

Hi,

On Tue, Aug 7, 2018 at 5:57 AM, Eric . < ... at gmail dot com> wrote:
Possibly you are hitting this bug in APR:
<a href="https://bz.apache.org/bugzilla/show_bug.cgi?id=61786" title="https://bz.apache.org/bugzilla/show_bug.cgi?id=61786">https://bz.apache.org/bugzilla/show_bug.cgi?id=61786</a>
It was fixed in: <a href="https://svn.apache.org/r1819938" title="https://svn.apache.org/r1819938">https://svn.apache.org/r1819938</a> (not in any released
APR for now)
A more targeted patch would be:
<a href="http://svn.apache.org/viewvc/apr/apr/trunk/poll/unix/port.c?r1=1819858&amp;r2=1819857&amp;pathrev=1819858&amp;view=patch" title="http://svn.apache.org/viewvc/apr/apr/trunk/poll/unix/port.c?r1=1819858&amp;r2=1819857&amp;pathrev=1819858&amp;view=patch">http://svn.apache.org/viewvc/apr/apr/trunk/poll/unix/port.c?r1=1819858&amp;r...</a>

Regards,
Yann.

Re: Apache HTTPD not responding after running for

By Eric . at 08/12/2018 - 22:16

Dear Yann,

Thanks for the info. I'm trying APR 1.6.2 (the previous version) and let's
see whether the problem would occur again.
Thank you.

Regards,
Eric

Re: Apache HTTPD not responding after running for

By Eric . at 08/20/2018 - 01:55

Hi Yann and all,

Understood that Apr-1.6.3 had a bug in Solaris, I compiled Apache 2.4.34
with apr-1.6.2 and apr-util-1.6.0 , but the problem still occurred.

OS: Solaris 10 and 11 (no problem on Redhat 6.7)
Symptom:

Upon starting, the HTTPDs initially run fine. After running for several
days, HTTPD would stop responding. It wouldn't accept any new connections
although the processes are running. In the whole period the incoming
traffic volumn is low.
When in the "not responding" situation, stopping and then starting the
HTTPD would workaround the issue temporarily.

The symptoms are:
1. About 5 HTTPD processes are still running (checked by the "ps -ef"
command)
2. The TCP port is still being listened (checked by the "netstat -an | grep
LISTEN" command)
3. Any new incoming request would result in a connection in SYN_SENT state
(checked by "netstat -an") which usually disappears 1 minute later.

I'd appreciate your help.
Thanks in advance.

Regards,
Eric

Re: Apache HTTPD not responding after running for

By Eric Covener at 08/20/2018 - 07:34

On Mon, Aug 20, 2018 at 1:56 AM Eric . < ... at gmail dot com> wrote:
I don't think the bug was introduced in 1.6.3. That's just where it
was reported. You need the patch.

Re: Apache HTTPD not responding after running for

By Eric . at 08/21/2018 - 03:49

Hi Eric,

Thanks for the info.

Since Apr-1.6.4, which should contain the patch, is not yet available as a
stable release, I'll try even older Apr versions.

I'm not sure whether it is a problem of Apr.

Thanks.

Regards,
Eric