Jboss / Wildfly errors when accepting socket java.io.IOException: Bad file descriptor problem

Jboss/Wildfly random crashes under heavy load:

Symptoms:

Jboss/Wildfly crashes randomly with  java.io.IOException: Bad file descriptor. Undertow HTTp server does not restart but admin console and everything else is working and nothing in log files.

How to debug it:

Put this in your configuration file (standalone.xml for Wildfly)

        <logger category="org.xnio.nio">
        <level name="DEBUG"/>
        </logger>

The error:

2018-03-03 17:19:15,273 DEBUG [org.xnio.nio.tcp.server] (default Accept) Exception accepting request, closing server channel TCP server (NIO) <69a407dd>: java.io.IOException: Bad file descriptor
    at sun.nio.ch.IOUtil.configureBlocking(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:264)
    at org.xnio.nio.QueuedNioTcpServer.handleReady(QueuedNioTcpServer.java:477)
    at org.xnio.nio.QueuedNioTcpServerHandle.handleReady(QueuedNioTcpServerHandle.java:38)
    at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)

What is the reason for that:

The reason is that you are probably running a BSD and the bug is in Java implementation of KQueueSelectorImpl.java in latest OpenJDK for BSD (openjdk-1.8.162 at the moment). This was tested on NetBSD 7.1.

Solution:

There is no solution at the moment. You need to move your Jboss/Wildfly to Linux. You can try changing default selector with java command line arguments (example is solution for Solaris):

java -Dxnio.nio.selector.provider=sun.nio.ch.PollSelectorProvider .......

At the moment the only solution to this is not to run it under BSD. Under Linux with EPollSelectorProvider everything works just fine.

Qmail on OpenBSD 5.7 qmail-queue problem.

OpenBSD 5.7 default install with netqmail-1.06 installation.

The problem: When you try to send mail the qmail server returns the following error:

geroy@newsrv:~$ telnet localhost 25
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
220 mx1.polycomp.bg ESMTP
helo
250 mx1.polycomp.bg
mail from: test@mail.bg
250 ok
rcpt to: testovacc@gmail.com
250 ok
data
354 go ahead
test test 1 2 3
.
451 qq trouble creating files in queue (#4.3.0)

The solution: Default installation of OpenBSD 5.7 makes /var directory mounted with nosuid option. Remove it from /etc/fstab and then reboot.

# cat /etc/fstab
f292a8d8eba5b8dc.b none swap sw
f292a8d8eba5b8dc.a / ffs rw 1 1
f292a8d8eba5b8dc.k /home ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.d /tmp ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.f /usr ffs rw,nodev 1 2
f292a8d8eba5b8dc.g /usr/X11R6 ffs rw,nodev 1 2
f292a8d8eba5b8dc.h /usr/local ffs rw,nodev 1 2
f292a8d8eba5b8dc.j /usr/obj ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.i /usr/src ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.e /var ffs rw,nodev,nosuid 1 2

remove nosuid to look like this:

f292a8d8eba5b8dc.e /var ffs rw,nodev 1 2

Save and reboot.