I need to create an UDP server working with multiple clients over DTLS. I found one note about net-snmp's DTLS Implementation Notes which almost has the same requirement as mime. But it seems it falls back to use memory BIO and cache the data, and Campgnol VPN's solution is making use of NAT traversal technique that open each new connection with a new UDP port, which does not fit my requirement to only use one UDP port for server.
From the notes, I notice the author is almost succeeded with peeking
the incoming traffic except the "Packets Pile Up" problem.
I think that is the right direction for me.
I tried on a single process server first. With recvmsg() with MSG_PEEK,
I could get the client's address and port number, from which I could select the
right context for the DTLS link. I realized that I need a custom BIO module
to avoid the"Packets Pile Up" problem.
I made a copy of crypto/bio/bss_dgram.c from OpenSSL's source and renamed it to my_udp.c,
then stripped away anything unrelated to my usage, basically removed all platform specific
and SCTP code except for Linux. Then further removed all code related to connected
filed flag. Then changed the reading function to peek the packet first and compare
the source IP and port number to the current one. If it is not the current one,
then fake an EAGAIN error.
Since server is listening on multiple interfaces, when sending data out,
it needs to set its source address correspondingly. But UDP does not
directly give the local address used for the data. With socket option
IP_PKTINFO turn on for UDP server socket, local server address
could be found with recvmsg() together with MSG_PEEK.
It is then set in my custom UDP BIO to be used when sending out data.
This seems to work fine with single server process.
But then I was facing the challenge to serve client on multiple processes
on the same UDP port. After some tried and errors, I settled on the following
The parent creates socketpair() with each child for
communication. For the start, only the first child would listen on
the UDP server port. After peeking from the incoming data, calculate
a child index from the client address and port. If the child index is its
own, it proceeds to process the data, if not, it stops monitoring the server
port for incoming data and send a command to parent and inform parent that
the other child should handle the incoming data. When parent receives
the command, it will send the information to the other child, the other child
gets the message and start monitoring server port and process incoming
data and so on.
It seems working, but have not test out the performance, whether it is worth
the trouble to have multiple processes.