The previous model was flawed. Consider the following case:
* The main loop looks as follows (pseudocode):
loop {
let _ = (tcp:1234).read_all()
wait(iface.poll())
}
* The remote end is continuously transmitting data and at some
point fills the window of (tcp:1234), stopping the transmission
afterwards.
* The local end processes the packets and, as a part of egress
routine, emits an ACK. That also updates the window, and
the socket's poll_at() routine returns None, since there is
nothing to transmit or retransmit.
* The local end now waits indefinitely even though it can start
processing the data in the socket buffers right now.
To be precise, I'm talking about IPX, AppleTalk and DECnet here,
not things like PPPoE, ATAoE, FCoE, or PTP, which make sense
to implement on top of EthernetInterface but do not work on
the same level on top of it as IP.
Before this commit, anything that touched RawSocket or TapInterface
worked partly by accident and partly because of a horrible crutch
that resulted in massive latencies as well as inevitable packet loss
every time an ARP request had to be issued. Also, there was no way
to use poll() other than by continuously calling it in a busy loop.
After this commit, poll() indicates when the earliest timer expires,
and so the caller can sleep until that moment (or until packets
arrive).
Note that there is a subtle problem remaining: every time poll()
is called, every socket with a pending outbound packet whose
IP address doesn't correspond to a MAC address will send a new
ARP request, resulting in potentially a whole lot of such requests.
ARP rate limiting is a separate topic though.
