Skip to content
  • Robert Watson's avatar
    Implement a CPU-affine TCP and UDP connection lookup data structure, · 52cd27cb
    Robert Watson authored
    struct inpcbgroup.  pcbgroups, or "connection groups", supplement the
    existing inpcbinfo connection hash table, which when pcbgroups are
    enabled, might now be thought of more usefully as a per-protocol
    4-tuple reservation table.
    
    Connections are assigned to connection groups base on a hash of their
    4-tuple; wildcard sockets require special handling, and are members
    of all connection groups.  During a connection lookup, a
    per-connection group lock is employed rather than the global pcbinfo
    lock.  By aligning connection groups with input path processing,
    connection groups take on an effective CPU affinity, especially when
    aligned with RSS work placement (see a forthcoming commit for
    details).  This eliminates cache line migration associated with
    global, protocol-layer data structures in steady state TCP and UDP
    processing (with the exception of protocol-layer statistics; further
    commit to follow).
    
    Elements of this approach were inspired by Willman, Rixner, and Cox's
    2006 USENIX paper, "An Evaluation of Network Stack Parallelization
    Strategies in Modern Operating Systems".  However, there are also
    significant differences: we maintain the inpcb lock, rather than using
    the connection group lock for per-connection state.
    
    Likewise, the focus of this implementation is alignment with NIC
    packet distribution strategies such as RSS, rather than pure software
    strategies.  Despite that focus, software distribution is supported
    through the parallel netisr implementation, and works well in
    configurations where the number of hardware threads is greater than
    the number of NIC input queues, such as in the RMI XLR threaded MIPS
    architecture.
    
    Another important difference is the continued maintenance of existing
    hash tables as "reservation tables" -- these are useful both to
    distinguish the resource allocation aspect of protocol name management
    and the more common-case lookup aspect.  In configurations where
    connection tables are aligned with hardware hashes, it is desirable to
    use the traditional lookup tables for loopback or encapsulated traffic
    rather than take the expense of hardware hashes that are hard to
    implement efficiently in software (such as RSS Toeplitz).
    
    Connection group support is enabled by compiling "options PCBGROUP"
    into your kernel configuration; for the time being, this is an
    experimental feature, and hence is not enabled by default.
    
    Subject to the limited MFCability of change dependencies in inpcb,
    and its change to the inpcbinfo init function signature, this change
    in principle could be merged to FreeBSD 8.x.
    
    Reviewed by:    bz
    Sponsored by:   Juniper Networks, Inc.
    52cd27cb