Skip to content
  • Mark Johnston's avatar
    ktls: Cache output buffers for software encryption · 49f6925c
    Mark Johnston authored
    Maintain a cache of physically contiguous runs of pages for use as
    output buffers when software encryption is configured and in-place
    encryption is not possible.  This makes allocation and free cheaper
    since in the common case we avoid touching the vm_page structures for
    the buffer, and fewer calls into UMA are needed.  gallatin@ reports a
    ~10% absolute decrease in CPU usage with sendfile/KTLS on a Xeon after
    this change.
    
    It is possible that we will not be able to allocate these buffers if
    physical memory is fragmented.  To avoid frequently calling into the
    physical memory allocator in this scenario, rate-limit allocation
    attempts after a failure.  In the failure case we fall back to the old
    behaviour of allocating a page at a time.
    
    N.B.: this scheme could be simplified, either by simply using malloc()
    and looking up the PAs of the pages backing the buffer, or by falling
    back to page by page allocation and creating a mapping in the cache
    zone.  This requires some way to save a mapping of an M_EXTPG page array
    in the mbuf, though.  m_data is not really appropriate.  The second
    approach may be possible by saving the mapping in the plinks union of
    the first vm_page structure of the array, but this would force a vm_page
    access when freeing an mbuf.
    
    Reviewed by:	gallatin, jhb
    Tested by:	gallatin
    Sponsored by:	Ampere Computing
    Submitted by:	Klara, Inc.
    Differential Revision:	https://reviews.freebsd.org/D28556
    49f6925c