The definition of "APPLE" was used by the Mac OSX port.
For FreeBSD, this definition is never used, so remove
the references to it to clean up the code.

This commit should not result in a semantics change.

Add kf_pipe_buffer_[in/out/size] fields to kf_pipe, and populate them.

Add a kf_kqueue struct to the kf_un union, to allow querying kqueue state,
and populate it.

Populate the kf_sock_rcv_sb_state and kf_sock_snd_sb_state fields in
kf_sock for INET/INET6 sockets, and populate all other fields for all
transport layer protocols, not just TCP.

Bump __FreeBSD_version.

Differential revision:	https://reviews.freebsd.org/D34184
Reviewed by:	jhb, kib, se
MFC after:	1 week

PR:	264723
Reviewed by:	kib
Discussed with:	markj
MFC after:	1 week

The current situation is fairly confusing, where an integer is interpreted
as a percent until you slap a decimal on it and magically it becomes an
absolute value.

Let's have a flag day in 14.0 and remove this shim entirely.  Setting with
percent can still be useful, so allow a trailing '%' to indicate as such.
As a side effect, we tighten down the format allowed in the volume a little
bit by ensuring there's no trailing garbage after the value once it's
separated into left and right components.

Reviewed by:	christos, hselasky, pauamma_gundo.com (manpages)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D35101

We've already read it and validated it ‒ re-reading is wasteful

PR:		264376

The "void *stuff" (also called fstuff and dstuff) argument
was used by the Mac OSX port.  For FreeBSD, this argument
is always NULL, so remove it to clean up the code.

This commit gets rid of "stuff" for assorted functions
defined in nfs_clrpcops.c and called in nfs_clvnops.c and
nfs_clstate.c.

This commit should not result in a semantics change.

Add definitions related to 802.11ax Control Trigger frame format
needed for rtw89.

MFC after:	3 days

The "void *stuff" (also called fstuff and dstuff) argument
was used by the Mac OSX port.  For FreeBSD, this argument
is always NULL, so remove it to clean up the code.

This commit gets rid of "stuff" for assorted functions
defined in nfs_clrpcops.c and called in nfs_clvnops.c and
nfs_clvfsops.c. Future commits will do the same for other functions.

This commit should not result in a semantics change.

This avoids relying on the `includes` target to generate this header
which must be in .OBJDIR for the ncurses library to find it.

With an upcoming change libcompat build won't build a seperate
sysroot and will instead share with the 64-bit target which means the
`includes` target will not be called.

Sponsored by:	DARPA, AFRL

Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D35510

If iommu_gas_find_space() ever called iommu_gas_uppermatch(), and it
succeeded in allocating space, then the subsequent KASSERT would be
triggered.  Change that KASSERT to accept either success or ENOMEM.

MFC after:	1 week

Honor -q parameter and do not display any warning messages when -q is
specified.

Approved by:		dfr
MFC after:		2 weeks
Sponsored by:		Resulta, s.r.o.
Differential Revision:	https://reviews.freebsd.org/D35511

To allow for a dynamic page size on arm64 remove the static valud from
the kcov tests

Sponsored by:	The FreeBSD Foundation

It may not be known at compile time so we should detect it at run time.

Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35118

Add clean up on failure and a detach function to the pci host generic
driver.

Reviewed by:	jhb (earlier version)
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35291

 - Add the missing RANDOM_PURE_QUALCOMM description
 - Make RANDOM_PURE_VMGENID consistent with the other pure sources
   by including "PURE_" in the description.

Approved by:	csprng (cem)
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35412

If isc-dhcp44-server is not installed (as is the case the project's CI
servers) we don't create ngctl.shutdown, causing the 'rm' to fail.
That in turn causes the cleanup function to return non-zero, which
causes kyua to consider the test failed.

Use 'rm -f' instead, so we don't error even if the file doesn't exist.

Sponsored by:	Rubicon Communications, LLC ("Netgate")

Eliminate trivial wrappers for several iommu_gas functions that serve
no functional purpose.

Reviewed by:	br, dougm, kib
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D35487

Reported by:	mhorne
Discussed with:	kib

The "void *stuff" (also called fstuff and dstuff) argument
was used by the Mac OSX port.  For FreeBSD, this argument
is always NULL, so remove it to clean up the code.

This commit gets rid of "stuff" for assorted functions
defined in nfs_clrpcops.c and called in nfs_clvnops.c.
Future commits will do the same for other functions.

This commit should not result in a semantics change.

The "void *stuff" (also called fstuff and dstuff) argument
was used by the Mac OSX port.  For FreeBSD, this argument
is always NULL, so remove it to clean up the code.

This commit gets rid of "stuff" for assorted functions
defined in nfs_clrpcops.c and called in nfs_clvnops.c.
Future commits will do the same for other functions.

This commit should not result in a semantics change.

Reviewed by:	rgrimes, grehan, jhb
Differential Revision:	https://reviews.freebsd.org/D30050

Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes
REP MOVSB the fastest way to copy memory in most of cases. However
Intel Optimization Reference Manual says: "setting the DF to force
REP MOVSB to copy bytes from high towards low addresses will expe-
rience significant performance degradation". Measurements on Intel
Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can
drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s
of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs.

This patch keeps ERMS use for forward ordered memory copies, but
removes it for backward overlapped moves where it does not work.

This is just a cosmetic sync with kernel, since libc does not use
ERMS at this time.

Reviewed by:    mjg
MFC after:	2 weeks

Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes
REP MOVSB the fastest way to copy memory in most of cases. However
Intel Optimization Reference Manual says: "setting the DF to force
REP MOVSB to copy bytes from high towards low addresses will expe-
rience significant performance degradation". Measurements on Intel
Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can
drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s
of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs.

This patch keeps ERMS use for forward ordered memory copies, but
removes it for backward overlapped moves where it does not work.

Reviewed by:	mjg
MFC after:	2 weeks

When locking the knote list for a socket, we check whether the socket is
a listening socket in order to select the appropriate mutex; a listening
socket uses the socket lock, while data sockets use socket buffer
mutexes.

If SOLISTENING(so) is false and the knote lock routine locks a socket
buffer, then it must re-check whether the socket is a listening socket
since solisten_proto() could have changed the socket's identity while we
were blocked on the socket buffer lock.

Reported by:	syzkaller
Reviewed by:	glebius
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35492

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation

Some drivers will collect multiple mbuf chains, linked by m_nextpkt,
before passing them to upper layers.  debugnet_pkt_in() didn't handle
this and would process only the first packet, typically leading to
retransmits.

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation

The "void *stuff" (also called fstuff and dstuff) argument
was used by the Mac OSX port.  For FreeBSD, this argument
is always NULL, so remove it to clean up the code.

This commit gets rid of "stuff" for assorted functions
defined in nfs_clrpcops.c and called in nfs_clvnops.c.
Future commits will do the same for other functions.

This commit should not result in a semantics change.

Reviewed by:		rscheff
MFC after:		3 days
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D35503

It seems we do not clear UPS_C_BH_PORT_RESET and UPS_C_PORT_RESET
conditions after warm or port reset.  Add that code.

Obtained from:	an old patch mainly debugging other problems
MFC after:	2 weeks
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D35483

While XHCI is very generic some revisions of chipsets have problems.
On dwc3 <= 3.00a Port Disable does not seem to work so we need to not
enable it.
For that introduce quirks to xhci so that controllers can steer
certain features.  I would hope that this is and remains the only one.

Obtained from:	an old patch mainly debugging other problems
MFC after:	2 weeks
Reviewed by:	hselasky
Differential Revision: https://reviews.freebsd.org/D35482

For as long as we do not implement the compat code for tx aggregation
return -EINVAL in ieee80211_start_tx_ba_session() as both rtw88 and
rtw89 check for this value and only then disable further attempts.

MFC after:	3 days

The "void *stuff" (also called fstuff and dstuff) argument
was used by the Mac OSX port.  For FreeBSD, this argument
is always NULL, so remove it to clean up the code.

This commit gets rid of "stuff" for assorted functions
defined in nfs_clrpcops.c and called in nfs_clvnops.c.
Future commits will do the same for other functions.

This commit should not result in a semantics change.

Reviewed By:	pauamma
Differential Revision: https://reviews.freebsd.org/D35409

Change the recursive implementation to one that uses parent pointers
to walk back up the rb-tree, to slightly improve performance.

Reviewed by:	alc, kib
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D35486

Doing otherwise may lead to lost interrupts, that in combination
with hardware PMCs freezing may leave them frozen forever.  It may
also slightly improve profiling accuracy.

MFC after:	2 weeks

MFC after:	2 weeks

The check fails in kernels compiled with KASAN because AddressSanitizer
inserts redzones around global variables, so the size of the "kernphys"
symbol is 32 rather than 8.  Thus we fall back to copying even though
it's not necessary.

Simply remove the size check.  I didn't want to extend the symbol size
check since there's no guarantee that AddressSanitizer will always emit
32 bytes for "kernphys".

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35448

When the kernel is compiled with -asan-stack=true, the address sanitizer
will emit inline accesses to the shadow map.  In other words, some
shadow map accesses are not intercepted by the KASAN runtime, so they
cannot be disabled even if the runtime is not yet initialized by
kasan_init() at the end of hammer_time().

This went unnoticed because the loader will initialize all PML4 entries
of the bootstrap page table to point to the same PDP page, so early
shadow map accesses do not raise a page fault, though they are silently
corrupting memory.  In fact, when the loader does not copy the staging
area, we do get a page fault since in that case only the first and last
PML4Es are populated by the loader.  But due to another bug, the loader
always treated KASAN kernels as non-relocatable and thus always copied
the staging area.

It is not really practical to annotate hammer_time() and all callees
with __nosanitizeaddress, so instead add some early initialization which
creates a shadow for the boot stack used by hammer_time().  This is only
needed by KASAN, not by KMSAN, but the shared pmap code handles both.

Reported by:	mhorne
Reviewed by:	kib
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35449

On arm64, testing pc_curpcb != NULL is not correct since pc_curpcb is
set in pmap_switch() while the bootstrap stack is still in use.  As a
result, smp_after_idle_runnable() can free the boot stack prematurely.

Take a different approach: use smp_rendezvous() to wait for all APs to
acknowledge an interrupt.  Since APs must not enable interrupts until
they've entered the scheduler, i.e., switched off the boot stack, this
provides the right guarantee without depending as much on the
implementation of cpu_throw().  And, this approach applies to all
platforms, so convert x86 and riscv as well.

Reported by:	mmel
Tested by:	mmel
Reviewed by:	kib
Fixes:		8db2e8fd ("Remove the secondary_stacks array in arm64 and riscv kernels.")
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35435

PTI page table pages are allocated from a VM object, so must be
exclusively busied when they are freed, e.g., when a thread loses a race
in pmap_pti_pde().  Simply keep PTPs busy at all times, as was done for
some other kernel allocators in commit
e9ceb9dd.

Also remove some redundant assertions on "ref_count":
vm_page_unwire_noq() already asserts that the page's reference count is
greater than zero.

Reported by:	syzkaller
Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35466