1. 01 Dec, 2021 1 commit
  2. 12 Aug, 2021 1 commit
    • Jessica Clarke's avatar
      bsd.compiler.mk: Fix cross-building from non-FreeBSD · cef65082
      Jessica Clarke authored
      On non-FreeBSD, the various MACHINE variables for the host when
      bootstrapping can be missing or not match FreeBSD's naming, causing
      bsd.endian.mk to be unable to infer the endianness. Work around this by
      assuming it's unsupported.
      
      Note that we can't check BOOTSTRAPPING here as Makefile.inc1 includes
      bsd.compiler.mk before that is set, and so we are unable to catch errors
      during buildworld itself when cross-building and bsd.endian.mk failed,
      but such errors should also show up when building on FreeBSD.
      
      Fixes:	47363e99 ("Enable compressed debug on little-endian targets")
      cef65082
  3. 08 Aug, 2021 1 commit
    • Ed Maste's avatar
      Enable compressed debug on little-endian targets · 47363e99
      Ed Maste authored
      Compressed debug was enabled by default in commit c910570e, but
      broke the build on big-endian targets, and so was disabled in
      89ed2ecb.
      
      Older versions of LLD fail with big-endian compressed debug sections.
      This was fixed in LLD upstream (commit c6ebc651b6fa) and merged to
      FreeBSD main (commit d69d0756) by dim.
      
      External toolchains (e.g. the llvm12 package) will not yet have the fix.
      These may be used to link against base system .a archives, so compressed
      debug sections would cause trouble even though the base system is fixed.
      
      Enable compressed debug sections again, for little-endian targets only.
      As discussed on freebsd-hackers[1] I expect to undo this in the future
      (using compressed debug everywhere), once fixed versions of lld are
      widely available.
      
      Note that to be pedantically correct we should check both the compiler
      and the linker for compressed debug support, but given the external
      toolchain constraint the extra complexity does not seem worthwhile.
      
      [1] https://lists.freebsd.org/archives/freebsd-hackers/2021-August/000188.html
      
      PR:		257638
      Reported by:	jrtc27 [impact of .a archives]
      Discussed with:	imp
      Relnotes:	Yes
      Sponsored by:	The FreeBSD Foundation
      Differential Revision:	https://reviews.freebsd.org/D31454
      
      Revert "Disable compressed debug by default"
      
      This reverts commit 89ed2ecb.
      47363e99
  4. 04 Aug, 2021 1 commit
  5. 12 Jul, 2021 1 commit
  6. 20 Apr, 2021 1 commit
  7. 17 Feb, 2021 1 commit
    • Dan McGregor's avatar
      Add fileprefixmap compiler feature · d616ae46
      Dan McGregor authored
      -ffile-prefix-map=<old>=<new> is a compiler feature first added in
      GCC 8, and implemented for clang 10. It remaps old paths to new paths
      in both debug information and __FILE__ and __BASE_FILE__ macros. It can
      be used to improve reproducibility or to hide local system directories.
      
      I intend to use it to replace the real source directory and real object
      directory with constant values across all builds.
      
      Reviewed by:	brooks
      Differential Revision:	https://reviews.freebsd.org/D28632
      d616ae46
  8. 09 Feb, 2021 1 commit
    • Jessica Clarke's avatar
      bsd.compiler.mk: Detect distribution-provided GCC when executed as cc · 9c695432
      Jessica Clarke authored
      Clang always prints "clang $VERSION" regardless of the name used to
      execute it, whereas GCC prints "$progname $VERSION", meaning if CC is
      set to cc and cc is GCC it will print "cc $VERSION". We are able to
      detect some of those cases since it then prints "($PKGVERSION)", where
      the default is "GCC", but many distributions override that to print
      their name and the package version number (e.g. "Debian 10.2.1-6"), so
      nothing tells us it's GCC other than the fact that it's not Clang (and
      that there's an FSF copyright disclaimer).
      
      However, GCC's -v option will always print "gcc version $VERSION", so
      fall back on using that to detect GCC. Whilst Clang also supports this
      option, we should never get here, so Clang handling is not added.
      
      Reviewed by:	brooks, emaste, arichardson
      Differential Revision:	https://reviews.freebsd.org/D28315
      9c695432
  9. 10 Nov, 2020 1 commit
    • Brooks Davis's avatar
      Support initializing stack variables on function entry · e268fd0a
      Brooks Davis authored
      There are two options:
       - WITH_INIT_ALL_ZERO: Zero all variables on the stack.
       - WITH_INIT_ALL_PATTERN: Initialize variables with well-defined patterns.
      
      The exact pattern are a compiler implementation detail and vary by type.
      They are somewhat documented in the LLVM commit message:
      https://reviews.llvm.org/rL349442
      I've used WITH_INIT_ALL_* to match Microsoft's InitAll feature rather
      than naming them after the LLVM specific compiler flags.
      
      In a range of consumer products, options like these are used in
      both debug and production builds with debugs builds using patterns
      (intended to provoke crashes on use of uninitialized values) and
      production using zeros (deemed more likely to lead to harmless
      misbehavior or NULL-pointer dereferences).
      
      Reviewed by:	emaste
      Obtained from:	CheriBSD
      Sponsored by:	DARPA
      Differential Revision:	https://reviews.freebsd.org/D27131
      e268fd0a
  10. 26 Aug, 2020 1 commit
    • Alex Richardson's avatar
      Avoid recomputing COMPILER_/LINKER_ variables when set explicitly · 489377c0
      Alex Richardson authored
      I noticed that when we build libraries for a different ABI (in CheriBSD) we
      were calling ${XCC}/${LD} --version for every directory. It turns out that
      this was caused by bsd.compat.mk explicitly setting (X_)COMPILER variables
      for that build stage and this stops the _can_export logic from working.
      To fix this, we change the check to only set _can_export=no if the variable
      is set and it is set to a different value than the cached value.
      This noticeably speeds up the tree walk while building compat libraries.
      During an upstream amd64 buildworld this also removes 8 --version calls.
      
      Obtained from:	CheriBSD
      Reviewed By:	brooks, emaste
      Differential Revision: https://reviews.freebsd.org/D25986
      489377c0
  11. 12 May, 2020 1 commit
  12. 06 Apr, 2020 1 commit
    • Brooks Davis's avatar
      Fix compilation with upstream clang builtin headers. · f382bac4
      Brooks Davis authored
      By using -nobuiltininc and adding the clang builtin headers resource dir
      to the end of the compiler header search path, we can still find headers
      such as immintrin.h but find the FreeBSD version of stddef.h/stdarg.h/..
      first.
      
      This is a workaround until we are able to settle on and complete a plan
      to harmonize guard macros with LLVM.  We've mostly worked out this on
      FreeBSD systems by removing select headers from the installed set of
      devel/llvm*, but that isn't a good solution for cross build.
      
      Submitted by:	arichardson
      Obtained from:	CheriBSD
      Sponsored by:	DARPA, AFRL
      Differential Revision:	https://reviews.freebsd.org/D17002
      f382bac4
  13. 15 Nov, 2019 1 commit
  14. 08 Sep, 2019 1 commit
    • Mitchell Horne's avatar
      Allow for compiler versions >= 10 · e90bb1ad
      Mitchell Horne authored
      Both clang and gcc development branches have reached version 10. Since we
      only parse for a single digit in the major version number, this causes
      COMPILER_VERSION to be set to its default of 0.0.0, meaning version checks
      fail with these newer compilers.
      
      Reviewed by:	emaste
      Approved by:	markj (mentor)
      MFC after:	2 weeks
      Differential Revision:	https://reviews.freebsd.org/D21413
      e90bb1ad
  15. 24 Jul, 2019 1 commit
  16. 25 May, 2019 1 commit
    • Alan Somers's avatar
      fusefs: misc build fixes · 93fecd02
      Alan Somers authored
      * Only build the tests on platforms with C++14 support
      * Fix an undefined symbol error on lint builds
      * Remove an unused function: fiov_clear
      
      Sponsored by:	The FreeBSD Foundation
      93fecd02
  17. 31 Oct, 2018 1 commit
  18. 23 Oct, 2018 1 commit
    • Alex Richardson's avatar
      Only compute the X_COMPILER_*/X_LINKER_* variables when needed · 74f65486
      Alex Richardson authored
      When building CheriBSD we have to set XLD/XCC/XCFLAGS on the command line.
      This triggers the $XCC != $CC case in bsd.compiler.mk (and the same for LD
      in bsd.linker.mk) which causes it to call ${XCC} --version and
      ${XLD} --version (plus various awk+sed+echo calls) in every subdirectory.
      For incremental builds and stages that only walk the source tree this is
      often the majority of the time spent in that directory.
      
      By only computing the value of the X_COMPILER_*/X_LINKER_* variables if
      _WANT_TOOLCHAIN_CROSS_VARS is set we can reduce the number of cc/ld calls
      to once per build stage instead of once per recursive make.
      
      With this change (and no changes to the sources) the `make includes` stage
      now takes 28 seconds at -j1 instead of 86 seconds.
      
      Approved By:	brooks (mentor)
      Differential Revision: https://reviews.freebsd.org/D17046
      74f65486
  19. 27 Jun, 2018 1 commit
  20. 20 Jun, 2018 2 commits
    • Bryan Drewery's avatar
      Reduce exec and fstat overhead for non-build targets. · 6e6be2bb
      Bryan Drewery authored
      This is mostly targetting 'make showconfig' and 'make test-system-*' for
      the benefit of makeman and universe-one-clang work.
      
      Sponsored by:	Dell EMC
      6e6be2bb
    • Bryan Drewery's avatar
      Add WITH_SYSTEM_LINKER, on by default, that avoids building lld when possible. · a7d84af8
      Bryan Drewery authored
      This works similar to WITH_SYSTEM_COMPILER added in r300354.  It only
      supports lld via WITH_LLD_BOOTSTRAP.
      
      When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
      will not build in cross-tools.  If either check fails though then
      libclang is built.
      
      The .info is reworked to notify when libclang will be built since if
      either clang or lld needs to be rebuilt, but not the other, the
      notification can lead to confusion on why "clang is building".
      
      -fuse-ld= is not used with this method so some combinations of compiler
      and linker are expected to fail.
      
      A new 'make test-system-linker' target is added to see the logic results.
      
      Makefile.inc1:
        CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
        could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
        logic as done with SYSTEM_COMPILER.  This also required moving where
        bsd.linker.mk was read since XLD needs to be set before parsing it.  This
        creates a situation where src.opts.mk can not test LINKER_FEATURES or
        add LLD_BOOTSTAP to BROKEN_OPTIONS.
      
      Reviewed by:	emaste (earlier version)
      Sponsored by:	Dell EMC
      Differential Revision:	https://reviews.freebsd.org/D15894
      a7d84af8
  21. 09 May, 2018 1 commit
  22. 07 Feb, 2018 1 commit
  23. 01 Aug, 2017 2 commits
    • Bryan Drewery's avatar
      CCACHE_BUILD: Follow-up r321880: Fix some PATH issues with buildworld. · 91234af7
      Bryan Drewery authored
      - bsd.compiler.mk: Must ensure that the CCACHE_WRAPPER_PATH comes first
        in PATH.
      - Makefile.inc1: Must prepend the CCACHE_WRAPPER_PATH into BPATH as it
        overrides the PATH set in bsd.compiler.mk in sub-makes.  The PATH
        set in bsd.compiler.mk is not exported and doing so would cause it to
        then override the BPATH set from environment.  The only sane solution
        is to prepend into BPATH as needed.
      CCACHE_PATH could possibly be used for some of this as well.
      
      Sponsored by:	Dell EMC Isilon
      91234af7
    • Bryan Drewery's avatar
      CCACHE_BUILD: Allow setting CCACHE_BUILD_TYPE=wrapper. · e2a659ea
      Bryan Drewery authored
      This uses the /usr/local/libexec/ccache/<cc,c++> wrappers rather than
      modifying CC to be '/usr/local/bin/ccache cc'.  Some forms of compilation
      do not support the 'command' type.
      
      Sponsored by:	Dell EMC Isilon
      e2a659ea
  24. 22 Jun, 2017 2 commits
  25. 24 Oct, 2016 1 commit
  26. 03 Jun, 2016 1 commit
  27. 29 May, 2016 1 commit
  28. 28 May, 2016 1 commit
    • Bryan Drewery's avatar
      Always export X_* vars. · 26816d33
      Bryan Drewery authored
      This fixes CROSS_TOOLCHAIN builds after r300886 since it
      relies on X_COMPILER_TYPE being set.
      
      The X_* vars will only represent the external compiler
      being used.
      
      Sponsored by:	EMC / Isilon Storage Division
      26816d33
  29. 23 May, 2016 1 commit
  30. 21 May, 2016 2 commits
  31. 01 Mar, 2016 1 commit
  32. 31 Dec, 2015 1 commit
  33. 08 Dec, 2015 2 commits
    • Bryan Drewery's avatar
      CCACHE_BUILD: Only export CCACHE_PATH= if it was already set with a value. · b7df39cc
      Bryan Drewery authored
      Older ccache don't work with an empty CCACHE_PATH value.  They will error with:
        ccache: FATAL: Could not find compiler "cc" in PATH
        make: "/mnt/bdrewery/git/onefs/src/share/mk/bsd.compiler.mk" line 134: Unable to determine compiler type for /usr/local/bin/ccache cc.  Consider setting COMPILER_TYPE.
      
      Sponsored by:	EMC / Isilon Storage Division
      b7df39cc
    • Bryan Drewery's avatar
      Fix some makeman issues. · 9229b179
      Bryan Drewery authored
      - Don't bother looking up REVISION/BRANCH/etc from release/, or the
        CPUTYPE check, as these are not used for makeman and wastes time.  The also
        invokes auto.obj.mk after I reverted auto.obj.mk ignoring -V in r291312.
      - Don't modify CC or PATH when WITH_CCACHE_BUILD or WITH_META_MODE is enabled
        as it leads to bsd.compiler.mk errors.
      
      Sponsored by:	EMC / Isilon Storage Division
      9229b179
  34. 08 Nov, 2015 1 commit
    • Bryan Drewery's avatar
      Add built-in ccache build support via WITH_CCACHE_BUILD option. · 9160419c
      Bryan Drewery authored
      ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
      used to achieve a safe pseudo-incremental build.  This is explained in
      more detail upstream [1] [2].  It incurs about a 20%-28% hit to populate the
      cache, but with a full cache saves 30-50% in build times.  When combined with
      the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
      resulting dependency file, which it does not do when using mkdep(1)/'CC
      -E'.  Stats are provided at the end of this message.
      
      This removes the need to modify /etc/make.conf with the CC:= and CXX:=
      lines which conflicted with external compiler support [3] (causing the
      bootstrap compiler to not be built which lead to obscure failures [4]),
      incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
      Clang errors with parenthesis, and did not work with META_MODE.
      
      The option name was picked to match the existing option in ports.  This
      feature is available for both in-src and out-of-src builds that use
      /usr/share/mk.
      
      Linking, assembly compiles, and pre-processing avoid using ccache since it is
      only overhead.  ccache does nothing special in these modes, although there is
      no harm in calling it for them.
      
      CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
      compiler to hash the content of the compiler binary to determine if it
      should be a cache miss.  For external compilers the 'mtime' option is used
      as it is more efficient and likely to be correct.  Future work may optimize the
      'content' check using the same checks as whether a bootstrap compiler is needed
      to be built.
      
      The CCACHE_CPP2 pessimization is currently default in our devel/ccache
      port due to Clang requiring it.  Clang's -Wparentheses-equality,
      -Wtautological-compare, and -Wself-assign warnings do not mix well with
      compiling already-pre-processed code that may have expanded macros that
      trigger the warnings.  GCC has so far not had this issue so it is allowed to
      disable the CCACHE_CPP2 default in our port.
      
      Sharing a cache between multiple checkouts, or systems, is explained in
      the ccache manual.  Sharing a cache over NFS would likely not be worth
      it, but syncing cache directories between systems may be useful for an
      organization.  There is also a memcached backend available [5].  Due to using
      an object directory outside of the source directory though you will need to
      ensure that both are in the same prefix and all users use the same layout.  A
      possible working layout is as follows:
        Source: /some/prefix/src1
        Source: /some/prefix/src2
        Source: /some/prefix/src3
        Objdir: /some/prefix/obj
        Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
      This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
      all absolute paths to be relative.  Using something like this is required due
      to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
      adds into its hash for the object without CCACHE_BASEDIR.
      
      distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
      I have not personally tested this and assume it will not mix well with
      using the bootstrap compiler.
      
      The cache from buildworld can be reused in a subdir by first running
      'make buildenv' (from r290424).
      
      Note that the cache is currently different depending on whether -j is
      used or not due to ccache enabling -fdiagnostics-color automatically if
      stderr is a TTY, which bmake only does if not using -j.
      
      The system I used for testing was:
        WITNESS
        Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
        DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
              The arc was fully populated with src tree files and ccache objects.
        RAM: 76GiB
        CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
             2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
      
      The WITH_FAST_DEPEND feature was used for comparison here as well to show
      the dramatic time savings with a full cache.
      
      buildworld:
        x buildworld-before
        + buildworld-ccache-empty
        * buildworld-ccache-full
        % buildworld-ccache-full-fastdep
        # buildworld-fastdep
        +-------------------------------------------------------------------------------+
        |%            *                               #                                +|
        |%            *                               #                                +|
        |%            *                               #             xxx                +|
        |                                                           |A                  |
        |                                                                              A|
        |             A                                                                 |
        |A                                                                              |
        |                                             A                                 |
        +-------------------------------------------------------------------------------+
            N           Min           Max        Median           Avg        Stddev
        x   3       3744.13       3794.31       3752.25     3763.5633     26.935139
        +   3          4519       4525.04       4520.73       4521.59     3.1104823
        Difference at 95.0% confidence
                758.027 +/- 43.4565
                20.1412% +/- 1.15466%
                (Student's t, pooled s = 19.1726)
        *   3       1823.08        1827.2       1825.62        1825.3     2.0785572
        Difference at 95.0% confidence
                -1938.26 +/- 43.298
                -51.5007% +/- 1.15045%
                (Student's t, pooled s = 19.1026)
        %   3       1266.96       1279.37       1270.47     1272.2667     6.3971113
        Difference at 95.0% confidence
                -2491.3 +/- 44.3704
                -66.1952% +/- 1.17895%
                (Student's t, pooled s = 19.5758)
        #   3       3153.34       3155.16        3154.2     3154.2333    0.91045776
        Difference at 95.0% confidence
                -609.33 +/- 43.1943
                -16.1902% +/- 1.1477%
                (Student's t, pooled s = 19.0569)
      
      buildkernel:
        x buildkernel-before
        + buildkernel-ccache-empty
        * buildkernel-ccache-empty-fastdep
        % buildkernel-ccache-full
        # buildkernel-ccache-full-fastdep
        @ buildkernel-fastdep
        +-------------------------------------------------------------------------------+
        |#                        @   %                  *                              |
        |#                        @   %                  *     x                      + |
        |#                        @   %                  *     xx                     ++|
        |                                                      MA                       |
        |                                                                             MA|
        |                                                A                              |
        |                             A                                                 |
        |A                                                                              |
        |                         A                                                     |
        +-------------------------------------------------------------------------------+
            N           Min           Max        Median           Avg        Stddev
        x   3        571.57        573.94        571.79     572.43333     1.3094401
        +   3        727.97        731.91        728.06     729.31333     2.2492295
        Difference at 95.0% confidence
                156.88 +/- 4.17129
                27.4058% +/- 0.728695%
                (Student's t, pooled s = 1.84034)
        *   3         527.1        528.29        528.08     527.82333    0.63516402
        Difference at 95.0% confidence
                -44.61 +/- 2.33254
                -7.79305% +/- 0.407478%
                (Student's t, pooled s = 1.02909)
        %   3         400.4        401.05        400.62        400.69     0.3306055
        Difference at 95.0% confidence
                -171.743 +/- 2.16453
                -30.0023% +/- 0.378128%
                (Student's t, pooled s = 0.954969)
        #   3        201.94        203.34        202.28        202.52    0.73020545
        Difference at 95.0% confidence
                -369.913 +/- 2.40293
                -64.6212% +/- 0.419774%
                (Student's t, pooled s = 1.06015)
        @   3        369.12        370.57         369.3     369.66333    0.79033748
        Difference at 95.0% confidence
                -202.77 +/- 2.45131
                -35.4225% +/- 0.428227%
                (Student's t, pooled s = 1.0815)
      
      [1] https://ccache.samba.org/performance.html
      [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
      [3] https://reviews.freebsd.org/D3484
      [5] https://github.com/jrosdahl/ccache/pull/30
      
      PR:		182944 [4]
      MFC after:	3 weeks
      Sponsored by:	EMC / Isilon Storage Division
      Relnotes:	yes
      9160419c
  35. 31 Oct, 2015 1 commit