1. 15 May, 2022 1 commit
  2. 10 Jan, 2022 1 commit
    • Kirk McKusick's avatar
      Avoid unnecessary setting of UFS flag requesting fsck(8) be run. · 7ef56fb0
      Kirk McKusick authored
      When the kernel is requested to mount a filesystem with a bad superblock
      check hash, it would set the flag in the superblock requesting that the
      fsck(8) program be run. The flag is only written to disk as part of a
      superblock update. Since the superblock always has its check hash updated
      when it is written to disk, the problem for which the flag has been set
      will no longer exist. Hence, it is counter-productive to set the flag
      as it will just cause an unnecessary run of fsck if it ever gets written.
      Sponsored by: Netflix
  3. 16 Nov, 2021 1 commit
  4. 15 Nov, 2021 2 commits
    • Kirk McKusick's avatar
      Add ability to suppress UFS/FFS superblock check-hash failure messages. · f2b39152
      Kirk McKusick authored
      When reading UFS/FFS superblocks that have check hashes, both the kernel
      and libufs print an error message if the check hash is incorrect. This
      commit adds the ability to request that the error message not be made.
      It is intended for use by programs like fsck that wants to print its
      own error message and by kernel subsystems like glabel that just wants
      to check for possible filesystem types.
      This capability will be used in followup commits.
      Sponsored by: Netflix
    • Kirk McKusick's avatar
      Consolodate four copies of the STDSB define into a single place. · b366ee48
      Kirk McKusick authored
      The STDSB macro is passed to the ffs_sbget() routine to fetch a
      UFS/FFS superblock "from the stadard place". It was identically defined
      in lib/libufs/libufs.h, stand/libsa/ufs.c, sys/ufs/ffs/ffs_extern.h,
      and sys/ufs/ffs/ffs_subr.c. Delete it from these four files and
      define it instead in sys/ufs/ffs/fs.h. All existing uses of this macro
      already include sys/ufs/ffs/fs.h so no include changes need to be made.
      No functional change intended.
      Sponsored by: Netflix
  5. 25 Oct, 2020 1 commit
    • Kirk McKusick's avatar
      Various new check-hash checks have been added to the UFS filesystem · 996d40f9
      Kirk McKusick authored
      over various major releases. Superblock check hashes were added for
      the 12 release and cylinder-group and inode check hashes will appear
      in the 13 release.
      When a disk with a UFS filesystem is writably mounted, the kernel
      clears the feature flags for anything that it does not support. For
      example, if a UFS disk from a 12-stable kernel is mounted on an
      11-stable system, the 11-stable kernel will clear the flag in the
      filesystem superblock that indicates that superblock check-hashs
      are being maintained. Thus if the disk is later moved back to a
      12-stable system, the 12-stable system will know to ignore its
      incorrect check-hash.
      If the only filesystem modification done on the earlier kernel is
      to run a utility such as growfs(8) that modifies the superblock but
      neither updates the check-hash nor clears the feature flag indicating
      that it does not support the check-hash, the disk will fail to mount
      if it is moved back to its original newer kernel.
      This patch moves the code that clears the filesystem feature flags
      from the mount code (ffs_mountfs()) to the code that reads the
      superblock (ffs_sbget()). As ffs_sbget() is used by the kernel mount
      code and is imported into libufs(3), all the filesystem utilities
      will now also clear these flags when they make modifications to the
      As suggested by John Baldwin, fsck_ffs(8) has been changed to accept
      and repair bad superblock check-hashes rather than refusing to run.
      This change allows fsck to recover filesystems that have been impacted
      by utilities older than those created after this change and is a
      sensible thing to do in any event.
      Reported by:  John Baldwin (jhb@)
      MFC after:    2 weeks
      Sponsored by: Netflix
  6. 01 Sep, 2020 1 commit
  7. 19 Jun, 2020 2 commits
    • Kirk McKusick's avatar
      The binary representation of the superblock (the fs structure) is written · 93440bbe
      Kirk McKusick authored
      out verbatim to the disk: see ffs_sbput() in sys/ufs/ffs/ffs_subr.c.
      It contains a pointer to the fs_summary_info structure. This pointer
      value inadvertently causes garbage to be stored. It is garbage because
      the pointer to the fs_summary_info structure is the address the then
      current stack or heap. Although a mere pointer does not reveal anything
      useful (like a part of a private key) to an attacker, garbage output
      deteriorates reproducibility.
      This commit zeros out the pointer to the fs_summary_info structure
      before writing the out the superblock.
      Reviewed by:  kib
      Tested by:    Peter Holm
      PR:           246983
      Sponsored by: Netflix
    • Kirk McKusick's avatar
      Move the pointers stored in the superblock into a separate · 34816cb9
      Kirk McKusick authored
      fs_summary_info structure. This change was originally done
      by the CheriBSD project as they need larger pointers that
      do not fit in the existing superblock.
      This cleanup of the superblock eases the task of the commit
      that immediately follows this one.
      Suggested by: brooks
      Reviewed by:  kib
      PR:           246983
      Sponsored by: Netflix
  8. 17 Jun, 2020 1 commit
  9. 25 May, 2020 1 commit
    • Chuck Silvers's avatar
      This commit enables a UFS filesystem to do a forcible unmount when · d79ff54b
      Chuck Silvers authored
      the underlying media fails or becomes inaccessible. For example
      when a USB flash memory card hosting a UFS filesystem is unplugged.
      The strategy for handling disk I/O errors when soft updates are
      enabled is to stop writing to the disk of the affected file system
      but continue to accept I/O requests and report that all future
      writes by the file system to that disk actually succeed. Then
      initiate an asynchronous forced unmount of the affected file system.
      There are two cases for disk I/O errors:
         - ENXIO, which means that this disk is gone and the lower layers
           of the storage stack already guarantee that no future I/O to
           this disk will succeed.
         - EIO (or most other errors), which means that this particular
           I/O request has failed but subsequent I/O requests to this
           disk might still succeed.
      For ENXIO, we can just clear the error and continue, because we
      know that the file system cannot affect the on-disk state after we
      see this error. For EIO or other errors, we arrange for the geom_vfs
      layer to reject all future I/O requests with ENXIO just like is
      done when the geom_vfs is orphaned. In both cases, the file system
      code can just clear the error and proceed with the forcible unmount.
      This new treatment of I/O errors is needed for writes of any buffer
      that is involved in a dependency. Most dependencies are described
      by a structure attached to the buffer's b_dep field. But some are
      created and processed as a result of the completion of the dependencies
      attached to the buffer.
      Clearing of some dependencies require a read. For example if there
      is a dependency that requires an inode to be written, the disk block
      containing that inode must be read, the updated inode copied into
      place in that buffer, and the buffer then written back to disk.
      Often the needed buffer is already in memory and can be used. But
      if it needs to be read from the disk, the read will fail, so we
      fabricate a buffer full of zeroes and pretend that the read succeeded.
      This zero'ed buffer can be updated and written back to disk.
      The only case where a buffer full of zeros causes the code to do
      the wrong thing is when reading an inode buffer containing an inode
      that still has an inode dependency in memory that will reinitialize
      the effective link count (i_effnlink) based on the actual link count
      (i_nlink) that we read. To handle this case we now store the i_nlink
      value that we wrote in the inode dependency so that it can be
      restored into the zero'ed buffer thus keeping the tracking of the
      inode link count consistent.
      Because applications depend on knowing when an attempt to write
      their data to stable storage has failed, the fsync(2) and msync(2)
      system calls need to return errors if data fails to be written to
      stable storage. So these operations return ENXIO for every call
      made on files in a file system where we have otherwise been ignoring
      I/O errors.
      Coauthered by: mckusick
      Reviewed by:   kib
      Tested by:     Peter Holm
      Approved by:   mckusick (mentor)
      Sponsored by:  Netflix
      Differential Revision:  https://reviews.freebsd.org/D24088
  10. 11 Mar, 2020 1 commit
  11. 06 Sep, 2019 1 commit
  12. 17 Jul, 2019 1 commit
    • Kirk McKusick's avatar
      The error reported in FS-14-UFS-3 can only happen on UFS/FFS · fdf34aa3
      Kirk McKusick authored
      filesystems that have block pointers that are out-of-range for their
      filesystem. These out-of-range block pointers are corrected by
      fsck(8) so are only encountered when an unchecked filesystem is
      A new "untrusted" flag has been added to the generic mount interface
      that can be set when mounting media of unknown provenance or integrity.
      For example, a daemon that automounts a filesystem on a flash drive
      when it is plugged into a system.
      This commit adds a test to UFS/FFS that validates all block numbers
      before using them. Because checking for out-of-range blocks adds
      unnecessary overhead to normal operation, the tests are only done
      when the filesystem is mounted as an "untrusted" filesystem.
      Reported by:  Christopher Krah, Thomas Barabosch, and Jan-Niclas Hilgert of Fraunhofer FKIE
      Reported as:  FS-14-UFS-3: Out of bounds read in write-2 (ffs_alloccg)
      Reviewed by:  kib
      Sponsored by: Netflix
  13. 17 Jun, 2019 1 commit
  14. 15 Dec, 2018 3 commits
    • Kirk McKusick's avatar
      Ensure that the inode check-hash is not left zeroed out in the case where · c8f55fc4
      Kirk McKusick authored
      the check-hash fails. Prior to the fix in -r342133 the inode with the
      zeroed out check-hash was written back to disk causing further confusion.
      Reported by:  Gary Jennejohn (gj)
      Sponsored by: Netflix
    • Kirk McKusick's avatar
      Reorder ffs_verify_dinode_ckhash() so that it checks the inode check-hash · 72d28f97
      Kirk McKusick authored
      before copying in the inode so that the mode and link-count are not set
      if the check-hash fails. This change ensures that the vnode will be properly
      unwound and recycled rather than being held in the cache.
      Initialize the file mode is zero so that if the loading of the inode
      fails (for example because of a check-hash failure), the vnode will be
      properly unwound and recycled.
      Reported by:  Gary Jennejohn (gj)
      Sponsored by: Netflix
    • Kirk McKusick's avatar
      Must set ip->i_effnlink = ip->i_nlink to avoid a soft updates · 6fa9bc99
      Kirk McKusick authored
      "panic: softdep_update_inodeblock: bad link count" when releasing
      a partially initialized vnode after an inode check-hash failure.
      Reported by:  Gary Jennejohn <gljennjohn@gmail.com>
      Reported by:  Peter Holm (pho)
      Sponsored by: Netflix
  15. 11 Dec, 2018 1 commit
    • Kirk McKusick's avatar
      Continuing efforts to provide hardening of FFS. This change adds a · 8f829a5c
      Kirk McKusick authored
      check hash to the filesystem inodes. Access attempts to files
      associated with an inode with an invalid check hash will fail with
      EINVAL (Invalid argument). Access is reestablished after an fsck
      is run to find and validate the inodes with invalid check-hashes.
      This check avoids a class of filesystem panics related to corrupted
      inodes. The hash is done using crc32c.
      Note this check-hash is for the inode itself and not any of its
      indirect blocks. Check-hash validation may be extended to also
      cover indirect block pointers, but that will be a separate (and
      more costly) feature.
      Check hashes are added only to UFS2 and not to UFS1 as UFS1 is
      primarily used in embedded systems with small memories and low-powered
      processors which need as light-weight a filesystem as possible.
      Reviewed by:  kib
      Tested by:    Peter Holm
      Sponsored by: Netflix
  16. 06 Dec, 2018 1 commit
    • Kirk McKusick's avatar
      Normally when an attempt is made to mount a UFS/FFS filesystem whose · fb14e73c
      Kirk McKusick authored
      superblock has a check-hash error, an error message noting the
      superblock check-hash failure is printed and the mount fails. The
      administrator then runs fsck to repair the filesystem and when
      successful, the filesystem can once again be mounted.
      This approach fails if the filesystem in question is a root filesystem
      from which you are trying to boot. Here, the loader fails when trying
      to access the filesystem to get the kernel to boot. So it is necessary
      to allow the loader to ignore the superblock check-hash error and make
      a best effort to read the kernel. The filesystem may be suffiently
      corrupted that the read attempt fails, but there is no harm in trying
      since the loader makes no attempt to write to the filesystem.
      Once the kernel is loaded and starts to run, it attempts to mount its
      root filesystem. Once again, failure means that it breaks to its prompt
      to ask where to get its root filesystem. Unless you have an alternate
      root filesystem, you are stuck.
      Since the root filesystem is initially mounted read-only, it is
      safe to make an attempt to mount the root filesystem with the failed
      superblock check-hash. Thus, when asked to mount a root filesystem
      with a failed superblock check-hash, the kernel prints a warning
      message that the root filesystem superblock check-hash needs repair,
      but notes that it is ignoring the error and proceeding. It does
      mark the filesystem as needing an fsck which prevents it from being
      enabled for writing until fsck has been run on it. The net effect
      is that the reboot fails to single user, but at least at that point
      the administrator has the tools at hand to fix the problem.
      Reported by:    Rick Macklem (rmacklem@)
      Discussed with: Warner Losh (imp@)
      Sponsored by:   Netflix
  17. 26 Nov, 2018 1 commit
  18. 25 Nov, 2018 1 commit
  19. 13 Nov, 2018 1 commit
    • Kirk McKusick's avatar
      In preparation for adding inode check-hashes, clean up and · 9fc5d538
      Kirk McKusick authored
      document the libufs interface for fetching and storing inodes.
      The undocumented getino / putino interface has been replaced
      with a new getinode / putinode interface.
      Convert the utilities that had been using the undocumented
      interface to use the new documented interface.
      No functional change (as for now the libufs library does not
      do inode check-hashes).
      Reviewed by:  kib
      Tested by:    Peter Holm
      Sponsored by: Netflix
  20. 23 Oct, 2018 1 commit
    • Kirk McKusick's avatar
      Continuing efforts to provide hardening of FFS, this change adds a · ec888383
      Kirk McKusick authored
      check hash to the superblock. If a check hash fails when an attempt
      is made to mount a filesystem, the mount fails with EINVAL (Invalid
      argument). This avoids a class of filesystem panics related to
      corrupted superblocks. The hash is done using crc32c.
      Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
      used in embedded systems with small memories and low-powered processors
      which need as light-weight a filesystem as possible.
      Reviewed by:  kib
      Tested by:    Peter Holm
      Sponsored by: Netflix
  21. 07 Jul, 2018 1 commit
    • Kirk McKusick's avatar
      Import commit from NetBSD with checkin message: · e1c27cf7
      Kirk McKusick authored
          Avoid Undefined Behavior in ffs_clusteracct()
          Change the type of 'bit' variable from int to unsigned int and use unsigned
          values consistently.
          sys/ufs/ffs/ffs_subr.c:336:10, shift exponent -1 is negative
          Detected with Kernel Undefined Behavior Sanitizer.
          Reported by <Harry Pantazis>
      Submitted by: Pedro Giffuni
  22. 02 Mar, 2018 1 commit
    • Kirk McKusick's avatar
      This change is some refactoring of Mark Johnston's changes in r329375 · efbf3964
      Kirk McKusick authored
      to fix the memory leak that I introduced in r328426. Instead of
      trying to clear up the possible memory leak in all the clients, I
      ensure that it gets cleaned up in the source (e.g., ffs_sbget ensures
      that memory is always freed if it returns an error).
      The original change in r328426 was a bit sparse in its description.
      So I am expanding on its description here (thanks cem@ and rgrimes@
      for your encouragement for my longer commit messages).
      In preparation for adding check hashing to superblocks, r328426 is
      a refactoring of the code to get the reading/writing of the superblock
      into one place. Unlike the cylinder group reading/writing which
      ends up in two places (ffs_getcg/ffs_geom_strategy in the kernel
      and cgget/cgput in libufs), I have the core superblock functions
      just in the kernel (ffs_sbfetch/ffs_sbput in ffs_subr.c which is
      already imported into utilities like fsck_ffs as well as libufs to
      implement sbget/sbput). The ffs_sbfetch and ffs_sbput functions
      take a function pointer to do the actual I/O for which there are
      four variants:
          ffs_use_bread / ffs_use_bwrite for the in-kernel filesystem
          g_use_g_read_data / g_use_g_write_data for kernel geom clients
          ufs_use_sa_read for the standalone code (stand/libsa/ufs.c
      	but not stand/libsa/ufsread.c which is size constrained)
          use_pread / use_pwrite for libufs
      Uses of these interfaces are in the UFS filesystem, geoms journal &
      label, libsa changes, and libufs. They also permeate out into the
      filesystem utilities fsck_ffs, newfs, growfs, clri, dump, quotacheck,
      fsirand, fstyp, and quot. Some of these utilities should probably be
      converted to directly use libufs (like dumpfs was for example), but
      there does not seem to be much win in doing so.
      Tested by: Peter Holm (pho@)
  23. 26 Feb, 2018 1 commit
  24. 24 Feb, 2018 1 commit
  25. 21 Feb, 2018 1 commit
  26. 19 Feb, 2018 1 commit
  27. 16 Feb, 2018 1 commit
    • Mark Johnston's avatar
      Fix a memory leak introduced in r328426. · 16759360
      Mark Johnston authored
      ffs_sbget() may return a superblock buffer even if it fails, so the
      caller must be prepared to free it in this case. Moreover, when tasting
      alternate superblock locations in a loop, ffs_sbget()'s readfunc
      callback must free the previously allocated buffer.
      Reported and tested by:	pho
      Reviewed by:		kib (previous version)
      Differential Revision:	https://reviews.freebsd.org/D14390
  28. 26 Jan, 2018 1 commit
    • Kirk McKusick's avatar
      Refactoring of reading and writing of the UFS/FFS superblock. · dffce215
      Kirk McKusick authored
      Specifically reading is done if ffs_sbget() and writing is done
      in ffs_sbput(). These functions are exported to libufs via the
      sbget() and sbput() functions which then used in the various
      filesystem utilities. This work is in preparation for adding
      subperblock check hashes.
      No functional change intended.
      Reviewed by: kib
  29. 20 Nov, 2017 1 commit
    • Pedro F. Giffuni's avatar
      sys: further adoption of SPDX licensing ID tags. · 51369649
      Pedro F. Giffuni authored
      Mainly focus on files that use BSD 3-Clause license.
      The Software Package Data Exchange (SPDX) group provides a specification
      to make it easier for automated tools to detect and summarize well known
      opensource licenses. We are gradually adopting the specification, noting
      that the tags are considered only advisory and do not, in any way,
      superceed or replace the license texts.
      Special thanks to Wind River for providing access to "The Duke of
      Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
      starting point.
  30. 31 Oct, 2016 1 commit
  31. 17 Sep, 2016 1 commit
    • Konstantin Belousov's avatar
      Reduce size of ufs inode. · e1db6897
      Konstantin Belousov authored
      Remove redunand i_dev and i_fs pointers, which are available as
      ip->i_ump->um_dev and ip->i_ump->um_fs, and reorder members by size to
      reduce padding.  To compensate added derefences, the most often i_ump
      access to differentiate between UFS1 and UFS2 dinode layout is
      removed, by addition of the new i_flag IN_UFS2.  Overall, this
      actually reduces the amount of memory dereferences.
      On 64bit machine, original struct inode size is 176, reduced to 152
      bytes with the change.
      Tested by:	pho (previous version)
      Reviewed by:	mckusick
      Sponsored by:	The FreeBSD Foundation
      MFC after:	2 weeks
  32. 29 Jul, 2015 1 commit
    • Jeff Roberson's avatar
      - Make 'struct buf *buf' private to vfs_bio.c. Having a global variable · 98082691
      Jeff Roberson authored
         'buf' is inconvenient and has lead me to some irritating to discover
         bugs over the years.  It also makes it more challenging to refactor
         the buf allocation system.
       - Move swbuf and declare it as an extern in vfs_bio.c.  This is still
         not perfect but better than it was before.
       - Eliminate the unused ffs function that relied on knowledge of the buf
       - Move the shutdown code that iterates over the buf array into vfs_bio.c.
      Reviewed by:	kib
      Sponsored by:	EMC / Isilon Storage Division
  33. 19 Oct, 2010 1 commit
  34. 24 Apr, 2010 1 commit
  35. 18 Dec, 2008 1 commit
    • Sam Leffler's avatar
      Apply the big hammer: · a6b691a8
      Sam Leffler authored
      o remove all of compat except for pwcache and strstuftoll; these might
        end up in libutil or similar so keep them in the subdir
      o mv getid.c up to the top level; this looks like something that'll be
      o eliminate private versions of .h files in sys; use system files instead
      o eliminate private ffs_tables.c; use the system version directly (might
        want to adopt const'ification at some point but that's the only diff I
        can see)
      o mv remaining code from sys to ffs and strip out unused bits; this now
        becomes part of makefs
      o add compat defs and shims to makefs.h
      o strip all vestiges of nbtool_config.h, compat_defs.h, etc.
      o fixup includes after file shuffling
      o rename system #defines that do implicit byte swapping to have an _swap
        suffix; e.g. DIRSIZ -> DIRSIZ_SWAP, cg_inosused -> cg_inosused_swap; if
        we ever add endian-agnostic support to the kernel these can go back to
        their original names
      o strip some netbsd'isms that aren't worth shim'ing (e.g. _DIAGASSERT)
      Code compiles w/o complaints but is untested.
  36. 23 Nov, 2008 1 commit