1. 19 Jun, 2020 2 commits
    • mckusick's avatar
      The binary representation of the superblock (the fs structure) is written · cfac6b7d
      mckusick authored
      out verbatim to the disk: see ffs_sbput() in sys/ufs/ffs/ffs_subr.c.
      It contains a pointer to the fs_summary_info structure. This pointer
      value inadvertently causes garbage to be stored. It is garbage because
      the pointer to the fs_summary_info structure is the address the then
      current stack or heap. Although a mere pointer does not reveal anything
      useful (like a part of a private key) to an attacker, garbage output
      deteriorates reproducibility.
      
      This commit zeros out the pointer to the fs_summary_info structure
      before writing the out the superblock.
      
      Reviewed by:  kib
      Tested by:    Peter Holm
      PR:           246983
      Sponsored by: Netflix
      cfac6b7d
    • mckusick's avatar
      Move the pointers stored in the superblock into a separate · fc56f5d1
      mckusick authored
      fs_summary_info structure. This change was originally done
      by the CheriBSD project as they need larger pointers that
      do not fit in the existing superblock.
      
      This cleanup of the superblock eases the task of the commit
      that immediately follows this one.
      
      Suggested by: brooks
      Reviewed by:  kib
      PR:           246983
      Sponsored by: Netflix
      fc56f5d1
  2. 17 Jun, 2020 1 commit
  3. 25 May, 2020 1 commit
    • chs's avatar
      This commit enables a UFS filesystem to do a forcible unmount when · c18fd016
      chs authored
      the underlying media fails or becomes inaccessible. For example
      when a USB flash memory card hosting a UFS filesystem is unplugged.
      
      The strategy for handling disk I/O errors when soft updates are
      enabled is to stop writing to the disk of the affected file system
      but continue to accept I/O requests and report that all future
      writes by the file system to that disk actually succeed. Then
      initiate an asynchronous forced unmount of the affected file system.
      
      There are two cases for disk I/O errors:
      
         - ENXIO, which means that this disk is gone and the lower layers
           of the storage stack already guarantee that no future I/O to
           this disk will succeed.
      
         - EIO (or most other errors), which means that this particular
           I/O request has failed but subsequent I/O requests to this
           disk might still succeed.
      
      For ENXIO, we can just clear the error and continue, because we
      know that the file system cannot affect the on-disk state after we
      see this error. For EIO or other errors, we arrange for the geom_vfs
      layer to reject all future I/O requests with ENXIO just like is
      done when the geom_vfs is orphaned. In both cases, the file system
      code can just clear the error and proceed with the forcible unmount.
      
      This new treatment of I/O errors is needed for writes of any buffer
      that is involved in a dependency. Most dependencies are described
      by a structure attached to the buffer's b_dep field. But some are
      created and processed as a result of the completion of the dependencies
      attached to the buffer.
      
      Clearing of some dependencies require a read. For example if there
      is a dependency that requires an inode to be written, the disk block
      containing that inode must be read, the updated inode copied into
      place in that buffer, and the buffer then written back to disk.
      
      Often the needed buffer is already in memory and can be used. But
      if it needs to be read from the disk, the read will fail, so we
      fabricate a buffer full of zeroes and pretend that the read succeeded.
      This zero'ed buffer can be updated and written back to disk.
      
      The only case where a buffer full of zeros causes the code to do
      the wrong thing is when reading an inode buffer containing an inode
      that still has an inode dependency in memory that will reinitialize
      the effective link count (i_effnlink) based on the actual link count
      (i_nlink) that we read. To handle this case we now store the i_nlink
      value that we wrote in the inode dependency so that it can be
      restored into the zero'ed buffer thus keeping the tracking of the
      inode link count consistent.
      
      Because applications depend on knowing when an attempt to write
      their data to stable storage has failed, the fsync(2) and msync(2)
      system calls need to return errors if data fails to be written to
      stable storage. So these operations return ENXIO for every call
      made on files in a file system where we have otherwise been ignoring
      I/O errors.
      
      Coauthered by: mckusick
      Reviewed by:   kib
      Tested by:     Peter Holm
      Approved by:   mckusick (mentor)
      Sponsored by:  Netflix
      Differential Revision:  https://reviews.freebsd.org/D24088
      c18fd016
  4. 11 Mar, 2020 1 commit
    • mckusick's avatar
      When mounting a UFS filesystem, return EINTEGRITY rather than EIO · 91ea5037
      mckusick authored
      when a superblock check-hash error is detected. This change clarifies
      a mount that failed due to media hardware failures (EIO) from a mount
      that failed due to media errors (EINTEGRITY) that can be corrected by
      running fsck(8).
      
      Sponsored by: Netflix
      91ea5037
  5. 06 Sep, 2019 1 commit
  6. 17 Jul, 2019 1 commit
    • mckusick's avatar
      The error reported in FS-14-UFS-3 can only happen on UFS/FFS · eb5503c1
      mckusick authored
      filesystems that have block pointers that are out-of-range for their
      filesystem. These out-of-range block pointers are corrected by
      fsck(8) so are only encountered when an unchecked filesystem is
      mounted.
      
      A new "untrusted" flag has been added to the generic mount interface
      that can be set when mounting media of unknown provenance or integrity.
      For example, a daemon that automounts a filesystem on a flash drive
      when it is plugged into a system.
      
      This commit adds a test to UFS/FFS that validates all block numbers
      before using them. Because checking for out-of-range blocks adds
      unnecessary overhead to normal operation, the tests are only done
      when the filesystem is mounted as an "untrusted" filesystem.
      
      Reported by:  Christopher Krah, Thomas Barabosch, and Jan-Niclas Hilgert of Fraunhofer FKIE
      Reported as:  FS-14-UFS-3: Out of bounds read in write-2 (ffs_alloccg)
      Reviewed by:  kib
      Sponsored by: Netflix
      eb5503c1
  7. 17 Jun, 2019 1 commit
  8. 15 Dec, 2018 3 commits
    • mckusick's avatar
      Ensure that the inode check-hash is not left zeroed out in the case where · bfbf739a
      mckusick authored
      the check-hash fails. Prior to the fix in -r342133 the inode with the
      zeroed out check-hash was written back to disk causing further confusion.
      
      Reported by:  Gary Jennejohn (gj)
      Sponsored by: Netflix
      bfbf739a
    • mckusick's avatar
      Reorder ffs_verify_dinode_ckhash() so that it checks the inode check-hash · b27d4425
      mckusick authored
      before copying in the inode so that the mode and link-count are not set
      if the check-hash fails. This change ensures that the vnode will be properly
      unwound and recycled rather than being held in the cache.
      
      Initialize the file mode is zero so that if the loading of the inode
      fails (for example because of a check-hash failure), the vnode will be
      properly unwound and recycled.
      
      Reported by:  Gary Jennejohn (gj)
      Sponsored by: Netflix
      b27d4425
    • mckusick's avatar
      Must set ip->i_effnlink = ip->i_nlink to avoid a soft updates · c376c836
      mckusick authored
      "panic: softdep_update_inodeblock: bad link count" when releasing
      a partially initialized vnode after an inode check-hash failure.
      
      Reported by:  Gary Jennejohn <gljennjohn@gmail.com>
      Reported by:  Peter Holm (pho)
      Sponsored by: Netflix
      c376c836
  9. 11 Dec, 2018 1 commit
    • mckusick's avatar
      Continuing efforts to provide hardening of FFS. This change adds a · 830a63af
      mckusick authored
      check hash to the filesystem inodes. Access attempts to files
      associated with an inode with an invalid check hash will fail with
      EINVAL (Invalid argument). Access is reestablished after an fsck
      is run to find and validate the inodes with invalid check-hashes.
      This check avoids a class of filesystem panics related to corrupted
      inodes. The hash is done using crc32c.
      
      Note this check-hash is for the inode itself and not any of its
      indirect blocks. Check-hash validation may be extended to also
      cover indirect block pointers, but that will be a separate (and
      more costly) feature.
      
      Check hashes are added only to UFS2 and not to UFS1 as UFS1 is
      primarily used in embedded systems with small memories and low-powered
      processors which need as light-weight a filesystem as possible.
      
      Reviewed by:  kib
      Tested by:    Peter Holm
      Sponsored by: Netflix
      830a63af
  10. 06 Dec, 2018 1 commit
    • mckusick's avatar
      Normally when an attempt is made to mount a UFS/FFS filesystem whose · 2c9178ed
      mckusick authored
      superblock has a check-hash error, an error message noting the
      superblock check-hash failure is printed and the mount fails. The
      administrator then runs fsck to repair the filesystem and when
      successful, the filesystem can once again be mounted.
      
      This approach fails if the filesystem in question is a root filesystem
      from which you are trying to boot. Here, the loader fails when trying
      to access the filesystem to get the kernel to boot. So it is necessary
      to allow the loader to ignore the superblock check-hash error and make
      a best effort to read the kernel. The filesystem may be suffiently
      corrupted that the read attempt fails, but there is no harm in trying
      since the loader makes no attempt to write to the filesystem.
      
      Once the kernel is loaded and starts to run, it attempts to mount its
      root filesystem. Once again, failure means that it breaks to its prompt
      to ask where to get its root filesystem. Unless you have an alternate
      root filesystem, you are stuck.
      
      Since the root filesystem is initially mounted read-only, it is
      safe to make an attempt to mount the root filesystem with the failed
      superblock check-hash. Thus, when asked to mount a root filesystem
      with a failed superblock check-hash, the kernel prints a warning
      message that the root filesystem superblock check-hash needs repair,
      but notes that it is ignoring the error and proceeding. It does
      mark the filesystem as needing an fsck which prevents it from being
      enabled for writing until fsck has been run on it. The net effect
      is that the reboot fails to single user, but at least at that point
      the administrator has the tools at hand to fix the problem.
      
      Reported by:    Rick Macklem (rmacklem@)
      Discussed with: Warner Losh (imp@)
      Sponsored by:   Netflix
      2c9178ed
  11. 26 Nov, 2018 1 commit
  12. 25 Nov, 2018 1 commit
  13. 13 Nov, 2018 1 commit
    • mckusick's avatar
      In preparation for adding inode check-hashes, clean up and · 80199cdc
      mckusick authored
      document the libufs interface for fetching and storing inodes.
      The undocumented getino / putino interface has been replaced
      with a new getinode / putinode interface.
      
      Convert the utilities that had been using the undocumented
      interface to use the new documented interface.
      
      No functional change (as for now the libufs library does not
      do inode check-hashes).
      
      Reviewed by:  kib
      Tested by:    Peter Holm
      Sponsored by: Netflix
      80199cdc
  14. 23 Oct, 2018 1 commit
    • mckusick's avatar
      Continuing efforts to provide hardening of FFS, this change adds a · ce1bca43
      mckusick authored
      check hash to the superblock. If a check hash fails when an attempt
      is made to mount a filesystem, the mount fails with EINVAL (Invalid
      argument). This avoids a class of filesystem panics related to
      corrupted superblocks. The hash is done using crc32c.
      
      Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
      used in embedded systems with small memories and low-powered processors
      which need as light-weight a filesystem as possible.
      
      Reviewed by:  kib
      Tested by:    Peter Holm
      Sponsored by: Netflix
      ce1bca43
  15. 07 Jul, 2018 1 commit
    • mckusick's avatar
      Import commit from NetBSD with checkin message: · 658dc4c0
      mckusick authored
          Avoid Undefined Behavior in ffs_clusteracct()
      
          Change the type of 'bit' variable from int to unsigned int and use unsigned
          values consistently.
      
          sys/ufs/ffs/ffs_subr.c:336:10, shift exponent -1 is negative
      
          Detected with Kernel Undefined Behavior Sanitizer.
      
          Reported by <Harry Pantazis>
      
      Submitted by: Pedro Giffuni
      658dc4c0
  16. 02 Mar, 2018 1 commit
    • mckusick's avatar
      This change is some refactoring of Mark Johnston's changes in r329375 · ea7d6715
      mckusick authored
      to fix the memory leak that I introduced in r328426. Instead of
      trying to clear up the possible memory leak in all the clients, I
      ensure that it gets cleaned up in the source (e.g., ffs_sbget ensures
      that memory is always freed if it returns an error).
      
      The original change in r328426 was a bit sparse in its description.
      So I am expanding on its description here (thanks cem@ and rgrimes@
      for your encouragement for my longer commit messages).
      
      In preparation for adding check hashing to superblocks, r328426 is
      a refactoring of the code to get the reading/writing of the superblock
      into one place. Unlike the cylinder group reading/writing which
      ends up in two places (ffs_getcg/ffs_geom_strategy in the kernel
      and cgget/cgput in libufs), I have the core superblock functions
      just in the kernel (ffs_sbfetch/ffs_sbput in ffs_subr.c which is
      already imported into utilities like fsck_ffs as well as libufs to
      implement sbget/sbput). The ffs_sbfetch and ffs_sbput functions
      take a function pointer to do the actual I/O for which there are
      four variants:
      
          ffs_use_bread / ffs_use_bwrite for the in-kernel filesystem
      
          g_use_g_read_data / g_use_g_write_data for kernel geom clients
      
          ufs_use_sa_read for the standalone code (stand/libsa/ufs.c
      	but not stand/libsa/ufsread.c which is size constrained)
      
          use_pread / use_pwrite for libufs
      
      Uses of these interfaces are in the UFS filesystem, geoms journal &
      label, libsa changes, and libufs. They also permeate out into the
      filesystem utilities fsck_ffs, newfs, growfs, clri, dump, quotacheck,
      fsirand, fstyp, and quot. Some of these utilities should probably be
      converted to directly use libufs (like dumpfs was for example), but
      there does not seem to be much win in doing so.
      
      Tested by: Peter Holm (pho@)
      ea7d6715
  17. 26 Feb, 2018 1 commit
  18. 24 Feb, 2018 1 commit
  19. 21 Feb, 2018 1 commit
  20. 19 Feb, 2018 1 commit
  21. 16 Feb, 2018 1 commit
    • markj's avatar
      Fix a memory leak introduced in r328426. · 65a57d39
      markj authored
      ffs_sbget() may return a superblock buffer even if it fails, so the
      caller must be prepared to free it in this case. Moreover, when tasting
      alternate superblock locations in a loop, ffs_sbget()'s readfunc
      callback must free the previously allocated buffer.
      
      Reported and tested by:	pho
      Reviewed by:		kib (previous version)
      Differential Revision:	https://reviews.freebsd.org/D14390
      65a57d39
  22. 26 Jan, 2018 1 commit
    • mckusick's avatar
      Refactoring of reading and writing of the UFS/FFS superblock. · f5e73a2c
      mckusick authored
      Specifically reading is done if ffs_sbget() and writing is done
      in ffs_sbput(). These functions are exported to libufs via the
      sbget() and sbput() functions which then used in the various
      filesystem utilities. This work is in preparation for adding
      subperblock check hashes.
      
      No functional change intended.
      
      Reviewed by: kib
      f5e73a2c
  23. 20 Nov, 2017 1 commit
    • pfg's avatar
      sys: further adoption of SPDX licensing ID tags. · 4736ccfd
      pfg authored
      Mainly focus on files that use BSD 3-Clause license.
      
      The Software Package Data Exchange (SPDX) group provides a specification
      to make it easier for automated tools to detect and summarize well known
      opensource licenses. We are gradually adopting the specification, noting
      that the tags are considered only advisory and do not, in any way,
      superceed or replace the license texts.
      
      Special thanks to Wind River for providing access to "The Duke of
      Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
      starting point.
      4736ccfd
  24. 31 Oct, 2016 1 commit
  25. 17 Sep, 2016 1 commit
    • kib's avatar
      Reduce size of ufs inode. · 20f1e8ac
      kib authored
      Remove redunand i_dev and i_fs pointers, which are available as
      ip->i_ump->um_dev and ip->i_ump->um_fs, and reorder members by size to
      reduce padding.  To compensate added derefences, the most often i_ump
      access to differentiate between UFS1 and UFS2 dinode layout is
      removed, by addition of the new i_flag IN_UFS2.  Overall, this
      actually reduces the amount of memory dereferences.
      
      On 64bit machine, original struct inode size is 176, reduced to 152
      bytes with the change.
      
      Tested by:	pho (previous version)
      Reviewed by:	mckusick
      Sponsored by:	The FreeBSD Foundation
      MFC after:	2 weeks
      20f1e8ac
  26. 27 May, 2016 1 commit
  27. 29 Jul, 2015 1 commit
    • jeff's avatar
      - Make 'struct buf *buf' private to vfs_bio.c. Having a global variable · 44267026
      jeff authored
         'buf' is inconvenient and has lead me to some irritating to discover
         bugs over the years.  It also makes it more challenging to refactor
         the buf allocation system.
       - Move swbuf and declare it as an extern in vfs_bio.c.  This is still
         not perfect but better than it was before.
       - Eliminate the unused ffs function that relied on knowledge of the buf
         array.
       - Move the shutdown code that iterates over the buf array into vfs_bio.c.
      
      Reviewed by:	kib
      Sponsored by:	EMC / Isilon Storage Division
      44267026
  28. 21 Feb, 2014 1 commit
  29. 27 Dec, 2013 1 commit
  30. 19 Oct, 2010 1 commit
  31. 24 Apr, 2010 1 commit
  32. 19 Mar, 2010 1 commit
  33. 18 Dec, 2008 1 commit
    • sam's avatar
      Apply the big hammer: · f700131f
      sam authored
      o remove all of compat except for pwcache and strstuftoll; these might
        end up in libutil or similar so keep them in the subdir
      o mv getid.c up to the top level; this looks like something that'll be
        makefs-specific
      o eliminate private versions of .h files in sys; use system files instead
      o eliminate private ffs_tables.c; use the system version directly (might
        want to adopt const'ification at some point but that's the only diff I
        can see)
      o mv remaining code from sys to ffs and strip out unused bits; this now
        becomes part of makefs
      o add compat defs and shims to makefs.h
      o strip all vestiges of nbtool_config.h, compat_defs.h, etc.
      o fixup includes after file shuffling
      o rename system #defines that do implicit byte swapping to have an _swap
        suffix; e.g. DIRSIZ -> DIRSIZ_SWAP, cg_inosused -> cg_inosused_swap; if
        we ever add endian-agnostic support to the kernel these can go back to
        their original names
      o strip some netbsd'isms that aren't worth shim'ing (e.g. _DIAGASSERT)
      
      Code compiles w/o complaints but is untested.
      f700131f
  34. 23 Nov, 2008 1 commit
  35. 22 Nov, 2008 1 commit
  36. 19 Oct, 2008 1 commit
  37. 07 Jan, 2005 1 commit