Search code examples
filesystemsstoragepartitionvolumes

Device vs Partition vs File System vs Volume: how do these concepts relate to each other, accurately


How do these concepts relate to each other, from a Java developer standpoint?

My question: Could someone provide explanations or some links for simple and accurate / generally accepted definitions? Tks.

for reference, documents I found but are not clear to me:

http://arxiv.org/ftp/cs/papers/0508/0508063.pdf http://arxiv.org/ftp/cs/papers/0508/0508063.pdf

Not very lucky on Google.

My initial assumptions:

  • In the beginning is the physical device, and its driver.
  • A partition is a view of a portion of the device provided by the driver, not related to file system concept
  • A volume is the name given to a set of partitions organized in some way (RAID for instance)
  • A file system goes on top of a volume to store data in file units.
  • A file system provides usually a tree view of the volume
  • A file system tree may silently include other file system trees using file system Junction / Link features

Solution

  • Overall picture (Windows like)

    enter image description here
    Icons source: vector.me


    Disk, drive, partition, volume

    Disk or drive: The physical device used to store data. Drive seems more generic than disk which is related to the storage technology, e.g. there are hard disk drive, floppy disk drive and USB Flash drives.

    Disks are divided into sectors, each sector contains the same number of bytes. Sectors have a sector number which can be used to reference them individually.

    Partition and volume: Often used interchangeably, but it's not the same, there can be multiple volumes within a single partition.

    • A partition is a chunk of a disk with a specific size (e.g. a specific sector range of a hard disk. Disk partitioning is the act of dividing a disk into multiple chunks as if there were multiple disks. Some partitions may be divided in turn into multiple separate logical chunks, it must be supported by the partitioning scheme used.

    • The effective chunk (regardless being physical or logical) is called a volume. The raw volume can be later formatted to contain a file system which can itself store actual data.

    The operating system needs to keep tracks of the volumes in the system. It's were files and directories are stored.

    Partitioning can be done using two main partitioning schemes:

    MBR

    MBR was used with the legacy BIOS firmware. MBR can create up to 4 partitions on a drive, either primary or extended. The visible space on the drive is limited to 2 TB, space in excess cannot be used by partitions.

    There can be only one extended partition per drive, this partition can be divided into up to 128 logical volumes.

    One primary partition can be selected as the active partition and be used to boot the computer.

    GPT

    GPT supports drives larger than 2 TB and up to 128 partitions per drive. GPT is not compatible with BIOS, the computer must be configured with the an EFI firmware.

    GPT contains a fake MBR at the beginning of its space. This MBR shows the drive as being a single MBR partition to cope with tools which do not recognize GPT.

    Image

    An image is a snapshot of a volume (files and other data) into a single file, similarly to a zip file. An image from a volume can be expanded on another volume and an image can also be “mounted” or “attached” to appear like any other volume, or appear as a directory of an existing volume.

    Additional volumes can be created (“mounted”) from image files without being linked to actual physical units (except the one where the image file is stored).


    File system

    The file system is used to control how data is stored and retrieved on a volume. It's the practical way to store data organized into files and directories instead of unordered and unrelated bytes.

    The file system takes care of the file content and structure (tree). Directories and files are given properties (like read only) and access permissions.

    The legacy FAT file system was used with DOS OS. It's still supported by modern devices for compatibility and exchange purposes. FAT versions: FAT12, FAT16, FAT32, correspond to the number of bits used in the file entries, determining the number of sectors which can be referenced. FAT32 can reference 232 = 4,294,967,296 sectors. With sectors of 512 bytes, the FAT32 can therefore manage 2TB.

    Modern Windows versions use NTFS. NTFS adds support for metadata, access control list (permissions) and journaling.

    • MacOS uses APFS.

    • Linux often defaults to ext4.

    • Android uses ext4.

    • Optical disks (CD, DVD, Blu-ray) often use UDF.

    Disk, partitions, volumes and file systems on Windows (MBR):

    enter image description here
    Source

    Fragmentation

    When the file system cannot allocate contiguous sectors for a file, file content is stored in distant sectors, this fragmentation slows down data access in mechanical devices.

    HFS+ and ext4 have fragmentation control mechanisms, but to limit fragmentation, most file systems allocate space for a file by complete blocks/clusters, a block containing a given number of contiguous sectors. For example, NTFS can be configured to allocate 4KB clusters. Some file systems are able to reduce the effective unused space size, but a file usually owns more space than actually required to store data.


    Boot support

    When BIOS/EFI firmware starts the computer, the file system, which is part of the OS, is not available. BIOS/EFI instead looks for a boot sector (master boot record on PC) written on the boot drive during partitioning and/or OS installation. This code is a bootstrap which is able to load and execute the appropriate code from the active partition to start the main OS components, of which the file system which provides functions to load files. Then the OS takes control of the computer.


    Additional resources:

    https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2003/cc787202(v=ws.10)https://en.wikiversity.org/wiki/IT_Fundamentals/2014/File_Systemshttps://www.howtogeek.com/school/using-windows-admin-tools-like-a-pro/lesson4/?PageSpeed=noscripthttps://www.lifewire.com/volume-vs-partition-2260237https://en.wikipedia.org/wiki/File_system_fragmentation