Search code examples
linuxfilesystemstarext4

I received a 'no space left on device' error during the untarring process, even though I have enough space and sufficient inodes

  1. I'm going to save a lot of small files on disk, so I formatted it with this command:

sudo mkfs.ext4 -m 0 -T small -i 8192 /dev/sdb

  1. I checked the disk as follows:

sudo fsck -f /dev/sdb

fsck from util-linux 2.36.1
e2fsck 1.46.2 (28-Feb-2021)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb: 11/65536000 files (0.0% non-contiguous), 8680071/524288000 blocks
  1. Then I mounted it:

sudo mount /dev/sdb /mnt/disk

  1. I started to untar an archive with ~17 million files:

tar -xvf archive.tar -C /mnt/disk

train_m/233557/10.jpg
tar: train_m/233557: Cannot mkdir: No space left on device
tar: train_m/233557/10.jpg: Cannot open: No such file or directory
train_m/233557/5.jpg
tar: train_m/233557: Cannot mkdir: No space left on device
tar: train_m/233557/5.jpg: Cannot open: No such file or directory
train_m/233557/0.jpg
tar: train_m/233557: Cannot mkdir: No space left on device
tar: train_m/233557/0.jpg: Cannot open: No such file or directory
train_m/492576/
train_m/492576/1.jpg
train_m/492576/2.jpg
train_m/492576/3.jpg
train_m/492576/0.jpg

As I understand it, this means that some directories couldn't be extracted while others could.

  1. When the archive had been extracted, I received the following message:

tar: Exiting with failure status due to previous errors

Apparently, a few thousand directories (out of ~600k directories) weren't extracted. How could this have happened?

After extraction, I checked the space:

df -i

Filesystem              Inodes   IUsed    IFree IUse%
/dev/sdb              65536000 8276432 57259568   13%

df -h

Filesystem             Size  Used Avail Use%
/dev/sdb               492G   97G  396G  20%

P.S. If I format the disk with the default parameters, the archive extracts successfully, but due to the large number of files, I reach the inode limit and cannot add any more data to the disk.

Solution

  • The problem was solved by adding the large_dir option during formatting:

    mkfs.ext4 -O large_dir -m 0 -T small -i 8192 /dev/sdb