read/write to a disk without a file system

Question

I would like to know if anybody has any experience writing data directly to disk without a file system - in a similar way that data would be written to a magnetic tape. In particular I would like to know if/how data is written in blocks, and whether a certain blocksize needs to be specified (like it does when writing to tape), and if there is a disk equivalent of a tape file mark, which separates the archives written to a tape.

We are creating a digital archive for over 1 PB of data, and we want redundancy built in to the system in as many levels as possible (by storing multiple copies using different storage media, and storage formats). Our current system works with tapes, and we have a mechanism for storing the block offset of each archive on each tape so we can restore it.

We'd like to extend the system to work with disk volumes without having to change much of the logic. Another advantage of not having a file system is that the solution would be portable across Operating Systems.

Note that the ability to browse the files on disk is not important in this application, since we are considering this for an archival copy of data which is not accessed independently. Also note that we would already have an index of the files stored in the application database, which we also write to the end of the tape/disk when it is almost full.

EDIT 27/05/2020: It seems that accessing the disk device as a raw/character device is what I'm looking for.

Answer 1

You can read/write to disk directly without going through the filesystem by opening the /dev/sda device directly, using lseek to seek to the block you want to read/write and calling read/write to do your read or write. Don't forget to close the device when you are done with it. In Linux everything is a file so the API you use is actually the same API you would use if you were writing to the file system. The only difference is that you open /dev/sda directly.

sample code:

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>

#define BLOCK_SIZE 512

int main(int argc, char *argv[]) {
    int fd;
    off_t offset;
    ssize_t nwritten;

    // Open the device file for the disk you want to write to
    fd = open("/dev/sda", O_WRONLY);
    if (fd == -1) {
        perror("open");
        return 1;
    }

    // Seek to the appropriate block
    offset = BLOCK_SIZE * 100;
    if (lseek(fd, offset, SEEK_SET) == -1) {
        perror("lseek");
        return 1;
    }

    // Write the data to the block
    char data[BLOCK_SIZE] = "Hello, World!";
    nwritten = write(fd, data, BLOCK_SIZE);
    if (nwritten == -1) {
        perror("write");
        return 1;
    }

    printf("Wrote %zd bytes to block %ld\n", nwritten, offset/BLOCK_SIZE);

    // Close the device file
    if (close(fd) == -1) {
        perror("close");
        return 1;
    }

    return 0;
}