Using syscall open() to open a file in FUSE

Question

I'm recently working on fuse, but yet still cant have a grasp of how it works since i have very little knowledge about filesystem in linux. One of the most confusing thing is that what will happen if i use syscall open() to open a file within the function for the fuse_operation "open", like the code below

int bb_open(const char *path, struct fuse_file_info *fi)
{
    int retstat = 0;
    int fd;
    char fpath[PATH_MAX];

    bb_fullpath(fpath, path);
    
    log_msg("bb_open(fpath\"%s\", fi=0x%08x)\n",
        fpath,  (int) fi);
    
    fd = open(fpath, fi->flags);
    if (fd < 0)
    retstat = bb_error("bb_open open");
    
    fi->fh = fd;
    log_fi(fi);
    
    return retstat;
}

My assumption was:

1. A file within this filesystem("myfilesystem") is being opened, the request eventually gets accepted by fuse and the bb_open() is invoked.
2. When the execution reaches "fd = open(fpath, fi->flags)", it is going to open() a file within this filesystem again which goes back to step one, then comes a loop.

BUT after trying the code, it actually end up opening the file successfully, which confuses me. Did i miss something important? Hope i make this problem understandable, and thanks in advance!

---

EDIT

thanks for the help guys! but i still couldnt get the deadlock that user253751 have metioned, i think i should post more details here:

static int hello_getattr(const char *path, struct stat *stbuf,
                         struct fuse_file_info *fi)
{
        (void) fi;
        int res = 0;
        memset(stbuf, 0, sizeof(struct stat));
        if (strcmp(path, "/") == 0) {
                stbuf->st_mode = S_IFDIR | 0755;
                stbuf->st_nlink = 2;
        } else if (strcmp(path+1, "hello") == 0) {
                stbuf->st_mode = S_IFREG | 0444;
                stbuf->st_nlink = 1;
                stbuf->st_size = strlen("hello there");
        } else
                res = -ENOENT;
        return res;
}
static int hello_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
                         off_t offset, struct fuse_file_info *fi,
                         enum fuse_readdir_flags flags)
{
        (void) offset;
        (void) fi;
        (void) flags;
        if (strcmp(path, "/") != 0)
                return -ENOENT;
        filler(buf, ".", NULL, 0, 0);
        filler(buf, "..", NULL, 0, 0);
        filler(buf, "hello", NULL, 0, 0);
        return 0;
}
static int hello_open(const char *path, struct fuse_file_info *fi)
{
        char* fpath = get_full_path(path);
        int fd = open(fpath,O_RDONLY);
        fi->fh = fd;
        log1(fpath);
        log1("  open\n");
        return 0;
}
static int hello_read(const char *path, char *buf, size_t size, off_t offset,
                      struct fuse_file_info *fi)
{
        char* fpath = get_full_path(path);
        read(fi->fh,buf,size); 
        log1(fpath);
        log1("  read\n");
        return size;
}

Basically i use the hello_readdir() to fill a filename to the directory,then tell the hello_getattr() to get some random info for the file. Then i "tail" the file five times, and this is what i get from the log

/home/P1G3s/WorkSpace/PIT/libfuse-fuse-3.9.2/example/hello  open
/home/P1G3s/WorkSpace/PIT/libfuse-fuse-3.9.2/example/hello  read
/home/P1G3s/WorkSpace/PIT/libfuse-fuse-3.9.2/example/hello  open
/home/P1G3s/WorkSpace/PIT/libfuse-fuse-3.9.2/example/hello  open
/home/P1G3s/WorkSpace/PIT/libfuse-fuse-3.9.2/example/hello  open

i thought it should not be opening a file that does not exist, but it seems the open() is doing fine and no deadlock happened, however the read wont show up after the first tail.

Answer 1

If you try to open a file inside your own FUSE filesystem, this will create a deadlock because open is waiting for your FUSE filesystem to finish processing the last open request before it sends the next one, but your FUSE filesystem is waiting for open to finish before it finishes that request.

But if you open another file that isn't inside your FUSE filesystem, there's no problem. You open the different file, and finish processing the other program's open request, and then open returns in the other program. It's just recursion.