Search code examples
linuxgccdynamiczipfuse

How to dynamic output a file content with replaced some bytes?


I used a fuse plugin named 'concatfs'

It can virtual merge some file with a specific filename like xxx-concat-.mkv, and this file's content is a TEXT:

1.mkv
2.mkv
3.mkv

system reads xxx-concat-.mkv, it outputs a merged file with 1.mkv 2.mkv .... but the files are not merged in fact, it's a virtual output.

So, My question is :

  • a static file, 1.zip > 300 MB.
  • It has so many copies(publish to many people), and the difference is only 4 bytes offset: 0xa 0xb 0xc 0xd in these copies. the 4 bytes is people's id
  • so I wanna make a file named '1-replaced-123.zip' like concatfs, content mostly like:(all binary)

1.zip

0xa 4 replaced-4bytes-binary-content

0x1111 10 replaced-10bytes-binary-content

  • system reads 1-replaced-123.zip, the fuse will outputing the replaced content.
  • It can saves so many disk space.
  • And I known PHP/JSP can read and echo with replaced, but php will using a high cpu/memory to echo big file stream.

do you have some suggestion?

Thanks a lot, and sorry for my english.


Solution

  • Use merged-fuse to virtual merge/replace files (C++ 11)

    https://github.com/fly-studio/merged-fuse

    Example

    1.txt

    12345678
    

    2.txt

    abcdefghi
    

    Make a file named 1-merged-.txt,

    -merged- is a special words in the file name

    content is JSON format

    [
        {
            "path": "1.txt"
        },
        {
            "path": "2.txt",
            "replaces": [
                 {
                     "offset": 2,
                     "length": 2,
                     "content": "MjE="
                 }
            ]
        }
    ]
    

    When cat 1-merged-.txt, output:

    12345678ab12efghi
    

    You can see the files were merged(virtual), and the content was replaced:

    cd replaced to 12