Hard limit of 16GB when uploading to Google Drive shared folder with a service account

I have written a Rust program using the crate google-drive3 to upload files (100GB each) to my personal google drive which has 10TB of space as part of my Google One plan. I am using a service account, and specifying the parent folder for the file upload as the folder in my Drive which has been shared with the service account. When I run the upload, it takes several hours (presumably uploading the whole file) then gives me the error "user storage quota exceeded" and the file is not saved on the drive. Querying the API for the storage quota information reveals that while my file is well within the max_upload_size, it is not within the Limit which is only 16106127360 bytes, or 16GB!

Other things I have tried:

The uploads worked fine when I was using interactive authentication, but now that I am trying to use a service account I get that error. Using the service account is preferred however because interactive authentication is just bad for my use case. Some people recommend that instead of sharing the folder with the service account, to impersonate the user using "domain-wide delegation" but the instructions to enable that involve a special admin panel and the login tells me my google account won't work for it, I need an account that has a "managed google service such as Workspaces" ie something for businesses I don't and will never have. I have read that despite the error wording, the storage quota being referenced here is not per-user but rather for the project in GCP. However when I look up the list of quotas for this project in GCP none of them are storage-related. There is another tab for requesting increasing of quotas but it's empty and has a message saying that there are no quotas that are extendable for this project. I have also read that the applicable quotas only show up here if you have billing info associated with the project but I am trying to upload to the big personal drive I already have from Google One, I am not trying to purchase storage on GCP. I have double checked against the solutions to similar questions on Stack Overflow, for example, I am already using .supports_all_drives(true) and the issue is not related to existing usage filling up the drive, the problem is the overall storage limit.

The main question, then, is how do I get around this 16GB limit that is only imposed when using a service account, so I can make full use of the 10TB I paid for.

Relevant parts of the code that show what I am doing:

Reading service account key from file: oauth2::read_service_account_key(&settings.gdrive.service_account_key_file).await

Building authenticator: oauth2::ServiceAccountAuthenticator::builder(service_account_key).persist_tokens_to_disk(cache_path).build().await

Connecting and creating the hub object: DriveHub::new(hyper::Client::builder().build(hyper_rustls::HttpsConnectorBuilder::new().with_native_roots().https_or_http().enable_http1().enable_http2().build()), auth)

Uploading:

hub.files().create(file_props)
    .use_content_as_indexable_text(false)
    .supports_all_drives(true)
    .keep_revision_forever(false)
    .ignore_default_visibility(false)
    .delegate(&mut UploadDelegate::new())
    .upload_resumable(
        file,
        mime_type.clone()
    )
    .await

Solution

There is no solution for fully automatic uploading to Drive storage purchased through Google One for regular gmail accounts when using the API via the crate google-drive3, and probably not in any other SDK either.

To answer the issues posed in the question:

The storage capacity that a file consumes is based on the owner and is unaffected by the fact that the file's parent is a shared folder owned by someone else.
A file uploaded by a service account will be owned by the service account.

Which suggests that you can get by by splitting your files into smaller pieces that fit in the service account's quota, then changing the owner to the gmail account after each upload. However,

Easy transferance of file ownership is only possible between "accounts in the same domain" ie when using Workspaces/corporate accounts.
For us plebs, the documentation says that setting a new owner on a file only initiates a request that has to be manually approved by the recipient, not feasible when you're trying to make a background process upload hundreds of files and each one must be reassigned immediately to make room for the next in the storage quota.
In practice, even that doesn't seem to be possible since, at least when using google-drive3, uploading files to a shared folder seems to be a "black hole" where the service account can not see these files once uploaded, even ones it still owns, and therefore gets Not Found errors trying to look for them to set their ownership or delete them to make room to upload anything else.

So it would seem that the only ways of fully automating your Google Drive uploads are:

Fully buy in to Workspaces/GCP so you can use Domain Wide Delegation to impersonate the target user with the service account and upload directly into their Drive without worrying about all these extra quota and ownership issues. Expensive, and inconvenient if you were expecting this to work alongside your regular personal account stuff.
Live with the difficulties of interactive authentication to upload as the user without the service account. Maybe you can make a separate workflow for authenticating, then store the token you get and use it asynchronously. This may work for small things, but Google imposes some strict limits on how long tokens can last unless you have an official Published app (The requirements are very onerous for someone just writing a server-to-server app for personal use -- but google expects server-to-server stuff would be done with a service account and not interactive auth anyway). Without this verification, however, the access your app gets via interactive auth doesn't last long enough to finish 1 job that is uploading a few TB of files and therefore doesn't work for the purposes laid out in the question.