I'm writing a rest api in Rust, and I'm connecting it to Firestore. I set up default application credentials via the gcloud cli and when running it locally (whether via cargo run or via executing the release binary), it connects to Firestore just fine and can read and write data.
However, the problem begins as soon as it is dockerized. I deployed it to Cloud Run but it fails when creating the Firestore client:
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
let db = create_firestore_db().await?;
// ...
}
async fn create_firestore_db() -> anyhow::Result<FirestoreDb> {
let project_id = get_project_id()?;
FirestoreDb::new(project_id)
.await
.map_err(|e| anyhow::anyhow!("Failed to create FirestoreDb: {}", e))
}
fn get_project_id() -> anyhow::Result<String> {
std::env::var("PROJECT_ID").map_err(|e| anyhow::anyhow!("Failed to find PROJECT_ID: {}", e))
}
The message I get is
Error: Failed to create FirestoreDb: Firestore system/internal error: GCloud system error: Tonic/gRPC error: transport error
I get the very same error if I run the container locally. At first I thought it might be the case that (somehow) the container can't reach the internet, but running a shell in the container, installing inetutils, and pinging google works just fine.
I thought it might be an issue with the base image: I tried debian bullseye and its slim version, as well as doing away with the multi-stage dockerfile and just trying cargo run instead of installing it.
I don't think it's auth issue because the message is saying that it's a transport error, which AFAICT basically means that the destination server closed the connection?
Trying to Google this issue in the context of docker is frustrating, since all the results are about docker's network mode (which made no difference to my issue) or some other docker configuration issue. I've written node and go services that run on cloud run and connect to firestore and I've never had this issue pop up.
For extra context, the spec for firestore in my Cargo.toml is firestore = "0.32.2" and my Dockerfile looks like this:
FROM rust:1.67 as builder
WORKDIR /usr/src/my-rest-api
COPY . .
RUN cargo install --path my-cargo-workspace-binary-project
# ---
FROM debian:bullseye-slim
COPY --from=builder /usr/local/cargo/bin/my-rest-api /usr/local/bin/my-rest-api
CMD ["my-rest-api"]
I'd be very grateful to hear if anyone else has come across this issue and how they've conquered it.
Disclaimer: Author of the crate firestore.
The common issue happens inside docker images that are built without Root CA certs inside, which means that all TLS connections may actually fail (not just calls to Google APIs or firestore).
So, to fix it you need to install CA root certificates when you build your image.
For Debian-based images this usually needed:
RUN apt-get install -y ca-certificates
Or you can use Google Distroless which already contains CA root certs, more secure and small.
Other reports and docker examples:
Hope this helps.