Search code examples
postgresqlrustrust-tokio

How do I prevent BB8 connections to break after several repeats


I have an application that should use a shared connection pool for all requests. I observe that at seemingly-random times, requests fail with the error type "Closed". I have isolated this behavior into the following example:

use lazy_static::lazy_static;

use bb8_postgres::bb8::Pool;
use bb8_postgres::PostgresConnectionManager;
use bb8_postgres::tokio_postgres::{NoTls, Client};

lazy_static! {
    static ref CONNECTION_POOL: Pool<PostgresConnectionManager<NoTls>> = {
        let manager = PostgresConnectionManager::new_from_stringlike("dbname=demodb host=localhost user=postgres", NoTls).unwrap();

        Pool::builder().build_unchecked(manager)
    };
}

fn main() {
    println!("Hello, world!");
}


#[cfg(test)]
mod test {
    use super::*;

    #[tokio::test]
    async fn much_insert_traffic() {
        much_traffic("INSERT INTO foo(a,b) VALUES (1, 2) RETURNING id").await
    }

    #[tokio::test]
    async fn much_select_traffic() {
        much_traffic("SELECT MAX(id) FROM foo").await
    }

    #[tokio::test]
    async fn much_update_traffic() {
        much_traffic("UPDATE foo SET a = 81 WHERE id = 1919 RETURNING b").await;
    }

    async fn much_traffic(stmt: &str) {
        let c = CONNECTION_POOL.get().await.expect("Get a connection");
        let client = &*c;

        for i in 0..10000i32 {
            let res = client.query_opt(stmt, &[]).await.expect(&format!("Perform repeat {} of {} ok", i, stmt));
        }
    }

}

When executing the tests, >50% one of the tests will fail in a later iteration with output similar to the following:

Perform repeat 8782 of UPDATE foo SET a = 81 WHERE id = 1919 RETURNING b ok: Error { kind: Closed, cause: None } thread 'test::much_update_traffic' panicked at 'Perform repeat 8782 of UPDATE foo SET a = 81 WHERE id = 1919 RETURNING b ok: Error { kind: Closed, cause: None }', src\main.rs:44:23


Solution

  • Turns out, the problem is completely predicated on the [tokio::test] annotation starting up a distinct runtime whenever a test is executed. The lazy static is initialized with one of these runtimes, and as soon as that runtime shuts down, the pool is destroyed. The other tests (with different runtimes) can use the value as long as the spawning test still runs, but are met with an invalid state once it has shut down.