Search code examples
sqlbenchmarkingrdbms

Looking for generic library for SQL script execution (a.k.a. RDBMS benchmarking library)

I wonder if there's a library that abstracts the following DBMS operations:

  1. Load data (a bunch of CSV files or SQL insert statements)
  2. Execute a bunch of queries
  3. Get measurements for the above (throughput, latency, etc)
  4. Report generation (graphs, etc.) based on the above.

I have a bunch of DDL/DML files for different databases (for example TPC-H, SSB, etc) and I want to be able to load the data and run queries to many DBMSs without having to worry about the particularities of the specific DBMS (MySQL,SQLServer,Oracle, etc).

The closes thing I've found so far are these. The problem with them is that they’re specific for a given workload (TPC in most of the cases) in the sense that the domain specific objects are modeled as classes (Java or C++). What I'm looking for is domain-independent and DBMS-agnostic.

Solution

  • OLTPBench fits the bill for all stated requirements:

    Benchmarking is incredibly useful, yet endlessly painful. This benchmark suite is the result of a group of Phd/post-docs/professors getting together and combining their workloads/frameworks/experiences/efforts. We hope this will save other people's time, and will provide an extensible platform, that can be grown in an open-source fashion.

    OLTPBenchmark is a multi-threaded load generator. The framework is designed to be able to produce variable rate, variable mixture load against any JDBC-enabled relational database. The framework also provides data collection features, e.g., per-transaction-type latency and throughput logs.

    Together with the framework we provide the following OLTP/Web benchmarks:

    • TPC-C
    • Wikipedia
    • Synthetic Resource Stresser
    • Twitter
    • Epinions.com
    • TATP
    • AuctionMark
    • SEATS
    • YCSB
    • JPAB (Hibernate)
    • CH-benCHmark
    • Voter (Japanese "American Idol")
    • SIBench (Snapshot Isolation)
    • SmallBank
    • LinkBench
    • CH-benCHmark

    This framework is design to allow easy extension, we provide stub code that a contributor can use to include a new benchmark, leveraging all the system features (logging, controlled speed, controlled mixture, etc.)