Best way to add and extend a generic writer trait for step by step data storage

Question

I need to store/write information on different and multiple data support in my program (H2, SQLite,NEO4J, Text, etc.) I create exporter one by one, so the interface need to be easily extendable.

I make simulation program, so i need to store information at each step (need the db object, and the various data to store).

My first idea is to use dependecy injection pattern with generic method initializeWriter(), stepWriter(), closeWriter() like this :

trait OutputWriter {  Simulation =>

  def path:String

  def initializeWriter()
  def stepWriter()
  def closeWriter()
}

User which need to implement writer extends this trait, and override method. Normally 'database val' contain the DB object after initialization by initializeWriter. Data object contain the all-data-possible i want to write.

I have dependency between OutputWriter and Simulation, because actually i need to access object and function from simulation to write final result. I thinks it's not a really good solution, and given the answer of @x3ro it's a better solution to remove these dependency to give some better generic DATA object to the writer.

My problem is in this case if i need to add some complex informations to write this DATA correctly into database, where can i put this specific code which contain for example sqlquery, or other specific writer command ?

So, at this point my problem is :

1 - I create a Data object which contain data to write at each step of my simulation. Structure of this data object is the same, only the result value move.

2 - Any output or multiple output writer, which contain all the method to write correctly on the output : TextWriter, SQLITEWriter, etc.

3 - A specific part of code which make the relation between (1) -> (2). There is no problem to write an indexedSeq of double into text file, but into an SQL Database, i need to give to the writer structure of table, query to insert data, etc.

My first idea is to set this code into the stepWriter() of the Writer's method, but perhaps there is better solution for this because i think here i break the genericity of the writer

object DB
object MyData

trait DBWriter extends OutputWriter {

 val database:DB = DB

 def initializeWriter() = { ... }
 def stepWriter(dataToWrite:MyData) = { ... }

 } 

After what, if i need to export my simulation, i add the good writer like this :

new Simulation (...) with DBWriter {
  override def path = "my-path-for-db"

  initializeWriter()

  // computation loop 
 (0 until 50){ var result:MyData= computeData() ; stepWriter(result) }

  closeWriter()

}

There is other pattern (in litterature or based on you're experience) you use regulary which are more robust or more flexible to do that ?

Thanks a lot for your experience return, SR.

Answer 1

I don't think that what you present in your example falls into the "Dependency Injection" (DI) category. This is because DI aims to reduce dependencies in your codebase, and your classes/traits are actually all dependant on each other:

I suggest you read this article on Dependency Injection if you want to learn more about it.

---

Possible issues

As to your example, the problem is that your Simulation depends on DBWriter, and you'd need to change your implementation in order to use another writer. The simulation should only depend on the OutputWriter trait.

Another problem seems to be that the stepWriter() method of your DBWriter implementation needs parameters specific to the database writer, and is therefore not generic (and does not conform to the trait OutputWriter at all).

A third issue is the fact that your OutputWriter trait actually depends on the simulation, for which I really can't find a reason. To keep your OutputWriter as generic and re-usable as possible, you shouldn't be making it dependant on the Simulation. What was your reason to add this dependency?

---

My approach

I would make the following changes in order to enhance your dependency situation.

Make the OutputWriter and DBWriter generic: Your OutputWriter should really be just an interface, like so:

trait OutputWriter {
    def initializeWriter()
    def stepWriter(dataToWrite:Data)
    def closeWriter()
}

Make the DBWriter an actual class:

class DBWriter(database:DB) extends OutputWriter {
    def initializeWriter() { ... }
    def stepWriter(dataToWrite:Data) { ... }
    def closeWriter() { ... }
}

Pass a writer to the simulation when instantiating it:

object Foo extends App {
    val database:DB = ...
    val writer:OutputWriter = new DBWriter(database)

    new Simulation(..., writer)
}

Like this, you'll be able to simply change the writer that is being passed to the Simulation constructor, or even use multiple writers at the same time! This also allows for the writer to be configurable through an external configuration file (see the article on dependency injection mentioned above).

---

To address the questions in your edit

1 - Data i want to write a each step of my simulation. Structure of this data is the same, only the result value move. For example the simulation step return the same indexedSeq[Double] which contain different values to write on output at each step.

2 - Any output or multiple output writer, which contain all the method to write correctly on the output : TextWriter, SQLITEWriter, etc.

3 - A specific part of code which make the relation between (1) -> (2). There is no problem to write an indexedSeq of double into text file, but into an SQL Database, i need to give to the writer structure of table, query to insert data, etc.

Putting the SQL logic in your DBWriter is totally fine (which I'd then call SimulationSQLWriter). This of course makes the actual implementation less generic, but this must not be a bad thing. Always keep your code as generic as possible, but as specific as necessary, so that you don't have to add a lot of complexity to make code generic that you don't need to be generic at all.

Always keep a reasonable ratio between time and effort and benefit!

Now for a concrete example of the SimulationSQLWriter:

class SimulationSQLWriter(database:DB, table:String) extends OutputWriter {
    def initialize() { ... }
    def stepWriter(dataToWrite:Data) {
        val preparedData = prepareDataForInsertion(dataToWrite)
        insertIntoDatabase(preparedData)
    }
    def closeWriter() { ... }

    def prepareDataForInsertion(dataToWrite:Data) = { ... }
    def insertIntoDatabase(preparedData:<whatever type you need>) = { ... }
}

prepareDataForInsertion could prepare the given data and put it, for example, into a map mapping SQL field name to value, or a list of such map elements. That result could then be thrown into insertIntoDatabase, which according to the parameters passed to the constructor would insert it into the database table table.