Search code examples
crystal-lang

Pass dynamic types into pipeline container


I want to make shard that help to process data step-by-step with workers organized into pipeline. I achieved some result, but it seems to me that it is possible to make it easier and more convenient. In the current solution I do not like that need specify the type of each Worker even if the Worker is wrapped in another class (online example):

class Worker
  def do_it(data : String)
    puts "#{data} from #{self.class}"
  end
end

class Worker1 < Worker; end
class Worker2 < Worker; end
class Worker3 < Worker; end

class Wrapper(T)
  property item : T
  def initialize(@item : T); end
  def worker
    @item
  end
end

class Pipeline(T)
  @stack = [] of T

  def self.build
    with self.new yield
  end

  def add(wrapper : T)
    @stack << wrapper
  end

  def run(data : String)
    @stack.each do |wrapper|
      wrapper.worker.do_it(data)
    end
  end

end

alias WrapperObject = Wrapper(Worker1) | Wrapper(Worker2) | Wrapper(Worker3)

Pipeline(WrapperObject).build do
  add Wrapper(Worker1).new(Worker1.new)
  add Wrapper(Worker2).new(Worker2.new)
  add Wrapper(Worker3).new(Worker3.new)
  run("Some string data")
end

I tried to solve this with macro that extract types from block and pass it to constant of Pipeline class and then i can use types into internal methods:

macro collect_types(workers)
  {% worker_types = workers.map{|k| k.id }.join(" | ") %}
  alias worker_types = {{ worker_types }}
  STACK = [] of {{ worker_types }}
end

but lost the possibility of inheritance Pipeline... In the light of my dreams shards should use like this:

require 'pipeline'

class Worker < Pipeline::Worker
  def process(data : String)
    # some business logic
  end
end

pipeline = Pipeline::Line.build do
  add Worker1
  add Worker2
  add Worker3
end

result = pipeline.run(data)

It is possible?


Solution

  • The problem has been solved with Command pattern for Crystal by @veelenga. Example on carc.in