I have a question about how to make a good design for my program. My program is quite simple but I want to have good architecture and make my program easily extensible in the future.
My program need to fetch data from external data sources (XML), extract information from these data and at the end it need to prepare SQL statements to import information to database. So for all the external data sources that there are now and there will be in the future there is simple 'flow' of my application: fetch, extract and load.
I was thinking about creating generic classes called: DataFetcher, DataExtractor and DataLoader and then write specific ones that will inherit from them. I suppose I will need some factory design pattern, but which? FactoryMethod or Abstract Factory?
I want also to not use code like this:
if data_source == 'X':
fetcher = XDataFetcher()
elif data_source == 'Y':
fetcher = YDataFetcher()
....
Ideally (I am not sure if this is easily possible), I would like to write new 'data source processor', add one or two lines in existing code and my program would load data from new data source.
How can I get use of design patterns to accomplish my goals? If you could provide some examples in python it would be great.
If the fetchers all have the same interface, you can use a dictionary:
fetcher_dict = {'X':XDataFetcher,'Y':YDataFetcher}
data_source = ...
fetcher = fetcher_dict[data_source]()
As far as keeping things flexible -- Just write clean idiomatic code. I tend to like the "You ain't gonna need it" (YAGNI) philosophy. If you spend too much time trying to look into the future to figure out what you're going to need, your code will end up too bloated and complex to make the simple adjustments when you find out what you actually need. If the code is clean up front, it should be easy enough to refactor later to suit your needs.