In the creation of a custom adapter for a said DB that will work correctly with the Microsoft Bot framework I am planning on creating something analogous to the cosmosDBPartitionedStorage class in the bot framework.
From what I see there are 3 operations of read, write, and delete that are inherited/implemented from botbuilder storage.
Is there anything from the database perspective that has to be considered in creating this adapter that isn't apparent from reading through a couple layers of source code. Such as initialization() for example. That is cosmos specific and how should I interpret that for the solution I need?
I plan on using 2 databases which of one is redis. I can test this in an Azure redis instance for my local development and I think that is a good place to get started. So in short, this would be for a redis adapter initially.
Update: I went with a Redis only cluster solution and it's solid. I was not able to achieve the concurrency checking because that would have to be a server side script, which I am using them for my CRUD operations, that will be more involved in a v2 update.
The help @mrichardson gave in the answer below was invaluable to creating your own data store. I was able to get most of the important base tests working in the unit testing for my TypeScript implementation too! All but the concurrency test.
In using Redis I was able to create an adapter that is compatible with JSON via the RedisJson module. This is a Redis module you have to install via your cmd or conf file configuration.
The library I went with was IORedis from Luin and the learning curve was steep not necessarily his library but the integration with what Redis does along with his library and being a cluster AND using the RedisJson module integration was a nice challenge!
Because of going with the RedisJson module I had to use LUA scripts load EVALSHA for every CRUD operation that falls back to EVAL if the script hasn't been loaded or is missing of any reason. It also reestablishes the script upon that failure.
I'm not sure if there is a great performance gain for using EVALSHA LUA scripting for just read and write operations but the Redis documentation seems to suggest it does.
A big advantage of scripting is that it is able to both read and write data with minimal latency, making operations like read, compute, write very fast (pipelining can't help in this scenario since the client needs the reply of the read command before it can call the write command).
However more importantly, the reason why I went with scripting in the first place was more to do with the IORedis client. It does pipelining but since there isn't native support for RedisJson commands I had to either do a custom script (in IORedis which doesn't allow pipelining but does the evalhas fallback for you) or create my own EVALSHA to EVAL fallback scenario.
Seems to work pretty awesome!
The codebase is for a RedisCluster and once I am finished putting a few tweaks on it I will post it as a typescript npm package via github and npm.
The inputs also take in a TTL setting which is a good security & performance abstraction for a messaging application such as is the Microsoft bot framework.
From what I see there are 3 operations of read, write, and delete that are inherited/implemented from botbuilder storage.
Correct. That's all you really need. So long as you can do those successfully, it will work just fine.
Is there anything from the database perspective that has to be considered in creating this adapter that isn't apparent from reading through a couple layers of source code. Such as initialization() for example. That is cosmos specific and how should I interpret that for the solution I need?
Also correct. That is Cosmos-specific. Basically, it:
You would want something like the initialization() function if you want to first check that the database exists prior to trying any kind of read/write. It's probably good practice to have something like this to future-proof your bot (if you change/add databases or something), but isn't required.
this would be for a redis adapter initially.
Unfortunately, we don't have any Redis storage adapters, but here's some additional storage adapters you can look at when you build yours:
When writing yours, if you want to make sure it works appropriately, we have a set of Storage Base Tests you can use. Your adapter should pass all of them.