We are looking at using Redis hosted in Azure. Using the StackExchange nuget, our first usage case is a search against our customer base. This is initially about 10,000 customers but will grow over time.
The customer will be a specific search model, something like this:
{
"Id": 123,
"Name": "Bob Smith",
"Age": 72,
"Postcode": "AB123EF",
"Email": "[email protected]",
"Telephones": [
"07710123456",
"01453123456"
]
}
My understanding is that you should avoid entering large blob objects into Redis, so storing the list as one object is not a great idea. So we have used the List* functions to store each item under a key and serializing the data.
Initial work looks something like this:
var tasks = new List<Task>();
foreach (var customer in customers)
{
tasks.Add(db.ListLeftPushAsync(_key, JsonConvert.SerializeObject(customer)));
}
await Task.WhenAll(tasks).ConfigureAwait(true);
This all works as we expect, I'm sure there's some optimization, but seems ok.
That has fundamentally left us with the question of querying the information. I understand that Redis stores everything as a string, but I don't understand how to search the list, as it is a Redis object and not a string. So we can't use ListRange(_key).Where(c => c.Contains("Bob")) for example.
Please let me know if our approach is wrong on this or there is a method I am just missing.
I'm completely agree with what @GuyKorland said that you should use a mature product like RediSearch, not to realize a custom search engine based on redis.
However, I'm very glad to simply introduce about how to implement a simple search engine by using the data structure of redis, just based on my knowledge for redis.
First, how to store a customer data like your sample customer into redis. You know, redis is a K-V database on memory, so you can flat store an object or a json data as a redis Hash structure with its key using the unique Id & a prefix, as below.
CUSTOM#123 =>
Id => 123
Name => Bob Smith
Age => 72,
Postcode => AB123EF
Email => [email protected]
Telephones#0 => 07710123456
Telephones#1 => 01453123456
Second, to build the indexes for all properties of the object class by using redis List or Sorted Set structure, as below.
For examples, using List for Name and Age properties. The idea is to use a prefix like NAME concat a Name value of a object like Bob with a symbol # to build a redis key and store the Id value as a redis List element.
\\ To build `NAME` index for all objects contains `Bob`, `Smith` and `Bob Smith` in the `Name` propertory
NAME#Bob => 123, ...
NAME#Smith => 123, ...
NAME#Bob Smith => 123, ...
\\ Do the same operation above to build `AGE` index
AGE#72 => 123, ...
Finally, you can search Bob in the redis keys having the prefix NAME# by using command GET NAME#Bob to get the list of customer Id exactly or command keys NAME#Bob* to get the list of customer Id fuzzily, then you can continous to get these customer data contains the Name property with Bob value. If you want to sort the list of customer Id, you should use Sorted Set instead of List to sort the result of Id list.
I still recommended you should use a mature product based on redis, such as RediSearch, it will help you to fast realize your needs.