How can I create a Grid with it's X and Y axis bound to different collections?

I need to display something that shows a Week of days along the Columns, and a list of Categories along the Rows, and a list of Tasks that are due in the Cells (See sketch below)

I feel like the solution should be simple, but I'm drawing a blank as to how to bind such a thing.

Both the Rows, Columns, and data items in the Cells are dynamic based on what week the user is viewing, and ideally I'd like to hide categories that do not have any tasks for the currently viewed Week. I have collection properties for the currently Viewed Week, the list of Categories, and the list of Tasks, and each Task has a DueDate and Category associated with it....

Task Scheduler Sketch

Edit

My database looks like this:

TASK TABLE

TaskId
CategoryId
OtherProperties

TASK INSTANCE TABLE

TaskInstanceId
ParentTaskId
DueDate

I have a stored procedure that accepts a range of dates and returns a list of the following class (need to calculate recurring events)

TaskInstanceId
ParentTaskId
Name
Category
DueDate
OtherProperties

I was trying to do this with a 3rd class which contains

Date
List<Task>

Solution

I think I might have found a way to do it... I need to rearrange my Data a bit

My main ViewModel will contain ObservableCollection<DateTime> CurrentWeek and ObservableCollection<Category> CurrentTasks
The Category class should have ObservableCollection<Task> Tasks
And Task should have ObservableCollection<TaskInstances> Occurences

The main ViewModel should also subscribe to CurrentWeek.CollectionChanged, and based on what the dates are in the collection, update each Category.Task[x].Occurences if Category.Task.IsRecurring == true

Then I think I can use the following structure to get what I want. I'll update this post when I know if this works or not

Update: It works :)

Sample Layout