I need to do it in a way that the binding target is given the value of the most recently changed SelectedValue.
Databind both your listbox's SelectedValue to a view model property,So that the most recently selected value will be available in your Property.Then update your binding target accordingly.All depends what and how you are trying to do things