Unexpected behaviour - unable to access new elements in an array driven by an NSFetchedResultController (SwiftUI)

Question

I have a SwiftUI app that uses the MVVM design pattern in places where the underlying logic driving the View is either verbose or unit testing is advisable. In certain places I have taken to using a NSFetchedResultsController in conjunction with @Published properties and, early in development, this behaved as I would expect.

However, I have now encountered a situation where an addition to the CoreData store triggers controllerDidChangeContent and the array populated by controller.fetchedObjects has an appropriate number of elements but, for reasons I cannot fathom, I am unable to access the newest elements.

There is a certain amount of data processing which, as I'm working with an array by this point, I didn't think would cause a problem. I'm more suspicious that relationships may be responsible in some way and/or faulting is responsible (although adjusting faulting behaviour on the underlying fetch request failed to resolve the issue).

Interestingly, some similar code elsewhere in the app that uses @FetchRequest (because the View is simpler and so a ViewModel wasn't considered necessary) doesn't seem to suffer from the same problem.

Normally scattering debugging around has put me back on track but not today! I've included the console output - as you can see, as new entries (timestamped) are added, the total observation count increases but the most property which should reflect the most recent observation does not change. Any pointers would be gratefully received as always.

I can't really prune the code on this without losing context - apologies in advance for the verbosity ;-)

ViewModel:

extension ParameterGridView {
    final class ViewModel: NSObject, ObservableObject, NSFetchedResultsControllerDelegate {
        @Published var parameters: [Parameter] = []

        @Published var lastObservation: [Parameter : Double] = [:]

        @Published var recentObservation: [Parameter : Double] = [:]

        let patient: Patient

        private let dataController: DataController

        private let viewContext: NSManagedObjectContext

        private let frc: NSFetchedResultsController<Observation>

        var observations: [Observation] = []

        init(patient: Patient, dataController: DataController) {
            self.patient = patient
            self.dataController = dataController
            self.viewContext = dataController.container.viewContext

            let parameterFetch = Parameter.fetchAll
            self.parameters = try! dataController.container.viewContext.fetch(parameterFetch)

            let observationFetch = Observation.fetchAllDateSorted(for: patient)
            self.frc = NSFetchedResultsController(
                fetchRequest: observationFetch,
                managedObjectContext: dataController.container.viewContext,
                sectionNameKeyPath: nil,
                cacheName: nil)
            try! self.frc.performFetch()

            observations = self.frc.fetchedObjects ?? []

            super.init()
            frc.delegate = self

            updateHistoricalObservations()
        }

        // MARK: - METHODS
        /// UI controls for entering new Observations default to the last value entered
        /// This function calculates the median value for the Parameter's reference range to be used in the event no historical observations are available
        /// - Parameter parameter: Parameter used to derive start value
        /// - Returns: median value for the Parameter's reference range
        func medianReferenceRangeFor(_ parameter: Parameter) -> Double {
            let rangeMagnitude = parameter.referenceRange.upperBound - parameter.referenceRange.lowerBound

            return parameter.referenceRange.lowerBound + (rangeMagnitude / 2)
        }

        /// Adds a new Observation to the Core Data store
        /// - Parameters:
        ///   - parameter: Parameter for the observation
        ///   - value: Observation value
        func addObservationFor(_ parameter: Parameter, with value: Double) {
            _ = Observation.create(in: viewContext,
                                   patient: patient,
                                   parameter: parameter,
                                   numericValue: value)

            try! viewContext.save()
        }

        /// Obtains clinically relevant historical observations from the dataset for each Parameter
        /// lastObservation = an observation within the last 15 minutes
        /// recentObservation= an observation obtained within the last 4 hours
        /// There may be better names for these!
        private func updateHistoricalObservations() {
            let lastObservationTimeLimit = Date.now.offset(.minute, value: -15)!.offset(.second, value: -1)!
            let recentObservationTimeLimit = Date.now.offset(.hour, value: -4)!.offset(.second, value: -1)!

            Logger.coreData.debug("New Observations.count = \(self.observations.count)")
            let sortedObs = observations.sorted(by: { $0.timestamp < $1.timestamp })
            let newestObs = sortedObs.first!
            let oldestObs = sortedObs.last!
            Logger.coreData.debug("Newest obs: \(newestObs.timestamp) || \(newestObs.numericValue)")
            Logger.coreData.debug("Oldest obs: \(oldestObs.timestamp) || \(oldestObs.numericValue)")

            for parameter in parameters {
                var twoMostRecentObservatonsForParameter = observations
                    .filter { $0.cd_Parameter == parameter }
                    .prefix(2)

                if let last = twoMostRecentObservatonsForParameter
                    .first(where: { $0.timestamp > lastObservationTimeLimit }) {
                    lastObservation[parameter] = last.numericValue
                    twoMostRecentObservatonsForParameter.removeAll(where: { $0.objectID == last.objectID })
                } else {
                    lastObservation[parameter] = nil
                }

                recentObservation[parameter] = twoMostRecentObservatonsForParameter
                    .first(where: { $0.timestamp > recentObservationTimeLimit })?.numericValue
            }
        }

        // MARK: - NSFetchedResultsControllerDelegate conformance
        internal func controllerDidChangeContent(_ controller: NSFetchedResultsController<NSFetchRequestResult>) {
            let newObservations = controller.fetchedObjects as? [Observation] ?? []
            observations = newObservations
            updateHistoricalObservations()
        }
    }
}

NSManagedObject subclass:

extension Observation {
    // Computed properties excluded to aid clarity

    class func create(in context: NSManagedObjectContext,
                      patient: Patient,
                      parameter: Parameter,
                      numericValue: Double? = nil,
                      stringValue: String? = nil) -> Observation {
        precondition(!((numericValue != nil) && (stringValue != nil)), "No values sent to initialiser")

        let observation = Observation(context: context)
        observation.cd_Patient = patient
        observation.timestamp = Date.now
        observation.parameter = parameter
        if let value = numericValue {
            observation.numericValue = value
        } else {
            observation.stringValue = stringValue!
        }

        try! context.save()
        
        return observation
    }

    static var fetchAll: NSFetchRequest<Observation> {
        let request = Observation.fetchRequest()
        request.sortDescriptors = [NSSortDescriptor(keyPath: \Observation.cd_timestamp, ascending: true)]

        return request
    }

    static func fetchAllDateSorted(for patient: Patient) -> NSFetchRequest<Observation> {
        let request = fetchAll
        request.sortDescriptors = [NSSortDescriptor(keyPath: \Observation.cd_timestamp, ascending: true)]
        request.predicate = NSPredicate(format: "%K == %@", #keyPath(Observation.cd_Patient), patient)

        return request
    }

    static func fetchDateSorted(for patient: Patient, and parameter: Parameter) -> NSFetchRequest<Observation> {
        let patientPredicate = NSPredicate(format: "%K == %@", #keyPath(Observation.cd_Patient), patient)
        let parameterPredicate = NSPredicate(format: "%K == %@", #keyPath(Observation.cd_Parameter), parameter)
        let compoundPredicate = NSCompoundPredicate(andPredicateWithSubpredicates: [patientPredicate, parameterPredicate])

        let request = fetchAll
        request.predicate = compoundPredicate

        return request
    }
}

Console output: (note observation count increments but the most recent observation does not change)

Answer 1

There is something wrong with your timestamps and/or sorting, the oldest observation is 4 days newer than the newest one (and it is in the future!)

Joakim was on the money - the timestamps are indeed incorrect; the problem was not in the logic but an error in the code (maths error relating to the TimeInterval between datapoints) that generated data for testing purposes. Garbage in, garbage out...

A lesson to me to be more careful - precondition now added to the function that generated the time series data (and a unit test!).

static func placeholderTimeSeries(for parameter: Parameter, startDate: Date, numberOfValues: Int) -> [(Date, Double)] {
    let observationTimeInterval: TimeInterval = (60*5) // 5 minute intervals, not 5 hours! Test next time!!
    let observationPeriodDuration: TimeInterval = observationTimeInterval * Double(numberOfValues)
    let observationEndDate = startDate.advanced(by: observationPeriodDuration)
    precondition(observationEndDate < Date.now, "Observation period end date is in the future")

    return placeholderTimeSeries(valueRange: parameter.referenceRange,
                                 valueDelta: parameter.controlStep...(3 * parameter.controlStep),
                                 numberOfValues: numberOfValues,
                                 startDate: startDate,
                                 dataTimeInterval: observationTimeInterval)
}