Getting data from FireStore and compile them takes too much time in Flutter

Question

I need several process before drawing UI in Flutter Web.

But the problem is that it takes too much time, So UI is drawn too late.

I need your advice my several process has something that I can reduce time.

Because I use same data and process with ReactJS before, and It doesn't takes this much at all compared to Flutter.

Or If there's no way of reducing time, then should I just show loading page before?

My data and process are below:

1. I need to get 3 data from 3 collections from FireStore of all users registered between two Dates: step, diary, and comment.

   • So I get all data between two dates from Database of each user by using for loop.

2. I re-calculate these 3 data with certain formula in order to make points of my app.

     Future<UserModel?> calculateItem(
          UserModel? user, DateTime startDate, DateTime endDate) async {
        final String userId = user!.userId;
    
        final List<int?> results = await Future.wait([
          _rankingRepository.getStepScores(userId, startDate, endDate),
          _rankingRepository.getDiaryScores(userId, startDate, endDate),
          _rankingRepository.getCommentScores(userId, startDate, endDate),
        ]);
    
        final int? stepScore = results[0];
        final int? diaryScore = results[1];
        final int? commentScore = results[2];
        final int totalScore = stepScore! + diaryScore! + commentScore!;
    
        final updatedUser = user.copyWith(
          totalScore: totalScore,
          stepScore: stepScore,
          diaryScore: diaryScore,
          commentScore: commentScore,
        );
     return updatedUser;
      }

• step / diary / comment -> each data and formula

     class RankingRepository {
          final FirebaseFirestore _db = FirebaseFirestore.instance;
        
          List<DateTime> getBetweenDays(DateTime startDate, DateTime endDate) {
            List<DateTime> dates = [];
            DateTime currentDate = startDate;
        
            while (currentDate.isBefore(endDate) ||
                currentDate.isAtSameMomentAs(endDate)) {
              dates.add(currentDate);
              currentDate = currentDate.add(const Duration(days: 1));
            }
            return dates;
          }
        
          Future<int> getStepScores(
              String userId, DateTime startDate, DateTime endDate) async {
            List<DateTime> betweenDates = getBetweenDays(startDate, endDate);
            int stepScores = 0;
        
            await Future.forEach(betweenDates, (DateTime date) async {
              final dailyScore = await calculateStepScore(userId, date);
              stepScores += dailyScore;
            });
        
            return stepScores;
          }
        
          Future<int> calculateStepScore(String userId, DateTime date) async {
            int dailyScore = 0;
            final dateString =
                "${date.year}-${date.month.toString().padLeft(2, '0')}-${date.day.toString().padLeft(2, '0')}";
            final query = await _db.collection("user_step_count").doc(userId).get();
        
            if (query.exists) {
              final dateExists = query.data()?.containsKey(dateString);
        
              if (dateExists!) {
                dynamic diaryStepString = query.get(dateString);
        
                final int dailyStepInt = diaryStepString as int;
        
                dailyScore = dailyStepInt < 0
                    ? 0
                    : dailyStepInt > 10000
                        ? 100
                        : ((dailyStepInt / 1000).floor()) * 10;
              }
            }
            return dailyScore;
          }
        
          Future<int> getDiaryScores(
              String userId, DateTime startDate, DateTime endDate) async {
            final query = await _db
                .collection("diary")
                .where("userId", isEqualTo: userId)
                .where("timestamp", isGreaterThanOrEqualTo: startDate)
                .where("timestamp", isLessThanOrEqualTo: endDate)
                .get();
            int docCount = query.docs.length;
            return docCount * 100;
          }
        
          Future<int> getCommentScores(
              String userId, DateTime startDate, DateTime endDate) async {
            final query = await _db
                .collectionGroup("comments")
                .where("userId", isEqualTo: userId)
                .where("timestamp", isGreaterThanOrEqualTo: startDate)
                .where("timestamp", isLessThanOrEqualTo: endDate)
                .get();
            int docCount = query.docs.length;
            return docCount * 20;
          }
        }

3. I run for loop to run this process of userList.

 for (UserModel? user in userDataList) {
          final updateScoreInUser = await ref
              .read(rankingProvider.notifier)
              .calculateItem(user, firstDateOfMonth, now);
          setState(() {
            _initialUserDataListState = false;
            _userContractType = getUserContractType;
            _userContractName = getUserContractName;
            _userDataList.add(updateScoreInUser);
          });
        }

4. after all, I also need to order list based on totalPoint to add step, diary, comment.

However even up to 3 process it takes too much time..

I think all other app has also some data process and this much time will never be taken.

Please help me..

Answer 1

Instead of making separate queries for each user and each data collection, try to minimize the number of queries by combining them. You can fetch all the required data for multiple users in a single Firestore query using the whereIn method to query multiple user IDs at once.

Also, instead of using a sequential loop to fetch data for each user, consider fetching the data for multiple users concurrently. You can use Future.wait to run multiple asynchronous tasks in parallel, which can significantly improve the performance.

Indexes! Do you have the proper indexes on the Firestore collections?

Then there is also cache the results using memoization techniques. This means when you do the same action to put something in a variable, it gets it from Firestore just once and the 2nd, 3rd, 4th etc times it comes from memory.

Pagination? Maybe you are just getting WAY too big a bite size.

If none of this works, add a spinner to the UI so the user knows to wait.