Lazy Loading data with a RxJS Subject. Last batch is not displayed, the subject never completes

Question

I'm lazy loading data using a fake backend. The backend returns an array. Because I want to lazy load the data I'm buffering it every 100 records. The event that will trigger the call for getting more data is going to be a custom event, but for the time being I'm testing it with a button.

multiselect.service.ts

import { Injectable } from '@angular/core';
import { HttpClient, HttpHeaders, HttpResponse } from '@angular/common/http';
import { BehaviorSubject } from 'rxjs';

const httpOptions = {
  headers: new HttpHeaders({ 'Content-Type': 'application/json' })
};

export interface ProductCategory {
  id: number;
  name: string;
}

@Injectable({
  providedIn: 'root'
})

export class MultiSelectService {

  constructor(private http: HttpClient) { }

  private readonly categorySubject = new BehaviorSubject(undefined);
  readonly categories$ = this.categorySubject.asObservable();

  // URL to web api - mocked. Categories is the object present in the mock server file
  private categoriesUrl = 'api/categories';

  /** GET heroes from the server */
  getCategories(): void {
    this.http.get<Array<ProductCategory>>(this.categoriesUrl, httpOptions)
    .subscribe( data => {
      data.map( category => this.categorySubject.next(category));
      this.categorySubject.subscribe( xyz => console.log(xyz));
    });
  }
}

multiselect.component.ts

import { Component, AfterViewInit } from '@angular/core';
import { zip, Observable, fromEvent } from 'rxjs';
import { MultiSelectService, ProductCategory } from './multiselect.service';
import { bufferCount, startWith, map, scan } from 'rxjs/operators';

@Component({
  selector: 'multiselect',
  templateUrl: './multiselect.component.html',
  styleUrls: ['./multiselect.component.scss']
})
export class MultiselectComponent implements AfterViewInit {

  SLICE_SIZE = 100;

  constructor(private data: MultiSelectService) { }

  ngAfterViewInit() {
    const loadMore$ = fromEvent(document.getElementsByTagName('button')[0], 'click');
    this.data.getCategories(); // loads the data

    zip(
      this.data.categories$.pipe(bufferCount(this.SLICE_SIZE)),
      loadMore$.pipe(startWith(0)),
    ).pipe(
      map(results => results[0]),
      scan((acc, chunk) => [...acc, ...chunk], []),
    ).subscribe({
      next: v => console.log(v),
      complete: () => console.log('complete'),
    });
  }

}

multiselect.component.html

<button>Fetch more results</button>

1) The elements count is 429, and I'm showing them in batches of 100 elements. After four clicks (4 x 100) the last 29 are never shown. What am I missing to display the last 29 elements? I was able to check that the subject produced all the values.

2) Is there any better way of achieving the same functionality that I'm developing here? I'm mapping (it can be a forEach loop instead) the array due to the fact that after requesting the data I will only get one item (the array) with all the elements (429). That won't allow me to buffer the items if I want to do lazy loading.

3) I'm required to provide an initial value for the behaviorSubject, is there any way to avoid that?

Thanks in advance!

Answer 1

This seems like a very convoluted way of doing things

data.map( category => this.categorySubject.next(category));

This line returns a new array the same length as data containing undefined in each slot, if you want to create an observable from an array then you go

this.categories$ = from(data);

A BehaviorSubject with no initial value is a Subject.

All that aside you already have your data in an array, no need to create a new observable from it.

Have a BehaviorSubject with a number in it and each key press increment the number and use a combineLatest to get the amout of elements you want from the array.

combineLatest(categories$, page$).pipe(map(([categories, page]) => categories.filter((item, index) => index < page * pageSize)))

Is all you really need to do where you start page at 1 and increment it each button click.