Unable to Return JSON Data from NestJS API

Question

I am building a NestJS API which will call Bing's Custom Search Engine endpoint and return the search results in JSON format. https://learn.microsoft.com/en-us/azure/cognitive-services/bing-custom-search/call-endpoint-nodejs

I am successfully able to call the endpoint using my API key and print the resulting JSON to my console. The issue I have is that the function returns the empty data before the request has a chance to populate the array with any data.

I apologize if the code format is completely wrong or if the entire thing is setup incorrectly. I've never programmed an API before and have to learn it on the fly.

Controller

import { Controller, Get } from '@nestjs/common';
import { SearchEngineService } from './search-engine.service';

@Controller('tasks') 
export class SearchEngineController {
    constructor(private searchEngineService: SearchEngineService) {}

    @Get()
    processRequest() {
        return this.searchEngineService.processRequest();
    }
}

Service

import { Injectable } from '@nestjs/common';

@Injectable()
export class SearchEngineService {
    processRequest() {
        var request = require("request"); 
        var searchResponse;
        var webPage;
        var parsedSearchResponse = [];
        var subscriptionKey = "SUBSCRIPTION_KEY";
        var customConfigId = "CONFIG_ID";
        var searchTerm = "partner";

        var info = {
            url: 'https://api.cognitive.microsoft.com/bingcustomsearch/v7.0/search?' 
                + 'q=' 
                + searchTerm 
                + "&" 
                + 'customconfig=' 
                + customConfigId,
            headers: {
                'Ocp-Apim-Subscription-Key' : subscriptionKey
            }
        }

        request(info, function(error, response, body) {
            searchResponse = JSON.parse(body);
            console.log("\n");
            for (var i = 0; i < searchResponse.webPages.value.length; ++i) {
                webPage = searchResponse.webPages.value[i];
                parsedSearchResponse.push(searchResponse.webPages.value[i]);
            
                console.log('name: ' + parsedSearchResponse[i].name);
                console.log('url: ' + parsedSearchResponse[i].url);
                console.log('displayUrl: ' + parsedSearchResponse[i].displayUrl); 
                console.log('snippet: ' + parsedSearchResponse[i].snippet);
                console.log('dateLastCrawled: ' + parsedSearchResponse[i].dateLastCrawled);
                console.log();
            
            }
            console.log("\n###################################");
        });

        return parsedSearchResponse;
    } 
}    

Answer 1

Welcome to the wonderful world of async programming.

Your function return before the request is finished. As a snippet is worth a thousands words, take a look at what's happening and in which order:

// 1. Start an http request and register a callback to back when complete
request(info, function(error, response, body) {
    // 3. This function is called once the request is complete
    searchResponse = JSON.parse(body);

    for (var i = 0; i < searchResponse.webPages.value.length; ++i) {
        webPage = searchResponse.webPages.value[i];
        parsedSearchResponse.push(searchResponse.webPages.value[i]);
    }

    // 4. Your array is ready but your function returned long ago
});

// 2. Return parsedSearchResponse, an empty array at this point
return parsedSearchResponse;

You need to wait for your request to finish before returning your function. Best way to do this is via promises. I usually don't recommend to swith to new libraries but as request is deprecated and doesn't support promise I will do an exception and advise you to swith to another http client. I like axios but any client that support promises will do.

Here is a possible solution for your problem, I didn't test it as I don't have credentials but feel free to comment if something is not working.

import { Injectable } from '@nestjs/common';
import axios from 'axios';

// Put this values in environment variable
const subscriptionKey = "SUBSCRIPTION_KEY";
const customConfigId = "CONFIG_ID";

@Injectable()
export class SearchEngineService {
  async processRequest() {
    let webPage;
    const parsedSearchResponse = [];
    const searchTerm = "partner";

    const url = `https://api.cognitive.microsoft.com/bingcustomsearch/v7.0/search?q=${searchTerm}&customconfig=${customConfigId}`

    const options = {
      headers: {
        'Ocp-Apim-Subscription-Key' : subscriptionKey
      }
    }

    try {
      const { data: searchResponse } = await axios.get(url, options);

      for (let i = 0; i < searchResponse.webPages.value.length; ++i) {
        webPage = searchResponse.webPages.value[i];
        parsedSearchResponse.push(searchResponse.webPages.value[i]);
      }
    } catch (e) {
        // Handle your error cases
    }

    return parsedSearchResponse;
  }
}