I am working on an internal console app that allows users to get a full set of results from our paged JSON API. We have many different types but all fall into the same base structure.
public class Tickets
{
public Ticket[] tickets { get; set; }
public string nextPageURL { get; set; }
public string previousPageURL { get; set; }
public int recordCount { get; set; }
}
public class Ticket
{
...
}
I have an Async task to make the call and loop through the paged results and fire the results into a single JSON file. But I'd like to make it generic instead of essentially the same code repeated 17 times, one for each type.
My current code is:
private static async Task<List<Ticket>> GetTicketsAsync(Action<Tickets> callBack = null)
{
var tickets = new List<Ticket>();
HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes($"-----")));
httpClient.BaseAddress = new Uri("-----");
var nextUrl = "/api/tickets";
do
{
await httpClient.GetAsync(nextUrl)
.ContinueWith(async (ticketSearchTask) =>
{
var response = await ticketSearchTask;
if (response.IsSuccessStatusCode)
{
string jsonString = await response.Content.ReadAsStringAsync();
try
{
var result = JsonSerializer.Deserialize<Tickets>(jsonString);
if (result != null)
{
// Build the full list to return later after the loop.
if (result.tickets.Any())
tickets.AddRange(result.tickets.ToList());
// Run the callback method, passing the current page of data from the API.
if (callBack != null)
callBack(result);
// Get the URL for the next page
nextUrl = (result.nextPageURL != null) ? result.nextPageURL : string.Empty;
}
} catch (Exception ex)
{
Console.WriteLine($"\n We ran into an error: {ex.Message}");
nextUrl = string.Empty;
}
}
else
{
// End loop if we get an error response.
nextUrl = string.Empty;
}
});
} while (!string.IsNullOrEmpty(nextUrl));
return tickets;
}
private static void TicketsCallBack(Tickets tickets)
{
if (tickets != null && tickets.count > 0)
{
foreach (var ticket in tickets.tickets)
{
Console.WriteLine($"fetched ticket: {ticket.id}");
}
}
}
I've made a start into a generic method but referencing the second level (In his instance the ticket object) is causing me problems as well as getting the nextPageURL. It would also be nice to keep the call back structure to keep the console ticking over with with the data that has been processed.
private static async Task<List<T>> GetAsync<T>(string type, Action<T> callBack = null, string ticketId = null)
{
var results = new List<T>();
HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes($"-----")));
httpClient.BaseAddress = new Uri("-----");
var nextUrl = $"/api/{type}.json";
do
{
await httpClient.GetAsync(nextUrl)
.ContinueWith(async (searchTask) =>
{
var response = await searchTask;
if (response.IsSuccessStatusCode)
{
string jsonString = await response.Content.ReadAsStringAsync();
try
{
var result = JsonSerializer.Deserialize<T>(jsonString);
if (result != null)
{
// Build the full list to return later after the loop.
if (result.tickets.Any())
results.AddRange(result.tickets.ToList());
// Run the callback method, passing the current page of data from the API.
if (callBack != null)
callBack(result);
// Get the URL for the next page
nextUrl = (result.GetType().GetProperty("nextPageURL") != null) ? result.GetType().GetProperty("nextPageURL").ToString() : string.Empty;
}
}
catch (Exception ex)
{
Console.WriteLine($"\nWe ran into an error: {ex.Message}");
nextUrl = string.Empty;
}
}
else
{
// End loop if we get an error response.
nextUrl = string.Empty;
}
});
} while (!string.IsNullOrEmpty(nextUrl));
return results;
}
Any help greatly appreciated.
To start with, if you have multiple objects following the same structure you could likely make that generic:
public class Paged<T>
{
T[] Data { get; set;}
string nextPageURL { get; set;}
string previousPageURL { get; set;}
int recordCount { get; set; }
}
You should be able to use it more or less in place of the tickets object:
private static async Task<List<T>> GetAsync<T>(string type, Action<Paged<T>> callBack = null)
...
var result = JsonSerializer.Deserialize<Paged<T>>(jsonString);
If you cant use a common type due to backwards compatibility, you could still create a common interface for all similar classes:
public interface IPaged<T>
{
public T[] Data{ get;}
public string nextPageURL { get; set; }
public string previousPageURL { get; set; }
public int recordCount { get; set; }
}
public class Tickets : IPaged<Ticket>
{
public Ticket[] Data => tickets;
public Ticket[] tickets { get; set; }
public string nextPageURL { get; set; }
public string previousPageURL { get; set; }
public int recordCount { get; set; }
}
private static async Task<List<TData>> GetAsync<T, TData>(string type, Action<T> callBack = null) where T : IPaged<TData>
{
...
var result = JsonSerializer.Deserialize<T>(jsonString);
...
}
var allTickets = await GetAsync<Tickets, Ticket>(...);
However, I would consider using the IAsyncEnumerable interface instead, something like:
private static async IAsyncEnumerable<T[]> GetAsync<T>(string type){
...
var result = JsonSerializer.Deserialize<Paged<T>>(jsonString);
if (result != null)
{
yield return result.Data;
}
...
That should provide cleaner separation between the code to fetch data, and the code that processes the data.