I am working on a C# Web API that performs a specific action using a third-party-api-1, executed concurrently for speed. Once I get my output from the third-party-api-1, I would further get additional processing using the third-party-api-2, which is again intended to be run concurrently, and its results would be appended to the results from third-party-api-1, and eventually the output from my API would be sent back to the caller.
So the path is:
- My API receives input
- My API calls third-party-api-1 concurrently, which I intend to keep for speed
- My API calls third-party-api-2, again wanting to keep it concurrent execution
I would like to identify my http requests after the calls to third-party-api-1 and third-party-api-2 are done so that I can tie back the outputs to input and return the result to the caller. If I had any control over third-party-api-1 & 2, I would have set some correlation-ids in the request header and persisted in the response too, but I can't do that with my third party APIs.
My issue is that once I construct my http get requests into a concurrent mode, I am no longer able to link back each response to its original request.
Is there any way to achieve what I am trying to do? Or is my attempt futile or bad design all together?
// I build my get query inputs from my user request
foreach (ParsedInputAddress ia in inaddr)
{
.
.
.
addresses.Add(addrbuilder.ToString());
addrbuilder.Clear();
}
.
// I call the http concurrently
IEnumerable<Task<HttpResponseMessage>> getTasks = addresses.Select(async ia => await httpClient.GetAsync(basepath + ia.ToString()));
var gets = await Task.WhenAll(getTasks);//this tasks will run concurrently
foreach (var getContent in gets)
{
jsonResponse = await getContent.Content.ReadAsStringAsync();
if (jsonResponse != null)
vaddr = JsonSerializer.Deserialize<ValidatedAddress>(jsonResponse);
if (vaddr != null && vaddr.address != null)
vaddresses.Add(vaddr.address);
}
I tried to set custom http request with an unique GUID but since I do not have control over third-party-api, I am unable to receive that ID in the response to tie them together.
My last resort is to make this a non-concurrent execution so that each request is processed by API-1 and 2 sequentially, preserving the link between request and response.