Search code examples
javahttpclient

How to prioritize a certain request in java?

I feel like this is such a niche question so I will try to explain it as best possible as I can.

Intro: I'm sending ~500 requests / second and I feel like the more requests I send the slower the requests are handled (it becomes noticeably slower at some point)

Question: So the question is in Java is there any way to prioritize a request? Any solution that I am seeking is to optimize the speed of such request.. So any answer that would take time before the request is sent is not of my concern.

INFO: (I hope this is sufficient if not please tell me!)

  • The library I am using is apache httpclients (however I can switch if the solutions calls for it)
  • I also am multi threading the requests on one server/pc. I hope this is helpful information.
  • CPU Usage varies from (5-15%) - I believe these are the measurements enter image description here

I am sending 2 types of request and I only need to prioritize 1 type

  1. HTTP GET Request - HTML Response expected
  2. HTTP POST Request - JSON response expected (although I do not need the response)

#2 is the request that I want to prioritize. I send this request very little but when I send it I need it to be as quick as possible.

Solutions thought of: The only solution I have come up with is to stop/end all of the live connections in order to execute the request I want, however I think that doing so will take a considerable amount of time causing the solution to become a waste of time.

Note: You could say I am an idiot in this area so if the solution is non existent or obvious I am sorry, also if there is a duplicate I am also sorry.. I could not find any questions even close to this.

Solution

  • Ideally, you never want to do that on the client. You want this on the server, but I do understand that this might not be an option.

    (Not going to mention HTTP/2 and priority since I already did in the comments).

    The easiest way to think about it is: "I'll just sort them based on some XXX rule". You will then realize you need a Queue/Deque implementation, most probably a thread-safe one. You will want to put entries in this queue by some threads, but remove them by others. Thus you will need a thread-safe PriorityQueue. And, afaik, there are only blocking implementations of such, which means - you can end-up artificially delaying non-priority requests for no reason. It gets funner, you have 100 PUT requests and only one has a HIGH priority. You have already received the requests, but since you have no control on how threads are scheduled (the ones that insert into this queue), your HIGH priority request is put last.

    What we did is slightly different. We get all requests and dispatch them to two different thread pools, based on their paths.

    .../abc -> place in queueA -> process by thread-pool-A
.../def -> place in queueB -> process by thread-pool-B

    thread-pool-A uses threads with Thread.MIN_PRIORITY and thread-pool-B uses Thread.MAX_PRIORITY. For that to sort of work, you need to read this, rather carefully. I wish I could tell you that this worked smoothly or that I have actual numbers from real production - but I have longed moved to a different workplace since then.

    This is just to give you an idea that there is yet another way to do it.