[ upstream commit 208c69ceddb127d811c7dc011c69bd51b14896e1 ]

So far, the rate limiting has been enforced before enforcing parallel
requests. In theory, this is better because we can return error code
429 earlier to tell callers to slow down.

In practice, many callers will retry immediately anyway which means that
all the limiting will happen by the rate limiter. The rate limiter
relies on reservations that need to be canceled. If too many requests
happen in parallel, reservations can't be canceled quickly enough.

By swapping the enforcement of parallel requests with the rate limiter,
all requests will block for at least MaxWaitDuration if more than the
allowed number of parallel requests are pending which will naturally
pace the callers.

Swapping the enforcement order requires the acquired semaphore to be
released in error cases of the rate limiter. This requires to change the
structure of Wait() to have a single error handling structure. By
reusing finishRequest(), the metrics handler has to be adjusted slightly
to account for new outcomes as it now bumps the metric for canceled
requests as well. What remains unchanged is that only successful API
requests are used to calculate the mean processing duration.

Fixes: 3141e6581e0 ("rate: Add API rate limiting system")
Signed-off-by: Thomas Graf <thomas@cilium.io>