Service current limit

The current limit is to limit the number of concurrent and requested when accessing scarce resources, such as spikes and snapped goods, so as to effectively cut the peak and smooth the flow curve. The purpose of current limiting is to limit the rate of concurrent access and concurrent requests, or to request a speed limit within a time window to protect the system. Once the rate limit is reached or exceeded, the service can be denied, or queued.

Current limiting algorithm

counter

Using a counter to achieve current limit is a bit simple and rude. Generally, we limit the number of requests that can pass in one second. For example, the current limit qps is 100. The implementation idea of the algorithm is to start timing from the first request, within the next 1 second. Each time a request is made, the count is incremented by 1. If the accumulated number reaches 100, then subsequent requests are rejected. Wait until the end of 1s, return the count to 0, and start counting again.

This kind of implementation, I believe everyone knows that there is a drawback: If I have passed 100 requests within the first 10ms of the unit time within 1s, then the next 990ms can only be rejected by the request, we call this phenomenon For "spurs"

Leaky bucket

In order to eliminate the "spurt phenomenon", the leaky bucket algorithm can be used to achieve the current limit. The name of the leaky bucket algorithm is very vivid. There is a container inside the algorithm, which is similar to the funnel used in life. When the request comes in, it is equivalent to pouring water into the funnel. Then slowly flow out from the lower end of the mouth. Regardless of the volume above, the speed of the outflow will remain the same.

Regardless of how unstable the service caller is, the flow is limited by the leaky bucket algorithm and the request is processed every 10 milliseconds. Because the processing speed is fixed, the speed of the incoming request is unknown, and many requests may come in suddenly. The request that has not been processed is first placed in the bucket. Since it is a bucket, there is definitely a capacity limit. If the bucket is full, then New incoming requests are discarded.

Leaky bucket algorithm

This kind of algorithm has its drawbacks after use: it can't cope with short-term burst traffic.

Token bucket

In a sense, the token bucket algorithm is an improvement to the leaky bucket algorithm. The bucket algorithm can limit the rate of request calls, while the token bucket algorithm can limit the average rate of calls while allowing a certain degree of bursts. transfer.

In the token bucket algorithm, there is a bucket for storing a fixed number of tokens. There is a mechanism in the algorithm to put tokens into the bucket at a certain rate. Each request call needs to obtain the token first. Only when the token is obtained, has the opportunity to continue execution. Otherwise, choose to wait for the available token or reject it directly.

The action of putting the token is continuous. If the number of tokens in the bucket reaches the upper limit, the token is discarded. Therefore, there is a case where there are a large number of available tokens in the bucket, and the incoming request can be directly Get the token execution, for example, set the qps to 100, then after the limiter is initialized for one second, there are already 100 tokens in the bucket. At this time, the service has not been fully started, and when the startup is completed, the service is provided. The current limiter can withstand up to 100 requests instantaneously. Therefore, only if there is no token in the bucket, the request will wait, and finally it is equivalent to execute at a certain rate.

Token bucket algorithm

The Swoft current limiter uses the token bucket algorithm at the bottom, and the underlying layer relies on Redis to implement distributed current limiting.

Token bucket implementation principle

There are two general approaches to the generation of tokens in a token bucket:

  • One solution is to turn on a timed task, and the token is continuously generated by the timed task. The problem is that the system resources are greatly consumed. For example, an interface needs to limit the access frequency of each user separately. If there are 6W users in the system, at most 6W scheduled tasks need to be opened to maintain the order in each bucket. The amount of cards, such overhead is huge.
  • The second solution is a delay calculation that defines a resync function. The function is called before each time the token is obtained. The implementation idea is: if the current time is later than nextFreeTicketMicros, calculate how many tokens can be generated during the period, add the generated token to the token bucket, and update the data. In this case, you only need to calculate it once when you get the token.

Swoft adopts the second type. Whenever a token is obtained, resync is executed to update the number of tokens in the token bucket, thereby achieving the purpose of asynchronously generating the token. Let's start with a detailed explanation of how to use the swoft-limiter component for development.

installation

The principle of the current limiter has been explained in detail above. Before using the current limiter, the swoft-limiter component must be installed. The installation commands are as follows:

 composer require swoft/limiter 

Configuration

The current limiter can be used without a configuration. However, in some service scenarios, you need to configure the current limiter globally. You can refer to the following configuration:

 return [
    'rateLimiter' => [
        'class'      => RateLimter::class,
        'rateLimter' => bean('redisRateLimiter'),
    ]
]; 

specification:

  • Name limiter name, default swoft:limiter
  • Rate How much request access is allowed, number of requests/sec
  • Max maximum number of requests
  • Default initialization request number

Global configuration will be @Limiter by local configuration annotated with @Limiter

The cache rate limit information is also configurable. The configuration is as follows:

Return [ 'redisRateLimiter' => [ 'pool' => 'redis.pool' ]; ];

  • Pool specifies the name of the cache connection pool used. By default, the framework default connection pool is used.

With a speed limiter, be sure to configure the redis component and configure the available redis cache

use

Controller speed limit

This section takes a common speed limit scenario as an example:

 <?php declare(strict_types=1);

namespace App\Http\Controller;

use Swoft\Http\Message\Request;
use Swoft\Http\Server\Annotation\Mapping\Controller;
use Swoft\Http\Server\Annotation\Mapping\RequestMapping;
use Swoft\Limiter\Annotation\Mapping\RateLimiter;

/**
 * Class LimiterController
 *
 * @since 2.0
 *
 * @Controller(prefix="limiter")
 */
class LimiterController
{
    /**
     * @RequestMapping()
     * @RateLimiter(key="request.getUriPath()")
     *
     * @param Request $request
     *
     * @return array
     */
    public function requestLimiter(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['requestLimiter', $uri];
    }

    /**
     * @RequestMapping()
     * @RateLimiter(rate=20, fallback="limiterFallback")
     *
     * @param Request $request
     *
     * @return array
     */
    public function requestLimiter2(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['requestLimiter2', $uri];
    }

    /**
     * @RequestMapping()
     * @RateLimiter(key="request.getUriPath()~':'~request.query('id')")
     *
     * @param Request $request
     *
     * @return array
     */
    public function paramLimiter(Request $request): array
    {
        $id = $request->query('id');
        return ['paramLimiter', $id];
    }

    /**
     * @param Request $request
     *
     * @return array
     */
    public function limiterFallback(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['limiterFallback', $uri];
    }
} 
  • requestLimiter method limits the flow based on the URI request address
  • The requestLimiter2 method restricts the flow based on the class name + method name. In fact, it has the same function as the requestLimiter current limit, but defines a degraded function.
  • The paramLimiter method is based on URI address + parameter limit

Key This supports symfony/expression-language expressions, which can implement many complicated functions. Detailed documentation will throw a Swoft\Limiter\Exception\RateLImiterException if it is limited. The key expression has two variables, CLASS (class name) and METHOD (method name), which are convenient for developers to use.

Method speed limit

The Swoft speed limiter not only limits the flow controller, but also limits the methods in any bean to control the access rate of the method. Here is a detailed explanation of the following examples.

 <?php declare(strict_types=1);

namespace App\Model\Logic;

use Swoft\Bean\Annotation\Mapping\Bean;
use Swoft\Limiter\Annotation\Mapping\RateLimiter;

/**
 * Class LimiterLogic
 *
 * @since 2.0
 *
 * @Bean()
 */
class LimiterLogic
{
    /**
     * @RateLimiter(fallback="limterFallback")
     *
     * @return array
     */
    public function limter(): array
    {
        // Do something

        return [];
    }

    /**
     * @RateLimiter(key="requestBean.getName('name')")
     *
     * @param RequestBean $requestBean
     *
     * @return array
     */
    public function limterParams(RequestBean $requestBean): array
    {
        // Do something

        return [];
    }

    /**
     * @return array
     */
    public function limterFallback(): array
    {
        return [];
    }
} 
  • Limtier method, which limits speed based on class name + method name and defines a degraded function
  • The limtierParams method, based on the method that calls the RequestBean object, returns the value as the current-limit key

The function specified by fallback must be the same as the limiter, including the name and parameters, the return value, and must be in the same class. If the speed limit is thrown, a `Swoft\Limiter\Exception\RateLImiterException` exception will be thrown. The key expression has two variables, `CLASS` (class name) and `METHOD` (method name), which are convenient for developers to use.

annotation

The current limiter annotation is very simple and only involves a @RateLimiter annotation.

@RateLimiter

Marking method, start the current limiting policy, detailed parameters are as follows

  • Name cache prefix
  • Rate How much request access is allowed, number of requests/sec
  • Key current limit key, support symfony/expression-language expression, key expression built-in CLASS (class name) and METHOD (method name) two variables, easy for developers to use.
  • Max maximum number of requests
  • Default initialization request number
  • Fallback downgrade function, same as breaker
/docs/2.x/en/ms/govern/limiter.html
progress-bar