pub trait BaseRateLimiter: Send + Sync {
// Required methods
fn acquire(&self, blocking: bool) -> Result<bool>;
fn aacquire<'life0, 'async_trait>(
&'life0 self,
blocking: bool,
) -> Pin<Box<dyn Future<Output = Result<bool>> + Send + 'async_trait>>
where Self: 'async_trait,
'life0: 'async_trait;
}Expand description
Base trait for all rate limiters.
Usage of the base limiter is through the acquire and aacquire methods depending on whether running in a sync or async context.
Implementations are free to add a timeout parameter to their initialize method to allow users to specify a timeout for acquiring the necessary tokens when using a blocking call.
Current limitations:
- Rate limiting information is not surfaced in tracing or callbacks. This means that the total time it takes to invoke a chat model will encompass both the time spent waiting for tokens and the time spent making the request.
Required Methods§
Sourcefn acquire(&self, blocking: bool) -> Result<bool>
fn acquire(&self, blocking: bool) -> Result<bool>
Attempt to acquire the necessary tokens for the rate limiter.
This method blocks until the required tokens are available if blocking
is set to true.
If blocking is set to false, the method will immediately return the result
of the attempt to acquire the tokens.
§Arguments
blocking- If true, the method will block until the tokens are available. If false, the method will return immediately with the result of the attempt. Defaults to true.
§Returns
True if the tokens were successfully acquired, false otherwise.
Sourcefn aacquire<'life0, 'async_trait>(
&'life0 self,
blocking: bool,
) -> Pin<Box<dyn Future<Output = Result<bool>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn aacquire<'life0, 'async_trait>(
&'life0 self,
blocking: bool,
) -> Pin<Box<dyn Future<Output = Result<bool>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Attempt to acquire the necessary tokens for the rate limiter. Async version.
This method blocks until the required tokens are available if blocking
is set to true.
If blocking is set to false, the method will return immediately with the result
of the attempt to acquire the tokens.
§Arguments
blocking- If true, the method will block until the tokens are available. If false, the method will return immediately with the result of the attempt. Defaults to true.
§Returns
True if the tokens were successfully acquired, false otherwise.