If speedtest fails, assume network speed of 100 Mbit/s (#404)

The value is chosen as some safe value below average at https://health.petals.dev/

Note that if a server uses relays, the effective throughput will be further divided by 2 (see #399).
pull/403/head^2
Alexander Borzunov 10 months ago committed by GitHub
parent 8666653cf5
commit 30b94ef18b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -138,7 +138,7 @@ def measure_throughput_info(
def measure_network_rps(
config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 25e6
config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 100e6 # 100 Mbit/s
) -> Optional[float]:
bits_per_request = config.hidden_size * 16 # Clients usually send 16-bit tensors for forward/backward
try:

Loading…
Cancel
Save