Benchmark a VPS without cherry-picking the fastest run
A useful VPS benchmark answers a defined workload question and gives another person enough environment detail, commands, raw output, and timing information to repeat the test. One unexplained score cannot represent every application or future performance on shared infrastructure.
Key facts
- Minimum publication unit
- Question, environment, command, duration, repetitions, date, and raw output
- Comparison rule
- Use the same VPS shape, image, tool version, and test window
- Honest summary
- Keep slow and failed runs; report a median and variability
- Safety boundary
- Bound resource use and respect provider and network limits
Begin with a workload question
Decide what the result should help you choose. A CPU compilation task, a latency-sensitive API, a transactional database, a sequential backup, and a multi-user file service stress different resources. Select a test whose access pattern resembles the workload and define the success threshold before seeing results. This reduces the temptation to promote whichever number looks largest.
Specify the VPS plan and region, test duration, concurrency, data size, read-write mix, block size, protocol, remote endpoint, and time window. Shared-host performance and internet routes can vary, so a benchmark is a dated sample under stated conditions, not a permanent guarantee.
- Write the decision and threshold before starting.
- Test an application-level transaction when possible.
- Avoid combining unrelated scores into one opaque rating.
Record enough context to reproduce the run
Capture the plan, region, provision time, operating-system image, kernel, CPU presentation, memory, swap, storage devices, filesystem, mount options, IP family, and benchmark versions. Useful inventory commands include lscpu, free -h, lsblk, and uname -r. Remove customer secrets and unique identifiers before publishing the record.
Note whether the server was newly provisioned, warmed up, recently resized, or carrying another workload. Record the start time in a clear timezone and any provider maintenance. For network tests, identify both endpoints, their networks, direction, protocol, and relevant rate limits. A city-to-city label alone is not reproducible.
- Pin tool versions or save package metadata.
- Keep the same operating-system state across comparisons.
- Publish material exceptions instead of silently rerunning them.
Run bounded CPU, storage, and network tests
For CPU, choose a tool such as sysbench with a stated thread count and duration. For storage, fio can model sequential or random access, queue depth, block size, direct I/O, file size, and read-write mix; save machine-readable output with --output-format=json. Ensure the test file is large enough for the question and explain cache effects rather than pretending one mode is universally correct.
For network capacity, iperf3 requires a controlled remote endpoint and should be rate- and time-bounded. Test both directions where relevant. For delay, ping can sample round-trip time and loss, but identify source, target, interval, count, address family, and path conditions. Intensive tests can affect neighbors or trigger limits, so obtain permission and stop if service health degrades.
- Use a dedicated test VPS with no customer workload.
- Repeat the same sequence at least five times for published comparisons.
- Preserve command output, errors, and exit status for every run.
Summarize variability instead of hiding it
Inspect every run for errors, throttling, memory pressure, full storage, background activity, and endpoint saturation. Do not delete a slow result merely because it is inconvenient. If a run is invalid, retain it, explain the exclusion rule, and rerun the full planned sequence rather than selecting a replacement value opportunistically.
Report the median for repeated central behavior and show a range or percentile that exposes variation. Storage summaries should include latency as well as throughput or operations per second. Network summaries should include loss and endpoint limitations. Compare only results produced with materially equivalent configurations and state when they are not equivalent.
- Separate invalid runs from valid but slow runs.
- Show units and aggregation methods beside every value.
- Avoid percentage claims based on incomparable environments.
Publish the method, raw data, and limits together
A credible evidence page links a versioned method, environment manifest, exact commands, individual raw files, summary table, test date, and responsible reviewer. Use stable file names and checksums when practical. Explain that results describe the tested sample and may change with hardware allocation, contention, routing, software, or provider policy.
Schedule a repeat after material infrastructure or image changes and retain prior results with their original dates. Corrections should remain visible. VPSEverywhere.com's transparency template stays excluded from indexing while it contains example values; only real measurements that pass the stated verification process should be presented as operational evidence.
- Publish failed and slow runs in the raw record.
- Give each methodology revision a date and version.
- Never backfill measurements that were not actually taken.
Sources
Frequently asked questions
Which VPS benchmark is the best?
No single benchmark is best. Choose tests that resemble the workload and publish their configuration. Application-level measurements usually answer buying questions better than one synthetic score.
How many benchmark runs should I publish?
This method uses at least five identical runs for a comparison and keeps every valid result. More samples may be necessary when variability is high or the decision is important.
Why publish median instead of the fastest result?
The fastest run rewards cherry-picking. A median describes the center of repeated observations, while a range or percentile shows variability. Neither guarantees future performance.
Can I run `fio` or `iperf3` on a production server?
Avoid disruptive tests on customer workloads. Use a dedicated environment, bound duration and rate, follow provider rules, and verify that the remote endpoint is authorized and not the bottleneck.