SMS Latency Test: Measure OTP and Alert Delivery

Measure the wait that matters to your application

An SMS latency test should measure how long a message takes to reach a defined step. Start by naming that step. A quick API response proves that a request received a quick response. It does not prove that an OTP reached a phone quickly, or that an account alert was seen.

For OTPs, measure controlled handset receipt separately from completed verification. A person may wait before entering a code, so verification completion includes more than message transport. For alerts, decide whether the measurement concerns receipt on a controlled device or the time the application learns a reported delivery state.

• Record API response time and handset receipt as separate measurements.
• Keep delivery event timestamps separate from the time a callback reaches your application.
• Report failures, missing receipts, and the number of observations alongside latency.
• Compare providers under equivalent conditions instead of ranking them from unrelated averages.

Keep each timestamp tied to the event it describes

Use timestamps the provider actually exposes. Do not reconstruct a missing carrier timestamp from the time a webhook arrived. Provider records and carrier receipts vary. Keep the available evidence and its definition with the measurement rather than silently treating unlike events as equivalent.

Use a controlled sample that represents your destinations

Send only to numbers you control or have authorization to contact. Choose devices and networks that represent the countries and carriers relevant to your application. Record whether a device is roaming or temporarily offline. Keep the sender type, message purpose, and message segment count comparable when comparing two routes or providers.

Use a harmless unique reference to match each observed message to its send record. Record the request start, response time, provider message reference, available delivery events, callback receipt, and handset observation. Do not collect real authentication secrets or expose complete recipient numbers in a shared results sheet.

Synchronize the clocks used by the sending system and the observation method. Record the timezone, timestamp precision, and any known clock uncertainty. Manual observation has human reaction time, while automated observation has its own collection delay. Document the method so another person can interpret a difference of a few seconds honestly.

Repeat the measurement during ordinary traffic and busier periods relevant to your service. Keep send volume within provider limits and your approved budget. Start with a small controlled sample, then expand it if the results need more confidence. A few successful sends cannot establish performance across an entire country or carrier.

Report slow, failed, and missing messages together

State the sample size, measurement window, destination groups, and observation deadline before interpreting the result. For messages observed within that window, report the median, also called p50, and a tail percentile such as p95. The median describes the middle observation. The p95 describes the point at or below which roughly 95% of measured observations fall, according to the calculation method used.

Show the number of observed messages beside those percentiles. A small sample cannot support a strong claim about rare slow deliveries. Keep failed requests, reported delivery failures, unobserved handset receipts, and missing delivery reports in separate counts. Never assign a missing message zero latency or remove it from the overall success calculation.

A hypothetical report might show 100 attempted sends, 90 handset receipts within the chosen window, four explicit delivery failures, and six messages not observed by the deadline. A latency percentile calculated from the 90 received messages describes those 90 receipts. It does not mean that 95 of all 100 attempted sends arrived within that time.

Keep retries visible as additional attempts attached to the same original verification or alert. Otherwise, a report may count a successful replacement while hiding the failed first send and the user's extra wait. Separate measurements by country, carrier, sender type, and test window before drawing conclusions from a combined average.

A late callback does not establish a late message

Consider a hypothetical timeline. The API responds after 0.2 seconds, the message appears on a controlled phone after four seconds, and a delivery callback reaches the application after eight seconds. Those observations describe three different waits. The API was responsive, the observed handset wait was four seconds, and the application learned the reported state later. These numbers illustrate the method and are not provider benchmarks.

If the callback is slow but controlled handset receipt is prompt, inspect the reporting path before changing the sending route. If handset receipt is slow, compare the available queue, carrier, destination, and device evidence. If receipt is missing entirely, report an incomplete observation rather than guessing when delivery happened.

For an OTP, compare observed receipt with the application's code expiry policy. An SMS arriving after expiry is unusable even if a later receipt says delivered. Keep the timeout, resend, and recovery decisions in the existing verification process. A delivery receipt must not extend a code's validity or prove successful verification.

Common questions about SMS latency measurements