A release passes only when the evidence follows one subscription end to end
A Web Push notification testing checklist must cover more than a successful send button. Record the browser and operating system, permission transition, worker version, subscription identity, provider response, push event, visible notification, click destination, and cleanup result.
- Create a clean state
- Exercise the requested flow
- Capture each boundary
- Reverse or expire the state
Define the release matrix before opening a browser
Choose representative combinations from actual supported traffic rather than claiming every browser from one desktop test. At minimum, include the current major desktop engines you support, Android behavior, and the installed Home Screen web-app path on supported iPhone or iPad devices. Add private browsing, managed devices, and older versions only when the product promises them.
| Record | Why it matters | Evidence |
|---|---|---|
| Browser, version, OS, and install state | Support and permission rules differ by environment. | Environment string plus a screenshot or screen recording when the interface matters. |
| Origin and worker scope | A valid worker at the wrong scope can create confusing partial success. | Registration scope, active script URL, and worker build identifier. |
| Permission and subscription state | Granted permission does not prove an active subscription. | Permission value and a redacted fingerprint of the current endpoint record. |
| Server attempt and provider response | Acceptance is not display. | Internal attempt ID, status, timestamp, and bounded response metadata. |
| Worker event and click result | The visible notification and destination are application responsibilities. | Event log, displayed fields, action selected, and final URL. |
Start from four controlled states
First visit with no permission decision
Confirm that the site explains a concrete benefit and waits for an affirmative user action. Declining the explanation must leave the site usable. If the native prompt is dismissed, the interface should not pretend that permission is denied or immediately ask again.
Permission granted with no stored subscription
This state can follow storage loss, failed registration, or server reconciliation. The interface should offer repair, create one current subscription, and avoid duplicate active records.
Active subscription with an older worker
Install a previous worker build, then deploy the candidate. Verify installation, waiting, activation, and event handling without dropping existing fetch handlers or breaking pages controlled by the worker. Use the service-worker implementation guide to check lifecycle boundaries.
Permission denied or revoked
Verify that site code stops calling the permission flow and presents quiet status help only when requested. A later server send should eventually retire an invalid endpoint under the normal lifecycle policy.
Run the happy path with traceable identifiers
- Clear only the test origin's site data and document the clean starting state.
- Trigger the explanation from the intended product action, then grant native permission.
- Confirm one subscription reaches the correct registered origin and account.
- Send a uniquely labeled test message to that test record, not a production audience.
- Match the server attempt to the push-service response and the worker's push event.
- Check title, body, icon, actions, direction, and truncation on the real device.
- Select the body and each supported action; confirm every destination is public HTTPS and fulfills the text.
- Unsubscribe, pause, or revoke permission and confirm that later eligibility changes accordingly.
Do not log full endpoints, authentication secrets, private VAPID keys, or authorization headers to make the trace easier. A stable internal record ID and a one-way endpoint fingerprint are enough for most release evidence.
Exercise offline, expiry, and replacement behavior
Take the test device offline, send a message with a short but nonzero TTL, and reconnect inside the useful window. Repeat after the window to confirm the product does not claim a guaranteed display. Test TTL zero separately. If the application uses Topic replacement, queue two updates with the same replacement key and verify that only the latest still represents valid state.
The TTL, urgency, and Topic policy explains how to choose the controls. The test should verify the chosen policy, not invent a deadline after observing the result.
Inject failures at each system boundary
- Serve an invalid application server key in a test environment and confirm subscription creation fails visibly.
- Use a stale endpoint fixture and confirm a permanent service response retires it.
- Simulate a transient provider response and verify bounded backoff without duplicate notification intent.
- Send a payload near the library's supported limit and reject oversize content before it enters a production queue.
- Return malformed notification data to the worker and confirm a safe fallback or explicit handled error.
- Use a blocked or mismatched destination and confirm validation stops the campaign.
Record expected and actual outcomes. A checklist with only check marks cannot reveal whether the tester knew what should happen.
Verify accessibility and subscriber control
Use concise text that remains meaningful when an icon or image is absent. Test keyboard access to the site-controlled permission explanation and preference center. Confirm visible focus, clear button labels, sufficient contrast, and a non-notification route to the same important information. Device notification surfaces are browser and operating-system UI, so test real output rather than relying only on a web preview.
Store a release record that can be repeated
Include a final check against the PushNotifications.org product boundary so the release record does not claim native-app delivery, guaranteed display, or an integration the service does not provide.
- Build
- Application version, worker version, server revision, environment, and test date.
- Matrix
- Every device/browser combination attempted, including unsupported results that match the published support boundary.
- Cases
- Starting state, steps, expected result, actual result, evidence link, and reviewer.
- Exceptions
- Known limitation, affected environments, user-facing fallback, owner, and review date.
If a failure appears only after release, follow the layered Web Push diagnostic and add the reproduced case to this matrix before shipping the correction.
Primary technical references
The W3C Push API specification defines subscription and worker behavior. MDN's Web Push best practices covers permission and opt-out expectations, while WebKit's iOS and iPadOS guidance documents the installed web-app path.