SCROLL/EditCheckHeadless
Appearance
From Wikitech
< SCROLL
π₯ SCROLL stage: requested. This SCROLL has been requested and is being scoped. The Prologue (scope, owners, target dates) is not yet complete.
Service Checklist for Readiness, Operations, Launch and Lifecycle
|
| ||||||||||||||||||||||||||
Status: π’ Done Β· π΄ Not Done Β· N/A
Priority: π Required for soft launch Β· π― Required for full launch Β· β Needs scoping / may not be applicable
Required for: βοΈ Service Β· π§© Extension Β· π» Core Feature Β· β¨ Feature
Service Summary
| # | Status | Priority | Required for | Item | Phab | Remarks | Assignee | Signed off by | Date |
|---|---|---|---|---|---|---|---|---|---|
| Documentation | |||||||||
| 1.1 | π’ | π | βοΈπ§©π»β¨ | Do you have a Wikitech page? | T434129 | A Wikitech page is the home for this component's documentation, Most of the items in this list are required to be present in the components Wikitech page. | Dev Team | ||
| 1.2 | β | π | βοΈπ§©π»β¨ | Is your service or component present on the Service Catalogue? | T000000 | The Service Catalogue is the canonical inventory of WMF services. The component should have an entry there. | Dev Team | ||
| Contacting The Team | |||||||||
| 1.3 | π’ | π | βοΈπ§©π»β¨ | Are the team's Phabricator tags and contact details documented on the Wikitech page and up to date in officewiki, so the right people can be found promptly when needed? | T434134 | Wikitech page should include Phabricator tags of the project and team, so bug reports are routed to the right people. Moreover, it should be clear in the wikitech page how to contact the team, or ensure there is a link pointing to the team's page. Lastly, it would be great if we are certain the team's office page is up to date. | Dev Team | ||
| External Reviews | |||||||||
| 1.4 | β | π | βοΈπ§©π»β¨ | Has the design been reviewed by SRE? | T000000 | Reach out to SRE early, both to agree target dates (soft launch, full launch) and to walk through the checklist together so you can confirm which items are relevant and which can be skipped. | SRE & Dev Team | ||
| 1.5 | β | πβ | βοΈπ§©π»β¨ | Has the design been reviewed by Security? | T432708 | The Security team is aware of this work and has communicated their requirements (if applicable). | |||
| 1.6 | β | πβ | βοΈπ§©π»β¨ | Has the design been reviewed by Data Persistence? | T414140 | Data Persistence is aware of this work and has communicated their requirements (if applicable). | |||
Operating Procedures
| # | Status | Priority | Required for | Item | Phab | Remarks | Assignee | Signed off by | Date |
|---|---|---|---|---|---|---|---|---|---|
| Interactions | |||||||||
| 2.1 | β | πβ | βοΈπ§©π»β¨ | Have you described how the service/feature interacts with common mediawiki userflows on wikitech? | T000000 | Engineers should be able to read the Wikitech page and understand where this component sits on the critical path and assess the impact when something goes wrong. | Dev Team | ||
| Infrastructure | |||||||||
| 2.2 | N/A | πβ | βοΈ | If your service runs on baremetal, are the server prefixes documented on wikitech? | T000000 | If the service runs on baremetal or VMs, document the server prefixes on Wikitech. | SRE | ||
| 2.3 | β | πβ | βοΈ | If your service runs on Kubernetes, has it been added to deployment charts (helm charts) and has the relevant kubernetes service defined? | T434109 | A Kubernetes deployment needs a Helm chart in deployment-charts, and a Kubernetes Service if it accepts traffic from outside the cluster. | SRE & Dev Team | ||
| 2.4 | β | πβ | βοΈ | If you have a Service URL? should it be present in service.yaml on Puppet? | T000000 | A standalone service accepting traffic must exist in Puppet's service.yaml; document the URL where the service can be reached. | SRE | ||
| 2.5 | N/A | πβ | βοΈ | Does your service have a staging environment? if yes, is it documented on Wikitech? | T000000 | If the service has a staging environment, document its URL on Wikitech. | SRE | ||
| Service Traffic | |||||||||
| 2.6 | π’ | π | βοΈ | Do you have an estimation of the traffic your service will be serving? | T433024 | Teams should be able to work out an estimation of the traffic they expect, and the methodology used. If that is not straightforward, reach out to SRE and we can work through it together. | Dev Team | ||
| 2.7 | N/A | π | βοΈ | Does your service accept traffic directly from the CDN? | T000000 | If the service has public endpoints, SRE Traffic may need to provide additional configuration for routing and caching. | SRE & Dev Team | ||
| 2.8 | β | π | βοΈ | If your service is in discovery, can the service be depooled safely and run from a single DC, any special consideration for DC Switchover? | T000000 | An active/active or active/passive service needs a discovery URL; confirm it can be depooled to a single datacentre without user-visible impact. | SRE & Dev Team | ||
| 2.9 | β | π | βοΈ | Have you documented with which systems/datastores your service needs to communicate with? | T000000 | A clear list of dependencies helps with capacity planning as well as monitoring. | Dev Team | ||
| Maintenance | |||||||||
| 2.10 | β | π―β | βοΈπ§©π»β¨ | If you have dependencies on maintenance scripts (mw-script) or crons (mw-cron), have they been documented and recently tested? | T000000 | Maintenance scripts and crons often go untested for long periods. Documenting and testing them prevents surprises when they fail or need to be re-run. | Dev Team | ||
Release Confidence
| # | Status | Priority | Required for | Item | Phab | Remarks | Assignee | Signed off by | Date |
|---|---|---|---|---|---|---|---|---|---|
| Building and Testing | |||||||||
| 3.1 | β | π | βοΈπ§©π»β¨ | Does your wikitech page link to the code repository and production branch, and recent image (if applicable)? | T000000 | Link the code repository and the production branch from your Wikitech page, and document the name and location of the most recent container image. | Dev Team | ||
| 3.2 | β | π―β | βοΈπ§©π»β¨ | Do you have integration and unit tests (CI)? | T000000 | Unit and integration tests in CI give confidence that changes are safe to deploy. | Dev Team | ||
| Initial Rollout | |||||||||
| 3.3 | β | π | βοΈπ§©π»β¨ | Do you have a rollout plan (soft and full launch), including the common failure points you have identified for launch day? | T000000 | A documented rollout plan for soft and full launch, covering the deployment sequence, smoke tests, rollback, communication, and the common failure points identified for launch day (eg cold caches or an overwhelmed dependency). | SRE & Dev Team | ||
| Deploying to Production | |||||||||
| 3.4 | β | π | βοΈπ§©π»β¨ | Do you have Configuration & Deployment Owners? | T000000 | See Configuration & Deployment Owners for more details. Please ensure this is also documented on Wikitechready and authorised to deploy means you can roll out changes and fixes on your own schedule, rather than waiting for help. | Dev Team | ||
Observability
| # | Status | Priority | Required for | Item | Phab | Remarks | Assignee | Signed off by | Date |
|---|---|---|---|---|---|---|---|---|---|
| Metrics, Instrumentation, Logging | |||||||||
| 4.1 | β | π | βοΈπ§©π»β¨ | Are you exporting Prometheus metrics and sending logs to Logstash? | T000000 | Prometheus and Logstash are WMF's standard tools for metrics and logs. Exporting to both is the baseline for any observable service. | Dev Team | ||
| 4.2 | β | π | βοΈπ§©π»β¨ | Are key user flows and business metrics instrumented and exported? | T000000 | Instrumenting user-facing flows and business outcomes helps measure what matters to users and assess impact during incidents. | Dev Team | ||
| Service Level Objectives | |||||||||
| 4.3 | β | π―β | βοΈπ§©π»β¨ | Have SLOs been agreed on and published to assist in evaluating the impact on end users? | T000000 | SLOs should be agreed on and published so the team can evaluate user-facing impact and reason about the error budget. | Dev Team | ||
| 4.4 | β | π―β | βοΈπ§©π»β¨ | Have the SLO dashboard and alerts been configured in Sloth? | T000000 | Configure the SLO dashboard and alerts in Sloth. | Dev Team | ||
| Monitoring | |||||||||
| 4.5 | β | ππ― | βοΈπ§©π»β¨ | Are your dashboards (Grafana, Logstash) documented on Wikitech page, and include links to related dashboards and runbooks? | T000000 | Document your Grafana and Logstash dashboards on the Wikitech page. The dashboards themselves should have cross-links to related dashboards, dependencies and other information. | Dev Team | ||
| 4.6 | β | π―β | βοΈπ§©π»β¨ | Do you monitor latency variations at the p50, p75, and p99 percentiles (eg via envoy, or other business metrics), as well as the status of external dependencies? | T000000 | Monitor latency at p50/p75/p99 (eg via Envoy or business metrics) and the status of any external dependencies. | Dev Team | ||
| Alerting | |||||||||
| 4.7 | β | ππ― | βοΈπ§©π»β¨ | Are your alerts documented on Wikitech and committed to Alertmanager: paging alerts, alerts for excessive errors (business, infrastructure, budget burn), and linked to runbooks? | T000000 | Document your alerts on Wikitech and commit them to Alertmanager: paging alerts, alerts for excessive errors (business, infrastructure, budget burn), each linked to a runbook. | SRE & Dev Team | ||
| Incident Response | |||||||||
| 4.8 | β | π | βοΈπ§©π»β¨ | Does your component have and Incident Response owner and Runbooks? | T000000 | See Incident Response Owner for more details and ensure this is documented on wikitech. A runbook documents common operational procedures and the responses to each alert. Check Category:Runbooks | Dev Team | ||
Reliability and performance
| # | Status | Priority | Required for | Item | Phab | Remarks | Assignee | Signed off by | Date |
|---|---|---|---|---|---|---|---|---|---|
| Resources | |||||||||
| 5.1 | β | π― | βοΈ | Do you have an estimation of the resources you will need? | T000000 | Estimated CPU, memory, and storage requirements drive capacity planning. Linked phab task with discussion. | Dev Team | ||
| 5.2 | β | π― | βοΈ | Is the service designed to scale up or down as needed? | T000000 | SRE should be able to add or remove resources on demand without contacting the team. | Dev Team | ||
| Reliability | |||||||||
| 5.3 | β | π | βοΈ | Does each component have its own health or liveness check to ensure production traffic does not reach an unhealthy endpoint? | T000000 | Liveness and readiness checks should be in place for routing and alerting purposes. | Dev Team | ||
| 5.4 | β | π | βοΈ | Have you identified your system SPOFs? | T000000 | Single Points of Failure are components whose loss takes down the service. Identifying them is the first step to mitigating or accepting the risk. Should be documented on wikitech | Dev Team | ||
| 5.5 | β | π― | βοΈπ§©π»β¨ | Does the component protect itself from being overwhelmed by incoming traffic (e.g. rate limiting, load shedding)? If yes, have they been documented? | T000000 | Protect the service from being overwhelmed by incoming traffic (eg rate limiting, load shedding), and document these protections. | Dev Team | ||
| 5.6 | β | π― | βοΈπ§©π»β¨ | Does the component degrade gracefully when its dependencies fail (e.g. timeouts, backoff/retry, fallback/failover, cached or default responses)? | T000000 | Well-defined timeout, retry, and fallback behaviour keeps the component working when its dependencies misbehave. | Dev Team | ||
| 5.7 | β | π― | βοΈπ§©π»β¨ | Is the Bus Factor for this service or feature at least 2? | T000000 | At least two people should understand the service well enough to ensure its operation and longevity. | Dev Team | ||