Talk:Developer Satisfaction Survey/2026

The graphic at Developer Satisfaction Survey/2026/Cloud Services#storage-caching misrepresents the options that were available: as can be seen at Developer Satisfaction Survey/2026/Questions (CLTBQ04), the first option was MySQL/MariaDB (not including ToolsDB or the wiki replicas) (emphasis added). I don’t understand why the option is phrased like that to begin with, when there is no separate ToolsDB/replicas option and I would expect that to be the most common response; but I find it even stranger that the result, already confounded by the available options, is then presented inaccurately. (My best guess is that a large part of the 29 "other" answers mean ToolsDB – at least that was my response. But we won’t know for sure until someone fixes this question, which has had this problem since 2021.) Lucas Werkmeister (talk) 11:58, 26 April 2026 (UTC) Reply

I'll add this to our planning doc for next round so we don't miss it. That is, add an option for toolsdb—would that fix this question for you?

"Other" is where responses with a low number of responses get lumped together. In this case, it captures the Postgres/Mongo/open answer folks.

FWIW, we still have anonymized answers in some raw form (i.e., we can see the answers for questions, but dissociated from demographics/row). There was only one toolsdb in there. Squinting hard at this, I think the biggest share of "Other" is something like: files of some format on disk. TCipriani (WMF) (talk) 17:39, 27 April 2026 (UTC) Reply

That one toolsdb mention is probably me. I’m really confused what’s going on, then... next year maybe just change the option label to "MySQL/MariaDB" and let that implicitly include ToolsDB and the replicas? Lucas Werkmeister (talk) 18:21, 28 April 2026 (UTC) Reply

Make sense to me. I'll flag for cloud folks for next year. TCipriani (WMF) (talk) 19:34, 28 April 2026 (UTC) Reply

The section Developer Satisfaction Survey/2026/Deployment#deployment-use-in-the-past-year-by-role seems to be empty? Lucas Werkmeister (talk) 12:09, 26 April 2026 (UTC) Reply

Whoops. Yes it was. I've now removed that section. We didn't upload that break out for two reasons:

1. n was low for most groups aside from MediaWiki core devs.

2. The shape of the data was the same as the overall. That is, the breakdown of use for all groups was the same as the breakdown of use overall—nothing interesting.

I've removed that section, sorry for missing that before upload :( Thanks for the note. TCipriani (WMF) (talk) 17:28, 27 April 2026 (UTC) Reply

I found myself mystified by these two entries from Developer Satisfaction Survey/2026/APIs#specific-api-use-frequency:

• Wikidata SPARQL Query Service (54% of respondents reported using very or somewhat frequently, up from 45% last year)
• Wiki Commons SPARQL Query Service (46% of respondents reported using very or somewhat frequently, up from 25% last year)

We can see the usage of both query services in Grafana: WDQS usually gets around 8000 requests per second; WCQS seems to average around one or two. That’s a difference of four orders of magnitude, so I wonder where all this frequent usage of WCQS is...

And I think part of the answer is that the presentation is mixing different response groups? In the chart we can see the respondent numbers: the Action API has N=71, WDQS has N=43, WCQS has N=26. So I think the summary should really be something like "46% of respondents who use WCQS at all reported using [it] very or somewhat frequently"? And in that case, I’m not sure that The APIs with the most frequency of use reported is a useful thing to list at all, to be honest. (Seeing the frequency of usage of the different APIs is useful, just not the pulling out of the three APIs with the highest percentages after those percentages have already been distorted by filtering out the "never" responses.) Lucas Werkmeister (talk) 12:38, 26 April 2026 (UTC) Reply

Since I left three criticism above, I thought I should also spend a moment to leave a positive comment and thank you for doing the survey :) interesting results as usual! Lucas Werkmeister (talk) 13:08, 26 April 2026 (UTC) Reply

Hey @TCipriani (WMF) et al., thanks for organizing the survey and putting together these results pages. In addition to those noted above this section, however, there are a number of potential issues that I've noticed with these results (and/or how some of the results are presented):

• Cloud Services:
  • #source-control: This section says that respondents could select more than one response, and that 198 people responded to this question; however, from what I can see, the numbers given alongside each answer in the graph only add up to 168.
  • #programming-languages: I'm confused by the first bullet-point here: it says that the majority (32%) said they use Python 3; however, the next bullet-point reports that 44% said they use Javascript (& IIUC, 32% would represent a minority of respondents). I'm speculatively wondering if the 32% figure may be the percentage of choices that were Python 3 (given that respondents could choose more than one language in answer to this question), rather than the percentage of respondents that reported using Python 3?
  • #cloudvps-tenure: From the numbers on the graph, it seems like the percentage of people that said between 4 and 7 years should maybe be $\frac{11+3}{48}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 29\mathrm{\%}$, rather than 27%.
  • #cloudvps-nfs-use: It seems like either the graph or the text may be slightly off here -- the text says [...] 29% said yes, and 13% were unsure, but the [numbers on the] graph suggest that it's the other way around (ie., that 29% were unsure & 13% said yes).
  • #why-wmcs: Ignoring possible minor rounding errors, some of the listed percentages don't appear to agree with the numbers on the graph:
    • Cost is listed as being said by 53% of respondents, but its number in the graph (& the reported number of respondents) suggest that it might've been said by $\frac{77}{126}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 61\mathrm{\%}$ of respondents;
    • Ease of collaborating with other Wikimedians is listed as being said by 48% of respondents, but the graph's number suggests $\frac{67}{126}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 53\mathrm{\%}$;
    • Privacy and security considerations is listed as being said by 37% of respondents, but the graph's number suggests $\frac{40}{126}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 32\mathrm{\%}$;
    • Ease of use is listed as being said by 27% of respondents, but the graph's number suggests $\frac{42}{126}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 33\mathrm{\%}$
    • Something else is listed as being said by 8% of respondents, but the graph's number suggests $\frac{7}{126}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 6\mathrm{\%}$.
    (It seems like some of the listed percentages may be from last year's published results.)
• Shared test environments#beta-auto-vs-manual-testing: Of the 44 who answered this question, 45 (80%) said they used beta cluster for manual testing of code -- from the number in the graph, I assume that 45 here is meant to say 35?
• Code review#gerrit-activity: [...] more than one-third (38%) said they had made 11-10 -- I assume that this is a typo for 11-100?
• Continuous integration#ci-queue-time-satisfaction: The text here says that 33% said they were dissatisfied (which is also stated in the 2026 entry for the 'satisfaction over time' graph); however, this doesn't appear to match up with the first graph included in this section, which (adding together the values for somewhat dissatisfied & very dissatisfied) reports that 31% of respondents said they were dissatisfied. (It seems like the 33% text may be from last year's published results .)
• Deployment#deployment-use-in-the-past-year: I'm confused by the text & the graph here -- they don't seem to agree with each other, FWICS:
  • The text reports that Of the 56 who opted in to the Deployment section of the survey and answered this question, about half (51%) said that did not use Wikimedia deployment train or backport windows in the past year. However, the numbers displayed within the graph indicate that only 6 respondents said that they did not use either, which seems like it'd be $\frac{6}{56}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 11\mathrm{\%}$, rather than 51%. (It seems like this text may be from last year's published results .)
  • The bullet-points in this section are preceded by the qualifier Of the respondents who said they did use Wikimedia deployment train or backport windows in the past year; however, from what I can tell based on the graph, the percentages listed in those bullet-points are those of all responses (rather than just of those that said they did use the deployment train and/or backport windows).
• MediaWiki Core backend#backend-component-satisfaction: Comparing the list of the top 10 backend components with the highest ratings to the graph, the list appears to be missing Deferred Updates & BagOStuff (which, from my calculations, both appear within the top 10).
• APIs#api-use-cases: A couple of things:
  • The first bullet-point here says that 54% reported using Wikimedia APIs to enhance on-wiki user experiences; however, from what I can see, the numbers in the graph seem to indicate that only 52% of respondents said that they very frequently or somewhat frequently used Wikimedia APIs to enable the use-case of enhancing on-wiki user experiences. (It seems like this text may be from last year's published results .)
  • Under Respondents’ least common use cases, one of the use-cases is listed as using Wikimedia APIs to automate tasks on a third party MediaWiki extension; however, in the graph, (what I believe is) the same use-case asked about is listed as automating tasks on a third party MediaWiki installation (both emphases mine).
• Technical documentation#reading-documentation-on-wikis: The third-bullet point here says that 3% said very infrequently; however, from what I can see, the graph reports that 4% of respondents selected very infrequently. (It seems like this text may be from last year's published results .)
• Overall: I was noticing numerous issues with the data presented on this page, and was getting through the process of typing all them out. And then, at some point, it (metaphorically) hit me: for what (AFAICS) is the entire page except for the new Building volunteer connections question & the Most important open-answer question, the results entered onto this page are identical to those on the initial version of Developer Satisfaction Survey/2025/Overall -- see this Special:ComparePages diff .
  IIUC, the initial version of that page was later revised due to some issues in how some of the data was presented. (So, in other words, much of this page appears to not just display last year's results, but - IIUC - a partially-inaccurate version of last year's results.)

I apologise if I have misunderstood anything here (and/or accidentally misquoted any of the published results). To be honest, though, it has felt at least slightly disheartening (if nothing else) that I've come across this many potential issues on reading through these published results (and, at this number of potential issues that I have found, it feels inevitable that there will be others that I will have missed). I don't want to be overly harsh, but I do feel at least slightly disappointed that these weren't caught before these documents were placed into publication (though I should be clear that this is addressed towards the WMF as an organization, rather than towards any individual person). Best, ‍—‍a smart kitten _[meow] 14:37, 27 April 2026 (UTC) Reply

#source-control: This section says that respondents could select more than one response, and that 198 people responded to this question; however, from what I can see, the numbers given alongside each answer in the graph only add up to 168.

Yes Done Special:Diff/8357531

#programming-languages: I'm confused by the first bullet-point here: it says that the majority (32%) said they use Python 3; however [...]

Yes Done Special:Diff/8357742

#cloudvps-tenure: From the numbers on the graph, it seems like the percentage of people that said between 4 and 7 years should maybe be $\frac{11+3}{48}<span\; class="naked\_sign">\times </span><span\; class="naked\_aural">ばつ</span>100\approx 29\mathrm{\%}$, rather than 27%.

Yes Done Special:Diff/8357532

#cloudvps-nfs-use: It seems like either the graph or the text may be slightly off here -- the text says [...] 29% said yes, and 13% were unsure, but the [numbers on the] graph suggest that it's the other way around (ie., that 29% were unsure & 13% said yes).

Yes Done Special:Diff/8357533

#why-wmcs: Ignoring possible minor rounding errors, some of the listed percentages don't appear to agree with the numbers on the graph [...]

Yes Done Special:Diff/8357769

Shared test environments#beta-auto-vs-manual-testing: Of the 44 who answered this question, 45 (80%) said they used beta cluster for manual testing of code -- from the number in the graph, I assume that 45 here is meant to say 35?

Yes Done Special:Diff/8357522

Code review#gerrit-activity: [...] more than one-third (38%) said they had made 11-10 -- I assume that this is a typo for 11-100?}}

Yes Done Special:Diff/8357525

Continuous integration#ci-queue-time-satisfaction: The text here says that 33% said they were dissatisfied [...]; however, this doesn't appear to match up with the first graph included in this section

Yes Done Special:Diff/8357590

Deployment#deployment-use-in-the-past-year: I'm confused by the text & the graph here -- they don't seem to agree with each other

Yes Done Special:Diff/8357529

MediaWiki Core backend#backend-component-satisfaction: Comparing the list of the top 10 backend components with the highest ratings to the graph, the list appears to be missing Deferred Updates & BagOStuff (which, from my calculations, both appear within the top 10).

discussing...

APIs#api-use-cases: The first bullet-point here says that 54% reported using Wikimedia APIs to enhance on-wiki user experiences; however [...] Under Respondents’ least common use cases, one of the use-cases is listed as using Wikimedia APIs to automate tasks on a third party MediaWiki extension; however, in the graph, (what I believe is) the same use-case asked about is listed as automating tasks on a third party MediaWiki installation (both emphases mine).

Yes Done Special:Diff/8357571

Technical documentation#reading-documentation-on-wikis: The third-bullet point here says that 3% said very infrequently; however, [...]

Yes Done Special:Diff/8357572

Overall [...] identical to those on the initial version of Developer Satisfaction Survey/2025/Overall -- see this Special:ComparePages diff

Yes Done Special:Diff/8357741

I don't want to be overly harsh, but I do feel at least slightly disappointed that these weren't caught before these documents were placed into publication (though I should be clear that this is addressed towards the WMF as an organization, rather than towards any individual person)

This year, I messed up. I kicked off the on-wiki survey draft by copy-pasting the 2025 version of the Developer Satisfaction Survey. Normally, we start from a blank page. This made it hard to spot when a section had been updated for publication vs. merely looked updated (since it was full of last year's information). Most of the issues you spotted were a direct result of that (modulo some small typos). This list of errata was accurate, but it was harsh to read. I feel bad since these errors were me trying to avoid translating a google doc into my complicated wiki templates. Mea culpa. TCipriani (WMF) (talk) 00:05, 28 April 2026 (UTC) Reply

Thank you for the updates to the pages @TCipriani (WMF) :) _/gen

I kicked off the on-wiki survey draft by copy-pasting the 2025 version of the Developer Satisfaction Survey. Normally, we start from a blank page. This made it hard to spot when a section had been updated for publication vs. merely looked updated (since it was full of last year's information). I don't want to unwarrantedly intrude in internal processes, but I naïvely wonder - if working up from a 'template on-wiki DSS' was/is helpful - whether it might be worth making a copy of DSS pages (to have as a starting-point for next year's draft), but replacing all of the numbers/graphs/answers/etc. in that copy with obvious placeholder values (I guess, so that it'd hopefully be more clear when something had[n't yet] been updated)?

This list of errata was accurate, but it was harsh to read. I feel bad since these errors were me trying to avoid translating a google doc into my complicated wiki templates. Mea culpa. To be clear, I don't blame you personally._/gen These things happen, & from what you've said I can easily understand how it happened on this occasion.

(FWIW, if you've got any feedback for me on how/if I could have presented these issues better [etc.], please genuinely feel free to let me know.) ‍—‍a smart kitten _[meow] 16:25, 28 April 2026 (UTC) Reply

I don't want to unwarrantedly intrude in internal processes, but I naïvely wonder - if working up from a 'template on-wiki DSS' was/is helpful - whether it might be worth making a copy of DSS pages (to have as a starting-point for next year's draft), but replacing all of the numbers/graphs/answers/etc. in that copy with obvious placeholder values (I guess, so that it'd hopefully be more clear when something had[n't yet] been updated)? Yes. Definitely what I'll do next year (or go back to the old method). In any event, not a mistake we'll make twice—all new mistakes is always my goal :)

Thank you for giving the results a deep read and flagging the issues you found, the results are much better for it. TCipriani (WMF) (talk) 19:32, 28 April 2026 (UTC) Reply