Talk:Toolhub: Difference between revisions

I am hoping to find some technical contributors who are interested in learning how they can use Toolhub's API to register more tools in the catalog, build an alternative user interface, or help figure out how to embed data from Toolhub into their local wiki. Part of what I am interested in is finding out how to explain the API and it's usage to others, so even if you do not have a concrete idea of how you might use the API, but are interested in learning a bit about it I would be glad to hear from you. --BDavis (WMF) (talk) 22:51, 7 October 2021 (UTC) Reply

Some tools are duplicated, for example whois [1] [2] or bullseye [3] [4]. Is there any way to merge them? Or remove one of them? Thank you. MarioGom (talk) 22:00, 14 October 2021 (UTC) Reply

The bullseye tools are written by the same person, so probably the author can be contacted to see which one is the up to date one, and the other removed. The two whois tools are by different authors, and I suspect they're not really the same tool, since I'm pretty sure I've seen at least three or four different whois-type tools in active use. Unless the code is identical - in which case we have a challenge deciding who the "main" author is. Risker (talk) 22:21, 14 October 2021 (UTC) Reply

cc GeneralNotability for the bullseye choice. The whois tools I linked both have links to whois.toolforge.org, so I assumed they are the same. MarioGom (talk) 22:53, 14 October 2021 (UTC) Reply

By the way, the linked whois tools are also the same author. It's just full name and nickname, see [5]. MarioGom (talk) 22:54, 14 October 2021 (UTC) Reply

These are indeed the same tool. The whois-gateway version should probably be considered more canonical as it is actually maintained with the source code of the tool itself.

• The https://toolhub.wikimedia.org/tools/whois-gateway record is imported from https://whois.toolforge.org/toolinfo.json which was registered with Hay's Directory.
• The https://toolhub.wikimedia.org/tools/toolforge-whois record is imported from the data at https://toolsadmin.wikimedia.org/tools/id/whois via the https://toolsadmin.wikimedia.org/tools/toolinfo/v1/toolinfo.json feed of tools hosted in Toolforge.

This general problem is likely to show up for a number of other tools which are hosted on Toolforge and have created toolinfo data using https://toolsadmin.wikimedia.org/ that were also manually registered with Hay's Directory. You can actually see the same duplication in Hay's Directory for these tools: https://hay.toolforge.org/directory/#/search/whois.

-- BDavis (WMF) (talk) 23:39, 14 October 2021 (UTC) Reply

BDavis (WMF), both bullseyes are mine - as with whois, one's an import, and the other was me filling out a Toolhub entry manually since I didn't know there was an automatic import. I expect I won't be the first tool author to do this, so how do we want to handle de-duplicating? GeneralNotability (talk) 00:28, 15 October 2021 (UTC) Reply

Same question for me, with WPCleaner. --NicoV (talk) 06:24, 15 October 2021 (UTC) Reply

I have created phab:T293518 to track the need for some mechanism to deal with duplicate records. I believe this is going to need some thought from SKim (WMF) and others to resolve. There are some technical things to figure out here, but the bigger picture need is thinking about workflows for reporting and resolving duplicates as well as the inevitable need to undo that resolution when it is done incorrectly. --BDavis (WMF) (talk) 20:10, 15 October 2021 (UTC) Reply

Is there any way to list multiple authors? It doesn't look like author can take an array. Sam Wilson 02:24, 15 October 2021 (UTC) Reply

The hack I used for this in toolsadmin is populating the author string with a comma separated list of names. It's not beautiful, but it is functional today. I have a couple ideas for future schema changes here:

1. Make author optionally an array of strings. This can be done in a backwards compatible way so that either a single string or an array of strings is accepted by the JSON Schema. We would canonicalize the input to always be an array of strings in storage and API responses.
2. Create a more detailed "person" object that can be used to give more than just a name string so that you don't have to choose between a SUL username, a Developer account username, or a 'real' name when providing data and you don't have to guess which it is when consuming that data.

A phabricator task to document the need would be a great start toward figuring out the next step solution.

-- BDavis (WMF) (talk) 15:53, 15 October 2021 (UTC) Reply

Thanks! Good ideas. I've created a task. I think a simple string is probably easiest, and I wonder if it'd be possible to match the behaviour of the author field in extension.json. But anyway, multiple is probably more important than that. Sam Wilson 01:57, 17 October 2021 (UTC) Reply

Hi! I noticed the footer on the mobile version of the page that contains some licensing information about structured data appears to be sticky. Is there a reason for this? It just seems like it's unnecessary adding a licensing footer that takes up 1/6 of my screen at all times. Otherwise it looks really great! I'm excited to see where this goes 😀 — Yours, Berrely • ^Talk∕_Contribs 06:18, 15 October 2021 (UTC) Reply

Also, does the language name need to take up 2/3 of the header on mobile? Surely it would make more sense to have just the icon. — Yours, Berrely • ^Talk∕_Contribs 06:21, 15 October 2021 (UTC) Reply

There is a lot of old data and general cleanup needed for many of the tools I maintain. I guess I should delete all the toolinfo.json's and re-create the tools one by one in the UI, or is it possible to fix these things in the UI now? What would take for me to get the requisite rights to do this? I plan only on touching mine or Community Tech tools. Thanks! — MusikAnimal ^talk 06:38, 15 October 2021 (UTC) Reply

The current ownership model for toolinfo records is:

• Records imported by crawling a toolinfo.json URL can only be changed by changing the external data. We made this choice to remove the difficult problem of 3-way merging changes made via the API and changes made to the crawled source material.
• Each record created directly via the API (and UI since it is just an API client) is owned by the user who created it and can only be edited by that user, Administrators, and Oversighers. Admins and Oversighters are really only expected to edit records created by others to remove or suppress "problematic" content.

The ideal current fix would be to update your toolinfo.json files to include better information. In the longer term I expect community discussions like this one to help us refine the permissions model and workflows to better suit the real world data and usage that we see. -- BDavis (WMF) (talk) 16:08, 15 October 2021 (UTC) Reply

Thanks for the info! I did correct some of the toolinfo.json data but they aren't using the new schema yet. I see that the new schema is still not "final" and subject to breaking changes? That's why I thought maybe doing it through the UI would be more future-proof. In particular I'd like to get images up.

I also wanted to fix the Edit Counter entry, which is currently featured. I see that's coming from wikitech:User:Magnus Manske/hay directory. If that gets removed, will the Coolest Tool Award list automatically pick up the other tool named xtools-ec that is defined at https://xtools.toolforge.org/toolinfo.json? Same for xtools-articleinfo. If the answer is yes, maybe I could get you update the hay directory page, as I see your an int-admin on Wikitech? — MusikAnimal ^talk 20:46, 15 October 2021 (UTC) Reply

I just updated Toolhub/Data_model#Version_1.2.0 to describe this as the stable 1.2.0 schema. The only difference was removing the -draft02 discriminator. It is stable to use and functionally the same schema used by the API and UI to create toolinfo records. The differences are really just that the API dropped support for legacy single value fields in properties which now accept lists as values.

Your intuition that the duplicate tool names on wikitech:User:Magnus Manske/hay directory were keeping the records in https://xtools.toolforge.org/toolinfo.json from being loaded was correct. I have removed both the xtools-ec and xtools-articleinfo records there. The next crawler run should notice that removal, delete the current records, and then find the new records from your toolinfo.json source and add them. (I also changed the content type back to wikitext on that page so that normal wikitech users can edit it as originally intended.) -- BDavis (WMF) (talk) 22:45, 15 October 2021 (UTC) Reply

Thanks for launching Toolhub! As a fanatic tool user, I am pumped about it. I noticed that OpenRefine was not in Toolhub yet, so I manually created its entry, which went really smoothly. However, I should not be the single person maintaining this entry at all - ideally, IMO it would be editable by trusted people in the OpenRefine team at large, and I wouldn't mind if it were editable by anyone (the wiki way). But there's no edit button on a tool entry page (which may be a conscious design decision?). Since we are resourceful people, we did figure out, through the page history, that manual entries are actually editable. Just saying that I personally wouldn't mind having an actual edit button on a tool's entry page in general. And perhaps it's nice to document somewhere how tool entries can actually be edited/updated? Thanks for all the good work! Cheers, Spinster (talk) 07:54, 18 October 2021 (UTC) Reply

#Multiple authors ^^^ ??? @Spinster: — billinghurst sDrewth 13:16, 18 October 2021 (UTC) Reply

I had the impression that that particular topic is about the field for tool authors (i.e. developers/maintainers) in json entries for tools - not about making a manually created tool entry on Toolhub editable by multiple people. Did I misunderstand? Spinster (talk) 15:37, 18 October 2021 (UTC) Reply

I am trying to create a list with some my tools. I have tried to add just a single tool and I get "Oops! An error occurred in tools: Unknown tool: Scholia". — Finn Årup Nielsen (fnielsen) (talk) 20:27, 18 October 2021 (UTC) Reply

I am trying to edit one of my tools. It is unclear where this should/could be done. I only see "Add or remove tools". — Finn Årup Nielsen (fnielsen) (talk) 20:31, 18 October 2021 (UTC) Reply