Classification of code challenges

Question

We've got a bunch of code-challenge problems. Furthermore, a lot of them have a bunch of similarities in their scoring systems. I've taken the first 50 of them (excluding off topic), and classified them:

• Accuracy based: You need to estimate the correct answer based on your input. Your score is the percentage of times you are correct.
• Smallest output: We score you based on the size of your output
• Fewest steps: You have a limited set of possible operations. Each operation counts as 1 point, lowest score wins.
• Image accuracy: Generate an image, we'll score it based on how close it is to the original
• Modified code-golf: We're scoring the size of your code, but tweaking it a bit.
• Other: These scoring systems are pretty unique and hard to classify

Therefore:

1. Does this seem like good information to tag?

2. Do my categories seem reasonable?

3. What should the tags be named, and what should their descriptions be?

4. Which of these tags (if any) should be a formal scoring system?

5. Should we retroactively go back and retag? (There are currently 559 of them)

• FryAmTheEggman: If this does happen, surely we don't need different tags for 1 and 4? Shouldn't it just be 1 with the various appropriate image tags used for other challenges?
• Martin Ender: When a scoring method has been used a lot and can be standardised easily (meaning there aren't half a dozen free parameters that would still need specifying in every single challenge of that genre) I'm definitely in favour of creating new challenge types and retagging applicable old challenges. code-challenge is a catch-all for scoring methods that don't match any existing challenge type, but necessarily some new challenge genres will always crystallise over time and it's good to go back and see if we can improve our categorisation every now and then.
• Nathan Merrill: @FryAmTheEggman I personally think they are different, but that's up to the community.
• Dave: Yay! I'm the first-linked question :D

Answer 1

I propose a new tag, fewest-operations, to be used as a scoring tag. This means it shouldn't be used with code-golf or fastest-code.

The OP needs to define what operation(s) are going to be scored. If there are multiple scored operations, then the OP can optionally define a different weight to each of the steps, otherwise it's assumed to be 1.

A submission's score is calculated by running all of the test cases and counting the total number of operations executed (multiplied by the weight, if applicable).

I prefer fewest-operations because I feel that it is clearer than fewest-steps. I also considered atomic-fastest-code, but it's a mouthful and not as immediately understandable (in my opinion).

I also propose that we should retroactively re-tag posts that use this scoring mechanism. There are some golf-cpu fastest-code challenges. I believe that they also lie under fewest-operations.

This answer is not intended to be all-inclusive of classification changes to be made, and is only addressing the "Fewest Steps" bullet point.

• Rohan Jhunjhunwala: Works, but could be dangerous for newbies who poorly pick the defined operations. +1

Answer 2

Here are my suggestions for tags:

• Accuracy-based: test-battery (presumably there would be a set of test cases to test against)
• Output size: optimized-output (yes, this tag exists)
• Fewest steps: I don't actually have a good suggestion for this one yet
• Image accuracy: image-estimation (note that this is the only new tag)
• Modified code golf: code-challenge (these don't have much of a common theme, so it's not worth making a separate tag)

• Martin Ender: I think "fewest steps" is "basically the same" as atomic-code-golf in the same way that fastest-code is "basically the same" as code-golf (i.e. I don't think they're remotely the same). It's actually closer to fastest-algorithm, but that one is scored by complexity class, so I think it's still different.
• Nathan Merrill: I don't much like wording of optimized-output, and it would certainly need a better description if we go with it. As far as I can tell, you aren't proposing these as a scoring tag, correct?
• user45941: @MartinEnder Fewest steps means using only a specified set of operations, and solving the problem in as few operations as possible. It's essentially restricted source + atomic code golf.
• trichoplax is on Codidact now: Having a look at the example links for "fewest steps", it appears they are mostly about minimising the number of steps in a task, rather than atomic-code-golf or restricted-source. Minimising the number of pixels checked before finding the target colour, for example, or minimising the number of moves made to solve a maze.

Answer 3

I propose two very similar tags, largest-output and smallest-output, to be used as scoring tags

These tags count the number of chunks (bytes, by default) in the final output. The person with the most/fewest chunks wins!

The term "chunk" here is intentionally broad: the OP can define what he wants "chunks" to be (as long as the chunks can be objectively counted).

Notes:

1. largest-output includes output-based busy-beaver challenges, but also includes challenges like Uniquely Separated Pixels, which scores by the number of pixels you can include in an image.
2. smallest-output includes all of the items listed under "Smallest output" above, as well as some of the "fewest steps" (I miscategorized when posting the question)
3. There is a important distinction to be made: fewest-operations is about performing a task in as few steps as possible (at runtime), while smallest-output is about calculating the fewest steps needed. A fewest-operations would be trying to get through a maze as fast as possible (with limited vision), while a smallest-output would be to calculate the shortest path through the maze.
4. I considered splitting these tags into byte/atomic based versions. However, in many challenges the fewest atoms is also the fewest bytes, and the line gets hard to define.
5. I considered using most-output/least-output, but I voted against them as the terms seem more broad to me.
6. meta-golf is a variety of smallest-output, so I'd argue to remove meta-golf as a scoring tag (keep it on the questions though), and add a smallest-output tag.

• trichoplax is on Codidact now: I like the idea of trying to tag more specifically, but it seems difficult to find a meaningful tag name that won't cause ambiguity. I know there's the tag wiki for that, but it would be great to have a catchy tag name that everyone agrees on the meaning of. All that comes to mind is "output-scored", which still doesn't seem to capture it.