Bash script to check available name in Github

Question

I just wrote this script to check if the name (that has 2 characters: 1 alphabet and 1 number) in Github is available or not so I can accompany this name.

Here is my script:

#!/bin/bash
for i in {a..z}
do
 echo 'running ...'
 for j in {0..9}
 do
 out=$(curl -s -o /dev/null -w "%{http_code}" "https://github.com/$i$j") &
 if [[ $out =~ .*4.* ]]
 then
 echo "$i$j is available"
 else
 echo "$i$j is not available"
 fi
 done
 echo 'finish batch ...'
 wait
done

Let me explain:

• I use 2 loops to generate all permutation of 2 characters.

• I use curl to check status of the the host. For example:

curl -s -o /dev/null -w "%{http_code}" "https://github.com/google"

it will return the status of the host, in case the username exist it will return 200 else it will return 404

• Then I compare the status code with any 4xx error:

$out =~ .*4.*

• If the status is 4xx then the user name is available so I can get it.

• Currently, it is running 10 background jobs at the same time by using curl with &.

As you can see I have found j3 and m6, n5, u6, y4, y6, y9 are available.

enter image description here

Could you please help me to review this script.

Answer 1

The code is nice and short, and it is easy to read.

I'm surprised that it works at all. Sending the out=$(...) command to the background sounds as if the code might continue before curl had the chance to fill the output. Do you get the same results without the &? I'd expect that more names are reported as available then.

As long as the & is in the program, I would verify the results manually.

I also don't see the point of running the requests in parallel. It's only 10 * 26 requests, which are answered anyway within 5 minutes.

---

Update: The Shell Command Language specification says:

If a command is terminated by the control operator ( '&' ), the shell shall execute the command asynchronously in a subshell.

This means that the variable assignment cannot affect the out variable at all. This in turn means that the output you posted does not correspond to the code you posted. Please check again.

If you can reproduce the behavior, please write down all details that might influence it. I tried to reproduce it, and I got exactly the result I expect after reading the specification: all names are not available.

• Mast: Sending them in parallel may not be much of an improvement now, but scripts like this tend to get reused for larger batches as well. If the next time there'll be 1000 requests and it doesn't hurt to add parallelisation now, why not? If there were only 10 requests being made, sure, it would be overkill. But now, sure, got to start somewhere.
• chicks: Scalability is great, but it doesn't look like the & is actually doing anything here or the code would break.