wget says 406 Not acceptable

Question

I have a simple file on my web server, and when I request it in a browser, it loads without problems:

http://example.server/report.php

But when I request the file with wget from a Raspberry Pi, I get this:

$ wget -d --spider http://example.server/report.php
Setting --spider (spider) to 1
DEBUG output created by Wget 1.18 on linux-gnueabihf.
Reading HSTS entries from /home/pi/.wget-hsts
URI encoding = 'ANSI_X3.4-1968'
converted 'http://example.server/report.php' (ANSI_X3.4-1968) -> 'http://example.server/report.php' (UTF-8)
Converted file name 'report.php' (UTF-8) -> 'report.php' (ANSI_X3.4-1968)
Spider mode enabled. Check if remote file exists.
--2018年06月03日 07:29:29-- http://example.server/report.php
Resolving example.server (example.server)... 49.132.206.71
Caching example.server => 49.132.206.71
Connecting to example.server (example.server)|49.132.206.71|:80... connected.
Created socket 3.
Releasing 0x00832548 (new refcount 1).
---request begin---
HEAD /report.php HTTP/1.1
User-Agent: Wget/1.18 (linux-gnueabihf)
Accept: */*
Accept-Encoding: identity
Host: example.server
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 406 Not Acceptable
Date: 2018年6月15日 08:25:17 GMT
Server: Apache
Keep-Alive: timeout=3, max=200
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
---response end---
406 Not Acceptable
Registered socket 3 for persistent reuse.
URI content encoding = 'iso-8859-1'
Remote file does not exist -- broken link!!!

I read somewhere that it might be an encoding problem, so I tried

$ wget -d --spider --header="Accept-encoding: *" http://example.server/report.php

but that gives me the exact same error.

Answer 1

That's because the server you're connecting to serves only to certain User-Agents.

Change the user agent and it works fine:

wget -d --user-agent="Mozilla/5.0 (Windows NT x.y; rv:10.0) Gecko/20100101 Firefox/10.0" http://example.server/report.php

• OZ1SEJ: Thank you very much! This did the trick! Is that common knowledge, that a 406 Not acceptable has to do with the type of user agent? All I was able to dig up was that it had something to do with encoding...
• Jay Brunet: This worked, but my shorter user-agent did not. All I can say is, weird.
• darnir: This heavily depends on the server on the other end. Some webmasters use robots.txt files which allow only web browsers to go through.