homepage

I've written a piece of code to POST request to a web service. 
===========================================================================
import json
import urllib
from urllib import request
from urllib import parse
def Payload(start_date, end_date, pnode_list):
 payload = {"startDate": start_date, 
 "endDate": end_date,
 "pnodelist": pnode_list}
 return json.dumps(payload)
def DownloadData(url, payload, header):
 data = []
 request = urllib.request.Request(url, payload, header)
 try:
 response = urllib.request.urlopen(request)
 except urllib.error.URLError as e:
 print("URLError occured.")
 except urllib.error.HTTPError as e:
 print("HTTPError occured.")
 else:
 #response.chunked = False #if this line is commented, ValueError will be thrown...
 data = json.loads(response.read().decode("utf-8"))
 return data
def main():
 url = "https://dataminer.pjm.com/dataminer/rest/public/api/markets/dayahead/lmp/daily"
 payload = Payload("2015-07-01", "2015-07-01", [135389795])
 header = {"Content-Type": "application/json"}
 data = DownloadData(url, payload.encode("utf-8"), header)
 print(data)
if __name__ == "__main__":
 main()
===========================================================================
However, "ValueError:invalid literal for int() with base 16: b'[\r\n'" is thrown when the HTTPResponse.read() is invoked:
Traceback (most recent call last):
 File "C:\Python34\lib\http\client.py", line 587, in _readall_chunked
 chunk_left = self._read_next_chunk_size()
 File "C:\Python34\lib\http\client.py", line 559, in _read_next_chunk_si
 return int(line, 16)
ValueError: invalid literal for int() with base 16: b'[\r\n'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "textpjm.py", line 34, in <module>
 main()
 File "textpjm.py", line 30, in main
 data = DownloadData(url, payload.encode("utf-8"), header)
 File "textpjm.py", line 23, in DownloadData
 data = json.loads(response.read().decode("utf-8"))
 File "C:\Python34\lib\http\client.py", line 506, in read
 return self._readall_chunked()
 File "C:\Python34\lib\http\client.py", line 591, in _readall_chunked
 raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(0 bytes read)
I've found a solution to avoid this exception: before HTTPResponse.read() is called, I have to set HTTPResponse.chunked to be False!
I wonder if there's something wrong in HTTPResponse.

It looks like this is a fault in the server. It does not appear to support Connection: close over HTTP 1.1, which is what Python’s "urllib.request" module uses. I think you will either have to get the server fixed, or work around the problem by using the low level "http.client" module, where you can avoid sending a "Connection: close" header field.
However see also Issue 12849, where various other people (including myself at one stage) have found this "Connection: close" flag triggers various server problems.
HTTP responses from the server with and without the "close" flag:
## HTTP 1.1 without Connection: close → good chunked response ##
POST /dataminer/rest/public/api/markets/dayahead/lmp/daily HTTP/1.1
Host: dataminer.pjm.com
Content-Type: application/json
Content-Length: 80
{"startDate": "2015-07-01", "endDate": "2015-07-01", "pnodelist": [135389795]}
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-UA-Compatible: IE=edge
Cache-Control: no-cache
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
X-Request-UUID: 09513fea-ac35-41ea-bf1b-b69357f42594
Transfer-Encoding: chunked
Server: Microsoft-IIS/7.5
Date: 2015年9月09日 04:12:34 GMT
Pragma: no-cache
Content-Type: application/json
Transfer-Encoding: chunked
Set-Cookie: dataminer=1923355820.36895.0000; path=/
1a44
[
 {
 "publishDate": "2015-07-01T04:00:00Z",
[. . .]
## HTTP 1.1 with Connection: close → invalid chunked response ##
POST /dataminer/rest/public/api/markets/dayahead/lmp/daily HTTP/1.1
Host: dataminer.pjm.com
Connection: close
Content-Type: application/json
Content-Length: 80
{"startDate": "2015-07-01", "endDate": "2015-07-01", "pnodelist": [135389795]}
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-UA-Compatible: IE=edge
Cache-Control: no-cache
Connection: close
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
X-Request-UUID: 0b43546c-d484-47a1-aff3-384f62a94a5a
Transfer-Encoding: chunked
Server: Microsoft-IIS/7.5
Date: 2015年9月09日 04:13:00 GMT
Pragma: no-cache
Content-Type: application/json
Connection: close
[
 {
 "publishDate": "2015-07-01T04:00:00Z",
[. . .]

FYI maybe your workaround of clearing HTTPResponse.chunked will work at the moment, but it would break if the server ever gets fixed and starts sending proper chunked responses.