Error getting text attributes from shapefile with python / pyshp

Question

I have a shapefile, containing only points in XY-coordinates and a string attribute for each point. I would like to read the coordinates and the attribute for postprocessing in python, but can't get it to work.

I have tried the following.

import shapefile
r = shapefile.Reader("subregion_59117")
print r.fields
shapes = r.shapes()
print len(shapes)
print shapes[0].points[0]

gives

[('DeletionFlag', 'C', 1, 0), ['LON', 'N', 16, 8], ['LAT', 'N', 16, 8],
['SUBREGION', 'C', 25, 47]]
59117
[176.0, 63.0]

So I can read the point coordinates, but

records = r.records()
sr = r.shapeRecords()
print len(records), len(sr)

gives just

0 0

meaning, these lists don't contain any elements. So I don't know how to get my hands on the string attributes of the points?

However, if I open the shapefile in QGIS, and save as geojson (a JSON textfile, thus human readable), it contains the features as a list of elements such as:

{ "type": "Feature", "id": 0, "properties": { "LON": 176.0, "LAT": 63.0, "SUBREGION": "Eastern Europe" }, "geometry": { "type": "Point", "coordinates": [ 176.0, 63.0 ] } }

So apparently my shapefile contains all the information, I just don't know how to read it with pyshp.

(So yes, I could always convert my shapefile to geojson with QGIS, and postprocess that with python, but this feels overtly complicated.)

(削除) My shapefile is here. (削除ここまで)

Edit: My shapefile was generated by Saga GIS, and as explained by Tuure Laurinolli's answer, Saga GIS seems to have a bug so the attributes in my shapefile were marked with an invalid flag, and also pyshp seems to be very picky in this, interpreting everything but the correct flag as "deleted" and refusing to read it.

Converting the broken shapefile to shapefile by e.g.

 ogr2ogr file_new.shp file_broken.shp

fixes the problem.

Answer 1

The DBF records in that Shapefile are apparently invalid. Their deleted flag values are 0x00 whereas the value should be either 0x20 (valid) or 0x2A (deleted) as per http://www.clicketyclick.dk/databases/xbase/format/dbf.html#DBF_STRUCT. Pyshp interprets everything except 0x20 as deleted.

Answer 2

You might try the Python package called Fiona. GeoJSON is its native format.