Find and replace text based on feature class attribute

Question

I've been struggling with this problem for a few days. It seems to be more programmatic than geospatial in nature, so I'm open to suggestions to redirect this elsewhere.

I have a text file representing a project file for another program. I'm trying to search that text file based on a feature class (or shapefile) with an ID field (ATTR_1) with elevation information (ElevMeters) and replace default terrain heights with heights pulled from the feature. I can't really search for the ID info alone because it appears numerous times throughout the project file. From my perspective, I have to search through the project to find lines with $TERRAIN_HEIGHT :, look three lines above to match $PHOTO_NUM with the equivalent ATTR_1, and then replace the $TERRAIN_HEIGHT value with ElevMeters.

I've gotten my script to property assign and print correct ElevMeters values for each ATTR_1 found. I've been unable to get it to replace the $TERRAIN_HEIGHT : line properly. It prints all new values at the end of the document rather than the line I thought I was directing it to.

import os, arcpy
pdir = r"input directory path"
infile = r"input project file path"
infc = r"input feature class"
fields = ['ATTR_1', 'ElevMeters']
fpath = os.path.join(pdir, infile)
f = open(fpath, 'r+')
searchlines = f.readlines()
for i, line in enumerate(searchlines):
 if "$TERRAIN_HEIGHT : " in line:
 print i #prints line number
 for l in searchlines[i-3:i-2:2]: #go up three lines, skip every other character
 print l #prints photo_num
 print line #prints original terrain_height
 with arcpy.da.SearchCursor(infc, fields) as cursor:
 for row in cursor:
 if str(row[0]) in l:
 newline = str(line)[:20] + str(row[1])
 f.write(line.replace(line, newline))
f.close()

To prove that I'm grabbing values correctly, when I run this code:

for i, line in enumerate(searchlines):
if "$TERRAIN_HEIGHT : " in line:
 print i,
 for l in searchlines[i-3:i-2:2]:
 print l,
 print line,
 with arcpy.da.SearchCursor(infc, fields) as cursor:
 for row in cursor:
 if str(row[0]) in l:
 newline = str(line)[:20] + str(row[1])
 print newline

I get an output that looks something like this, with line number, photo number, a default $TERRAIN_HEIGHT : of 650.000000 and correct new values underneath:

...
4153 $PHOTO_NUM : 60858
 $TERRAIN_HEIGHT : 650.000000
 $TERRAIN_HEIGHT : 446.301
4165 $PHOTO_NUM : 60859
 $TERRAIN_HEIGHT : 650.000000
 $TERRAIN_HEIGHT : 447.882
4181 $PHOTO_NUM : 60860
 $TERRAIN_HEIGHT : 650.000000
 $TERRAIN_HEIGHT : 448.934
4197 $PHOTO_NUM : 60861
 $TERRAIN_HEIGHT : 650.000000
 $TERRAIN_HEIGHT : 441.156
...

• phloem: I don't believe 'f' knows which line it is meant to replace, so it appends to the end of the file. You need to loop through lines individually, as outlined here: stackoverflow.com/questions/17140886/…
• Emil Brundage: Will another ID ever occur in the three lines between an ID and $TERRAIN_HEIGHT :?

Answer 1

This code will do the trick I think. It looks for any line with $PHOTO_NUM, and when it finds one it checks three lines below with the use of the linecache module. If $TERRAIN_HEIGHT : is found three lines down, the script performs a cursor to find the replacement value, and calculates the index of the line to do the replacement in. Once this index is reached the replacement is made.

I haven't tested this out on data, so there may be a coding mistake or two.

Code:

import os, arcpy, linecache
pdir = r"input directory path"
infile = r"input project file path"
infc = r"input feature class"
fields = ['ATTR_1', 'ElevMeters']
newTxtfile = r"{New\text\file\full\path}"
nf = open(newTxtfile, "w")
di = {}
fpath = os.path.join(pdir, infile)
f = open(fpath, 'r')
searchlines = f.readlines()
cur_di = {}
with arcpy.da.SearchCursor(infc, fields) as cursor:
 for row in cursor:
 cur_di[row[0]] = row[1]
for i, line in enumerate(searchlines):
 if "$PHOTO_NUM" in line: #Look for all lines with a photo number
 thirdline = linecache.getline(fpath, i + 4) #Check third line down
 if "$TERRAIN_HEIGHT :" in thirdline: #Check if terrain height is three lines down
 for key in cur_di:
 if key in line:
 di[thirdline] = cur_di[key] 
 if i in di: #if third line down from a match
 newline = str(line)[:20] + di[i] #replace value with new
 nf.write(newline) 
 else: #else if not third line down
 nf.write(line) #write line
f.close()
nf.close()

I hope this helps!

Edit: Whoops, I forgot that you had to write everything to a new text file. Code updated. You'll have to delete the old and rename the new if you don't want the second, updated text file.

Edit2: Updated the code to work regardless of whether or not another $PHOTO_NUM occurs in the three lines between a match $PHOTO_NUM and a $TERRAIN_HEIGHT, with use of a dictionary.

• Emil Brundage: Note that you can increase the efficiency of this code with use of another dictionary. The second dictionary would have keys of your IDs and values of your elevations. Populate this dictionary with an initial cursor, and then you could skip all the nested cursors later in the code.
• Wes: I made some minor adjustments (replaced textfile with fpath in line 19 and added () to f.close() and nf.close(). I also made open(fpath, 'r') rather than 'r+' since it's a new text file. Works great. Thanks for the help
• Emil Brundage: Cool. I've updated my code with your fixes, as well as pulled out the searchcursor and made use of a second dictionary to speed things up. You might want to check that out. Hopefully I didn't break as much as I fixed.