parsing log file and get time difference between two timestamp actions in python

Question

I am quite a beginner in python, so probably the taks I would like to perform is relative simple. I have log files with first column timestamp and second column describes the action, and I need to get the time difference between two actions. The log file called log.txt looks like:

2017年05月11日T12:22:12.760 End step: first action
2017年05月11日T12:22:13.724 Start step: other action
2017年05月11日T12:22:15.069 End step: other action
2017年05月11日T12:22:15.933 Start step: first action

I wrote a basic script that parses trough the directories, searches for specific words and calculate the time difference. However, the code is really basic and would like to improve it a little bit, for example by defining a config function that arises error and keeps running when the key words are missing. Any suggestion would be really appreciate. the code I use looks like:

 from datetime import datetime
 from datetime import timedelta
 import re
 import os
 import numpy as np
 inputDir = os.path.dirname(os.path.realpath(__file__))
 for subdir, dirs, files in os.walk(inputDir):
 for file in files:
 filepath = subdir + os.sep + file
 if 'log.txt' in filepath:
 filename=open(os.path.join(filepath))
 with open("OutputTime.csv","a") as outfile:
 for line in filename:
 line = line.rstrip()
 if re.search('Start step: first action', line):
 start_first_step=line[:23]
 start_step=datetime.strptime(start_first_step, "%Y-%m-%dT%H:%M:%S.%f")
 if re.search('End step: first action', line):
 end_first_step=line[:23]
 end_step=datetime.strptime(end_first_step, "%Y-%m-%dT%H:%M:%S.%f")
 minutes=end_step - start_step
 minutes=minutes.total_seconds()/60
 print minutes

• alecxe: Could you please fix the indentation in the script? Thanks.

Answer 1

In general, the indentation should be 4 spaces for each indentation level in Python. See PEP-0008 (the Python style guide). Loops and if statements should count as a level. The whole script could be shifted to the left, as well. Some of the variable names are a bit confusing because 'files' and 'file' can become easy to mix up as loop variables. One other thing of note is the spacing between things. I'd consider making fewer new lines between the bits of code. Some of the indentations seem like they should throw exceptions, so if you see IndentationErrors, that's what it's chirping about, I'd look at indenting after the blocks (for loops, and such).