How to parse a HTML file with table using Python

Question

I have got a html file with table ( its a large one, so only sample code is given ). I want to retrieve the values in tables. I tried the HTMLParser library from python.

I started coding like below. Then I found that the attribute "class" is same as system defined keyword. So its giving me error.

class MyHTMLParser(HTMLParser):
 def handle_starttag(self, tag, attrs):
 if tag == 'tr':
 for class in attrs:
 if class == 'Table_row'
p = MyHTMLParser()
p.feed(ht) 

HTML code for table

<table class="Table_rows" cellspacing="0" rules="all" border="1" id="MyDataGrid" style="width:700px;border-collapse:collapse;">
 <tr class="Table_Heading">
 <td>STATION CODE</td><td>STATION NAME</td><td>SCHEDULED ARRIVAL</td><td>SCHEDULED DEPARTURE</td><td>ACTUAL/ EXPECTED ARRIVAL</td><td>ACTUAL/ EXPECTED DEPARTURE</td>
 </tr><tr class="Table_row">
 <td>TVC </td><td style="width:160px;">ORIGON</td><td>Starting Station </td><td>05:00, 07 May 2011</td><td>Starting Station</td><td>05:00, 07 May 2011</td>
 </tr><tr class="alternat_table_row">
 <td>TVP </td><td>NEY YORK</td><td>05:04, 07 May 2011</td><td>05:05, 07 May 2011</td><td>05:04, 07 May 2011</td><td>05:05, 07 May 2011</td>
</tr> 
</table>

UPDATE

How could I get data between the tags?

• schmijos: I wrote a small and simple HTML table parser not requiring any external module: github.com/schmijos/html-table-parser-python3/blob/master/…

Answer 1

Note that the documentation of the handle_starttag method states:

The tag argument is the name of the tag converted to lower case. The attrs argument is a list of (name, value) pairs containing the attributes found inside the tag’s <> brackets.

So, you're probably looking for something like:

from HTMLParser import HTMLParser
class MyHTMLParser(HTMLParser):
 def handle_starttag(self, tag, attrs):
 if tag == 'tr':
 for name, value in attrs:
 if name == 'class':
 print 'Found class', value
p = MyHTMLParser()
p.feed(ht) 

Prints:

Found class Table_Heading
Found class Table_row
Found class alternat_table_row

---

P.S. I also recommend BeautifulSoup for parsing HTML with Python.

• user567879: How to print the values like STATION CODE STATION NAME ORIGON ...?
• Eli Bendersky: @user567879: you can find td tags and process them
• user567879: Sorry for the dumb question. What I want is that I need to process ( couldnt find a method to print the value b/w tags) td tag between the table tag. How to handle that nesting?
• Eli Bendersky: @user567879: not sure what you're asking exactly... Still I recommend to take a look at BeautifulSoup - it provides a much higher-level API for HTML processing. HTMLParser is quite awkward to use compared to it
• user567879: I want to print the data in tr tag when class=Table_Heading or class=Table_row or class=alternate_table_row only. Can I use and cluause to make it work?

Answer 2

How to print the values like STATION CODE STATION NAME ORIGON ...?.

You can do it like this with BeautifulSoup.

from BeautifulSoup import BeautifulSoup
html = '''\
<td>STATION CODE</td><td>STATION NAME</td><td>SCHEDULED ARRIVAL</td><td>SCHEDULED DEPARTURE</td><td>ACTUAL/ EXPECTED ARRIVAL</td><td>ACTUAL/ EXPECTED DEPARTURE</td>
</tr><tr class="Table_row">
<td>TVC </td><td style="width:160px;">ORIGON</td><td>Starting Station </td><td>05:00, 07 May 2011</td><td>Starting Station</td><td>05:00, 07 May 2011</td>
'''
soup = BeautifulSoup(html)
tag = soup.findAll('td', limit=2)
tag_O = soup.findAll('td')[7]
for i in range(len(tag)):
 print tag[i].string
print tag_O.string
'''Output-->
STATION CODE
STATION NAME
ORIGON
'''

Answer 3

I would highly recommend using the BeautifulSoup library. It handles even broken HTML with ease.

http://www.crummy.com/software/BeautifulSoup/