Find element by XPath in Python

Question

I would like to extract some odds from the website:

https://www.oddsportal.com/soccer/europe/champions-league-2015-2016/real-madrid-manchester-city-dhJZn1pn/#1X2;2

For example, the home odds for 1xbet is 1.5, and I copy the XPath expression for this odds and insert it into find_element_by_Xpath.

Here is my little code:

> from selenium import webdriver from bs4 import BeautifulSoup
>
> driver = webdriver.Firefox() url =
> 'https://www.oddsportal.com/soccer/europe/champions-league-2015-2016/real-madrid-manchester-city-dhJZn1pn/#1X2;2'
> driver.get(url) print(driver.title)
>
> element =
> driver.find_element_by_xpath("/html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[7]/div[1]/table/tbody/tr[2]/td[1]/div/a[2]")
>
> print(element.text)
>
> driver.close()

Unfortunately the answer is an error:

hao@hao-ThinkPad-T420:~$ ./testodds.py Real Madrid - Manchester City Betting Odds, Soccer - Champions League 2015/2016 Traceback (most recent call last): File "./testodds.py", line 12, in element = driver.find_element_by_xpath("/html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[7]/div[1]/table/tbody/tr[2]/td[1]/div/a[2]") File "/home/hao/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "/home/hao/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element 'value': value})['value'] File "/home/hao/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/home/hao/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: /html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[7]/div[1]/table/tbody/tr[2]/td[1]/div/a[2]

Answer 1

Try this XPath expression:

.//table[@class='table-main detail-odds sortable']/tbody/tr/td[2]/div

Answer 2

Your XPath expression is not correct.

Your XPath expression is this: "/html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[7]/div[1]/table/tbody/tr[2]/td[1]/div/a[2]"

You want this: /html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[8]/div[1]/table/tbody/tr[1]/td[2]/div

Here is the full working code. I successfully scraped the odds with it.

from selenium import webdriver
from bs4 import BeautifulSoup
driver = webdriver.Firefox()
url ='https://www.oddsportal.com/soccer/europe/champions-league-2015-2016/real-madrid-manchester-city-dhJZn1pn/#1X2;2'
driver.get(url)
print(driver.title)
element = driver.find_element_by_xpath("/html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[8]/div[1]/table/tbody/tr[1]/td[2]/div")
print(element.text)
driver.close()

• ahmedaao: Another question :
• ahmedaao: When I use the inspector to copy the xpath on the odd 1.5 It gives me :
• ahmedaao: /html/body/div[1]/div/div[2]/div[6]/div[1]/div/div[1]/div[2]/div[1]/div[7]/div[1]/table/tbody/tr[2]/td[1]/div/ the miskate come from div[7] why ?
• Matthew Gaiser: @ahmedaao I am not quite sure which element that is, but they often throw in fake elements to deter scrapers like you. Companies are generally not happy that people do this.

Answer 3

Try this XPath expression.

//div[@id="odds-data-table"]//td//a[contains(text(), "1xBet")]//ancestor::tr//td[2]

It's better to write your own XPath expression rather than copying from the browser as the browser gives a very tightly constructed XPath expression based on the element position relative to the nearest element with id or from your root. If any change occurs at any place, your XPath expression will not work.