Python Selenium. Scraping web page

Question

I want to get the data from the box inside 'Stock Style - Weight' from the url 'https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000NF9P&tab=3' using Selenium

This data is in an iframe. I ́m able to switch to the iframe and click the button = 'Weight' but i can ́t get the nine figures

Below is my code

driver = webdriver.Chrome(chromedriver)
driver.get("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000NF9P&tab=3")
iframe = WebDriverWait(driver, 10).until(
 EC.presence_of_element_located((By.XPATH, "//iframe[@id='portfolio']")))
driver.switch_to.frame(iframe)
element1=driver.find_element_by_xpath('/html/body/div/sal-components-pillar-cards-process/div/div[2]/div/div[2]/div[2]')
element2=element1.find_element_by_css_selector("input[type='radio'][value='Weight']").click()

I ́ve tried several options

driver.find_element_by_xpath('*//div/div[2]/div/div[2]/div/svg/g/g[3]/g[2]/g[1]/text')

driver.find_element_by_css_selector("mbc-chart-group> g.style-box-text-layer > g:nth-child(1)")

but i get the same error

NoSuchElementException: no such element: Unable to locate element

• Ghost Ops: print out the page source of the webpage using selenium and check whether your xpath/class name matches to yours or not

Answer 1

The Elements are in svg and text tags. To access the same you need to use:

//*[local-name()='svg'] or //*[name()='svg']

Link to refer

Xpath for those number would be:

//div[@class='sal-stock-style__weight']//*[name()='svg' and @role='chart']//*[name()='g' and @class='style-box-text-layer']//*[name()='text']

Try like below and confirm:

numbers = driver.find_elements_by_xpath("//div[@class='sal-stock-style__weight']//*[name()='svg' and @role='chart']//*[name()='g' and @class='style-box-text-layer']//*[name()='text']")
for num in numbers:
 print(num.text)

15
6
4
22
14
2
19
13
2

Answer 2

You need to add these two lines to click accept cookies button and investor type button

WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='onetrust-accept-btn-handler']"))).click()
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='btn_professional']"))).click()

Full Code

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000NF9P&tab=3")
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='onetrust-accept-btn-handler']"))).click()
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='btn_professional']"))).click()
iframe = WebDriverWait(driver, 10).until(
 EC.presence_of_element_located((By.XPATH, "//iframe[@id='portfolio']")))
driver.switch_to.frame(iframe)
element1=driver.find_element_by_xpath('/html/body/div/sal-components-pillar-cards-process/div/div[2]/div/div[2]/div[2]')
element2=element1.find_element_by_css_selector("input[type='radio'][value='Weight']").click()

• jccigh: This solves the way i access to the web. I was doing it manually before. But my main problem is how to get the data once I enter to the iframe and the Weight option