How to scrape text from html using Selenium or Beautiful Soup

Question

here is the html

<div data-v-026c45df="" class="col text-undefined">682ドル.0</div>

I need the '682ドル.0'. I have tried :

start_aum = driver.find_element_by_class_name('col text-undefined').get_attribute
print(start_aum)

and

start_aum = driver.find_element_by_css_selector('col text-undefined').get_attribute
print(start_aum)

both return

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".col text-undefined"}
 (Session info: chrome=88.0.4324.190)

I am open to using beautiful soup or selenium but I can't seem to understand why I am having problems selecting the element.

Answer 1

There might some issue with the element visibility etc., but I can't see the whole code, so I'll react to only what's in your question:

• get_attribute is a method, you are not calling it, you'd need something like get_attribute("class")
• 682ドル.0 is not an attribute, it's text, you need to get it with .text, e.g. driver.find_element_by_css_selector('.col.text-undefined').text
• in find_element_by_css_selector('col text-undefined') there's an incorrect selector, the correct one would be .col.text-undefined

• MeaColley: to be able to know if it is an issue with element visibility do you need to see the html, or my script?
• pavelsaman: I'd fix these things first, then study a bit along these topics stackoverflow.com/a/59130336/10401931 and then possibly change the script further.
• MeaColley: After doing some research I am trying to utilize this line of code from the link you attached for me. check_box = WebDriverWait(Browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="tableview-3687-record-2128"]/tbody/tr/td[1]/div/div'))) check_box.click() but I am getting the error that 'By' is not defined. I found the dependencies for the other bit in that line of code but I'm not sure what I need in order to use the 'By' part
• pavelsaman: Does this answer your question? selenium-python.readthedocs.io/locating-elements.html from selenium.webdriver.common.by import By You also have one extra pair of brackets there, although Python should not complain in this situation.
• ernie: also worth noting that find_element_by_class_name() can't take multiple classes: stackoverflow.com/questions/37771604/…

Answer 2

Your locator is wrong, use

 start_aum = driver.find_element_by_css_selector('[class="col text-undefined"]').Text
 print(start_aum)

Answer 3

driver.find_element_by_css_selector(".col.text-undefined").text

• corsiKa: What would be the difference between this and pavel's answer?
• IAmMilinPatel: Please add some explanation to the answer that might be helpful.
• vitaliis: Pavels answer is correct, no difference