"driver.find_element_by_css_selector" not working. How to fix this exception?

Question

I am extracting the data of the trending videos on YouTube. I am able to extract the data of the "first video" but on extracting data of the "second video" there is exception raising. How to fix it?

from selenium import webdriver
chromedriver="C:/Users/asus/Downloads/chromedriver_win32 (2)/chromedriver"
driver=webdriver.Chrome(chromedriver)
driver.get("https://www.youtube.com/feed/trending")
print("Trending 1: ")
title1=driver.find_element_by_css_selector('#video-title')
print(title1.text)
views1=driver.find_element_by_css_selector('#metadata-line > span:nth-child(1)')
print("Views: ",(views1.text))
up_t1=driver.find_element_by_css_selector('#metadata-line > span:nth-child(2)')
print("Time: ",(up_t1.text))
second_link=driver.find_element_by_css_selector('#grid-container > ytd-video-renderer:nth-child(2)')
second_link.click()
new_second_link=driver.current_url
print(new_second_link)
driver.get(new_second_link)
#error part
#title2=driver.find_element_by_xpath('#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer').text
title2=driver.find_element_by_css_selector('#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer').text
print(title2)

• Fenio: What exception?
• Kamal: You get exception with error 'not a valid XPath expression' because the CSS selector(#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer) is not a valid xpath.
• Parthav: @Fenio NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer"} but i have copied the correct selector
• Fenio: @Parthav Add HTML source with the element you want to locate
• Parthav: @Fenio Thank you for your response. I have got the solution of the problem.

Answer 1

The css selector you have used was correct.The problem it is throwing error due to loading the page and webdriver unable to find the element.

To overcome such problems it is always adviced that to use WebDriverWait and element_to_be_clickable.

Try the below code.It will return expected results.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
chromedriver="C:/Users/asus/Downloads/chromedriver_win32 (2)/chromedriver"
driver=webdriver.Chrome(chromedriver)
driver.get("https://www.youtube.com/feed/trending")
print("Trending 1: ")
title1=driver.find_element_by_css_selector('#video-title')
print(title1.text)
views1=driver.find_element_by_css_selector('#metadata-line > span:nth-child(1)')
print("Views: ",(views1.text))
up_t1=driver.find_element_by_css_selector('#metadata-line > span:nth-child(2)')
print("Time: ",(up_t1.text))
second_link=driver.find_element_by_css_selector('#grid-container > ytd-video-renderer:nth-child(2)')
second_link.click()
new_second_link=driver.current_url
print(new_second_link)
driver.get(new_second_link)
wait=WebDriverWait(driver,30)
title2=wait.until(expected_conditions.element_to_be_clickable((By.CSS_SELECTOR,'h1.ytd-video-primary-info-renderer yt-formatted-string'))) #driver.find_element_by_css_selector('h1.ytd-video-primary-info-renderer yt-formatted-string').text
print("Title : " + title2.text)
count=wait.until(expected_conditions.element_to_be_clickable((By.CSS_SELECTOR,'#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer')))
print("Views Count :" + count.text)

Output from console:

Trending 1: 
Film Theory: The Rat That Beat Thanos! (Marvel Endgame)
Views: 1.2M views
Time: 15 hours ago
https://www.youtube.com/watch?v=UkhYYwF7XaU
Title : Eurovision Song Contest 2019 - First Semi-Final - Live Stream
Views Count :1,290,319 views