I have a file full of Urls (about some 3000 urls) and the requirement is to get each url invoked in a browser tab and click on each page (just the click is enough on the loaded page), which redirects to a different page. This is all I need to do.
I have written a code for the urls to retrieve one by one and clink on the page (page is designed to click anywhere to land or redirect to a different page). However, while running the code an error is being thrown with "invalid argument". Any thoughts, how can I correct the error ?
Appreciate your support.
with open(dataFile, 'r') as urlFile:
urlFile.readline()
for url in urlFile:
driver = webdriver.Chrome('C:\\chromedriver_win32\\chromedriver.exe')
driver.get(url)
link = driver.find_element_by_xpath("/html/body")
link.click()
urlFile.close()
The above code should pull each url from the file on a new browser tab and click. Repeat the same until all the urls in the file are done.
-
2which line you are getting the error? and is there any blank lines in the text file. also, check the URL format and it should be starting link http:// or https://Murthi– Murthi2019年09月10日 05:15:25 +00:00Commented Sep 10, 2019 at 5:15
-
1@Murthi, you are correct. I have the /r (carriage return) in my code and I have it removed and it is working good. Thank you for pointing this out.user1911509– user19115092019年09月10日 05:38:13 +00:00Commented Sep 10, 2019 at 5:38
-
1@Murthi, I have another problem with this code. Let's say, I have n lines of Urls in the file - however, the code executes n-1 and leaves the last Urls from the file. Any suggestions, please ?user1911509– user19115092019年09月10日 05:42:17 +00:00Commented Sep 10, 2019 at 5:42
-
2remove the code urlFile.readline() in the loop(second line in your code). then it will execute all the lines.Murthi– Murthi2019年09月10日 08:47:14 +00:00Commented Sep 10, 2019 at 8:47
-
1Thank you very much. It works brilliantly.user1911509– user19115092019年09月10日 19:05:42 +00:00Commented Sep 10, 2019 at 19:05
1 Answer 1
I did something like this Code
from selenium import webdriver
import time
driver = webdriver.Chrome('driver_location')
f = open("dataFile.txt", 'r')
for line in f:
driver.get(line)
link = driver.find_element_by_css_selector("#header > div:nth-child(3) > div > div > div:nth-child(3) > div > a")
link.click()
print(driver.current_url)
f.close()
url in dataFile.txt file
http://automationpractice.com/index.php
http://automationpractice.com/index.php
http://automationpractice.com/index.php
output
http://automationpractice.com/index.php?controller=order
http://automationpractice.com/index.php?controller=order
http://automationpractice.com/index.php?controller=order
Hope this helps.