我試圖找出如何將影像 ID 添加到串列中并在下一次搜索中跳過它。到目前為止,這是我的代碼,我嘗試了很多......機器人應該總是將他最近復制的影像添加到“使用”黑名單中,并且下次不要復制它。
search = True
used = []
driver = webdriver.Chrome()
driver.get('https://9gag.com/funny')
time.sleep(2)
driver.find_element(By.XPATH,value='//*[@id="qc-cmp2-ui"]/div[2]/div/button[1]/span').click()
time.sleep(2)
while True:
while search:
post = driver.find_element(By.CSS_SELECTOR,value='.post-container a img')
if post.id in used:
search = True
else:
search = False
post_url = post.get_attribute('src')
post_title = post.get_attribute('alt')
used.append(post.id)
print(post_url)
print(post_title)
print('......')
print(used)
print(post.id)
time.sleep(20)
問題:他將使用過的影像添加到串列中,但他仍然找到并復制它......
https://img-9gag-fun.9cache.com/photo/aWgNx36_460s.jpg
Fat acceptance activist on the news was so fat they had to put her in landscape mode
......
['4d17ee3f-213a-4d54-a18f-425f8f4dea4b']
4d17ee3f-213a-4d54-a18f-425f8f4dea4b
https://img-9gag-fun.9cache.com/photo/aWgNx36_460s.jpg
Fat acceptance activist on the news was so fat they had to put her in landscape mode
......
['4d17ee3f-213a-4d54-a18f-425f8f4dea4b', '4d17ee3f-213a-4d54-a18f-425f8f4dea4b']
4d17ee3f-213a-4d54-a18f-425f8f4dea4b
https://img-9gag-fun.9cache.com/photo/aWgNx36_460s.jpg
Fat acceptance activist on the news was so fat they had to put her in landscape mode
......
['4d17ee3f-213a-4d54-a18f-425f8f4dea4b', '4d17ee3f-213a-4d54-a18f-425f8f4dea4b', '4d17ee3f-213a-4d54-a18f-425f8f4dea4b']
4d17ee3f-213a-4d54-a18f-425f8f4dea4b
編輯:代碼:
while True:
driver.switch_to.window(gag_tab)
post = driver.find_elements(By.CSS_SELECTOR,value='.post-container a img')
for post in post:
post_url = post.get_attribute('src')
post_title = post.get_attribute('alt')
#paste the the url and title in to another site
time.sleep(20)
錯誤:
Traceback (most recent call last):
File "main.py", line 86, in <module>
post_url = post.get_attribute('src')
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=101.0.4951.67)
Stacktrace:
Backtrace:
Ordinal0 [0x009CB8F3 2406643]
Ordinal0 [0x0095AF31 1945393]
Ordinal0 [0x0084C748 837448]
Ordinal0 [0x0084F154 848212]
Ordinal0 [0x0084F012 847890]
Ordinal0 [0x0084F98A 850314]
Ordinal0 [0x008A50C9 1200329]
Ordinal0 [0x0089427C 1131132]
Ordinal0 [0x008A4682 1197698]
Ordinal0 [0x00894096 1130646]
Ordinal0 [0x0086E636 976438]
Ordinal0 [0x0086F546 980294]
GetHandleVerifier [0x00C39612 2498066]
GetHandleVerifier [0x00C2C920 2445600]
GetHandleVerifier [0x00A64F2A 579370]
GetHandleVerifier [0x00A63D36 574774]
Ordinal0 [0x00961C0B 1973259]
Ordinal0 [0x00966688 1992328]
Ordinal0 [0x00966775 1992565]
Ordinal0 [0x0096F8D1 2029777]
BaseThreadInitThunk [0x75B9FA29 25]
RtlGetAppContainerNamedObjectPath [0x77C77A7E 286]
RtlGetAppContainerNamedObjectPath [0x77C77A4E 238]
uj5u.com熱心網友回復:
首先:您search = True在列印最后一篇文章后忘記放了,所以它總是會跳過回圈并列印出第一篇文章。但即便如此,你還沒有完成,因為driver.find_element()總是搜索與你的引數匹配的第一個元素,所以它會陷入無限回圈,因為第一個帖子在used串列中并且會設定search為True無限回圈。
嘗試driver.find_elements()改用。這將創建一個包含所有帖子的串列,因此您可以遍歷串列并像這樣列印每個帖子:
posts = driver.find_elements(by=By.CSS_SELECTOR, value='.post-container a img')
for post in posts:
post_url = post.get_attribute('src')
post_title = post.get_attribute('alt')
used.append(post.id)
print(post_url)
print(post_title)
print('......')
print(used)
print(post.id)
time.sleep(2)
編輯:
由于driver.find_elements()到目前為止只會收到加載在網站上的帖子,因此您需要在向下滾動頁面時再次呼叫它。這就是為什么我放入一個while回圈并忽略已經列印的帖子。關于StaleElementReferenceException我放一個try-except塊來忽略不再可參考的元素。當您向下滾動網站太快時,可能會發生這種情況。您像這樣匯入這些例外:
from selenium.common.exceptions import StaleElementReferenceException
from selenium.common.exceptions import WebDriverException
只要確保沒有命名沖突。
這是我目前的解決方案:
used = []
while True:
posts = driver.find_elements(by=By.CSS_SELECTOR, value='.post-container a img')
for post in posts:
if not post.id in used:
try:
post_url = post.get_attribute('src')
post_title = post.get_attribute('alt')
except StaleElementReferenceException or WebDriverException:
continue
used.append(post.id)
print(post_title)
print(post_url)
print('__________')
time.sleep(2)
您需要手動或自動向下滾動站點(Selenium 具有驅動程式功能execute_script(),您可以在其中逐步執行滾動命令)以加載更多可以列印的帖子。
uj5u.com熱心網友回復:
變數“post”沒有相對的背景關系(值以句點開頭)。由于沒有描述實際網頁的結構,因此很難確定您需要的正確代碼。
我發現這兩個 YouTube 剪輯很有啟發性:
- 我如何使用 SELENIUM 通過 PYTHON 自動化 Web。Pt1:https ://www.youtube.com/watch?v=pUUhvJvs-R4
- 如何使用 Selenium 抓取動態網站:https ://www.youtube.com/watch?v=lTypMlVBFM4
轉載請註明出處,本文鏈接:https://www.uj5u.com/net/485442.html
