我正在嘗試從IMDB他們的評論中抓取劇集資料。我想獲取所有劇集并將它們存盤在dataframe. 但是我遇到了一個問題：每集僅洗掉 1 條評論。當我在測驗時，有一個實體，所有評論都被刮掉了，但它不再起作用了。有誰知道我怎樣才能刮掉所有的評論并將其存盤在一個dataframe?
這是代碼：
library(dplyr)
library(rvest)
library(tidyverse)
getReviewLink = function(episodeLink) {
episodePage = read_html(episodeLink)
container = episodePage %>%
html_nodes(".Hero__WatchContainer__NoVideo-sc-kvkd64-9.cTdSBT")
reviewLinks = episodePage %>%
html_nodes(".Hero__WatchContainer__NoVideo-sc-kvkd64-9.cTdSBT > ul > li:nth-child(1) > a") %>%
html_attr("href") %>%
paste("https://www.imdb.com", ., sep="")
print(reviewLinks)
cleanedReviewLink = ifelse(reviewLinks == "https://www.imdb.com", NA, reviewLinks)
print(cleanedReviewLink)
get_reviews = ifelse(is.na(cleanedReviewLink), NA, read_html(cleanedReviewLink) %>% html_nodes(".show-more__control") %>%
html_text() %>% str_trim())
print(get_reviews)
return(get_reviews)
}
episodes = data.frame()
for (page_result in seq(from = 1, to = 51, by = 50)){
link = paste0("https://www.imdb.com/search/title/?title_type=tv_episode&num_votes=1000,&sort=user_rating,desc&start"
,page_result,"&ref_=adv_nxt")
page = read_html(link)
show_name = page %>% html_nodes(".lister-item-index a") %>% html_text() %>% str_trim
episode_name = page %>% html_nodes("small a") %>% html_text()
episode_links = page %>% html_nodes("small a") %>% html_attr("href") %>%
paste("https://www.imdb.com", ., sep="")
episodeReview = sapply(episode_links, FUN = getReviewLink, USE.NAMES = FALSE)
print(episodeReview)
episodes = rbind(episodes, data.frame(show_name, episode_name, episodeReview, stringsAsFactors = FALSE))
print(paste("Page:", page_result))
}
任何幫助表示贊賞。
uj5u.com熱心網友回復：
我運行了你的代碼，你的函式有一個小錯誤getReviewLink。
以下部分是洗掉所有評論并僅重新調整第一個評論。
get_reviews = ifelse(is.na(cleanedReviewLink), NA, read_html(cleanedReviewLink) %>% html_nodes(".show-more__control") %>%
html_text() %>% str_trim())
將其替換為
get_reviews = read_html(cleanedReviewLink) %>% html_nodes(".show-more__control") %>%
html_text() %>% str_trim() %>% str_subset(". ")
[1] "I haven't seen every episode in the world, but this is as close to perfect as I have ever seen. Never thought I would say something could match the likes of Lord of the Rings. It's the most cinematic episode I've ever seen with maybe \"Battle of the Bastards\" being alongside it for obvious reasons, but for an animated episode to do this is even more shocking. It would be hard for someone to imagine an animated episode being as cinematic as an HBO production and this episode made it possible. This episode was also very emotional as two of my favorite characters' (Armin and Erwin) lives were on the line. I even cried when Armin was gonna make the sacrifice. It was also sad to see Erwin's unfortunate plan come to fruition, but he did it knowing it was for a greater good. I also loved the parallels of all the main characters sacrificing themselves in this episode."
[2] "I cant understand how anyone can rate something as incredible like this below 10. This episode is amazingly godly and will go down in history as one of the greats"
此外，您實際上并沒有抓取所有評論。例如，這一集有 951 條評論https://www.imdb.com/title/tt9906260/reviews?ref_=tt_ov_rt
但是您的代碼只能獲得前 25 條評論。如果您需要顯示的所有評論，您需要繼續單擊加載更多。這可以是RSelenium或可能是imdbapi。
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/442700.html