我想在應用如下過濾器后抓取http://csla.history.ox.ac.uk/search.php
- 點擊“圣”
- 在“出生/埋葬地區”下選擇“高盧和法蘭克王國”
- 點擊“應用搜索”
我很掙扎，因為 URL 沒有相應地更新。
原始碼<option value="Gaul">Gaul and Frankish kingdoms</option>如下
<div class="section colm colm6" id="fl-page4-12">
<label for="item_12"class="field-label">Region of Birth/Burial</label>
<label class="field select">
<select id="text-nine" name="form[item_89]">
<option value=""></option>
<option value="East">'The East' (unspecified)</option>
<option value="West">'The West' (unspecified)</option>
<option value="Britain">Britain and Ireland</option>
<option value="Gaul">Gaul and Frankish kingdoms</option>
從選定的網頁中，我想點擊標記為藍色的 ID（例如第一個是 E06478）。
從然后選擇的網頁（例如http://csla.history.ox.ac.uk/record.php?recid=E06478）中，我想單擊“相關圣記錄”表中寫入的 ID（例如，這里的一個是 S01319）。
從然后選擇的網頁（例如http://csla.history.ox.ac.uk/record.php?recid=S01319）中，我想抓取 Saint ID（例如 'S01319'）、姓名（例如 'Orientius , 奧赫主教, 5th c.'), Reported Death Not Before, Reported Death Not After, Gender, Type of Saint and present them in a dataframe.
uj5u.com熱心網友回復：
我知道您之前曾問過類似的問題，我將繼續之前給出的解決方案
（開頭的代碼是從這個解決方案復制的，在這個擴展中，我們為附加資料創建新列，并使用 再次抓取它們rvest）
library(httr)
library(rvest)
items <- c(998, 1, 18,89, 90, 2, 88, 20, 3, 4, 5, 6, 12, 13, 11, 999, 213, 214)
contents <- c('\nE\n', '\n\n', '\n\n', '\nGaul\n', rep('\n\n', 11), '\nOr\n',
'\n\n', '\n\n')
s <- paste0("-----------------------------39565121210000504382566389445\n",
"Content-Disposition: form-data; name=\"form[item_", items,
']\"\n', contents,
collapse = '')
s <- paste0(s, '-----------------------------39565121210000504382566389445--')
type <- paste0('multipart/form-data; boundary=---------------------------',
'39565121210000504382566389445')
res <- POST('http://csla.history.ox.ac.uk/results.php',
body = charToRaw(s),
content_type(type))
df <- res %>%
read_html() %>%
html_elements(xpath = "//td[not(contains(@style, 'LightGray'))]") %>%
html_text() %>%
matrix(ncol = 2, byrow = TRUE) %>%
as.data.frame() %>%
setNames(c('ID', 'Title')) %>%
dplyr::as_tibble()
urls <- paste0("http://csla.history.ox.ac.uk/record.php?recid=", df$ID)
all_results <- list()
for(i in seq_along(urls)) {
all_results[[i]] <- read_html(urls[i]) %>%
html_elements("td") %>%
html_text() %>%
matrix(ncol = 4, byrow = TRUE) %>%
as.data.frame() %>%
setNames(c("ID", "Name", "Name_in_source", "Identity"))
}
final_result <- dplyr::bind_rows(all_results)
# continued solution ----------------------
additional_columns <- c("Name", "Number in BH", "Reported Death Not Before", "Reported Death Not After", "Gender", "Type of Saint")
final_result[, additional_columns] <- NA
for (i in seq_along(final_result$ID)) {
web_page <- read_html(paste0("http://csla.history.ox.ac.uk/record.php?recid=", final_result$ID[[i]]))
temp_res <- sapply(additional_columns, function(col) web_page %>%
html_element(xpath = paste0("//div[contains(text(),'", col, "')]")) %>%
html_children() %>% html_text())
final_result[i, additional_columns] <- lapply(temp_res, function(x) ifelse(!length(x), NA, x))
}
轉載請註明出處，本文鏈接：https://www.uj5u.com/qukuanlian/492301.html