我正在嘗試使用 scrapy FormRequest 模擬 ajax 請求，以獲取本網站https://www.the-academy.nl/trainingen上的下一頁。我的標題看起來像這樣
headers = {
'path': 'https://www.the-academy.nl/Page?$$ajaxid=view:_id1:_id2:_id3:_id4:_id5:6:_id114:_id116:tblView',
'authority': 'www.the-academy.nl',
'accept-encoding': 'gzip, deflate, br',
'content-length': '1225',
'content-type': 'multipart/form-data'
}
和這樣的formdata
formdata = {
'$$viewid': '!1rjej6ewgse3x0h6r86gfzlst!',
'$$xspsubmitid': 'view:_id1:_id2:_id3:_id4:_id5:6:_id114:_id116:viewPager__Group__lnk__1',
'$$xspexecid': 'view:_id1:_id2:_id3:_id4:_id5:6:_id114:_id116:viewPager',
'$$xspsubmitvalue':'',
'$$xspsubmitscroll': '0|1272',
}
我收到了回復，但它是 404 頁面。先感謝您）
uj5u.com熱心網友回復：
我
java用作搜索詞。我只選擇具有鍵值對的表單資料。不要注入
'content-length'標頭添加方法：“POST”
稱呼
FormRequest.from_response以下是 200 回應狀態的示例
腳本：
from scrapy.crawler import CrawlerProcess
import scrapy
class AspSpider(scrapy.Spider):
name = 'asp'
def start_requests(self):
yield scrapy.FormRequest(
url='https://www.the-academy.nl/zoekresultatenpagina?text=java',
formdata= {
'view:_id1:_id2:_id3:_id4:_id5:2:_id86:_id88:query': "",
'view:_id1:_id2:_id3:_id4:_id5:3:_id94:_id96:query': "",
'$viewid': '!eaie1cfxpuckx0dbjrxsxrw60!',
'$$xspsubmitid': 'view:_id1:_id2:_id3:_id199:_id200:0:_id201:_id203:viewPager__Next',
'$$xspexecid': 'view:_id1:_id2:_id3:_id199:_id200:0:_id201:_id203:viewPager',
'$$xspsubmitscroll': '0|1500',
'view:_id1': 'view:_id1',
'$$xspsubmitvalue': ""
},
callback=self.parse_item,
headers={
'accept': '*/*',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9',
'content-type': 'multipart/form-data; boundary=----WebKitFormBoundary2aCYMIdAcbwx4FjO',
'referer': 'https://www.the-academy.nl/zoekresultatenpagina?text=java'
},
method='POST'
)
def parse_item(self,response):
pass
if __name__ == "__main__":
process =CrawlerProcess(AspSpider)
process.crawl()
process.start()
輸出：
DEBUG: Crawled (200) <POST https://www3.hkexnews.hk/sdw/search/searchsdw.aspx> (referer: https://www.the-academy.nl/zoekresultatenpagina?text=java)
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/481939.html