我正在嘗試抓取這個網站：https ://bartleby.com ，我嘗試使用 Python 請求撰寫代碼并且它可以作業。但是我正在嘗試將其轉換為 PHP，因為我希望將結果列印在我的網站上并且我的 Cpanel 不讀取 python，所以我被迫使用 CURL 來執行此操作但沒有作業，下面的代碼回傳：
Not Found
This page you were trying to reach at this address doesn't seem to exist.
What can I do now?
Sign up for your own free account.
所以我只是想知道這個網站如何阻止 PHP 上的 CURL 而不是 Python 上的請求？PHP 上是否有任何無法檢測到 CURL 的替代方法？謝謝。
我的 PHP 代碼（不作業）：
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.bartleby.com/questions-and-answers/1.-a-given-the-lines-l-7-124-tk-13k-1-k-3-and-l-x2-3s-y-1-10s-z-3-5s-determine-the-values-of-k-if-po/b88e3e3d-bfd6-4158-8335-6a3ca420430e');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'authority' => 'www.bartleby.com',
'accept' => 'text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'accept-language' => 'en-US;q=0.6',
'cache-control' => 'max-age=0',
'sec-fetch-dest' => 'document',
'sec-fetch-mode' => 'navigate',
'sec-fetch-site' => 'same-origin',
'sec-fetch-user' => '?1',
'sec-gpc' => '1',
'upgrade-insecure-requests' => '1',
'user-agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.115 Safari/537.36',
'Accept-Encoding' => 'gzip',
]);
curl_setopt($ch, CURLOPT_COOKIE, 'G_ENABLED_IDPS=google; refreshToken=330bb387263aa6673c3e39e975d729f723b38002; userId=4c28bc2c-1eec-4d2c-b44d-7bfa78216ba3; userStatus=A1; promotionId=; sku=bb999_bookstore; endCycleWhenQuestionsRemainingWasClosed=2022-06-19T07:00:00.000Z; btbHomeDashboardTooltipAnimationCount=0; isNoQuestionAskedModalClosed=true; accessToken=34ceed9609a07bd0238a74b5650d5c5362990498; bartlebyRefreshTokenExpiresAt=2022-07-16T12:37:57.217Z; btbHomeDashboardAnimationTriggerDate=2022-06-17T12:39:25.907Z; OptanonConsent=isGpcEnabled=1&datestamp=Thu Jun 16 2022 20:39:43 GMT+0800 (China Standard Time)&version=6.32.0&isIABGlobal=false&hosts=&consentId=9432e357-0639-4883-9f99-39bed0bb5cd9&interactionCount=0&landingPath=NotLandingPage&groups=C0001:1,C0003:1,BG142:0,C0002:0,C0005:0,C0004:0&AwaitingReconsent=false');
$response = curl_exec($ch);
echo $response;
curl_close($ch);
我也嘗試使用 file_get_contents() 但它回傳錯誤：警告：file_get_contents（https://bartleby.com）：無法打開流：HTTP 請求失敗！HTTP/1.1 503 服務在第 11 行的 D:\xampp\htdocs\bartleby\index.php 中暫時不可用
第 11 行是 $response = file_get_contents($url, false, stream_context_create($arrContextOptions));
完整代碼（不作業）：
<?php
$url= 'https://bartleby.com';
$arrContextOptions=array(
"ssl"=>array(
"verify_peer"=>false,
"verify_peer_name"=>false,
),
);
$response = file_get_contents($url, false, stream_context_create($arrContextOptions));
echo $response;
我的 Python 代碼（作業）：
import requests
cookies = {
'G_ENABLED_IDPS': 'google',
'refreshToken': '330bb387263aa6673c3e39e975d729f723b38002',
'userId': '4c28bc2c-1eec-4d2c-b44d-7bfa78216ba3',
'userStatus': 'A1',
'promotionId': '',
'sku': 'bb999_bookstore',
'endCycleWhenQuestionsRemainingWasClosed': '2022-06-19T07:00:00.000Z',
'btbHomeDashboardTooltipAnimationCount': '0',
'isNoQuestionAskedModalClosed': 'true',
'accessToken': '34ceed9609a07bd0238a74b5650d5c5362990498',
'bartlebyRefreshTokenExpiresAt': '2022-07-16T12:37:57.217Z',
'btbHomeDashboardAnimationTriggerDate': '2022-06-17T12:39:25.907Z',
'OptanonConsent': 'isGpcEnabled=1&datestamp=Thu Jun 16 2022 20:39:43 GMT+0800 (China Standard Time)&version=6.32.0&isIABGlobal=false&hosts=&consentId=9432e357-0639-4883-9f99-39bed0bb5cd9&interactionCount=0&landingPath=NotLandingPage&groups=C0001:1,C0003:1,BG142:0,C0002:0,C0005:0,C0004:0&AwaitingReconsent=false',
}
headers = {
'authority': 'www.bartleby.com',
'accept': 'text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'accept-language': 'en-US;q=0.6',
'cache-control': 'max-age=0',
# Requests sorts cookies= alphabetically
# 'cookie': 'G_ENABLED_IDPS=google; refreshToken=330bb387263aa6673c3e39e975d729f723b38002; userId=4c28bc2c-1eec-4d2c-b44d-7bfa78216ba3; userStatus=A1; promotionId=; sku=bb999_bookstore; endCycleWhenQuestionsRemainingWasClosed=2022-06-19T07:00:00.000Z; btbHomeDashboardTooltipAnimationCount=0; isNoQuestionAskedModalClosed=true; accessToken=34ceed9609a07bd0238a74b5650d5c5362990498; bartlebyRefreshTokenExpiresAt=2022-07-16T12:37:57.217Z; btbHomeDashboardAnimationTriggerDate=2022-06-17T12:39:25.907Z; OptanonConsent=isGpcEnabled=1&datestamp=Thu Jun 16 2022 20:39:43 GMT+0800 (China Standard Time)&version=6.32.0&isIABGlobal=false&hosts=&consentId=9432e357-0639-4883-9f99-39bed0bb5cd9&interactionCount=0&landingPath=NotLandingPage&groups=C0001:1,C0003:1,BG142:0,C0002:0,C0005:0,C0004:0&AwaitingReconsent=false',
'sec-fetch-dest': 'document',
'sec-fetch-mode': 'navigate',
'sec-fetch-site': 'same-origin',
'sec-fetch-user': '?1',
'sec-gpc': '1',
'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.115 Safari/537.36',
}
response = requests.get('https://www.bartleby.com/questions-and-answers/1.-a-given-the-lines-l-7-124-tk-13k-1-k-3-and-l-x2-3s-y-1-10s-z-3-5s-determine-the-values-of-k-if-po/b88e3e3d-bfd6-4158-8335-6a3ca420430e', cookies=cookies, headers=headers)
print(response.text)
uj5u.com熱心網友回復：
您沒有設定用戶代理。
看起來該網站需要來自真實用戶的用戶代理，例如Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0。
這是我的代碼，它只是作業。
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.bartleby.com/questions-and-answers/1.-a-given-the-lines-l-7-124-tk-13k-1-k-3-and-l-x2-3s-y-1-10s-z-3-5s-determine-the-values-of-k-if-po/b88e3e3d-bfd6-4158-8335-6a3ca420430e');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);// this is needed.
// But in my code is using user agent from web browser directly.
// You may change this to other.
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'headerFunction');// for debug response headers only.
$response = curl_exec($ch);
if (curl_errno($ch)) {
echo 'cURL error: ' . curl_error($ch);
echo '<br>';
exit();
}
echo '<hr>' . PHP_EOL;
echo '<h4>cURL response body</h4>' . PHP_EOL;
echo $response;
curl_close($ch);
unset($ch, $response);
/**
* Header function for debugging
*/
function headerFunction($ch, $header)
{
echo $header;
echo '<br>';
return mb_strlen($header);
}
您的代碼集請求標頭使用了錯誤的陣列格式。
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'authority' => 'www.bartleby.com',
//...
]);
這是錯誤的！
它應該是...
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'authority: www.bartleby.com',
//...
]);
您可以使用$reqHeaders = curl_getinfo($ch, CURLINFO_HEADER_OUT);除錯請求標頭。
您當前的代碼根本沒有發送user-agent，這就是它不起作用的原因。
轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/492272.html