Scrapy在嵌套的json物件中運行for回圈-有解無憂

我正在使用 scrapy 抓取 json api，并希望回圈訪問優惠，然后獲得如下面的螢屏截圖所示的結果。我正在接受報價，但不確定要為 get() 寫什么，因為它沒有標記。我嘗試過的一切都會導致“串列”物件沒有屬性 get 的錯誤。

Scrapy 在嵌套的 json 物件中運行 for 回圈

我的代碼如下：

import scrapy
import json

class DkSpider(scrapy.Spider):
    name = 'dk'
    allowed_domains = ['sportsbook.draftkings.com']
    start_urls = ['https://sportsbook.draftkings.com//sites/US-SB/api/v4/eventgroups/88670846/categories/583/subcategories/4991']

    def parse(self, response):
        items = json.loads(response.body)
        cats = items.get('eventGroup').get('offerCategories')

        for cat in cats:
            groups = str(cat.get('name'))

            if groups == "Player Props":
                subcats = cat.get('offerSubcategoryDescriptors')

                for subcat in subcats:
                    markets = str(subcat.get('name'))

                    if markets == "Points":
                        games = subcat.get('offerSubcategory').get('offers')

                        for game in games:
                            outcomes = game.get('outcomes')

uj5u.com熱心網友回復：

如果只有一個空白條目而不是您需要撰寫的條目for game in games[0]:，但是由于您有多個空白鍵，那么您需要遍歷它們以獲取您想要的所有資訊。

用你的方法解決：

import scrapy


class DkSpider(scrapy.Spider):
    name = 'dk'
    allowed_domains = ['sportsbook.draftkings.com']
    start_urls = ['https://sportsbook.draftkings.com//sites/US-SB/api/v4/eventgroups/88670846/categories/583/subcategories/4991']

    def parse(self, response):
        items = response.json()
        cats = items.get('eventGroup').get('offerCategories')

        for cat in cats:
            groups = str(cat.get('name'))

            if groups == "Player Props":
                subcats = cat.get('offerSubcategoryDescriptors')

                for subcat in subcats:
                    markets = str(subcat.get('name'))

                    if markets == "Points":
                        games = subcat.get('offerSubcategory').get('offers')

                        for game in games:
                            for in_game in game:
                                outcomes = in_game.get('outcomes')
                                for outcome in outcomes:
                                    print(outcome['participant'])

但請注意，您執行的迭代次數比實際需要的多，因此運行時間會更長。要么休息一下，要么只是做這樣的事情：

import scrapy
import json


class DkSpider(scrapy.Spider):
    name = 'dk'
    allowed_domains = ['sportsbook.draftkings.com']
    start_urls = ['https://sportsbook.draftkings.com//sites/US-SB/api/v4/eventgroups/88670846/categories/583/subcategories/4991']

    def parse(self, response):
        # from scrapy.shell import inspect_response
        # inspect_response(response, self)
        games = json.loads(response.body)['eventGroup']['offerCategories'][1]['offerSubcategoryDescriptors'][1]['offerSubcategory']['offers']
        for game in games:
            for in_game in game:
                outcomes = in_game.get('outcomes')
                for outcome in outcomes:
                    # Get whatever info you want here
                    print(outcome['participant'])

轉載請註明出處，本文鏈接：https://www.uj5u.com/net/395915.html

標籤：Python json 刮的

上一篇：輸入陣列的震動以命名陣列的值對

下一篇：jq映射來自不同陣列的值