嘗試使用beautifulsoup抓取2個標簽并將它們放在同一個csv中

我目前正在學習 python 并嘗試通過獲取其他代碼來完成我自己的專案，所以在我學習的時候不要責怪我。

我從tickers.csv中獲取股票串列并抓取了一個網站以獲取部門和行業并將它們放在stocks.csv中

問題是我只能通過以下方式將行業或行業（通過選擇一個）放入stocks.csv

if __name__ == '__main__':
        to_csv(list(map(lambda ticker: get_sector(ticker), get_stocks())))
        # to_csv(list(map(lambda ticker: get_industry(ticker), get_stocks())))

我想同時完成部門和行業這是整個代碼

# dependencies
import pandas as pd
import requests
from bs4 import BeautifulSoup as bs

LSE = 'https://csimarket.com/stocks/at_glance.php?code='


def get_stocks():
    df = pd.read_csv('watchlist/tickers.csv')
    return list(df['ticker'])


def to_csv(stocks):
    df = pd.DataFrame(stocks)
    df.to_csv('stocks.csv', index=False)


def get_soup(url):
    return bs(requests.get(url).text, 'html.parser')


def get_sector(ticker):
    soup = get_soup(LSE   ticker)
    try:
        sector = soup.find('span', text='Sector').find_next('a').text.replace('\n', '').replace('?', '').strip()
    except:
        print('No sector information availible for ', ticker)
        return {'ticker': ticker, 'sector': ''}

    print(ticker, sector)
    return {'ticker': ticker, 'sector': sector}


def get_industry(ticker):
    soup1 = get_soup(LSE   ticker)
    try:
        industry = soup1.find('span', text='Industry').find_next('a').text.replace('\n', '').replace('?', '').strip()
    except:
        print('No industry information availible for ', ticker)
        return {'ticker': ticker, 'industry': ''}

    print(ticker, industry)
    return {'ticker': ticker, 'industry': industry}


if __name__ == '__main__':
    to_csv(list(map(lambda ticker: get_sector(ticker), get_stocks())))
    # to_csv(list(map(lambda ticker: get_industry(ticker), get_stocks())))

這是tickers.csv

ticker,
A
AA
AADI
AAIC
AAL
AAN
AAOI
AAON
AAP
AAPL
AAT
AAU
AAWW
AB
ABB
ABBV
ABC
ABCB
ABCL
ABEO
ABEV
ABG
ABIO
ABM
ABMD
ABNB
ABOS
ABR
ABSI
ABST
ABT
ABTX
ABUS
ACA
ACAD
ACB
ACC
ACCD
ACCO
ACEL
ACER
ACET
ACEV
ACGL
ACH
ACHC
ACHR
ACHV
ACI
ACIU

這是我得到扇區時的stocks.csv

ticker,sector
A,Healthcare
AA,Basic Materials
AADI,
AAIC,Services
AAL,Transportation
AAN,Services
AAOI,Technology
AAON,Capital Goods
AAP,Retail
AAPL,Technology
AAT,Financial
AAU,Basic Materials
AAWW,Transportation
AB,Financial
ABB,Consumer Discretionary
ABBV,Healthcare
ABC,Retail
ABCB,Financial
ABCL,Healthcare
ABEO,Healthcare
ABEV,Consumer Non Cyclical
ABG,Retail
ABIO,Healthcare
ABM,Services
ABMD,Healthcare
ABNB,Services
ABOS,Healthcare
ABR,Financial
ABSI,Healthcare
ABST,
ABT,Healthcare
ABTX,Financial
ABUS,Healthcare
ACA,Basic Materials
ACAD,Healthcare
ACB,
ACC,Financial
ACCD,Financial
ACCO,Basic Materials
ACEL,Services
ACER,Healthcare
ACET,Retail
ACEV,Technology
ACGL,Financial
ACH,Basic Materials
ACHC,Healthcare
ACHR,Capital Goods
ACHV,Healthcare
ACI,Energy
ACIU,

這是我得到行業時的stocks.csv

ticker,industry
A,Laboratory Analytical Instruments
AA,Aluminum
AADI,
AAIC,Real Estate Operations
AAL,Airline
AAN,Rental & Leasing
AAOI,Computer Networks
AAON,Industrial Machinery and Components
AAP,Automotive Aftermarket
AAPL,Computer Hardware
AAT,Real Estate Investment Trusts
AAU,Metal Mining
AAWW,Special Transportation Services
AB,Investment Services
ABB,Electric & Wiring Equipment
ABBV,Biotechnology & Pharmaceuticals
ABC,Pharmacy Services & Retail Drugstore
ABCB,Regional Banks
ABCL,Major Pharmaceutical Preparations
ABEO,Major Pharmaceutical Preparations
ABEV,Nonalcoholic Beverages
ABG,Automotive Aftermarket
ABIO,In Vitro & In Vivo Diagnostic Substances
ABM,Professional Services
ABMD,Medical Equipment & Supplies
ABNB,Real Estate Operations
ABOS,Biotechnology & Pharmaceuticals
ABR,Real Estate Investment Trusts
ABSI,Medical Laboratories
ABST,
ABT,Major Pharmaceutical Preparations
ABTX,Commercial Banks
ABUS,Major Pharmaceutical Preparations
ACA,Miscellaneous Fabricated Products
ACAD,Major Pharmaceutical Preparations
ACB,
ACC,Real Estate Investment Trusts
ACCD,Blank Checks
ACCO,Paper & Paper Products
ACEL,Casinos & Gaming
ACER,Major Pharmaceutical Preparations
ACET,Pharmacy Services & Retail Drugstore
ACEV,Semiconductors
ACGL,Property & Casualty Insurance
ACH,Aluminum
ACHC,Healthcare Facilities
ACHR,Aerospace & Defense
ACHV,In Vitro & In Vivo Diagnostic Substances
ACI,Coal Mining
ACIU,

uj5u.com熱心網友回復：

只需將現有的兩個函式合二為一，然后通過單個湯物件回傳決議結果

import pandas as pd
import requests
from bs4 import BeautifulSoup as bs

LSE = 'https://csimarket.com/stocks/at_glance.php?code='

def get_stocks():
    df = pd.read_csv('watchlist/tickers.csv')
    return list(df['ticker'])


def to_csv(stocks):
    df = pd.DataFrame(stocks)
    df.to_csv('stocks.csv', encoding='utf-8-sig', index=False)


def get_soup(url):
    return bs(requests.get(url, headers = {'User-Agent':'Mozilla/5.0'}).text, 'html.parser')


def get_data(ticker):
    soup = get_soup(LSE   ticker)
    try:
        sector = soup.find('span', text='Sector').find_next('a').text.replace('\n', '').replace('?', '').strip()
    except:
        print('No sector information availible for ', ticker)
        return {'ticker': ticker, 'sector': ''}

    print(ticker, sector)
    
    try:
        industry = soup.find('span', text='Industry').find_next('a').text.replace('\n', '').replace('?', '').strip()
    except:
        print('No industry information availible for ', ticker)
        return {'ticker': ticker, 'industry': ''}

    print(ticker, industry)
    return {'ticker': ticker, 'sector': sector,  'industry': industry}

if __name__ == '__main__':
    to_csv(list(map(lambda ticker: get_data(ticker), get_stocks())))

標籤：Python CSV 网页抓取 美丽的汤 标签

上一篇：在保留html物件的同時洗掉特定的“span”標簽

下一篇：Selenium無法提取文本