并行檔案下載損壞

我在下面有這段代碼，它能夠并行下載檔案的多個部分并使用記憶體映射檔案寫入它們。問題發生在DownloadFile()函式中。該檔案正在正常開始下載，但在此程序中它已損壞。例如，如果我嘗試下載影像，它的某些部分將被損壞。我不確定這是否來自代碼中的某種種族傳導，或者是否與零件的內容范圍計算有關。任何有關問題如何發生或是否發生的幫助將不勝感激，謝謝！

最小的、可重現的示例：

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading;
using System.Threading.Tasks;
using static System.Console;

namespace ZenTester
{
internal class FileChunk
{
    public long Start { get; set; }
    public long End { get; set; }
    public FileChunk(){}
    public int Id { get; set; }

    public FileChunk(long startByte, long endByte)
    {
        Start = startByte;
        End = endByte;
    }
}

internal class RetryHandler : DelegatingHandler
{
    private int _maxRetries = 3;

    public RetryHandler(HttpMessageHandler innerHandler) : base(innerHandler) { }

    public RetryHandler(HttpMessageHandler innerHandler, int maxRetries) : base(innerHandler)
    {
        _maxRetries = maxRetries;
    }

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        HttpResponseMessage response = null;
        for (var i = 0; i < _maxRetries; i  )
        {
            response = await base.SendAsync(request, cancellationToken);
            if (response.IsSuccessStatusCode)
            {
                return response;
            }
        }

        return response;
    }
}

public static class ZenTester
{
    private static async Task DownloadFile(string url, int parts, string outFile = null!)
    {
        var responseLength = (await WebRequest.Create(url).GetResponseAsync()).ContentLength;
        var partSize = (long)Math.Floor(responseLength / (parts   0.0));
        var pieces = new List<FileChunk>();
        var uri = new Uri(url);
        
        WriteLine(responseLength.ToString(CultureInfo.InvariantCulture)   " TOTAL SIZE");
        WriteLine(partSize.ToString(CultureInfo.InvariantCulture)   " PART SIZE"   "\n");
        
        string filename = outFile ?? Path.GetFileName(uri.LocalPath);

        var mmf = MemoryMappedFile.CreateFromFile(filename, FileMode.OpenOrCreate, null, responseLength);

        var httpPool = new HttpClient(new RetryHandler(new HttpClientHandler(), 10)) {MaxResponseContentBufferSize = 1000000000};
        
        //Loop to add all the events to the queue
        for (long i = 0; i < responseLength; i  = partSize)
        {
            pieces.Add(i   partSize < responseLength
                ? new FileChunk(i, i   partSize)
                : new FileChunk(i, responseLength));
        }

        await Parallel.ForEachAsync(pieces, parallelOptions, async (piece, cancellationToken) =>
        {
            var client = httpPool.Get();
            var request = new HttpRequestMessage { RequestUri = new Uri(url) };
            request.Headers.Range = new RangeHeaderValue(piece.Start, piece.End);
                
            var message = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).Result;

            if (message.IsSuccessStatusCode)
            {
                await using var streamToRead = await message.Content.ReadAsStreamAsync();
                var streams = mmf.CreateViewStream(piece.Start, piece.End-piece.Start - 1);
                var T = streamToRead.CopyToAsync(streams);
                T.Wait();
                if (T.IsCompletedSuccessfully)
                {
                    streams.Flush();
                    streams.Close();
                }
            }
        });
        
    }
    
    public static void Main(string[] args)
    {
        var url = "https://wallpaperaccess.com/full/2159447.jpg";

        var s = DownloadFile(url, 8);
        
        s.Wait();
    }
}
}

uj5u.com熱心網友回復：

我發現損壞正在發生，因為某些范圍會大于 Start-End。我解決了這個問題，但向記憶體映射檔案和視圖流添加了讀寫修飾符。然后我將視圖流的大小更改為 sendAsync 請求回傳的內容的長度：

var streams = mmf.CreateViewStream(piece.Start, message.Content.Headers.ContentLength!.Value,
                        MemoryMappedFileAccess.ReadWrite);

標籤：C＃ 异步 .net核心 联网 并行处理

上一篇：在tokio-tungstenite下一個原因退出

下一篇：如果C#async/await仍然等待上一次執行完成，它有什么好處？