各位好，
初學Spark，想讓Spark從S3上的資料
Spark搭建在EC2上
我在spark-shell下執行：
sc.hadoopConfiguration.set("fs.s3a.access.key", "xxxx")
sc.hadoopConfiguration.set("fs.s3a.secret.key", "yyyy")
val textFile = sc.textFile("s3a://...")
textFile.count()
例外：
15/12/21 13:29:58 INFO S3AFileSystem: Caught an AmazonServiceException, which means your request made it to Amazon S3, but was rejected with an error response for some reason.
15/12/21 13:29:58 INFO S3AFileSystem: Error Message: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: A5C5253A63B271A6, AWS Error Code: null, AWS Error Message: Forbidden
15/12/21 13:29:58 INFO S3AFileSystem: HTTP Status Code: 403
15/12/21 13:29:58 INFO S3AFileSystem: AWS Error Code: null
15/12/21 13:29:58 INFO S3AFileSystem: Error Type: Client
15/12/21 13:29:58 INFO S3AFileSystem: Request ID: A5C5253A63B271A6
15/12/21 13:29:58 INFO S3AFileSystem: Class Name: com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception
com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: A5C5253A63B271A6, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: 7zaafP60a1U8pt02jayWuWtf1ONVd1g5n6/xAbutkxtU/M1ZvtSV0pjVzPN2aFOpANTj5R7Cikg=
at com.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
at com.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
看起來是被服務器拒絕（403）了
請問是不是我操作哪里錯了？
是否跟北京區S3有關，默認訪問了global的？Endpoint／Region？
另外有鏈接給我學習一下s3://、s3n://、s3a://的區別嗎？我該用哪個？
謝謝！
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/83336.html
標籤：Spark