輸入資料如下：
1 5
2 8
1 3
2 7
4 9
目標是讓每行的第一個數字為鍵，每行的第二個數字為值。洗牌后我想輸出（鍵，值串列）
但我不知道如何輸出值串列。
我期望的輸出是：
1,[5,3]
2,[7,8]
4,[9]
映射后我得到：
1 5
1 3
2 7
2 8
4 9
public static class Map extends Mapper<LongWritable, Text, IntWritable, Text> {
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
String token1 = tokenizer.nextToken();
String token2 = tokenizer.nextToken();
context.write(new IntWritable(Integer.parseInt(token1)), new Text(token2));
}
}
public static class Reduce extends Reducer<IntWritable, Text, IntWritable, Text> {
String iterableToString(Iterable<Text> values) {
StringBuilder sb = new StringBuilder("[");
for (Text val : values) {
sb.append(val.get()).append(",");
}
sb.setLength(sb.length() - 2);
sb.append("]");
return sb.toString();
}
public void reduce(IntWritable key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
// context.write(key, new Text(iterableToString(values)));
}
}
但是有一個錯誤資訊：
compile:
[javac] /home/zih-yan/hadoop_tutorial/build.xml:12: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 1 source file to /home/zih-yan/hadoop_tutorial/bin
[javac] /home/zih-yan/hadoop_tutorial/src/f.java:32: error: cannot find symbol
[javac] sb.append(val.get()).append(",");
[javac] ^
[javac] symbol: method get()
[javac] location: variable val of type Text
[javac] Note: /home/zih-yan/hadoop_tutorial/src/f.java uses or overrides a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] 1 error
謝謝！
uj5u.com熱心網友回復：
輸出值串列。
那么你的 reducer 必須輸出 Text 型別，而不是ArrayList. 您只能將 Hadoop 可序列化型別作為輸入和輸出。
話雖如此，可以提取問題并單獨進行單元測驗。
String iterableToString(Iterable<Text> values) {
StringBuilder sb = new StringBuilder("[");
for (Text val : values) {
sb.append(val.get()).append(",");
}
sb.setLength(sb.length() - 2);
sb.append("]");
return sb.toString();
}
然后一旦測驗通過，你就可以使用它了
public void reduce(IntWritable key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
context.write(key, new Text(iterableToString(values)));
}
關于您的編譯錯誤，請確保您為 Text 類使用正確的匯入
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/525647.html
標籤：爪哇Hadoop映射减少
上一篇：nodecrypto=require('crypto');ReferenceError要求未定義
下一篇：如何以編程方式終止EMR任務