扫描二维码下载沐宇APP

沐宇

微信扫码使用沐宇小程序

沐宇

hadoop涓璻educe鏂规硶鎬庝箞浣跨敤

扬州沐宇科技
2024-01-24 13:29:30
hadoop

鍦℉adoop涓紝Reduce鏂规硶鏄疢apReduce浣滀笟鐨勪竴閮ㄥ垎銆傚畠鐢ㄤ簬瀵筂ap浠诲姟鐨勮緭鍑鸿繘琛屽悎骞跺拰澶勭悊锛屼互鐢熸垚鏈€缁堢殑杈撳嚭銆?/p>

鍦℉adoop涓娇鐢≧educe鏂规硶锛岄渶瑕佹寜鐓т互涓嬫楠よ繘琛屾搷浣滐細

  1. 鍒涘缓涓€涓猂educe绫伙紝璇ョ被闇€瑕佺户鎵?code>org.apache.hadoop.mapreduce.Reducer绫伙紝骞堕噸鍐?code>reduce鏂规硶銆?/li>
public class MyReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
    @Override
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
        // 鍦ㄨ繖閲岀紪鍐橰educe閫昏緫
        // key锛氳緭鍏ラ敭绫诲瀷
        // values锛氳緭鍏ュ€肩被鍨嬭凯浠e櫒
        // context锛氱敤浜庤緭鍑虹粨鏋滅殑涓婁笅鏂?/span>
    }
}
  1. 鍦?code>reduce鏂规硶涓紝缂栧啓Reduce閫昏緫銆俁educe鏂规硶鐨勮緭鍏ュ寘鎷竴涓敭锛坘ey锛夊拰璇ラ敭瀵瑰簲鐨勪竴缁勫€硷紙values锛夈€傚湪Reduce閫昏緫涓紝鍙互鏍规嵁闇€瑕佸杩欑粍鍊艰繘琛屽悎骞躲€佽仛鍚堟垨鍏朵粬澶勭悊锛屽苟灏嗙粨鏋滆緭鍑哄埌涓婁笅鏂囷紙context锛変腑銆?/p>

  2. 鍦ㄤ綔涓氱殑椹卞姩绋嬪簭涓紝閫氳繃璋冪敤job.setReducerClass()鏂规硶锛屽皢鍒涘缓鐨凴educe绫绘寚瀹氫负浣滀笟鐨凴educer銆?/p>

Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "MyJob");

// 璁剧疆Mapper鍜孯educer
job.setMapperClass(MyMapper.class);
job.setReducerClass(MyReducer.class);

// 璁剧疆Mapper鍜孯educer鐨勮緭鍑虹被鍨?/span>
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);

// ...

// 鎻愪氦浣滀笟
System.exit(job.waitForCompletion(true) ? 0 : 1);

浠ヤ笂鏄疕adoop涓娇鐢≧educe鏂规硶鐨勫熀鏈楠わ紝鍏蜂綋鐨勯€昏緫鏍规嵁瀹為檯闇€姹傝繘琛岀紪鍐欍€?/p>

扫码添加客服微信