MapReduce处理社交关系的一道问题

给定一个社交关系(A,B),A和B分别代表2个人的id或名字(如果是名字,要求名字不能有重复)。要求只使用一次MapReduce,过滤掉里面的双向关系,只保留单向关系。比如现在这个关系里包含如下内容:

(张三,李四)

(张三,王五)

(张三,赵六)

(李四,张三)

里面的第1条和第4条构成了双向关系,要过滤掉,剩下第2和第3条。

思路1:对每一条关系(A, B),在map阶段生成(A, B) 和(B, A) 。如果A和B是双向的,会生成4条记录,分别是(A, B),  (B, A), (B, A), (A,B)。

在reduce阶段,我们发现key A对应的values里,B有2条。我们知道这是由双向关系产生的,我们不应该有任何输出。我们只输出在values里只有1个的。假如C在values里只有1个。则reduce阶段,我们输出(A,C)。

带来的问题:以上的办法能过滤掉双向关系,但是会输出多余的内容。比如一个单项关系 (A, B)。按照以上方法, reduce阶段最后会输出(A,B)和(B,A)。实际上不应该输出(B, A)。这个(B,A)是我们人为加进去的。

思路2:在思路1的基础上,map 阶段加入标记。对每一条关系(A, B),在map阶段生成(A, B|1) 和(B, A|0)。1代表真实存在的关系,0代表为了解决问题加入的虚构关系。在reduce阶段,我们按照思路1过滤掉有2个值的,然后看标记是不是1,如果是1则输出;如果是0,不输出。

Hadoop MapReduce代码如下:

import java.io.IOException;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;

public class SingleRelationshipMapper extends MapReduceBase implements
Mapper<LongWritable, Text, Text, Text>{

    @Override
    public void map(LongWritable key, Text value, OutputCollector<Text, Text> output, Reporter arg3)
        throws IOException {
        String line = value.toString();
        String[] names = line.split(" ");
        Text text1 = new Text(names[0]);
        Text text2 = new Text(names[1]);
        output.collect(new Text(text1), new Text(text2+"|1")); //1代表真实关系
        output.collect(new Text(text2), new Text(text1+"|0")); //0代表虚构关系
    }
}

import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapred.Reporter;

public class SingleRelationshipReducer extends MapReduceBase implements
    Reducer<Text, Text, Text, Text>{

    @Override
    public void reduce(Text key, Iterator<Text> values, OutputCollector<Text, Text> output, Reporter arg3) throws IOException {
        List<String> list = new ArrayList<String>();
        while (values.hasNext()) {
            list.add(values.next().toString());
       }
       for(String item:list){
           String[] array = item.split("\\|");
           if(onlyOne(array[0], list)){
               if(array[1].equals("1")){
                   output.collect(key, new Text(array[0]));
               }
           }
       }
    }

    private boolean onlyOne(String str, List<String> list){
        int count=0;
        for(String item: list){
            String array[] = item.split("\\|");
            if(array[0].equals(str)){
                count++;
            }
        }
        return count==1;
     }
}
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;

public class SingleRelationshipDriver extends Configured implements Tool{

    /**
    * @param args
    * @throws Exception
    */
    public static void main(String[] args) throws Exception {
        int exitCode = ToolRunner.run(new SingleRelationshipDriver(), args);
        System.exit(exitCode);
    }

    @Override
    public int run(String[] arg) throws Exception {

        JobConf conf = new JobConf();
        conf.setJobName("singleRelation");
        conf.setJarByClass(SingleRelationshipDriver.class);
        conf.setMapperClass(SingleRelationshipMapper.class);
        conf.setReducerClass(SingleRelationshipReducer.class);
        conf.setOutputKeyClass(Text.class);
        conf.setOutputValueClass(Text.class);
        FileInputFormat.addInputPaths(conf, arg[0]);
        FileOutputFormat.setOutputPath(conf, new Path(arg[1]));
        JobClient.runJob(conf);
        return 0;
    }

}

输入:

John Tom
John Amily
John George
Amily John

输出:

John Tom
John George

频繁项集挖掘(Frequent Itemsets mining)之A-Priori算法java实现

Frequent Itemsets mining 最常见的形式就是给定market-basket形式的数据(每一行相当于一个购物篮,包含多个商品item),然后我们找出关联度大于某个值的所有item集合。A-Priori算法是Frequent Itemsets mining里最基本的算法。

一、A-Priori的基本思路

第一遍(pass 1),扫描文件,统计单项(single item)的出现次数(使用1个map进行统计,key就是item,value就是出现次数)。最后,过滤掉小于最小支持度的,得到频繁单项集。

第二遍(pass 2),扫描文件,对于每一行,对任意两个item组合得到pair item。如果pair item的2个单项都在频繁单项集里,则统计这个pair item的出现次数;否则略过。最后得到所有pair item的出现频率,过滤掉小于最小支持度的,得到频繁2项集。

第N遍,如果发现频繁N-1项集不为空,则说明mining还没有完成,需要进行第n次扫描。从每行中取得任意N个item,如果这N项的所有N-1项子集都在频繁N-1项集里,则统计其出现次数,否则略过。最后,过滤掉小于最小支持度的,得到频繁N项集。

直到频繁项集为空才停止循环。

二、java实现

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;

public class Apriori {

	private String inputFile;

	private double minSupport;

	private BufferedWriter bw;

	private HashSet<Set<Integer>> frequentItems = new HashSet<Set<Integer>>(); //频繁项集

	private Set<Integer> frequentSingleItems = new HashSet<Integer>(); //频繁单项集

	private int totalCount; //项目数量

	public Apriori(String inputFile, double minSupport) {
		this.inputFile = inputFile;
		this.minSupport = minSupport;
	}

	/**
	 * 找出频繁一项集
	 * @return
	 * @throws IOException
	 */
	public Map<Set<Integer>,Integer> findF1Item() throws IOException {
		Map<Set<Integer>,Integer> result = new LinkedHashMap<Set<Integer>, Integer>();
		Map<Integer, Integer> map = new HashMap<Integer, Integer>();
		BufferedReader reader = new BufferedReader(new FileReader(inputFile));
		String line;
		int numberOfLine = 0;
		while ((line = reader.readLine()) != null) {
			numberOfLine++;
			String[] items = line.split(" ");
			for(String item : items){
				int intItem = Integer.valueOf(item);
				if (map.containsKey(intItem)) {
					map.put(intItem, map.get(intItem) + 1);
				} else {
					map.put(intItem, 1);
				}
			}
		}
		totalCount = numberOfLine;
		reader.close();
		//使用TreeSet按照item编号从小到大排序
		TreeSet<Integer> treeSet = new TreeSet<Integer>(map.keySet());
		for (Integer item : treeSet) {
			int count = map.get(item);
			//过滤掉出现频率小于最小支持度的item
			if (count >= minSupport*totalCount) {
				Set<Integer> f1Set = new TreeSet<Integer>();
				f1Set.add(item);
				result.put(f1Set, count);
				frequentItems.add(f1Set);
				frequentSingleItems.add(item);
			}
		}
		return result;
	}

	public Map<Set<Integer>,Integer> generateNextPass(int k) throws Exception{
		Map<Set<Integer>, Integer> map = new HashMap<Set<Integer>, Integer>();
		BufferedReader reader = new BufferedReader(new FileReader(inputFile));
		String line;
		while ((line = reader.readLine()) != null) {
			String[] items = line.split(" ");
			List<Set<Integer>> list = generateSubset(items, k);
			for(Set<Integer> set :list){
				if(map.containsKey(set)){
					map.put(set, map.get(set)+1);
				}
				else{
					map.put(set, 1);
				}
			}
		}
		reader.close();
		frequentItems = new HashSet<Set<Integer>>();
		frequentSingleItems = new HashSet<Integer>();
		Iterator<Set<Integer>> ite = map.keySet().iterator();
		while(ite.hasNext()){
			Set<Integer> key = ite.next();
			int value = map.get(key);
			if(value<totalCount*minSupport){
				ite.remove();
			}
			else{
				frequentItems.add(key);
				for(int item: key){
					frequentSingleItems.add(item);
				}
			}
		}
		return map;
	}

	private List<Set<Integer>> generateSubset(String[] array, int k) throws Exception{
		List<Set<Integer>> result = new ArrayList<Set<Integer>>();
		int[] intArray = new int[array.length];
		for(int i=0; i<array.length;i++){
			intArray[i] = Integer.parseInt(array[i]);
		}
		int[] newArray = filterItems(intArray);
		List<Set<Integer>> list = generateSubSets(newArray,k);
		for(Set<Integer> set: list){
			//将set变成数组
			int smallArray[] = new int[set.size()];
			int i=0;
			Iterator<Integer> ite = set.iterator();
			while(ite.hasNext()){
				smallArray[i]= ite.next();
				i++;
			}
			//找出set的所有k-1次subItemSet
			List<Set<Integer>> smallList = generateSubSets(smallArray, k-1);
			//如果有1个subItemSet不是频繁的,则判断set不是频繁的
			boolean flag = true;
			for(Set<Integer> item: smallList){
				if(!frequentItems.contains(item)){
					flag = false;
					break;
				}
			}
			if(flag){
				result.add(set);
			}
		}
		return result;
	}

	public void printFrequentItems(Map<Set<Integer>,Integer> itemSets, int i) throws FileNotFoundException, IOException {
		if(bw == null){
			bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream("output.txt")));
		}
		StringBuffer sb = new StringBuffer();
		for (Set<Integer> set : itemSets.keySet()) {
			sb.append("[");
			for (Integer str : set) {
				sb.append(str + " ");
			}
			sb.append("], count:");
					sb.append(itemSets.get(set));
					sb.append("\n");
		}
		bw.write(sb.toString());
		bw.flush();
	}

	/**
	* 过滤掉不在前一次频繁单项集里的item
	* @param array
	* @return
	*/
	private int[] filterItems(int[] array){
		List<Integer> list = new ArrayList<Integer>();
		for(int i=0;i<array.length;i++){
			if(frequentSingleItems.contains(array[i])){
				list.add(array[i]);
			}
		}
		int[] newArray = new int[list.size()];
		for(int i=0;i<newArray.length;i++){
			newArray[i] = list.get(i);
		}
		return newArray;
	}

	public void closeOutputWriter() throws IOException{
		if(bw == null){
			bw.close();
		}
	}

	private List<Set<Integer>> generateSubSets(int[] array, int k){
		List<Set<Integer>> list = new ArrayList<Set<Integer>>();
		if(array.length<k){
			return list;
		}
		// 初始化移位法需要的数组
		byte[] bits = new byte[array.length];
		for (int i = 0; i < bits.length; i++) {
			bits[i] = i < k ? (byte) 1 : (byte) 0;
		}
		boolean find = false;
		do {
			// 找到10,换成01
			Set<Integer> set = getCombination(array, bits);
			if(set!=null && set.size()!=0){
				list.add(set);
			}
			find = false;
			for (int i = 0; i < array.length - 1; i++) {
				if (bits[i] == 1 && bits[i+1] == 0) {
					find = true;
					bits[i] = 0;
					bits[i+1] = 1;
					if(bits[0] == 0){
						for (int p=0, q=0; p < i; p++){
							if(bits[p] == 1){
								byte temp = bits[p];
								bits[p] = bits[q];
								bits[q] = temp;
								q++;
							}
						}
					}
					break;
				}
			}

		} while (find);
		return list;
	}

	private Set<Integer> getCombination(int[] array, byte[] bits) {
		Set<Integer> set = new TreeSet<Integer>();
		for (int i = 0; i < bits.length; i++) {
			if (bits[i] == (byte) 1) {
				set.add(array[i]);
			}
		}
		return set;
	}

}



import java.util.Map;
import java.util.Set;

public class Main {

	public static void main(String[] args) {
		System.out.println("program starts…");
		long startTime = System.currentTimeMillis();
		String inputFile = "src/test.txt";
		double minSupport = 0.02;
		Apriori apriori = new Apriori(inputFile, minSupport);
		try {
			System.out.println("pass 1");
			Map<Set<Integer>, Integer> f1Set = apriori.findF1Item();
			apriori.printFrequentItems(f1Set, 1);
			Map<Set<Integer>, Integer> result = f1Set;
			int i = 2;
			do {
				System.out.println("pass " + i);
				result = apriori.generateNextPass(i);
				apriori.printFrequentItems(result, i);
				i++;
			} while (result.size() != 0);
			apriori.closeOutputWriter();
		} catch (Exception e) {
			e.printStackTrace();
		}
		long endTime = System.currentTimeMillis();
		System.out.println("execution time:" + (endTime - startTime) + "ms");
	}
}

输入文件:test.txt

1 2 5
2 4
2 3
1 2 4
1 3
2 3
1 3
1 2 3 5
1 2 3

输出文件:output.txt

[1 ], count:6
[2 ], count:7
[3 ], count:6
[4 ], count:2
[5 ], count:2
[1 2 ], count:4
[1 3 ], count:4
[1 4 ], count:1
[2 3 ], count:4
[2 4 ], count:2
[1 5 ], count:2
[2 5 ], count:2
[3 5 ], count:1
[1 2 3 ], count:2
[1 2 4 ], count:1
[1 2 5 ], count:2
[1 3 5 ], count:1
[2 3 5 ], count:1
[1 2 3 5 ], count:1

使用jquery和spring实现图片上传功能

步骤如下:

(1)在html中定义1个form表单

<form id=”picForm” name=”picForm” action=”uploadPic.htm” enctype=”multipart/form-data” method=”post”>
    <table>
        <tr>
            <td>
                <label>图片:</label>
            </td>
            <td>
                <input id="picUpload" name="picUpload" type="file">
            </td>
        </tr>
        <tr>
            <td colspan=”2″>
            <input type="button" onclick="uploadPic();" value="上传">
            </td>
        </tr>
    </table>
</form>

(2)定义js函数

function uploadPic(){
    if($("#picUpload").val()==''){
        alert("please select picture first!");
        return;
    }
    var data = new FormData();
    data.append('file', document.picForm.picUpload.files[0]);
    $.ajax({
        type: "POST",
        url: "uploadPic.htm",
        processData:false,
        contentType:false,
        data: data,
        success: function (data) {
            var obj = jQuery.parseJSON(data);
            var result = obj.result;
            if(result !="1"){
                alert("Uploade file fail");
            }
            else{
                alert("Uploade file succeed");

            }

        }
    });
}

注意上面的document.picForm.picUpload.files[0],这里,javascript是通过name属性来找元素的。

(3)服务器端配置

在spring的配置文件里加入以下内容。

<bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
    <!-- one of the properties available; the maximum file size in bytes -->
    <property name="maxUploadSize" value="1000000"/>
</bean>

(4)服务器端代码

@Controller
public class MyController {

@RequestMapping("/uploadPic.htm")
@ResponseBody
public void uploadPic(ModelMap modelMap, HttpServletRequest req, HttpServletResponse res)
throws Exception {
      JSONObject json = new JSONObject();
      InputStream input = null;
      OutputStream output = null;
      try{
          MultipartHttpServletRequest multipartReq = (MultipartHttpServletRequest) req;
          MultipartFile rawFile = multipartReq.getFile("file");
          String rawFilename = rawFile.getOriginalFilename();
          String saveFileName = "d:\\"+rawFilename;
          input = rawFile.getInputStream();
          output = new BufferedOutputStream(new FileOutputStream(saveFileName));
          byte[] buffer = new byte[1024];
          int n = -1;
          while ((n = input.read(buffer)) != -1) {
              output.write(buffer, 0, n);
          }
          json.put("result", 1);
          res.setContentType("text/html;charset=utf-8");
          res.getWriter().write(json.toString());
      }
      catch(Exception e){
          json.put("result", 0);
          res.setContentType("text/html;charset=utf-8");
          res.getWriter().write(json.toString());
          return;
      }
      finally{
          input.close();
          output.close();
      }
   }
}

Hibernate中使用addEntity方法解决使用SQL查询时对象映射问题

我们在使用hibernate时,有时遇到比较复杂的查询,不好写HQL语句,只能写SQL。但是这时查询出来的是Object对象,我们需要将这些Ojbect对象转换成我们想要的域对象。

如果我们已经在hbm.xml文件里定义过表和领域类的映射关系,这时就很简单,只要加上addEntity方法就可以解决,如下。为了简化,下面的sql语句比较简单。

String sql = "select s.* from software s";
Query query = getHibernateTemplate().getSessionFactory().openSession().createSQLQuery(sql).addEntity(Software.class);
return (List<Software>)query.list();

需要强调的是:必须已经定义过software表和Software类的映射关系。

比如我已经在software.hbm.xml有如下定义了:

<hibernate-mapping>
    <class name="com.lixiaodong.Software" table="software" catalog="lixiaodong">
        <id name="id" type="int">
            <column name="id" />
            <generator class="identity" />
        </id>
        <property name="name" type="string">
            <column name="name" length="50" />
        </property>
    </class>
</hibernate-mapping>

2PMMS(2-phase multiway merge sort)算法java实现

2PMMS(2-phase multiway merge sort),顾名思义,由2个阶段组成。第一个阶段将源文件分批次读到内存中,采用某种内部排序算法进行排序,然后将每次排序后的结果写到硬盘上,1个批次对应1个文件;第二个阶段将n个文件批量读到缓存里,从N个缓存序列里找到最小数,放到输出缓存里。每次放完后,要检查每个输入缓存是否读完,如果读完则从对应的文件里继续读数据到缓存,直到文件全部读完为止;并检查输出缓存是否已满,如满则输出到文件并清空输出缓存。

4个类:

(1)

public class Main {

/*
* 2 arguments
* first argument: inputfile ex: d:\\data\\input.dat
* second argument: output folder ex: d:\\data
*/
public static void main(String[] args) throws Exception {
    if(args.length!=2){
        System.out.println("arguments are not valid");
        return;
    }
    long startTime = System.currentTimeMillis();
    //read data, sort, output sorted data to files
    PhaseOne phaseOne = new PhaseOne();
    List<String> fileNameList = phaseOne.generateSortedFile(args[0], args[1]);
    //merge
    PhaseTwo phaseTwo = new PhaseTwo();
    phaseTwo.merge(fileNameList, args[1]);
    //delete temp files
    for(String fileName: fileNameList){
        new File(fileName).delete();
    }
    long finishTime = System.currentTimeMillis();
    double totalTime = (double)(finishTime - startTime) / 1000;
    System.out.println("total time: " + totalTime + " seconds");
}

}

(2)

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class PhaseOne {

private final int amountOnce = 550000;

public List<String> generateSortedFile(String inputFile, String outputFolder) {
    List<String> fileNameList = new ArrayList<String>();
    BufferedReader reader = null;
    try {
        reader = new BufferedReader(new FileReader(inputFile));
        String tempString = null;
        int line = 0;
        int[] array = new int[amountOnce];
        while ((tempString = reader.readLine()) != null) {
        //System.out.println("line " + line + ": " + tempString);
        array[line % amountOnce] = Integer.parseInt(tempString);
        if(line % amountOnce == amountOnce -1){
            String fileNO = (line/amountOnce)<10 ? "0"+ line/amountOnce : String.valueOf(line/amountOnce);
            String fileName = outputFolder+ "/tmp"+ fileNO+".dat";
            writeData(array, line%amountOnce+1, fileName,fileNameList);
        }
        line++;
    }
    String fileNO = (line/amountOnce)<10 ? "0"+ line/amountOnce : String.valueOf(line/amountOnce);
    String fileName = outputFolder+ "/tmp"+ fileNO+".dat";
    writeData(array, line%amountOnce, fileName, fileNameList);
    reader.close();
} catch (IOException e) {
    e.printStackTrace();
} finally {
    if (reader != null) {
    try {
        reader.close();
    }
    catch (IOException e1) {
    }
}
return fileNameList;
}
}

private void writeData(int[] array, int length, String fileName, List<String> fileNames) throws IOException{
    Quicksort.quickSort(array, length);
    BufferedWriter output = new BufferedWriter(new FileWriter(fileName));
    for(int i=0;i<length;i++){
        output.write(String.valueOf(array[i]));
        output.write("\r\n");
    }
    output.close();
    fileNames.add(fileName);
    array = new int[amountOnce];
}
}

(3)

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.util.ArrayList;
import java.util.List;

public class PhaseTwo {

public static int inputBufferSize = 30000;

public static int outputBufferSize = 100000;

public static boolean finishFlag = false;

public void merge(List<String> fileNameList, String outputFolder) {
    int[] outputBuffer = new int[outputBufferSize];
    int[][] inputBuffers = new int[fileNameList.size()][inputBufferSize];
    List<BufferedReader> readers = new ArrayList<BufferedReader>();
    try {
        for(int i=0;i<fileNameList.size();i++){
            BufferedReader reader = new BufferedReader(new FileReader(fileNameList.get(i)));
            readers.add(reader);
            String tempString = null;
            int line = 0;
            while ((tempString = readers.get(i).readLine()) != null) {
            inputBuffers[i][line] = Integer.parseInt(tempString);
            line++;
            if(line >= inputBufferSize){
                break;
            }
        }
    }


    int[] pointers = new int[fileNameList.size()];

    int outputBufferIndex = 0;
    while(true){
        int min = findMin(inputBuffers,pointers, readers);
        outputBuffer[outputBufferIndex % outputBufferSize] = min;
        if(outputBufferIndex % outputBufferSize == outputBufferSize-1){
            outputData(outputBuffer, outputBufferSize, outputFolder+"/result.dat");
        }
        outputBufferIndex++;
        if(finishFlag){
            break;
        }
    }
    outputData(outputBuffer, outputBufferIndex%outputBufferSize, outputFolder+"/result.dat");

    for(int i=0;i<fileNameList.size();i++){
        readers.get(i).close();
    }
    System.out.println("------------------");

} catch (IOException e) {
    e.printStackTrace();
} finally {
    for(int i=0;i<fileNameList.size();i++){
    if (readers.get(i) != null) {
    try {
        readers.get(i).close();
    }
    catch (IOException e1) {
    }
}
}
}
}

private int findMin(int[][] arrays, int[] pointers, List<BufferedReader> readers) throws IOException{
    int min = 0;
    int index = 0;
    for(int i=0;i<arrays.length;i++){
        if(arrays[i][pointers[i]]>0){
            min = arrays[i][pointers[i]];
            index = i;
            break;
        }
    }

    for(int i=index+1;i<arrays.length;i++){
        if(arrays[i][pointers[i]]<=0){
            continue;
        }
        if(arrays[i][pointers[i]] < min){
            min = arrays[i][pointers[i]];
            index = i;
        }
    }
    pointers[index] = pointers[index]+1;

    for(int i=0;i<pointers.length;i++){
        if(pointers[i]>=inputBufferSize){
            arrays[i] = new int[inputBufferSize];
            String tempString = null;
            int line = 0;
            while ((tempString = readers.get(i).readLine()) != null) {
                arrays[i][line] = Integer.parseInt(tempString);
                line++;
                if(line >= inputBufferSize){
                    break;
                }
            }
            pointers[i]= 0;
        }
    } 

    //update finishFlag
    if(!finishFlag){
        boolean flag = true;
        for(int i=0;i<arrays.length;i++){
        if(arrays[i][pointers[i]]>0){
            flag = false;
            break;
        }
    }
    finishFlag = flag;
}
return min;
}

private void outputData(int[] outputBuffer, int length, String outputFile) throws IOException{
BufferedWriter output = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(outputFile, true)));
for(int i=0;i<length;i++){
    StringBuffer sb = new StringBuffer();
    sb.append(outputBuffer[i]);
    sb.append("\r\n");
    output.write(sb.toString());
}
output.close();
outputBuffer = new int[outputBufferSize];
}

}

(4)

public class Quicksort {
    public static void quickSort(int[] data, int length) {
        recQuickSort(data, 0, length-1);
    }

private static void recQuickSort(int[] data,int left, int right) {
    if(right-left <= 0){
    return;
}
else {
    long pivot = data[right];
    int partition = partitionIt(data, left, right, pivot);
    recQuickSort(data, left, partition-1);
    recQuickSort(data, partition+1, right);
}
}

private static int partitionIt(int[] data, int left, int right, long pivot) {
    int leftPtr = left-1;
    int rightPtr = right;
    while(true) {
        while(data[++leftPtr] < pivot )
        ;
        while(rightPtr > 0 && data[--rightPtr] > pivot)
        ;
        if(leftPtr >= rightPtr){
        break;
    }
    else{
        swap(data, leftPtr, rightPtr);
    }
}
swap(data, leftPtr, right);
    return leftPtr;
}

private static void swap(int[] data, int dex1, int dex2) {
    int temp = data[dex1];
    data[dex1] = data[dex2];
    data[dex2] = temp;
}
}