org.sfj

Class ReplacementDiskSort<E extends ReplacementDiskSort.Element>

java.lang.Object

org.sfj.ReplacementDiskSort<E>

Type Parameters:

E - element subclass

public class ReplacementDiskSort<E extends ReplacementDiskSort.Element>
extends Object

External merge sort implementation. Mostly as Knuth describes, at least the run generation phase. Uses the replacement selection method to generate runs of sorted elements from an unsorted source N at a time, then merges them M at a time.

The input and output is delegated to a pair of interfaces, ExternalIterator, and ExternalAppender. You actually use the class by providing callbacks that make arbitrary iterators and appenders, which are then used read and write your subclass of Element. SO if you have binary ints, fine. Strings, fine. Just provide the appender/iterators to make it happen.

Author:

cschanck

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ReplacementDiskSort.AppenderMaker<EE extends ReplacementDiskSort.Element> Make an appender for a specific file.
static class	ReplacementDiskSort.Element This is the class you need to subclass; you will provide file iterators (readers) and file appenders for your subclass.
static interface	ReplacementDiskSort.ExternalAppender<EE extends ReplacementDiskSort.Element> An external appender.
static interface	ReplacementDiskSort.ExternalIterator<EE extends ReplacementDiskSort.Element> An external iterator.
static interface	ReplacementDiskSort.IterMaker<EE extends ReplacementDiskSort.Element> Make an iterator for a specific file.
static class	ReplacementDiskSort.PassInfo

Field Summary

Fields

Modifier and Type	Field and Description
protected File	workDirectory

Constructor Summary

Constructors

Constructor and Description
ReplacementDiskSort(ReplacementDiskSort.IterMaker<E> iteratorMaker, ReplacementDiskSort.AppenderMaker<E> appenderMaker, Comparator<E> comp, boolean deleteFiles) Constructor.

Method Summary

Modifier and Type	Method and Description
List<ReplacementDiskSort.PassInfo>	getPassInfo()
protected List<File>	makeRuns(File src, int maxElementsForRuns)
protected File	mergePass(int pass, List<File> inputFiles, File dest)
void	run(File src, int maxElementsForRuns, int maxElementsForMerges, File dest, File workingDirectory)
static <E extends ReplacementDiskSort.Element> ReplacementDiskSort<E>	runOnce(ReplacementDiskSort.IterMaker<E> iteratorMaker, ReplacementDiskSort.AppenderMaker<E> appenderMaker, Comparator<E> comp, boolean deleteFiles, File src, int maxElementsForRuns, int maxElementsForMerges, File dest, File workDir) Single method to do a single run.
ReplacementDiskSort<E>	setVerbose(PrintStream verbose)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

workDirectory

protected File workDirectory

Constructor Detail

ReplacementDiskSort

public ReplacementDiskSort(ReplacementDiskSort.IterMaker<E> iteratorMaker,
                           ReplacementDiskSort.AppenderMaker<E> appenderMaker,
                           Comparator<E> comp,
                           boolean deleteFiles)

Constructor.

Parameters:

iteratorMaker - maker for iterator.

appenderMaker - maker for appender

comp - Comparator for your subclass of Element

deleteFiles - delete the intermediate files as you go

Method Detail

runOnce

public static <E extends ReplacementDiskSort.Element> ReplacementDiskSort<E> runOnce(ReplacementDiskSort.IterMaker<E> iteratorMaker,
                                                                                     ReplacementDiskSort.AppenderMaker<E> appenderMaker,
                                                                                     Comparator<E> comp,
                                                                                     boolean deleteFiles,
                                                                                     File src,
                                                                                     int maxElementsForRuns,
                                                                                     int maxElementsForMerges,
                                                                                     File dest,
                                                                                     File workDir)
                                                                              throws IOException

Single method to do a single run.

Type Parameters:

E - Element subclass

Parameters:

iteratorMaker - maker for iterator.

appenderMaker - maker for appender

comp - Comparator for your subclass of Element

deleteFiles - delete the intermediate files as you go

src - source file

maxElementsForRuns - max number of elements to have in memory for generating initial run files. This controls how much memory the sort takes, as it includes maxElementsForRuns element overhead, plus 1X the buffering needed for an appender, and 1X the buffering needed for an iterator.

maxElementsForMerges - max number of elements to have in memory for merge passes. This controls how much memory the sort takes, as it includes maxElementsForRuns element overhead, plus 1X the buffering needed for an iterator, and maxElementsForMerges X the buffering needed for an iterator. So the merge pass, depending on the buffering you do in your appender, can have a bigger footprint if you are not careful.

dest - destination file. Cannot be the same as source file.

workDir - working directory

Returns:

the disk sort object used for the sort

Throws:

IOException - on exception

setVerbose

public ReplacementDiskSort<E> setVerbose(PrintStream verbose)

run

public void run(File src,
                int maxElementsForRuns,
                int maxElementsForMerges,
                File dest,
                File workingDirectory)
         throws IOException

Throws:

IOException

makeRuns

protected List<File> makeRuns(File src,
                              int maxElementsForRuns)
                       throws IOException

Throws:

IOException

getPassInfo

public List<ReplacementDiskSort.PassInfo> getPassInfo()

mergePass

protected File mergePass(int pass,
                         List<File> inputFiles,
                         File dest)
                  throws IOException

Throws:

IOException