Module me.lemire.integercompression
Package me.lemire.integercompression

Class BinaryPacking

java.lang.Object
me.lemire.integercompression.BinaryPacking
All Implemented Interfaces:
IntegerCODEC, SkippableIntegerCODEC
public final class BinaryPacking extends Object implements IntegerCODEC, SkippableIntegerCODEC
Scheme based on a commonly used idea: can be extremely fast. It encodes integers in blocks of 32 integers. For arrays containing an arbitrary number of integers, you should use it in conjunction with another CODEC: 
IntegerCODEC ic = 
  new Composition(new BinaryPacking(), new VariableByte()).
Note that this does not use differential coding: if you are working on sorted lists, use IntegratedBinaryPacking instead.

For details, please see

Daniel Lemire and Leonid Boytsov, Decoding billions of integers per second through vectorization Software: Practice & Experience http://onlinelibrary.wiley.com/doi/10.1002/spe.2203/abstract http://arxiv.org/abs/1209.2137

Daniel Lemire, Leonid Boytsov, Nathan Kurz, SIMD Compression and the Intersection of Sorted Integers http://arxiv.org/abs/1401.6399

Author:
Daniel Lemire

  • Field Details

  • Constructor Details

    • BinaryPacking

      public BinaryPacking()

  • Method Details

    • compress

      public void compress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos)
      Description copied from interface: IntegerCODEC
      Compress data from an array to another array. Both inpos and outpos are modified to represent how much data was read and written to. If 12 ints (inlength = 12) are compressed to 3 ints, then inpos will be incremented by 12 while outpos will be incremented by 3. We use IntWrapper to pass the values by reference.
      Specified by:
      compress in interface IntegerCODEC
      Parameters:
      in - input array
      inpos - where to start reading in the array
      inlength - how many integers to compress
      out - output array
      outpos - where to write in the output array

    • headlessCompress

      public void headlessCompress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos)
      Description copied from interface: SkippableIntegerCODEC
      Compress data from an array to another array. Both inpos and outpos are modified to represent how much data was read and written to. If 12 ints (inlength = 12) are compressed to 3 ints, then inpos will be incremented by 12 while outpos will be incremented by 3. We use IntWrapper to pass the values by reference. Implementation note: contrary to IntegerCODEC.compress(int[], me.lemire.integercompression.IntWrapper, int, int[], me.lemire.integercompression.IntWrapper), this may skip writing information about the number of encoded integers.
      Specified by:
      headlessCompress in interface SkippableIntegerCODEC
      Parameters:
      in - input array
      inpos - where to start reading in the array
      inlength - how many integers to compress
      out - output array
      outpos - where to write in the output array

    • uncompress

      public void uncompress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos)
      Description copied from interface: IntegerCODEC
      Uncompress data from an array to another array. Both inpos and outpos parameters are modified to indicate new positions after read/write.
      Specified by:
      uncompress in interface IntegerCODEC
      Parameters:
      in - array containing data in compressed form
      inpos - where to start reading in the array
      inlength - length of the compressed data (ignored by some schemes)
      out - array where to write the compressed output
      outpos - where to start writing the uncompressed output in out

    • headlessUncompress

      public void headlessUncompress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos, int num)
      Description copied from interface: SkippableIntegerCODEC
      Uncompress data from an array to another array. Both inpos and outpos parameters are modified to indicate new positions after read/write.
      Specified by:
      headlessUncompress in interface SkippableIntegerCODEC
      Parameters:
      in - array containing data in compressed form
      inpos - where to start reading in the array
      inlength - length of the compressed data (ignored by some schemes)
      out - array where to write the uncompressed output
      outpos - where to start writing the uncompressed output in out
      num - number of integers we want to decode. May be less than the actual number of compressed integers

    • maxHeadlessCompressedLength

      public int maxHeadlessCompressedLength(IntWrapper compressedPositions, int inlength)
      Description copied from interface: SkippableIntegerCODEC
      Compute the maximum number of integers that might be required to store the compressed form of a given input array segment, without headers.

      This is useful to pre-allocate the output buffer before calling SkippableIntegerCODEC.headlessCompress(int[], IntWrapper, int, int[], IntWrapper).

      Specified by:
      maxHeadlessCompressedLength in interface SkippableIntegerCODEC
      Parameters:
      compressedPositions - since not all schemes compress every input integer, this parameter returns how many input integers will actually be compressed. This is useful when composing multiple schemes.
      inlength - number of integers to be compressed
      Returns:
      the maximum number of integers needed in the output array

    • toString

      public String toString()
      Overrides:
      toString in class Object