Class Zstd
- java.lang.Object
-
- org.lwjgl.util.zstd.Zstd
-
public class Zstd extends java.lang.Object
Native bindings to Zstandard (zstd), a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios.Introduction
zstd, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. The zstd compression library provides in-memory compression and decompression functions.
The library supports regular compression levels from 1 up to
maxCLevel
, which is currently 22. Levels ≥ 20, labeled--ultra
, should be used with caution, as they require more memory. The library also offers negative compression levels, which extend the range of speed vs. ratio preferences. The lower the level, the faster the speed (at the cost of compression).Compression can be done in:
- a single step (described as Simple API)
- a single step, reusing a context (described as Explicit context)
- unbounded multiple steps (described as Streaming compression)
The compression ratio achievable on small data can be highly improved using a dictionary. Dictionary compression can be performed in:
- a single step (described as Simple dictionary API)
- a single step, reusing a dictionary (described as Bulk-processing dictionary API)
Advanced experimental functions can be accessed using
#define ZSTD_STATIC_LINKING_ONLY
before includingzstd.h
. Advanced experimental APIs should never be used with a dynamically-linked library. They are not "stable", their definitions or signatures may change in the future. Only static linking is allowed.Streaming compression - HowTo
A
ZSTD_CStream
object is required to track streaming operation.Use
createCStream
andfreeCStream
to create/release resources.ZSTD_CStream
objects can be reused multiple times on consecutive compression operations. It is recommended to re-useZSTD_CStream
since it will play nicer with system's memory, by re-using already allocated memory.For parallel execution, use one separate
ZSTD_CStream
.Since v1.3.0,
ZSTD_CStream
andZSTD_CCtx
are the same thing.Parameters are sticky: when starting a new compression on the same context, it will re-use the same sticky parameters as previous compression session. When in doubt, it's recommended to fully initialize the context before usage. Use
CCtx_reset
to reset the context andCCtx_setParameter
,CCtx_setPledgedSrcSize
, orCCtx_loadDictionary
and friends to set more specific parameters, the pledged source size, or load a dictionary.Use
compressStream2
withe_continue
as many times as necessary to consume input stream. The function will automatically update bothpos
fields withininput
andoutput
. Note that the function may not consume the entire input, for example, because the output buffer is already full, in which caseinput.pos < input.size
. The caller must check if input has been entirely consumed. If not, the caller must make some room to receive more compressed data, and then present again remaining input data.Note:
ZSTD_e_continue
is guaranteed to make some forward progress when called, but doesn't guarantee maximal forward progress. This is especially relevant when compressing with multiple threads. The call won't block if it can consume some input, but if it can't it will wait for some, but not all, output to be flushed.At any moment, it's possible to flush whatever data might remain stuck within internal buffer, using
ZSTD_compressStream2()
withe_flush
.output->pos
will be updated. Note that, ifoutput->size
is too small, a single invocation withZSTD_e_flush
might not be enough (return code > 0). In which case, make some room to receive more compressed data, and call againZSTD_compressStream2()
withZSTD_e_flush
. You must continue callingZSTD_compressStream2()
withZSTD_e_flush
until it returns 0, at which point you can change the operation.Note:
ZSTD_e_flush
will flush as much output as possible, meaning when compressing with multiple threads, it will block until the flush is complete or the output buffer is full.Calling
ZSTD_compressStream2()
withe_end
instructs to finish a frame. It will perform a flush and write frame epilogue. The epilogue is required for decoders to consider a frame completed. Flush operation is the same, and follows same rules as callingZSTD_compressStream2()
withZSTD_e_flush
. You must continue callingZSTD_compressStream2()
withZSTD_e_end
until it returns 0, at which point you are free to start a new frame.Note:
ZSTD_e_end
will flush as much output as possible, meaning when compressing with multiple threads, it will block until the flush is complete or the output buffer is full.Streaming decompression - HowTo
A
ZSTD_DStream
object is required to track streaming operations.Use
createDStream
andfreeDStream
to create/release resources.ZSTD_DStream
objects can be re-used multiple times.Use
DCtx_reset
andDCtx_refDDict
to start a new decompression operation. Alternatively, use advanced API to set specific properties.Use
decompressStream
repetitively to consume your input. The function will update bothpos
fields. Ifinput.pos < input.size
, some input has not been consumed. It's up to the caller to present again remaining data. The function tries to flush all data decoded immediately, respecting output buffer size. Ifoutput.pos < output.size
, decoder has flushed everything it could. But ifoutput.pos == output.size
, there might be some data left within internal buffers. In which case, callZSTD_decompressStream()
again to flush whatever remains in the buffer.Note: with no additional input provided, amount of data flushed is necessarily ≤
BLOCKSIZE_MAX
.
-
-
Field Summary
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static long
nZSTD_CCtx_loadDictionary(long cctx, long dict, long dictSize)
Unsafe version of:CCtx_loadDictionary
static long
nZSTD_CCtx_refCDict(long cctx, long cdict)
Unsafe version of:CCtx_refCDict
static long
nZSTD_CCtx_refPrefix(long cctx, long prefix, long prefixSize)
Unsafe version of:CCtx_refPrefix
static long
nZSTD_CCtx_reset(long cctx, int reset)
Unsafe version of:CCtx_reset
static long
nZSTD_CCtx_setParameter(long cctx, int param, int value)
Unsafe version of:CCtx_setParameter
static long
nZSTD_CCtx_setPledgedSrcSize(long cctx, long pledgedSrcSize)
Unsafe version of:CCtx_setPledgedSrcSize
static long
nZSTD_compress(long dst, long dstCapacity, long src, long srcSize, int compressionLevel)
Unsafe version of:compress
static long
nZSTD_compress_usingCDict(long cctx, long dst, long dstCapacity, long src, long srcSize, long cdict)
Unsafe version of:compress_usingCDict
static long
nZSTD_compress_usingDict(long ctx, long dst, long dstCapacity, long src, long srcSize, long dict, long dictSize, int compressionLevel)
Unsafe version of:compress_usingDict
static long
nZSTD_compress2(long cctx, long dst, long dstCapacity, long src, long srcSize)
Unsafe version of:compress2
static long
nZSTD_compressCCtx(long ctx, long dst, long dstCapacity, long src, long srcSize, int compressionLevel)
Unsafe version of:compressCCtx
static long
nZSTD_compressStream2(long cctx, long output, long input, int endOp)
Unsafe version of:compressStream2
static void
nZSTD_cParam_getBounds(int cParam, long __result)
Unsafe version of:cParam_getBounds
static long
nZSTD_createCDict(long dictBuffer, long dictSize, int compressionLevel)
Unsafe version of:createCDict
static long
nZSTD_createDDict(long dictBuffer, long dictSize)
Unsafe version of:createDDict
static long
nZSTD_DCtx_loadDictionary(long dctx, long dict, long dictSize)
Unsafe version of:DCtx_loadDictionary
static long
nZSTD_DCtx_refDDict(long dctx, long ddict)
Unsafe version of:DCtx_refDDict
static long
nZSTD_DCtx_refPrefix(long dctx, long prefix, long prefixSize)
Unsafe version of:DCtx_refPrefix
static long
nZSTD_DCtx_reset(long dctx, int reset)
Unsafe version of:DCtx_reset
static long
nZSTD_DCtx_setParameter(long dctx, int param, int value)
Unsafe version of:DCtx_setParameter
static long
nZSTD_decompress(long dst, long dstCapacity, long src, long compressedSize)
Unsafe version of:decompress
static long
nZSTD_decompress_usingDDict(long dctx, long dst, long dstCapacity, long src, long srcSize, long ddict)
Unsafe version of:decompress_usingDDict
static long
nZSTD_decompress_usingDict(long dctx, long dst, long dstCapacity, long src, long srcSize, long dict, long dictSize)
Unsafe version of:decompress_usingDict
static long
nZSTD_decompressDCtx(long ctx, long dst, long dstCapacity, long src, long srcSize)
Unsafe version of:decompressDCtx
static long
nZSTD_decompressStream(long zds, long output, long input)
Unsafe version of:decompressStream
static void
nZSTD_dParam_getBounds(int dParam, long __result)
Unsafe version of:dParam_getBounds
static long
nZSTD_findFrameCompressedSize(long src, long srcSize)
Unsafe version of:findFrameCompressedSize
static long
nZSTD_freeCCtx(long cctx)
Unsafe version of:freeCCtx
static long
nZSTD_freeCDict(long CDict)
Unsafe version of:freeCDict
static long
nZSTD_freeCStream(long zcs)
Unsafe version of:freeCStream
static long
nZSTD_freeDCtx(long dctx)
Unsafe version of:freeDCtx
static long
nZSTD_freeDDict(long ddict)
Unsafe version of:freeDDict
static long
nZSTD_freeDStream(long zds)
Unsafe version of:freeDStream
static int
nZSTD_getDictID_fromDDict(long ddict)
Unsafe version of:getDictID_fromDDict
static int
nZSTD_getDictID_fromDict(long dict, long dictSize)
Unsafe version of:getDictID_fromDict
static int
nZSTD_getDictID_fromFrame(long src, long srcSize)
Unsafe version of:getDictID_fromFrame
static long
nZSTD_getErrorName(long code)
Unsafe version of:getErrorName
static long
nZSTD_getFrameContentSize(long src, long srcSize)
Unsafe version of:getFrameContentSize
static int
nZSTD_isError(long code)
Unsafe version of:isError
static long
nZSTD_sizeof_CCtx(long cctx)
static long
nZSTD_sizeof_CDict(long cdict)
static long
nZSTD_sizeof_CStream(long zcs)
static long
nZSTD_sizeof_DCtx(long dctx)
static long
nZSTD_sizeof_DDict(long ddict)
static long
nZSTD_sizeof_DStream(long zds)
static long
nZSTD_versionString()
Unsafe version of:versionString
static long
ZSTD_CCtx_loadDictionary(long cctx, java.nio.ByteBuffer dict)
Creates an internalCDict
fromdict
buffer.static long
ZSTD_CCtx_refCDict(long cctx, long cdict)
References a prepared dictionary, to be used for all next compressed frames.static long
ZSTD_CCtx_refPrefix(long cctx, java.nio.ByteBuffer prefix)
References a prefix (single-usage dictionary) for next compressed frame.static long
ZSTD_CCtx_reset(long cctx, int reset)
There are 2 different things that can be reset, independently or jointly : The session: will stop compressing current frame, and makeCCtx
ready to start a new one.static long
ZSTD_CCtx_setParameter(long cctx, int param, int value)
Set one compression parameter, selected by enumZSTD_cParameter
.static long
ZSTD_CCtx_setPledgedSrcSize(long cctx, long pledgedSrcSize)
Total input data size to be compressed as a single frame.static long
ZSTD_compress(java.nio.ByteBuffer dst, java.nio.ByteBuffer src, int compressionLevel)
Compressessrc
content as a single zstd compressed frame into already allocateddst
.static long
ZSTD_compress_usingCDict(long cctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, long cdict)
Compression using a digested Dictionary.static long
ZSTD_compress_usingDict(long ctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, java.nio.ByteBuffer dict, int compressionLevel)
Compression at an explicit compression level using a Dictionary.static long
ZSTD_compress2(long cctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src)
Behaves the same ascompressCCtx
, but compression parameters are set using the advanced API.static long
ZSTD_compressBound(long srcSize)
Returns the maximum compressed size in worst case single-pass scenario.static long
ZSTD_COMPRESSBOUND(long srcSize)
Pure Java version ofZSTD_compressBound(long)
.static long
ZSTD_compressCCtx(long ctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, int compressionLevel)
Same ascompress
, using an explicitZSTD_CCtx
.static long
ZSTD_compressStream2(long cctx, ZSTDOutBuffer output, ZSTDInBuffer input, int endOp)
Behaves about the same asZSTD_compressStream()
, with additional control on end directive.static ZSTDBounds
ZSTD_cParam_getBounds(int cParam, ZSTDBounds __result)
All parameters must belong to an interval with lower and upper bounds, otherwise they will either trigger an error or be automatically clamped.static long
ZSTD_createCCtx()
Creates a compression context.static long
ZSTD_createCDict(java.nio.ByteBuffer dictBuffer, int compressionLevel)
When compressing multiple messages / blocks using the same dictionary, it's recommended to load it only once.static long
ZSTD_createCStream()
AZSTD_CStream
object is required to track streaming operation.static long
ZSTD_createDCtx()
Creates a decompression context.static long
ZSTD_createDDict(java.nio.ByteBuffer dictBuffer)
Creates a digested dictionary, ready to start decompression operation without startup delay.static long
ZSTD_createDStream()
AZSTD_DStream
object is required to track streaming operations.static long
ZSTD_CStreamInSize()
Returns the recommended size for input buffer.static long
ZSTD_CStreamOutSize()
Returns the recommended size for output buffer.static long
ZSTD_DCtx_loadDictionary(long dctx, java.nio.ByteBuffer dict)
Create an internalDDict
fromdict
buffer, to be used to decompress next frames.static long
ZSTD_DCtx_refDDict(long dctx, long ddict)
References a prepared dictionary, to be used to decompress next frames.static long
ZSTD_DCtx_refPrefix(long dctx, java.nio.ByteBuffer prefix)
References a prefix (single-usage dictionary) to decompress next frame.static long
ZSTD_DCtx_reset(long dctx, int reset)
Returns aDCtx
to clean state.static long
ZSTD_DCtx_setParameter(long dctx, int param, int value)
Set one compression parameter, selected byenum ZSTD_dParameter
.static long
ZSTD_decompress(java.nio.ByteBuffer dst, java.nio.ByteBuffer src)
static long
ZSTD_decompress_usingDDict(long dctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, long ddict)
Decompression using a digested Dictionary.static long
ZSTD_decompress_usingDict(long dctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, java.nio.ByteBuffer dict)
Decompression using a known Dictionary.static long
ZSTD_decompressDCtx(long ctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src)
Same asdecompress
, requires an allocatedZSTD_DCtx
.static long
ZSTD_decompressStream(long zds, ZSTDOutBuffer output, ZSTDInBuffer input)
UseZSTD_decompressStream()
repetitively to consume your input.static ZSTDBounds
ZSTD_dParam_getBounds(int dParam, ZSTDBounds __result)
All parameters must belong to an interval with lower and upper bounds, otherwise they will either trigger an error or be automatically clamped.static long
ZSTD_DStreamInSize()
Returns the recommended size for input buffer.static long
ZSTD_DStreamOutSize()
Returns the recommended size for output buffer.static long
ZSTD_findFrameCompressedSize(java.nio.ByteBuffer src)
static long
ZSTD_freeCCtx(long cctx)
Frees memory allocated bycreateCCtx
.static long
ZSTD_freeCDict(long CDict)
Frees memory allocated bycreateCDict
.static long
ZSTD_freeCStream(long zcs)
Frees memory allocated bycreateCStream
.static long
ZSTD_freeDCtx(long dctx)
Frees memory allocated bycreateDCtx
.static long
ZSTD_freeDDict(long ddict)
Frees memory allocated withcreateDDict
.static long
ZSTD_freeDStream(long zds)
Frees memory allocated bycreateDStream
.static int
ZSTD_getDictID_fromDDict(long ddict)
Provides thedictID
of the dictionary loaded intoddict
.static int
ZSTD_getDictID_fromDict(java.nio.ByteBuffer dict)
Provides thedictID
stored within dictionary.static int
ZSTD_getDictID_fromFrame(java.nio.ByteBuffer src)
Provides the dictID required to decompressed the frame stored withinsrc
.static java.lang.String
ZSTD_getErrorName(long code)
Provides readable string from an error code.static long
ZSTD_getFrameContentSize(java.nio.ByteBuffer src)
Notes: a 0 return value means the frame is valid but "empty" decompressed size is an optional field, it may not be present, typically in streaming mode.static boolean
ZSTD_isError(long code)
Tells if asize_t
function result is an error code.static int
ZSTD_maxCLevel()
Returns the maximum compression level available.static int
ZSTD_minCLevel()
Returns the minimum compression level available.static long
ZSTD_sizeof_CCtx(long cctx)
static long
ZSTD_sizeof_CDict(long cdict)
static long
ZSTD_sizeof_CStream(long zcs)
static long
ZSTD_sizeof_DCtx(long dctx)
static long
ZSTD_sizeof_DDict(long ddict)
static long
ZSTD_sizeof_DStream(long zds)
static int
ZSTD_versionNumber()
Returns the version number.static java.lang.String
ZSTD_versionString()
Returns the version string.
-
-
-
Field Detail
-
ZSTD_VERSION_MAJOR, ZSTD_VERSION_MINOR, ZSTD_VERSION_RELEASE
Version number part.
-
ZSTD_VERSION_NUMBER
Version number.
-
ZSTD_VERSION_STRING
Version string.
-
ZSTD_CLEVEL_DEFAULT
Default compression level.
-
ZSTD_MAGICNUMBER
- See Also:
- Constant Field Values
-
ZSTD_MAGIC_DICTIONARY
- See Also:
- Constant Field Values
-
ZSTD_MAGIC_SKIPPABLE_START
- See Also:
- Constant Field Values
-
ZSTD_MAGIC_SKIPPABLE_MASK
- See Also:
- Constant Field Values
-
ZSTD_BLOCKSIZELOG_MAX
- See Also:
- Constant Field Values
-
ZSTD_BLOCKSIZE_MAX
- See Also:
- Constant Field Values
-
ZSTD_CONTENTSIZE_UNKNOWN, ZSTD_CONTENTSIZE_ERROR
Content size.
-
ZSTD_fast, ZSTD_dfast, ZSTD_greedy, ZSTD_lazy, ZSTD_lazy2, ZSTD_btlazy2, ZSTD_btopt, ZSTD_btultra, ZSTD_btultra2
-
ZSTD_c_compressionLevel, ZSTD_c_windowLog, ZSTD_c_hashLog, ZSTD_c_chainLog, ZSTD_c_searchLog, ZSTD_c_minMatch, ZSTD_c_targetLength, ZSTD_c_strategy, ZSTD_c_enableLongDistanceMatching, ZSTD_c_ldmHashLog, ZSTD_c_ldmMinMatch, ZSTD_c_ldmBucketSizeLog, ZSTD_c_ldmHashRateLog, ZSTD_c_contentSizeFlag, ZSTD_c_checksumFlag, ZSTD_c_dictIDFlag, ZSTD_c_nbWorkers, ZSTD_c_jobSize, ZSTD_c_overlapLog, ZSTD_c_experimentalParam1, ZSTD_c_experimentalParam2, ZSTD_c_experimentalParam3, ZSTD_c_experimentalParam4, ZSTD_c_experimentalParam5, ZSTD_c_experimentalParam6
Compression parameters. (ZSTD_cParameter
)Note: When compressing with a
ZSTD_CDict
these parameters are superseded by the parameters used to construct theZSTD_CDict
. SeeCCtx_refCDict
for more info.Enum values:
c_compressionLevel
- Update all compression parameters according to pre-defined cLevel table Default level isCLEVEL_DEFAULT
==3
. Special: value 0 means default, which is controlled byZSTD_CLEVEL_DEFAULT
.Note 1: it's possible to pass a negative compression level.
Note 2 : setting a level sets all default values of other compression parameters
c_windowLog
- Maximum allowed back-reference distance, expressed as power of 2. Must be clamped betweenWINDOWLOG_MIN
andWINDOWLOG_MAX
. Special: value 0 means "use defaultwindowLog
".Note: Using a
windowLog
greater thanWINDOWLOG_LIMIT_DEFAULT
requires explicitly allowing such window size at decompression stage if using streaming.c_hashLog
- Size of the initial probe table, as a power of 2. Resulting memory usage is (1 <<(hashLog+2)
). Must be clamped betweenHASHLOG_MIN
and HASHLOG_MAX. Larger tables improve compression ratio of strategies ≤ dFast, and improve speed of strategies > dFast. Special: value 0 means "use defaulthashLog
".c_chainLog
- Size of the multi-probe search table, as a power of 2. Resulting memory usage is (1 <<(chainLog+2)
). Must be clamped betweenCHAINLOG_MIN
andCHAINLOG_MAX
. Larger tables result in better and slower compression. This parameter is useless when using "fast" strategy. It's still useful when using "dfast" strategy, in which case it defines a secondary probe table. Special: value 0 means "use defaultchainLog
".c_searchLog
- Number of search attempts, as a power of 2. More attempts result in better and slower compression. This parameter is useless when using "fast" and "dFast" strategies. Special: value 0 means "use defaultsearchLog
".c_minMatch
- Minimum size of searched matches. Note that Zstandard can still find matches of smaller size, it just tweaks its search algorithm to look for this size and larger. Larger values increase compression and decompression speed, but decrease ratio. Must be clamped betweenMINMATCH_MIN
andMINMATCH_MAX
. Note that currently, for all strategies <btopt, effective minimum is 4. , for all strategies > fast, effective maximum is 6. Special: value 0 means "use defaultminMatchLength
".c_targetLength
- Impact of this field depends on strategy. For strategies btopt, btultra & btultra2: Length of Match considered "good enough" to stop search. Larger values make compression stronger, and slower. For strategy fast: Distance between match sampling. Larger values make compression faster, and weaker. Special: value 0 means "use default targetLength".c_strategy
- SeeZSTD_strategy
enum definition. The higher the value of selected strategy, the more complex it is, resulting in stronger and slower compression. Special: value 0 means "use default strategy".c_enableLongDistanceMatching
- Enable long distance matching. This parameter is designed to improve compression ratio for large inputs, by finding large matches at long distance. It increases memory usage and window size. Note: enabling this parameter increases defaultc_windowLog
to 128 MB except when expressly set to a different value.c_ldmHashLog
- Size of the table for long distance matching, as a power of 2. Larger values increase memory usage and compression ratio, but decrease compression speed. Must be clamped betweenHASHLOG_MIN
andHASHLOG_MAX
default: windowlog - 7. Special: value 0 means "automatically determine hashlog".c_ldmMinMatch
- Minimum match size for long distance matcher. Larger/too small values usually decrease compression ratio. Must be clamped betweenLDM_MINMATCH_MIN
andLDM_MINMATCH_MAX
. Special: value 0 means "use default value" (default: 64).c_ldmBucketSizeLog
- Log size of each bucket in the LDM hash table for collision resolution. Larger values improve collision resolution but decrease compression speed. The maximum value isLDM_BUCKETSIZELOG_MAX
. Special: value 0 means "use default value" (default: 3).c_ldmHashRateLog
- Frequency of inserting/looking up entries into the LDM hash table. Must be clamped between 0 and (WINDOWLOG_MAX
-HASHLOG_MIN
). Default isMAX(0, (windowLog - ldmHashLog))
, optimizing hash table usage. Larger values improve compression speed. Deviating far from default value will likely result in a compression ratio decrease. Special: value 0 means "automatically determinehashRateLog
".c_contentSizeFlag
- Content size will be written into frame header _whenever known_ (default:1) Content size must be known at the beginning of compression. This is automatically the case when usingcompress2
, For streaming variants, content size must be provided withCCtx_setPledgedSrcSize
.c_checksumFlag
- A 32-bits checksum of content is written at end of frame (default:0)c_dictIDFlag
- When applicable, dictionary's ID is written into frame header (default:1)c_nbWorkers
- Select how many threads will be spawned to compress in parallel. WhennbWorkers ≥ 1
, triggers asynchronous mode when used withZSTD_compressStream*()
:ZSTD_compressStream*()
consumes input and flush output if possible, but immediately gives back control to caller, while compression work is performed in parallel, within worker threads. (note: a strong exception to this rule is when first invocation ofcompressStream2
setse_end
: in which case,ZSTD_compressStream2()
delegates tocompress2
, which is always a blocking call). More workers improve speed, but also increase memory usage. Default value is0
, aka "single-threaded mode": no worker is spawned, compression is performed inside Caller's thread, all invocations are blocking.c_jobSize
- Size of a compression job. This value is enforced only whennbWorkers ≥ 1
. Each compression job is completed in parallel, so this value can indirectly impact the nb of active threads. 0 means default, which is dynamically determined based on compression parameters. Job size must be a minimum of overlap size, or 1 MB, whichever is largest. The minimum size is automatically and transparently enforced.c_overlapLog
- Control the overlap size, as a fraction of window size. The overlap size is an amount of data reloaded from previous job at the beginning of a new job. It helps preserve compression ratio, while each job is compressed in parallel. This value is enforced only whennbWorkers ≥ 1
. Larger values increase compression ratio, but decrease speed. Possible values range from 0 to 9:- 0 means "default" : value will be determined by the library, depending on strategy
- 1 means "no overlap"
- 9 means "full overlap", using a full window size. Each intermediate rank increases/decreases load size by a factor 2: 9: full window; 8: w/2; 7: w/4; 6: w/8; 5:w/16; 4: w/32; 3:w/64; 2:w/128; 1:no overlap; 0:default default value varies between 6 and 9, depending on strategy
c_experimentalParam1
c_experimentalParam2
c_experimentalParam3
c_experimentalParam4
c_experimentalParam5
c_experimentalParam6
-
ZSTD_d_windowLogMax, ZSTD_d_experimentalParam1
The advanced API pushes parameters one by one into an existingDCtx
context. Parameters are sticky, and remain valid for all following frames using the sameDCtx
context. It's possible to reset parameters to default values usingDCtx_reset
.Note: This API is compatible with existing
decompressDCtx
anddecompressStream
. Therefore, no new decompression function is necessary.(
ZSTD_dParameter
)Enum values:
d_windowLogMax
- Select a size limit (in power of 2) beyond which the streaming API will refuse to allocate memory buffer in order to protect the host from unreasonable memory requirements.This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode. By default, a decompression context accepts window sizes ≤ (1 <<
WINDOWLOG_LIMIT_DEFAULT
)Special: value 0 means "use default maximum windowLog".
d_experimentalParam1
- Note: additional experimental parameters are also available within the experimental section of the API. At the time of this writing, they include:c_format
Note: never ever use
experimentalParam
? names directly
-
ZSTD_e_continue, ZSTD_e_flush, ZSTD_e_end
ZSTD_EndDirective
Enum values:
e_continue
- collect more data, encoder decides when to output compressed result, for optimal compression ratioe_flush
- flush any data provided so far, it creates (at least) one new block, that can be decoded immediately on reception; frame will continue: any future data can still reference previously compressed data, improving compression.e_end
- flush any remaining data and close current frame. note that frame is only closed after compressed data is fully flushed (returnvalue == 0
). After that point, any additional data starts a new frame.Note: each frame is independent (does not reference any content from previous frame).
-
-
Method Detail
-
ZSTD_versionNumber
public static int ZSTD_versionNumber()
Returns the version number.
-
nZSTD_versionString
public static long nZSTD_versionString()
Unsafe version of:versionString
-
ZSTD_versionString
public static java.lang.String ZSTD_versionString()
Returns the version string.
-
nZSTD_compress
public static long nZSTD_compress(long dst, long dstCapacity, long src, long srcSize, int compressionLevel)
Unsafe version of:compress
-
ZSTD_compress
public static long ZSTD_compress(java.nio.ByteBuffer dst, java.nio.ByteBuffer src, int compressionLevel)
Compressessrc
content as a single zstd compressed frame into already allocateddst
.Hint: compression runs faster if
dstCapacity
≥compressBound
(srcSize)
- Returns:
- compressed size written into
dst
(≤dstCapacity
), or an error code if it fails (which can be tested usingisError
).
-
nZSTD_decompress
public static long nZSTD_decompress(long dst, long dstCapacity, long src, long compressedSize)
Unsafe version of:decompress
- Parameters:
dstCapacity
- is an upper bound oforiginalSize
to regenerate. If user cannot imply a maximum upper bound, it's better to use streaming mode to decompress data.compressedSize
- must be the exact size of some number of compressed and/or skippable frames
-
ZSTD_decompress
public static long ZSTD_decompress(java.nio.ByteBuffer dst, java.nio.ByteBuffer src)
- Returns:
- the number of bytes decompressed into
dst
(≤dstCapacity
), or anerrorCode
if it fails (which can be tested usingisError
).
-
nZSTD_getFrameContentSize
public static long nZSTD_getFrameContentSize(long src, long srcSize)
Unsafe version of:getFrameContentSize
- Parameters:
srcSize
- must be at least as large as the frame header. Hint: any size ≥FRAMEHEADERSIZE_MAX
is large enough.
-
ZSTD_getFrameContentSize
public static long ZSTD_getFrameContentSize(java.nio.ByteBuffer src)
Notes:- a 0 return value means the frame is valid but "empty"
- decompressed size is an optional field, it may not be present, typically in streaming mode. When
return==ZSTD_CONTENTSIZE_UNKNOWN
, data to decompress could be any size. In which case, it's necessary to use streaming mode to decompress data. Optionally, application can rely on some implicit limit, asdecompress
only needs an upper bound of decompressed size. (For example, data could be necessarily cut into blocks ≤ 16 KB). - decompressed size is always present when compression is completed using single-pass functions, such as
compress
,compressCCtx
,compress_usingDict
orcompress_usingCDict
. - decompressed size can be very large (64-bits value), potentially larger than what local system can handle as a single memory segment. In which case, it's necessary to use streaming mode to decompress data.
- If source is untrusted, decompressed size could be wrong or intentionally modified. Always ensure return value fits within application's authorized limits. Each application can set its own limits.
- Parameters:
src
- should point to the start of a ZSTD encoded frame- Returns:
- decompressed size of
src
frame content, if knownCONTENTSIZE_UNKNOWN
if the size cannot be determinedCONTENTSIZE_ERROR
if an error occurred (e.g. invalid magic number,srcSize
too small)
-
nZSTD_findFrameCompressedSize
public static long nZSTD_findFrameCompressedSize(long src, long srcSize)
Unsafe version of:findFrameCompressedSize
- Parameters:
srcSize
- must be ≥ first frame size
-
ZSTD_findFrameCompressedSize
public static long ZSTD_findFrameCompressedSize(java.nio.ByteBuffer src)
- Parameters:
src
- should point to the start of a ZSTD frame or skippable frame- Returns:
- the compressed size of the first frame starting at
src
, suitable to pass assrcSize
todecompress
or similar, or an error code if input is invalid
-
ZSTD_compressBound
public static long ZSTD_compressBound(long srcSize)
Returns the maximum compressed size in worst case single-pass scenario.
-
nZSTD_isError
public static int nZSTD_isError(long code)
Unsafe version of:isError
-
ZSTD_isError
public static boolean ZSTD_isError(long code)
Tells if asize_t
function result is an error code.
-
nZSTD_getErrorName
public static long nZSTD_getErrorName(long code)
Unsafe version of:getErrorName
-
ZSTD_getErrorName
public static java.lang.String ZSTD_getErrorName(long code)
Provides readable string from an error code.
-
ZSTD_minCLevel
public static int ZSTD_minCLevel()
Returns the minimum compression level available.
-
ZSTD_maxCLevel
public static int ZSTD_maxCLevel()
Returns the maximum compression level available.
-
ZSTD_createCCtx
public static long ZSTD_createCCtx()
Creates a compression context.When compressing many times, it is recommended to allocate a context just once, and re-use it for each successive compression operation. This will make workload friendlier for system's memory. Use one context per thread for parallel execution in multi-threaded environments.
-
nZSTD_freeCCtx
public static long nZSTD_freeCCtx(long cctx)
Unsafe version of:freeCCtx
-
ZSTD_freeCCtx
public static long ZSTD_freeCCtx(long cctx)
Frees memory allocated bycreateCCtx
.
-
nZSTD_compressCCtx
public static long nZSTD_compressCCtx(long ctx, long dst, long dstCapacity, long src, long srcSize, int compressionLevel)
Unsafe version of:compressCCtx
-
ZSTD_compressCCtx
public static long ZSTD_compressCCtx(long ctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, int compressionLevel)
Same ascompress
, using an explicitZSTD_CCtx
. The function will compress at requested compression level, ignoring any other parameter.
-
ZSTD_createDCtx
public static long ZSTD_createDCtx()
Creates a decompression context.When decompressing many times, it is recommended to allocate a context only once, and re-use it for each successive compression operation. This will make workload friendlier for system's memory. Use one context per thread for parallel execution.
-
nZSTD_freeDCtx
public static long nZSTD_freeDCtx(long dctx)
Unsafe version of:freeDCtx
-
ZSTD_freeDCtx
public static long ZSTD_freeDCtx(long dctx)
Frees memory allocated bycreateDCtx
.
-
nZSTD_decompressDCtx
public static long nZSTD_decompressDCtx(long ctx, long dst, long dstCapacity, long src, long srcSize)
Unsafe version of:decompressDCtx
-
ZSTD_decompressDCtx
public static long ZSTD_decompressDCtx(long ctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src)
Same asdecompress
, requires an allocatedZSTD_DCtx
. Compatible with sticky parameters.
-
nZSTD_cParam_getBounds
public static void nZSTD_cParam_getBounds(int cParam, long __result)
Unsafe version of:cParam_getBounds
-
ZSTD_cParam_getBounds
public static ZSTDBounds ZSTD_cParam_getBounds(int cParam, ZSTDBounds __result)
All parameters must belong to an interval with lower and upper bounds, otherwise they will either trigger an error or be automatically clamped.- Parameters:
cParam
- one of:__result
- a structure,ZSTD_bounds
, which contains- an error status field, which must be tested using
isError
- lower and upper bounds, both inclusive
- an error status field, which must be tested using
-
nZSTD_CCtx_setParameter
public static long nZSTD_CCtx_setParameter(long cctx, int param, int value)
Unsafe version of:CCtx_setParameter
-
ZSTD_CCtx_setParameter
public static long ZSTD_CCtx_setParameter(long cctx, int param, int value)
Set one compression parameter, selected by enumZSTD_cParameter
.All parameters have valid bounds. Bounds can be queried using
cParam_getBounds
. Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter). Setting a parameter is generally only possible during frame initialization (before starting compression). Exception: when using multi-threading mode (nbWorkers ≥ 1), the following parameters can be updated during compression (within same frame): =< compressionLevel, hashLog, chainLog, searchLog, minMatch, targetLength and strategy. new parameters will be active for next job only (after aflush()
).- Parameters:
param
- one of:- Returns:
- an error code (which can be tested using
isError
)
-
nZSTD_CCtx_setPledgedSrcSize
public static long nZSTD_CCtx_setPledgedSrcSize(long cctx, long pledgedSrcSize)
Unsafe version of:CCtx_setPledgedSrcSize
-
ZSTD_CCtx_setPledgedSrcSize
public static long ZSTD_CCtx_setPledgedSrcSize(long cctx, long pledgedSrcSize)
Total input data size to be compressed as a single frame.Value will be written in frame header, unless if explicitly forbidden using
c_contentSizeFlag
. This value will also be controlled at end of frame, and trigger an error if not respected.Note 1:
pledgedSrcSize==0
actually means zero, aka an empty frame. In order to mean "unknown content size", pass constantCONTENTSIZE_UNKNOWN
.ZSTD_CONTENTSIZE_UNKNOWN
is default value for any new frame.Note 2:
pledgedSrcSize
is only valid once, for the next frame. It's discarded at the end of the frame, and replaced byZSTD_CONTENTSIZE_UNKNOWN
.Note 3 : Whenever all input data is provided and consumed in a single round, for example with
compress2
, or invoking immediatelyZSTD_compressStream2(,,,ZSTD_e_end)
, this value is automatically overridden bysrcSize
instead.- Returns:
- 0, or an error code (which can be tested with
isError
).
-
nZSTD_CCtx_reset
public static long nZSTD_CCtx_reset(long cctx, int reset)
Unsafe version of:CCtx_reset
-
ZSTD_CCtx_reset
public static long ZSTD_CCtx_reset(long cctx, int reset)
There are 2 different things that can be reset, independently or jointly :- The session: will stop compressing current frame, and make
CCtx
ready to start a new one. Useful after an error, or to interrupt any ongoing compression. Any internal data not yet flushed is cancelled. Compression parameters and dictionary remain unchanged. They will be used to compress next frame. Resetting session never fails. - The parameters: changes all parameters back to "default". This removes any reference to any dictionary too. Parameters can only be changed between
2 sessions (i.e. no compression is currently ongoing) otherwise the reset fails, and function returns an error value (which can be tested using
isError
) - Both: similar to resetting the session, followed by resetting parameters.
- Parameters:
reset
- one of:reset_session_only
reset_parameters
reset_session_and_parameters
- The session: will stop compressing current frame, and make
-
nZSTD_compress2
public static long nZSTD_compress2(long cctx, long dst, long dstCapacity, long src, long srcSize)
Unsafe version of:compress2
-
ZSTD_compress2
public static long ZSTD_compress2(long cctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src)
Behaves the same ascompressCCtx
, but compression parameters are set using the advanced API.ZSTD_compress2()
always starts a new frame. Should cctx hold data from a previously unfinished frame, everything about it is forgotten.- Compression parameters are pushed into
CCtx
before starting compression, usingZSTD_CCtx_set*()
- The function is always blocking, returns when compression is completed. Hint: compression runs faster ifdstCapacity
≥ZSTD_compressBound(srcSize)
.- Returns:
- compressed size written into
dst
(≤dstCapacity
), or an error code if it fails (which can be tested usingisError
)
-
nZSTD_dParam_getBounds
public static void nZSTD_dParam_getBounds(int dParam, long __result)
Unsafe version of:dParam_getBounds
-
ZSTD_dParam_getBounds
public static ZSTDBounds ZSTD_dParam_getBounds(int dParam, ZSTDBounds __result)
All parameters must belong to an interval with lower and upper bounds, otherwise they will either trigger an error or be automatically clamped.- Parameters:
__result
- returns a structure,ZSTD_bounds
, which contains - an error status field, which must be tested usingisError
- both lower and upper bounds, inclusive
-
nZSTD_DCtx_setParameter
public static long nZSTD_DCtx_setParameter(long dctx, int param, int value)
Unsafe version of:DCtx_setParameter
-
ZSTD_DCtx_setParameter
public static long ZSTD_DCtx_setParameter(long dctx, int param, int value)
Set one compression parameter, selected byenum ZSTD_dParameter
.All parameters have valid bounds. Bounds can be queried using
dParam_getBounds
. Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter). Setting a parameter is only possible during frame initialization (before starting decompression).- Parameters:
param
- one of:d_windowLogMax
d_experimentalParam1
- Returns:
- 0, or an error code (which can be tested using
isError
)
-
nZSTD_DCtx_reset
public static long nZSTD_DCtx_reset(long dctx, int reset)
Unsafe version of:DCtx_reset
-
ZSTD_DCtx_reset
public static long ZSTD_DCtx_reset(long dctx, int reset)
Returns aDCtx
to clean state.Session and parameters can be reset jointly or separately. Parameters can only be reset when no active frame is being decompressed.
- Parameters:
reset
- one of:reset_session_only
reset_parameters
reset_session_and_parameters
- Returns:
- 0, or an error code, which can be tested with
isError
-
ZSTD_createCStream
public static long ZSTD_createCStream()
AZSTD_CStream
object is required to track streaming operation.Use
ZSTD_createCStream()
andfreeCStream
to create/release resources.ZSTD_CStream
objects can be reused multiple times on consecutive compression operations. It is recommended to re-useZSTD_CStream
in situations where many streaming operations will be achieved consecutively, since it will play nicer with system's memory, by re-using already allocated memory. Use one separateZSTD_CStream
per thread for parallel execution.
-
nZSTD_freeCStream
public static long nZSTD_freeCStream(long zcs)
Unsafe version of:freeCStream
-
ZSTD_freeCStream
public static long ZSTD_freeCStream(long zcs)
Frees memory allocated bycreateCStream
.
-
nZSTD_compressStream2
public static long nZSTD_compressStream2(long cctx, long output, long input, int endOp)
Unsafe version of:compressStream2
-
ZSTD_compressStream2
public static long ZSTD_compressStream2(long cctx, ZSTDOutBuffer output, ZSTDInBuffer input, int endOp)
Behaves about the same asZSTD_compressStream()
, with additional control on end directive.- Compression parameters are pushed into
CCtx
before starting compression, usingZSTD_CCtx_set*()
. - Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode).
outpot->pos
must be ≤dstCapacity
,input->pos
must be ≤srcSize
.outpot->pos
andinput->pos
will be updated. They are guaranteed to remain below their respective limit.- When
nbWorkers==0
(default), function is blocking: it completes its job before returning to caller. - When
nbWorkers≥1
, function is non-blocking: it just acquires a copy of input, and distributes jobs to internal worker threads, flush whatever is available, and then immediately returns, just indicating that there is some data remaining to be flushed. The function nonetheless guarantees forward progress: it will return only after it reads or write at least 1+ byte. - Exception: if the first call requests a
e_end
directive and provides enoughdstCapacity
, the function delegates tocompress2
which is always blocking.
- Parameters:
endOp
- one of:e_continue
e_flush
e_end
- Returns:
- provides a minimum amount of data remaining to be flushed from internal buffers or an error code, which can be tested using
isError
.If
!= 0
, flush is not fully completed, there is still some data left within internal buffers. This is useful fore_flush
, since in this case more flushes are necessary to empty all buffers. Fore_end
,== 0
when internal buffers are fully flushed and frame is completed.- after a
ZSTD_e_end
directive, if internal buffer is not fully flushed (!= 0
), onlyZSTD_e_end
orZSTD_e_flush
operations are allowed. Before starting a new compression job, or changing compression parameters, it is required to fully flush internal buffers.
- Compression parameters are pushed into
-
ZSTD_CStreamInSize
public static long ZSTD_CStreamInSize()
Returns the recommended size for input buffer.These buffer sizes are softly recommended. They are not required:
ZSTD_compressStream*()
happily accepts any buffer size, for both input and output. Respecting the recommended size just makes it a bit easier forZSTD_compressStream*()
, reducing the amount of memory shuffling and buffering, resulting in minor performance savings.However, note that these recommendations are from the perspective of a C caller program. If the streaming interface is invoked from some other language, especially managed ones such as Java or Go, through a foreign function interface such as jni or cgo, a major performance rule is to reduce crossing such interface to an absolute minimum. It's not rare that performance ends being spent more into the interface, rather than compression itself. In which cases, prefer using large buffers, as large as practical, for both input and output, to reduce the nb of roundtrips.
-
ZSTD_CStreamOutSize
public static long ZSTD_CStreamOutSize()
Returns the recommended size for output buffer. Guarantee to successfully flush at least one complete compressed block.
-
ZSTD_createDStream
public static long ZSTD_createDStream()
AZSTD_DStream
object is required to track streaming operations.Use
ZSTD_createDStream()
andfreeDStream
to create/release resources.ZSTD_DStream
objects can be re-used multiple times.
-
nZSTD_freeDStream
public static long nZSTD_freeDStream(long zds)
Unsafe version of:freeDStream
-
ZSTD_freeDStream
public static long ZSTD_freeDStream(long zds)
Frees memory allocated bycreateDStream
.
-
nZSTD_decompressStream
public static long nZSTD_decompressStream(long zds, long output, long input)
Unsafe version of:decompressStream
-
ZSTD_decompressStream
public static long ZSTD_decompressStream(long zds, ZSTDOutBuffer output, ZSTDInBuffer input)
UseZSTD_decompressStream()
repetitively to consume your input.The function will update both
pos
fields. Ifinput.pos < input.size
, some input has not been consumed. It's up to the caller to present again remaining data. The function tries to flush all data decoded immediately, respecting output buffer size. Ifoutput.pos < output.size
, decoder has flushed everything it could. But ifoutput.pos == output.size
, there might be some data left within internal buffers. In which case, callZSTD_decompressStream()
again to flush whatever remains in the buffer. With no additional input provided, amount of data flushed is necessarily ≤BLOCKSIZE_MAX
.- Returns:
- 0 when a frame is completely decoded and fully flushed, an error code, which can be tested using
isError
, any other value > 0, which means there is still some decoding to do to complete current frame. The return value is a suggested next input size (just a hint to improve latency) that will never request more than the remaining frame size.
-
ZSTD_DStreamInSize
public static long ZSTD_DStreamInSize()
Returns the recommended size for input buffer.
-
ZSTD_DStreamOutSize
public static long ZSTD_DStreamOutSize()
Returns the recommended size for output buffer. Guarantee to successfully flush at least one complete compressed block in all circumstances.
-
nZSTD_compress_usingDict
public static long nZSTD_compress_usingDict(long ctx, long dst, long dstCapacity, long src, long srcSize, long dict, long dictSize, int compressionLevel)
Unsafe version of:compress_usingDict
-
ZSTD_compress_usingDict
public static long ZSTD_compress_usingDict(long ctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, @Nullable java.nio.ByteBuffer dict, int compressionLevel)
Compression at an explicit compression level using a Dictionary.A dictionary can be any arbitrary data segment (also called a prefix), or a buffer with specified information (see
dictBuilder/zdict.h
).This function loads the dictionary, resulting in significant startup delay. It's intended for a dictionary used only once.
When
dict == NULL || dictSize < 8
no dictionary is used.
-
nZSTD_decompress_usingDict
public static long nZSTD_decompress_usingDict(long dctx, long dst, long dstCapacity, long src, long srcSize, long dict, long dictSize)
Unsafe version of:decompress_usingDict
-
ZSTD_decompress_usingDict
public static long ZSTD_decompress_usingDict(long dctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, @Nullable java.nio.ByteBuffer dict)
Decompression using a known Dictionary. Dictionary must be identical to the one used during compression.This function loads the dictionary, resulting in significant startup delay. It's intended for a dictionary used only once.
When
dict == NULL || dictSize < 8
no dictionary is used.
-
nZSTD_createCDict
public static long nZSTD_createCDict(long dictBuffer, long dictSize, int compressionLevel)
Unsafe version of:createCDict
-
ZSTD_createCDict
public static long ZSTD_createCDict(java.nio.ByteBuffer dictBuffer, int compressionLevel)
When compressing multiple messages / blocks using the same dictionary, it's recommended to load it only once.ZSTD_createCDict()
will create a digested dictionary, ready to start future compression operations without startup cost.ZSTD_CDict
can be created once and shared by multiple threads concurrently, since its usage is read-only.dictBuffer
can be released afterZSTD_CDict
creation, because its content is copied within CDict. Consider experimental functioncreateCDict_byReference
if you prefer to not duplicatedictBuffer
content.Note: A
ZSTD_CDict
can be created from an emptydictBuffer
, but it is inefficient when used to compress small data.
-
nZSTD_freeCDict
public static long nZSTD_freeCDict(long CDict)
Unsafe version of:freeCDict
-
ZSTD_freeCDict
public static long ZSTD_freeCDict(long CDict)
Frees memory allocated bycreateCDict
.
-
nZSTD_compress_usingCDict
public static long nZSTD_compress_usingCDict(long cctx, long dst, long dstCapacity, long src, long srcSize, long cdict)
Unsafe version of:compress_usingCDict
-
ZSTD_compress_usingCDict
public static long ZSTD_compress_usingCDict(long cctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, long cdict)
Compression using a digested Dictionary. Recommended when same dictionary is used multiple times.Compression level is decided at dictionary creation time, and frame parameters are hardcoded (
dictID=yes, contentSize=yes, checksum=no
)
-
nZSTD_createDDict
public static long nZSTD_createDDict(long dictBuffer, long dictSize)
Unsafe version of:createDDict
-
ZSTD_createDDict
public static long ZSTD_createDDict(java.nio.ByteBuffer dictBuffer)
Creates a digested dictionary, ready to start decompression operation without startup delay.dictBuffer
can be released afterDDict
creation, as its content is copied insideDDict
.
-
nZSTD_freeDDict
public static long nZSTD_freeDDict(long ddict)
Unsafe version of:freeDDict
-
ZSTD_freeDDict
public static long ZSTD_freeDDict(long ddict)
Frees memory allocated withcreateDDict
.
-
nZSTD_decompress_usingDDict
public static long nZSTD_decompress_usingDDict(long dctx, long dst, long dstCapacity, long src, long srcSize, long ddict)
Unsafe version of:decompress_usingDDict
-
ZSTD_decompress_usingDDict
public static long ZSTD_decompress_usingDDict(long dctx, java.nio.ByteBuffer dst, java.nio.ByteBuffer src, long ddict)
Decompression using a digested Dictionary.Recommended when same dictionary is used multiple times.
-
nZSTD_getDictID_fromDict
public static int nZSTD_getDictID_fromDict(long dict, long dictSize)
Unsafe version of:getDictID_fromDict
-
ZSTD_getDictID_fromDict
public static int ZSTD_getDictID_fromDict(java.nio.ByteBuffer dict)
Provides thedictID
stored within dictionary.- Returns:
- if
== 0
, the dictionary is not conformant with Zstandard specification. It can still be loaded, but as a content-only dictionary.
-
nZSTD_getDictID_fromDDict
public static int nZSTD_getDictID_fromDDict(long ddict)
Unsafe version of:getDictID_fromDDict
-
ZSTD_getDictID_fromDDict
public static int ZSTD_getDictID_fromDDict(long ddict)
Provides thedictID
of the dictionary loaded intoddict
.- Returns:
- if
== 0
, the dictionary is not conformant to Zstandard specification, or empty.Non-conformant dictionaries can still be loaded, but as content-only dictionaries.
-
nZSTD_getDictID_fromFrame
public static int nZSTD_getDictID_fromFrame(long src, long srcSize)
Unsafe version of:getDictID_fromFrame
-
ZSTD_getDictID_fromFrame
public static int ZSTD_getDictID_fromFrame(java.nio.ByteBuffer src)
Provides the dictID required to decompressed the frame stored withinsrc
.- Returns:
- if
== 0
, thedictID
could not be decoded. This could for one of the following reasons :- The frame does not require a dictionary to be decoded (most common case).
- The frame was built with
dictID
intentionally removed. Whatever dictionary is necessary is a hidden information.Note: this use case also happens when using a non-conformant dictionary.
srcSize
is too small, and as a result, the frame header could not be decoded (only possible ifsrcSize
<FRAMEHEADERSIZE_MAX
).- This is not a Zstandard frame. When identifying the exact failure cause, it's possible to use
getFrameHeader
, which will provide a more precise error code.
-
nZSTD_CCtx_loadDictionary
public static long nZSTD_CCtx_loadDictionary(long cctx, long dict, long dictSize)
Unsafe version of:CCtx_loadDictionary
-
ZSTD_CCtx_loadDictionary
public static long ZSTD_CCtx_loadDictionary(long cctx, @Nullable java.nio.ByteBuffer dict)
Creates an internalCDict
fromdict
buffer. Decompression will have to use same dictionary.Special: Loading a
NULL
(or 0-size) dictionary invalidates previous dictionary, meaning "return to no-dictionary mode".Note 1: Dictionary is sticky, it will be used for all future compressed frames. To return to "no-dictionary" situation, load a
NULL
dictionary (or reset parameters).Note 2: Loading a dictionary involves building tables. It's also a CPU consuming operation, with non-negligible impact on latency. Tables are dependent on compression parameters, and for this reason, compression parameters can no longer be changed after loading a dictionary.
Note 3:
dict
content will be copied internally. Use experimentalCCtx_loadDictionary_byReference
to reference content instead. In such a case, dictionary buffer must outlive its users.Note 4: Use
CCtx_loadDictionary_advanced
to precisely select how dictionary content must be interpreted.- Returns:
- 0, or an error code (which can be tested with
isError
).
-
nZSTD_CCtx_refCDict
public static long nZSTD_CCtx_refCDict(long cctx, long cdict)
Unsafe version of:CCtx_refCDict
-
ZSTD_CCtx_refCDict
public static long ZSTD_CCtx_refCDict(long cctx, long cdict)
References a prepared dictionary, to be used for all next compressed frames.Note that compression parameters are enforced from within CDict, and supercede any compression parameter previously set within
CCtx
. The dictionary will remain valid for future compressed frames using sameCCtx
.Special: Referencing a
NULL
CDict
means "return to no-dictionary mode".Note 1: Currently, only one dictionary can be managed. Referencing a new dictionary effectively "discards" any previous one.
Note 2:
CDict
is just referenced, its lifetime must outlive its usage withinCCtx
.- Returns:
- 0, or an error code (which can be tested with
isError
).
-
nZSTD_CCtx_refPrefix
public static long nZSTD_CCtx_refPrefix(long cctx, long prefix, long prefixSize)
Unsafe version of:CCtx_refPrefix
-
ZSTD_CCtx_refPrefix
public static long ZSTD_CCtx_refPrefix(long cctx, @Nullable java.nio.ByteBuffer prefix)
References a prefix (single-usage dictionary) for next compressed frame.A prefix is only used once. Tables are discarded at end of frame (
e_end
). Decompression will need same prefix to properly regenerate data. Compressing with a prefix is similar in outcome as performing a diff and compressing it, but performs much faster, especially during decompression (compression speed is tunable with compression level).Special: Adding any prefix (including
NULL
) invalidates any previous prefix or dictionaryNote 1: Prefix buffer is referenced. It must outlive compression. Its content must remain unmodified during compression.
Note 2: If the intention is to diff some large
src
data blob with some prior version of itself, ensure that the window size is large enough to contain the entire source. Seec_windowLog
.Note 3: Referencing a prefix involves building tables, which are dependent on compression parameters. It's a CPU consuming operation, with non-negligible impact on latency. If there is a need to use the same prefix multiple times, consider
loadDictionary
instead.Note 4: By default, the prefix is interpreted as raw content (
dct_rawContent
). Use experimentalCCtx_refPrefix_advanced
to alter dictionary interpretation.- Returns:
- 0, or an error code (which can be tested with
isError
).
-
nZSTD_DCtx_loadDictionary
public static long nZSTD_DCtx_loadDictionary(long dctx, long dict, long dictSize)
Unsafe version of:DCtx_loadDictionary
-
ZSTD_DCtx_loadDictionary
public static long ZSTD_DCtx_loadDictionary(long dctx, @Nullable java.nio.ByteBuffer dict)
Create an internalDDict
fromdict
buffer, to be used to decompress next frames. The dictionary remains valid for all future frames, until explicitly invalidated.Special: Adding a
NULL
(or 0-size) dictionary invalidates any previous dictionary, meaning "return to no-dictionary mode".Note 1: Loading a dictionary involves building tables, which has a non-negligible impact on CPU usage and latency. It's recommended to "load once, use many times", to amortize the cost.
Note 2:
dict
content will be copied internally, sodict
can be released after loading. UseDCtx_loadDictionary_byReference
to reference dictionary content instead.Note 3: Use
DCtx_loadDictionary_advanced
to take control of how dictionary content is loaded and interpreted.- Returns:
- 0, or an error code (which can be tested with ZSTD_isError())
-
nZSTD_DCtx_refDDict
public static long nZSTD_DCtx_refDDict(long dctx, long ddict)
Unsafe version of:DCtx_refDDict
-
ZSTD_DCtx_refDDict
public static long ZSTD_DCtx_refDDict(long dctx, long ddict)
References a prepared dictionary, to be used to decompress next frames. The dictionary remains active for decompression of future frames using sameDCtx
.Note 1: Currently, only one dictionary can be managed. Referencing a new dictionary effectively "discards" any previous one. Special: referencing a
NULL
DDict
means "return to no-dictionary mode".Note 2:
DDict
is just referenced, its lifetime must outlive its usage fromDCtx
.- Returns:
- 0, or an error code (which can be tested with
isError
)
-
nZSTD_DCtx_refPrefix
public static long nZSTD_DCtx_refPrefix(long dctx, long prefix, long prefixSize)
Unsafe version of:DCtx_refPrefix
-
ZSTD_DCtx_refPrefix
public static long ZSTD_DCtx_refPrefix(long dctx, java.nio.ByteBuffer prefix)
References a prefix (single-usage dictionary) to decompress next frame.This is the reverse operation of
CCtx_refPrefix
, and must use the same prefix as the one used during compression. Prefix is only used once. Reference is discarded at end of frame. End of frame is reached whendecompressStream
returns 0.Note 1: Adding any prefix (including
NULL
) invalidates any previously set prefix or dictionary.Note 2: Prefix buffer is referenced. It must outlive decompression. Prefix buffer must remain unmodified up to the end of frame, reached when
ZSTD_decompressStream()
returns 0.Note 3: By default, the prefix is treated as raw content (
dct_rawContent
). UseCCtx_refPrefix_advanced
to alterdictMode
.Note 4: Referencing a raw content prefix has almost no cpu nor memory cost. A full dictionary is more costly, as it requires building tables.
- Returns:
- 0, or an error code (which can be tested with
isError
)
-
nZSTD_sizeof_CCtx
public static long nZSTD_sizeof_CCtx(long cctx)
-
ZSTD_sizeof_CCtx
public static long ZSTD_sizeof_CCtx(long cctx)
-
nZSTD_sizeof_DCtx
public static long nZSTD_sizeof_DCtx(long dctx)
-
ZSTD_sizeof_DCtx
public static long ZSTD_sizeof_DCtx(long dctx)
-
nZSTD_sizeof_CStream
public static long nZSTD_sizeof_CStream(long zcs)
-
ZSTD_sizeof_CStream
public static long ZSTD_sizeof_CStream(long zcs)
-
nZSTD_sizeof_DStream
public static long nZSTD_sizeof_DStream(long zds)
-
ZSTD_sizeof_DStream
public static long ZSTD_sizeof_DStream(long zds)
-
nZSTD_sizeof_CDict
public static long nZSTD_sizeof_CDict(long cdict)
-
ZSTD_sizeof_CDict
public static long ZSTD_sizeof_CDict(long cdict)
-
nZSTD_sizeof_DDict
public static long nZSTD_sizeof_DDict(long ddict)
-
ZSTD_sizeof_DDict
public static long ZSTD_sizeof_DDict(long ddict)
-
ZSTD_COMPRESSBOUND
public static long ZSTD_COMPRESSBOUND(long srcSize)
Pure Java version ofZSTD_compressBound(long)
.
-
-