Class OpusMultistream
- java.lang.Object
-
- org.lwjgl.util.opus.OpusMultistream
-
public class OpusMultistream extends java.lang.Object
Opus Multistream APIThe multistream API allows individual Opus streams to be combined into a single packet, enabling support for up to 255 channels. Unlike an elementary Opus stream, the encoder and decoder must negotiate the channel configuration before the decoder can successfully interpret the data in the packets produced by the encoder. Some basic information, such as packet duration, can be computed without any special negotiation.
The format for multistream Opus packets is defined in RFC 7845 and is based on the self-delimited Opus framing described in Appendix B of RFC 6716. Normal Opus packets are just a degenerate case of multistream Opus packets, and can be encoded or decoded with the multistream API by setting
streams
to1
when initializing the encoder or decoder.Multistream Opus streams can contain up to 255 elementary Opus streams. These may be either "uncoupled" or "coupled", indicating that the decoder is configured to decode them to either 1 or 2 channels, respectively. The streams are ordered so that all coupled streams appear at the beginning.
A
mapping
table defines which decoded channeli
should be used for each input/output (I/O) channelj
. This table is typically provided as an unsigned char array. Leti = mapping[j]
be the index for I/O channelj
. Ifi < 2*coupled_streams
, then I/O channelj
is encoded as the left channel of stream(i/2)
ifi
is even, or as the right channel of stream(i/2)
ifi
is odd. Otherwise, I/O channelj
is encoded as mono in stream(i - coupled_streams)
, unless it has the special value 255, in which case it is omitted from the encoding entirely (the decoder will reproduce it as silence). Each valuei
must either be the special value 255 or be less thanstreams + coupled_streams
.The output channels specified by the encoder should use the Vorbis channel ordering. A decoder may wish to apply an additional permutation to the mapping the encoder used to achieve a different output channel order (e.g. for outputing in WAV order).
Each multistream packet contains an Opus packet for each stream, and all of the Opus packets in a single multistream packet must have the same duration. Therefore the duration of a multistream packet can be extracted from the TOC sequence of the first stream, which is located at the beginning of the packet, just like an elementary Opus stream:
int nb_samples, nb_frames; nb_frames = opus_packet_get_nb_frames(data, len); if (nb_frames < 1) return nb_frames; nb_samples = opus_packet_get_samples_per_frame(data, 48000) * nb_frames;
The general encoding and decoding process proceeds exactly the same as in the normal encoder and decoder APIs. See their documentation for an overview of how to use the corresponding multistream functions.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
OpusMultistream.Functions
Contains the function pointers loaded fromOpus.getLibrary()
.
-
Field Summary
Fields Modifier and Type Field Description static int
OPUS_MULTISTREAM_GET_DECODER_STATE_REQUEST
OPUS_MULTISTREAM_GET_ENCODER_STATE_REQUESTMultistream CTLs
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static int
nopus_multistream_decode(long st, long data, int len, long pcm, int frame_size, int decode_fec)
Unsafe version of:multistream_decode
static int
nopus_multistream_decode_float(long st, long data, int len, long pcm, int frame_size, int decode_fec)
Unsafe version of:multistream_decode_float
static long
nopus_multistream_decoder_create(int Fs, int channels, int streams, int coupled_streams, long mapping, long error)
Unsafe version of:multistream_decoder_create
static int
nopus_multistream_decoder_init(long st, int Fs, int channels, int streams, int coupled_streams, long mapping)
Unsafe version of:multistream_decoder_init
static int
nopus_multistream_encode(long st, long pcm, int frame_size, long data, int max_data_bytes)
Unsafe version of:multistream_encode
static int
nopus_multistream_encode_float(long st, long pcm, int frame_size, long data, int max_data_bytes)
Unsafe version of:multistream_encode_float
static long
nopus_multistream_encoder_create(int Fs, int channels, int streams, int coupled_streams, long mapping, int application, long error)
Unsafe version of:multistream_encoder_create
static int
nopus_multistream_encoder_init(long st, int Fs, int channels, int streams, int coupled_streams, long mapping, int application)
Unsafe version of:multistream_encoder_init
static long
nopus_multistream_surround_encoder_create(int Fs, int channels, int mapping_family, long streams, long coupled_streams, long mapping, int application, long error)
static int
nopus_multistream_surround_encoder_init(long st, int Fs, int channels, long streams, long coupled_streams, long mapping, int application)
static int
opus_multistream_decode(long st, java.nio.ByteBuffer data, java.nio.ShortBuffer pcm, int frame_size, int decode_fec)
Decode a multistream Opus packet.static int
opus_multistream_decode_float(long st, java.nio.ByteBuffer data, java.nio.FloatBuffer pcm, int frame_size, int decode_fec)
Decode a multistream Opus packet with floating point output.static long
opus_multistream_decoder_create(int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping, java.nio.IntBuffer error)
Allocates and initializes a multistream decoder state.static int
opus_multistream_decoder_ctl(long st, int request)
Performs a CTL function on a multistream Opus decoder.static int
opus_multistream_decoder_ctl(long st, Opus.CTLRequest request)
Performs a CTL function on a multistream Opus decoder.static void
opus_multistream_decoder_destroy(long st)
Frees anOpusMSDecoder
allocated bymultistream_decoder_create
.static int
opus_multistream_decoder_get_size(int streams, int coupled_streams)
Gets the size of anOpusMSDecoder
structure.static int
opus_multistream_decoder_init(long st, int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping)
Intialize a previously allocated decoder state object.static int
opus_multistream_encode(long st, java.nio.ShortBuffer pcm, int frame_size, java.nio.ByteBuffer data)
Encodes a multistream Opus frame.static int
opus_multistream_encode_float(long st, java.nio.FloatBuffer pcm, int frame_size, java.nio.ByteBuffer data)
Encodes a multistream Opus frame from floating point input.static long
opus_multistream_encoder_create(int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping, int application, java.nio.IntBuffer error)
Allocates and initializes a multistream encoder state.static int
opus_multistream_encoder_ctl(long st, int request)
Performs a CTL function on an multistream Opus encoder.static int
opus_multistream_encoder_ctl(long st, Opus.CTLRequest request)
Performs a CTL function on an multistream Opus encoder.static void
opus_multistream_encoder_destroy(long st)
Frees anOpusMSEncoder
allocated bymultistream_encoder_create
.static int
opus_multistream_encoder_get_size(int streams, int coupled_streams)
Gets the size of anOpusMSEncoder
structure.static int
opus_multistream_encoder_init(long st, int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping, int application)
Initialize a previously allocated multistream encoder state.static long
opus_multistream_surround_encoder_create(int Fs, int mapping_family, java.nio.IntBuffer streams, java.nio.IntBuffer coupled_streams, java.nio.ByteBuffer mapping, int application, java.nio.IntBuffer error)
static int
opus_multistream_surround_encoder_get_size(int channels, int mapping_family)
static int
opus_multistream_surround_encoder_init(long st, int Fs, java.nio.IntBuffer streams, java.nio.IntBuffer coupled_streams, java.nio.ByteBuffer mapping, int application)
-
-
-
Method Detail
-
opus_multistream_encoder_get_size
public static int opus_multistream_encoder_get_size(int streams, int coupled_streams)
Gets the size of anOpusMSEncoder
structure.- Parameters:
streams
- the total number of streams to encode from the input. This must be no more than 255.coupled_streams
- number of coupled (2 channel) streams to encode. This must be no larger than the total number of streams. Additionally, The total number of encoded channels (streams + coupled_streams
) must be no more than 255.- Returns:
- the size in bytes on success, or a negative error code on error
-
opus_multistream_surround_encoder_get_size
public static int opus_multistream_surround_encoder_get_size(int channels, int mapping_family)
-
nopus_multistream_encoder_create
public static long nopus_multistream_encoder_create(int Fs, int channels, int streams, int coupled_streams, long mapping, int application, long error)
Unsafe version of:multistream_encoder_create
- Parameters:
channels
- number of channels in the input signal. This must be at most 255. It may be greater than the number of coded channels (streams + coupled_streams
).
-
opus_multistream_encoder_create
public static long opus_multistream_encoder_create(int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping, int application, @Nullable java.nio.IntBuffer error)
Allocates and initializes a multistream encoder state.Call
multistream_encoder_destroy
to release this object when finished.- Parameters:
Fs
- sampling rate of the input signal (in Hz). One of:8000 12000 16000 24000 48000 streams
- the total number of streams to encode from the input. This must be no more than the number of channels.coupled_streams
- number of coupled (2 channel) streams to encode. This must be no larger than the total number of streams. Additionally, The total number of encoded channels (streams + coupled_streams
) must be no more than the number of input channels.mapping
- Mapping from encoded channels to input channels. As an extra constraint, the multistream encoder does not allow encoding coupled streams for which one channel is unused since this is never a good idea.application
- the target encoder applicationerror
- returnsOK
on success, or a negative error code on failure
-
nopus_multistream_surround_encoder_create
public static long nopus_multistream_surround_encoder_create(int Fs, int channels, int mapping_family, long streams, long coupled_streams, long mapping, int application, long error)
-
opus_multistream_surround_encoder_create
public static long opus_multistream_surround_encoder_create(int Fs, int mapping_family, java.nio.IntBuffer streams, java.nio.IntBuffer coupled_streams, java.nio.ByteBuffer mapping, int application, @Nullable java.nio.IntBuffer error)
-
nopus_multistream_encoder_init
public static int nopus_multistream_encoder_init(long st, int Fs, int channels, int streams, int coupled_streams, long mapping, int application)
Unsafe version of:multistream_encoder_init
- Parameters:
channels
- number of channels in the input signal. This must be at most 255. It may be greater than the number of coded channels (streams + coupled_streams
).
-
opus_multistream_encoder_init
public static int opus_multistream_encoder_init(long st, int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping, int application)
Initialize a previously allocated multistream encoder state.The memory pointed to by
st
must be at least the size returned bymultistream_encoder_get_size
. This is intended for applications which use their own allocator instead ofmalloc
.To reset a previously initialized state, use the
RESET_STATE
CTL.- Parameters:
st
- multistream encoder state to initializeFs
- sampling rate of input signal (Hz). One of:8000 12000 16000 24000 48000 streams
- the total number of streams to encode from the input. This must be no more than the number of channels.coupled_streams
- number of coupled (2 channel) streams to encode. This must be no larger than the total number of streams. Additionally, The total number of encoded channels (streams + coupled_streams
) must be no more than the number of input channels.mapping
- mapping from encoded channels to input channels. As an extra constraint, the multistream encoder does not allow encoding coupled streams for which one channel is unused since this is never a good idea.application
- the target encoder application- Returns:
OK
on success, or a negative error code on failure
-
nopus_multistream_surround_encoder_init
public static int nopus_multistream_surround_encoder_init(long st, int Fs, int channels, long streams, long coupled_streams, long mapping, int application)
-
opus_multistream_surround_encoder_init
public static int opus_multistream_surround_encoder_init(long st, int Fs, java.nio.IntBuffer streams, java.nio.IntBuffer coupled_streams, java.nio.ByteBuffer mapping, int application)
-
nopus_multistream_encode
public static int nopus_multistream_encode(long st, long pcm, int frame_size, long data, int max_data_bytes)
Unsafe version of:multistream_encode
- Parameters:
max_data_bytes
- size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as the only bitrate control. UseSET_BITRATE_REQUEST
to control the bitrate.
-
opus_multistream_encode
public static int opus_multistream_encode(long st, java.nio.ShortBuffer pcm, int frame_size, java.nio.ByteBuffer data)
Encodes a multistream Opus frame.- Parameters:
st
- multistream encoder statepcm
- the input signal as interleaved samples. This must containframe_size*channels
samples.frame_size
- number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the permitted values are 120, 240, 480, 960, 1920, and 2880. Passing in a duration of less than 10 ms (480 samples at 48 kHz) will prevent the encoder from using the LPC or hybrid modes.data
- output payload. This must contain storage for atmax_data_bytes
.- Returns:
- the length of the encoded packet (in bytes) on success or a negative error code on failure
-
nopus_multistream_encode_float
public static int nopus_multistream_encode_float(long st, long pcm, int frame_size, long data, int max_data_bytes)
Unsafe version of:multistream_encode_float
- Parameters:
max_data_bytes
- size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as the only bitrate control. UseSET_BITRATE_REQUEST
to control the bitrate.
-
opus_multistream_encode_float
public static int opus_multistream_encode_float(long st, java.nio.FloatBuffer pcm, int frame_size, java.nio.ByteBuffer data)
Encodes a multistream Opus frame from floating point input.- Parameters:
st
- multistream encoder statepcm
- the input signal as interleaved samples with a normal range of+/-1.0
. Samples with a range beyond+/-1.0
are supported but will be clipped by decoders using the integer API and should only be used if it is known that the far end supports extended dynamic range. This must containframe_size*channels
samples.frame_size
- number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the permitted values are 120, 240, 480, 960, 1920, and 2880. Passing in a duration of less than 10 ms (480 samples at 48 kHz) will prevent the encoder from using the LPC or hybrid modes.data
- output payload. This must contain storage for atmax_data_bytes
.- Returns:
- the length of the encoded packet (in bytes) on success or a negative error code on failure
-
opus_multistream_encoder_destroy
public static void opus_multistream_encoder_destroy(long st)
Frees anOpusMSEncoder
allocated bymultistream_encoder_create
.- Parameters:
st
- multistream encoder state to be freed
-
opus_multistream_decoder_get_size
public static int opus_multistream_decoder_get_size(int streams, int coupled_streams)
Gets the size of anOpusMSDecoder
structure.- Parameters:
streams
- the total number of streams coded in the input. This must be no more than 255.coupled_streams
- number streams to decode as coupled (2 channel) streams. This must be no larger than the total number of streams. Additionally, The total number of coded channels (streams + coupled_streams
) must be no more than 255.- Returns:
- the size in bytes on success, or a negative error code on failure
-
nopus_multistream_decoder_create
public static long nopus_multistream_decoder_create(int Fs, int channels, int streams, int coupled_streams, long mapping, long error)
Unsafe version of:multistream_decoder_create
- Parameters:
channels
- number of channels to decode. One of:1 2
-
opus_multistream_decoder_create
public static long opus_multistream_decoder_create(int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping, @Nullable java.nio.IntBuffer error)
Allocates and initializes a multistream decoder state.Call
multistream_decoder_destroy
to release this object when finished.- Parameters:
Fs
- sampling rate of input signal (Hz). One of:8000 12000 16000 24000 48000 streams
- the total number of streams coded in the input. This must be no more than 255.coupled_streams
- number streams to decode as coupled (2 channel) streams. This must be no larger than the total number of streams. Additionally, The total number of coded channels (streams + coupled_streams
) must be no more than 255.mapping
- mapping from coded channels to output channelserror
- returnsOK
on success, or a negative error code on failure
-
nopus_multistream_decoder_init
public static int nopus_multistream_decoder_init(long st, int Fs, int channels, int streams, int coupled_streams, long mapping)
Unsafe version of:multistream_decoder_init
- Parameters:
channels
- number of channels to decode. One of:1 2
-
opus_multistream_decoder_init
public static int opus_multistream_decoder_init(long st, int Fs, int streams, int coupled_streams, java.nio.ByteBuffer mapping)
Intialize a previously allocated decoder state object.The memory pointed to by
st
must be at least the size returned bymultistream_encoder_get_size
. This is intended for applications which use their own allocator instead ofmalloc
.To reset a previously initialized state, use the
RESET_STATE
CTL.- Parameters:
st
- multistream decoder state to initializeFs
- sampling rate of input signal (Hz). One of:8000 12000 16000 24000 48000 streams
- the total number of streams coded in the input. This must be no more than 255.coupled_streams
- number streams to decode as coupled (2 channel) streams. This must be no larger than the total number of streams. Additionally, The total number of coded channels (streams + coupled_streams
) must be no more than 255.mapping
- mapping from coded channels to output channels- Returns:
OK
on success, or a negative error code on failure
-
nopus_multistream_decode
public static int nopus_multistream_decode(long st, long data, int len, long pcm, int frame_size, int decode_fec)
Unsafe version of:multistream_decode
- Parameters:
len
- number of bytes in payload
-
opus_multistream_decode
public static int opus_multistream_decode(long st, @Nullable java.nio.ByteBuffer data, java.nio.ShortBuffer pcm, int frame_size, int decode_fec)
Decode a multistream Opus packet.- Parameters:
st
- multistream decoderdata
- input payload (Use aNULL
pointer to indicate packet loss)pcm
- output signal, with interleaved samples. This must contain room forframe_size*channels
samples.frame_size
- number of samples per channel of available space inpcm
.If this is less than the maximum packet duration (120ms; 5760 for 48kHz), this function will not be capable of decoding some packets. In the case of PLC (
data==NULL
) or FEC (decode_fec=1
), thenframe_size
needs to be exactly the duration of audio that is missing, otherwise the decoder will not be in the optimal state to decode the next incoming packet. For the PLC and FEC cases,frame_size
must be a multiple of 2.5 ms.decode_fec
- flag (0 or 1) to request that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it were lost.- Returns:
- number of decoded samples on success or a negative error code on failure
-
nopus_multistream_decode_float
public static int nopus_multistream_decode_float(long st, long data, int len, long pcm, int frame_size, int decode_fec)
Unsafe version of:multistream_decode_float
- Parameters:
len
- number of bytes in payload
-
opus_multistream_decode_float
public static int opus_multistream_decode_float(long st, @Nullable java.nio.ByteBuffer data, java.nio.FloatBuffer pcm, int frame_size, int decode_fec)
Decode a multistream Opus packet with floating point output.- Parameters:
st
- multistream decoderdata
- input payload. Use aNULL
pointer to indicate packet losspcm
- output signal, with interleaved samples. (This must contain room forframe_size*channels
samples.)frame_size
- number of samples per channel of available space in pcm.If this is less than the maximum packet duration (120ms; 5760 for 48kHz), this function will not be capable of decoding some packets. In the case of PLC (
data==NULL
) or FEC (decode_fec=1
), thenframe_size
needs to be exactly the duration of audio that is missing, otherwise the decoder will not be in the optimal state to decode the next incoming packet. For the PLC and FEC cases,frame_size
must be a multiple of 2.5 ms.decode_fec
- flag (0 or 1) to request that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it were lost.- Returns:
- number of decoded samples on success or a negative error code on failure
-
opus_multistream_decoder_destroy
public static void opus_multistream_decoder_destroy(long st)
Frees anOpusMSDecoder
allocated bymultistream_decoder_create
.- Parameters:
st
- multistream decoder state to be freed
-
opus_multistream_encoder_ctl
public static int opus_multistream_encoder_ctl(long st, int request) public static int opus_multistream_encoder_ctl(long st, Opus.CTLRequest request)
Performs a CTL function on an multistream Opus encoder.- Parameters:
st
- encoder staterequest
- CTL request
-
opus_multistream_decoder_ctl
public static int opus_multistream_decoder_ctl(long st, int request) public static int opus_multistream_decoder_ctl(long st, Opus.CTLRequest request)
Performs a CTL function on a multistream Opus decoder.- Parameters:
st
- decoder staterequest
- CTL request
-
-