16. Compression

Allows the user to open a Gzip archive and read and write it's contents in an uncompressed form, emulating the Stdio.File interface.

An important limitation on this class is that it may only be used for reading or writing, not both at the same time. Please also note that if you want to reopen a file for reading after a write, you must close the file before calling open or strange effects might be the result.

Filename or filedescriptor of the gzip file to open, or an already open Stream.

mode for the file. Defaults to "rb".

open Stdio.File

Returns an iterator that will loop over the lines in this file. If trim is true, all '\r' characters will be removed from the input.

Filename or filedescriptor of the gzip file to open, or an already open Stream.

mode for the file. Defaults to "rb". May be one of the following:

rb

read mode

wb

write mode

ab

append mode

For the wb and ab mode, additional parameters may be specified. Please se zlib manual for more info.

non-zero if successful.

Reads data from the file. If no argument is given, the whole file is read.

Returns a function that when called will call read with nbytes as argument. Can be used to get various callback functions, eg for the fourth argument to String.SplitIterator.

Low-level implementation of read/write support for GZip files

closes the file

1 if successful

Opens a gzip file for reading.

1 if EOF has been reached.

Opens a file for I/O.

The filename or an open filedescriptor or Stream for the GZip file to use.

Mode for the file operations. Defaults to read only. The following mode characters are unique to Gz.File.

If the object already has been opened, it will first be closed.

Reads len (uncompressed) bytes from the file. If read is unsuccessful, 0 is returned.

Seeks within the file.

Position relative to the searchtype.

SEEK_SET = set current position in file to pos SEEK_CUR = new position is current+pos SEEK_END is not supported.

New position or negative number if seek failed.

Sets the encoding level, strategy and window_size.

Gz.deflate

the current position within the file.

Writes the data to the file.

the number of bytes written to the file.

This class interfaces with the compression routines in the libz library.

This class is only available if libz was available and found when Pike was compiled.

Gz.inflate(), Gz.compress(), Gz.uncompress()

Clones the deflate object. Typically used to test compression of new content using the same exact state.

This function can also be used to re-initialize a Gz.deflate object so it can be re-used.

If a mapping is passed as the only argument, it will accept the parameters described below as indices, and additionally it accepts a string as dictionary.

Indicates the level of effort spent to make the data compress well. Zero means no packing, 2-3 is considered 'fast', 6 is default and higher is considered 'slow' but gives better packing.

If the argument is negative, no headers will be emitted. This is needed to produce ZIP-files, as an example. The negative value is then negated, and handled as a positive value.

The strategy to be used when compressing the data. One of the following.

Defines the size of the LZ77 window from 256 bytes to 32768 bytes, expressed as 2^x.

This function performs gzip style compression on a string data and returns the packed data. Streaming can be done by calling this function several times and concatenating the returned data.

The optional argument flush should be one of the following:

Gz.inflate->inflate()

This class interfaces with the uncompression routines in the libz library.

This program is only available if libz was available and found when Pike was compiled.

deflate, compress, uncompress

If called with a mapping as only argument, create accepts the entries window_size (described below) and dictionary, which is a string to be set as dictionary.

The window_size value is passed down to inflateInit2 in zlib.

If the argument is negative, no header checks are done, and no verification of the data will be done either. This is needed for uncompressing ZIP-files, as an example. The negative value is then negated, and handled as a positive value.

Positive arguments set the maximum dictionary size to an exponent of 2, such that 8 (the minimum) will cause the window size to be 256, and 15 (the maximum, and default value) will cause it to be 32Kb. Setting this to anything except 15 is rather pointless in Pike.

It can be used to limit the amount of memory that is used to uncompress files, but 32Kb is not all that much in the great scheme of things.

To decompress files compressed with level 9 compression, a 32Kb window size is needed. level 1 compression only requires a 256 byte window.

If the options version is used you can specify your own dictionary in addition to the window size.

This function returns 0 if the end of stream marker has not yet been encountered, or a string (possibly empty) containg any extra data received following the end of stream marker if the marker has been encountered. If the extra data is not needed, the result of this function can be treated as a logical value.

This function performs gzip style decompression. It can inflate a whole file at once or in blocks.

Gz.deflate->deflate(), Gz.uncompress

Bz2.Deflate is a builtin program written in C. It interfaces the packing routines in the bzlib library.

This program is only available if libz was available and found when Pike was compiled.

Bz2.Inflate()

If given, block_size should be a number from 1 to 9 indicating the block size used when doing compression. The actual block size will be a 100000 times this number. Low numbers are considered 'fast', higher numbers are considered 'slow' but give better packing. The parameter is set to 9 if it is omitted.

This function can also be used to re-initialize a Bz2.Deflate object so it can be re-used.

This function performs bzip2 style compression on a string data and returns the packed data. Streaming can be done by calling this function several times and concatenating the returned data.

The optional argument flush_mode should be one of the following:

Bz2.Inflate->inflate()

This function feeds the data to the internal buffers of the Deflate object. All data is buffered until a read or a finish is done.

Bz2.Deflate->read() Bz2.Deflate->finish()

This method feeds the data to the internal buffers of the Deflate object. Then it compresses all buffered data adds a end of data marker ot it, returns the compressed data as a string, and reinitializes the deflate object.

Bz2.Deflate->feed() Bz2.Deflate->read()

This function feeds the data to the internal buffers of the Deflate object. Then it compresses all buffered data and returns the compressed data as a string

Bz2.Deflate->feed() Bz2.Deflate->finish()

Low-level implementation of read/write support for Bzip2 files

This class is currently not available on Windows.

closes the file

Creates a Bz2.File object

1 if EOF has been reached, 0 otherwise

Returns an iterator that will loop over the lines in this file. If trim is true, all '\r' characters will be removed from the input.

Opens a file for I/O.

The name of the file to be opened

Mode for the file operations. Can be either "r" (read) or "w". Read is default.

Reads len (uncompressed) bytes from the file. If len is omitted the whole file is read. If read is unsuccessful, 0 is returned.

Returns a function that when called will call read with nbytes as argument. Can be used to get various callback functions, eg for the fourth argument to String.SplitIterator.

Opens a file for reading.

The name of the file to be opened

Writes the data to the file.

the number of bytes written to the file.

Opens a file for writing.

The name of the file to be opened

Bz2.Inflate is a builtin program written in C. It interfaces the unpacking routines in the libz library.

This program is only available if bzlib was available and found when Pike was compiled.

Deflate

This function performs bzip2 style decompression. It can do decompression with arbitrarily large pieces of data. When fed with data, it decompresses as much as it can and buffers the rest.

Bz2.Deflate->deflate()

Flags for Context()->encode_header() et al.

Indexed header.

Bitmask for indexing mode.

Never indexed header.

Unindexed header.

Context for an HPack encoder or decoder.

This class implements the majority of RFC 7541.

Functions of interest are typically encode() and decode().

Table of currently available dynamically defined headers.

New entries are appended last, and the first dynamic_prefix elements are not used.

header_index, add_header()

Current upper size limit in bytes for dynamic_headers.

set_dynamic_size()

Index of first avaiable header in dynamic_headers.

Current size in bytes of dynamic_headers.

Index into dynamic_headers and static_headers.

The index values in turn are coded as follows:

dynamic_headers, static_header_tab, add_header()

Static upper size limit in bytes for dynamic_headers.

create(), set_dynamic_size()

Add a header to the table of known headers and to the header index.

Name of header to add.

Value of the header.

Returns 0 (zero) if the header was too large to store. Returns the encoding key for the header on success (this is always sizeof(static_header_tab + 1 (ie 62), as new headers are prepended to the dynamic header table.

Adding a header may cause old headers to be evicted from the table.

get_indexed_header()

Create a new HPack Context.

This is the static maximum size in bytes (as calculated by RFC 7541 section 4.1) of the dynamic header table. It defaults to DEFAULT_HEADER_TABLE_SIZE, and is the upper limit for set_dynamic_size().

set_dynamic_size()

Decode a HPack header block.

Input buffer.

Returns an array of headers. Cf decode_header().

decode_header(), encode()

Decode a single HPack header.

Input buffer.

Returns UNDEFINED on empty buffer. Returns an array with a header and value otherwise:

The elements in the array are in the same order and compatible with the arguments to encode_header().

Throws on encoding errors.

The returned array MUST NOT be modified.

In future implementations the result array may get extended with a flag field.

The in-band signalling of encoding table sizes is handled internally.

decode(), encode_header()

Encode a full set of headers.

An array of ({ header, value })-tuples.

Output buffer.

encode_header(), decode()

Convenience variant of encode().

An array of ({ header, value })-tuples.

Returns the corresponding HPack encoding.

Encode a single HPack header.

Output buffer.

Name of header. This should under normal circumstances be a lower-case string, but this is currently not checked.

Header value.

Optional encoding flags.

encode(), decode_header()

Evict dynamic headers until dynamic_size goes below dynamic_max_size.

Lookup a known header.

Encoding key for the header to retrieve.

Returns UNDEFINED on unknown header. Returns an array with a header and value otherwise:

add_header()

Encode an integer with the HPack integer encoding.

Output buffer.

Bits that should always be set in the first byte of output.

Bitmask for the value part of the first byte of output.

Integer value to encode.

Encode a string with the HPack string encoding.

Output buffer.

String to output.

The encoder will huffman_encode() the string if that renders a shorter encoding than the verbatim string.

Set the dynamic maximum size of the dynamic header lookup table.

Output buffer.

New dynamic maximum size in bytes (as calculated by RFC 7541 section 4.1).

This function can be used to clear the dynamic header table by setting the size to zero.

Also note that the new_max_size has an upper bound that is limited by static_max_size.

encode_header(), encode(), create().