Efficient in-memory storage of multiple similar arrays. More...

#include "BLI_sys_types.h"

Typedefs
typedef struct BArrayState	BArrayState
typedef struct BArrayStore	BArrayStore

Functions
BArrayStore *	BLI_array_store_create (unsigned int stride, unsigned int chunk_count)
void	BLI_array_store_destroy (BArrayStore *bs)
void	BLI_array_store_clear (BArrayStore *bs)
size_t	BLI_array_store_calc_size_expanded_get (const BArrayStore *bs)
size_t	BLI_array_store_calc_size_compacted_get (const BArrayStore *bs)
BArrayState *	BLI_array_store_state_add (BArrayStore bs, const void data, size_t data_len, const BArrayState *state_reference)
void	BLI_array_store_state_remove (BArrayStore bs, BArrayState state)
size_t	BLI_array_store_state_size_get (const BArrayState *state)
void	BLI_array_store_state_data_get (const BArrayState state, void data)
void *	BLI_array_store_state_data_get_alloc (const BArrayState state, size_t r_data_len)
bool	BLI_array_store_is_valid (BArrayStore *bs)
uint8_t *	BLI_array_store_rle_encode (const uint8_t data_dec, size_t data_dec_len, size_t data_enc_extra_size, size_t r_data_enc_len)
void	BLI_array_store_rle_decode (const uint8_t data_enc, const size_t data_enc_len, void data_dec_v, const size_t data_dec_len)

Detailed Description

Efficient in-memory storage of multiple similar arrays.

Definition in file BLI_array_store.h.

Typedef Documentation

◆ BArrayState

typedef struct BArrayState BArrayState

Definition at line 14 of file BLI_array_store.h.

◆ BArrayStore

typedef struct BArrayStore BArrayStore

Definition at line 15 of file BLI_array_store.h.

Function Documentation

◆ BLI_array_store_calc_size_compacted_get()

size_t BLI_array_store_calc_size_compacted_get ( const BArrayStore * bs )

Returns: the amount of memory used by all BChunk.data (duplicate chunks are only counted once).

Definition at line 1607 of file array_store.cc.

References BLI_assert, BLI_mempool_iternew(), BLI_mempool_iterstep(), BArrayMemory::chunk, BChunk::data_len, BArrayStore::memory, and BChunk::users.

Referenced by BLI_array_store_at_size_calc_memory_usage(), random_chunk_mutate_helper(), TEST(), TEST(), TEST(), and text_undosys_step_encode_to_state().

◆ BLI_array_store_calc_size_expanded_get()

size_t BLI_array_store_calc_size_expanded_get ( const BArrayStore * bs )

Find the memory used by all states (expanded & real).

Returns: the total amount of memory that would be used by getting the arrays for all states.

Definition at line 1598 of file array_store.cc.

References LISTBASE_FOREACH, state, and BArrayStore::states.

Referenced by BLI_array_store_at_size_calc_memory_usage(), TEST(), and TEST().

◆ BLI_array_store_clear()

void BLI_array_store_clear ( BArrayStore * bs )

Clear all contents, allowing reuse of bs.

Definition at line 1581 of file array_store.cc.

References array_store_free_data(), BLI_listbase_clear(), BLI_mempool_clear(), BArrayMemory::chunk, BArrayMemory::chunk_list, BArrayMemory::chunk_ref, BArrayStore::memory, and BArrayStore::states.

◆ BLI_array_store_create()

BArrayStore * BLI_array_store_create	(	unsigned int	stride,
		unsigned int	chunk_count )

Create a new array store, which can store any number of arrays as long as their stride matches.

Parameters

stride sizeof() each element,

Note: while a stride of 1 will always work, its less efficient since duplicate chunks of memory will be searched at positions unaligned with the array data.

Parameters

chunk_count

Number of elements to split each chunk into.

A small value increases the ability to de-duplicate chunks, but adds overhead by increasing the number of chunks to look up when searching for duplicates, as well as some overhead constructing the original array again, with more calls to memcpy.
Larger values reduce the book keeping overhead, but increase the chance a small, isolated change will cause a larger amount of data to be duplicated.

Returns: A new array store, to be freed with BLI_array_store_destroy.

Definition at line 1493 of file array_store.cc.

References BArrayInfo::accum_read_ahead_bytes, BArrayInfo::accum_read_ahead_len, BArrayInfo::accum_steps, BCHUNK_HASH_TABLE_ACCUMULATE_STEPS_16BITS, BCHUNK_HASH_TABLE_ACCUMULATE_STEPS_32BITS, BCHUNK_HASH_TABLE_ACCUMULATE_STEPS_8BITS, BCHUNK_HASH_TABLE_ACCUMULATE_STEPS_DEFAULT, BCHUNK_SIZE_MAX_MUL, BCHUNK_SIZE_MIN_DIV, BLI_assert, BLI_MEMPOOL_ALLOW_ITER, BLI_mempool_create(), BLI_MEMPOOL_NOP, BArrayMemory::chunk, BArrayInfo::chunk_byte_size, BArrayInfo::chunk_byte_size_max, BArrayInfo::chunk_byte_size_min, BArrayMemory::chunk_list, BArrayMemory::chunk_ref, BArrayInfo::chunk_stride, BArrayStore::info, MEM_callocN(), BArrayStore::memory, and UNLIKELY.

Referenced by BLI_array_store_at_size_ensure(), random_chunk_mutate_helper(), TEST(), TEST(), TEST(), TEST(), TEST(), TEST(), testbuffer_run_tests_simple(), and text_undosys_step_encode_to_state().

◆ BLI_array_store_destroy()

void BLI_array_store_destroy ( BArrayStore * bs )

Free the BArrayStore, including all states and chunks.

Definition at line 1570 of file array_store.cc.

References array_store_free_data(), BLI_mempool_destroy(), BArrayMemory::chunk, BArrayMemory::chunk_list, BArrayMemory::chunk_ref, MEM_freeN(), and BArrayStore::memory.

Referenced by BLI_array_store_at_size_clear(), random_chunk_mutate_helper(), TEST(), TEST(), TEST(), TEST(), TEST(), TEST(), testbuffer_run_tests_simple(), and text_undosys_step_free().

◆ BLI_array_store_is_valid()

bool BLI_array_store_is_valid ( BArrayStore * bs )

◆ BLI_array_store_rle_decode()

void BLI_array_store_rle_decode	(	const uint8_t *	data_enc,
		const size_t	data_enc_len,
		void *	data_dec_v,
		const size_t	data_dec_len )

Decode a run-length encoded array, writing the result into data_dec_v.

Parameters

data_enc	The data to encode (returned by BLI_array_store_rle_encode).
data_enc_len	The size of data_enc.
data_dec	The destination for the decoded data to be written to.
data_dec_len	The size of the destination (as passed to BLI_array_store_rle_encode).

Definition at line 380 of file array_store_rle.cc.

References BLI_assert, e, and UNUSED_VARS_NDEBUG.

Referenced by rle_encode_decode_test(), and um_arraystore_cd_expand().

◆ BLI_array_store_rle_encode()

uint8_t * BLI_array_store_rle_encode	(	const uint8_t *	data_dec,
		size_t	data_dec_len,
		size_t	data_enc_extra_size,
		size_t *	r_data_enc_len )

Return a run-length encoded copy of data_dec.

Parameters

data_dec	The data to encode.
data_dec_len	The size of the data to encode.
data_enc_extra_size	Allocate extra memory at the beginning of the array. This doesn't impact the value of r_data_enc_len. This must be skipped when decoding.
r_data_enc_len	The size of the resulting RLE encoded data.

Definition at line 244 of file array_store_rle.cc.

References BLI_assert, e, find_byte_not_equal_to(), LIKELY, MEM_malloc_arrayN(), rle_link_chunk_elem_new(), rle_link_chunk_free_all(), rle_link_chunk_iter_new(), rle_link_chunk_iter_step(), rle_link_chunk_new(), UNLIKELY, and value_start.

Referenced by rle_encode_decode_test(), and um_arraystore_cd_compact().

◆ BLI_array_store_state_add()

BArrayState * BLI_array_store_state_add	(	BArrayStore *	bs,
		const void *	data,
		size_t	data_len,
		const BArrayState *	state_reference )

Parameters

data	Data used to create
state_reference	The state to use as a reference when adding the new state, typically this is the previous state, however it can be any previously created state from this bs.

Returns: The new state, which is used by the caller as a handle to get back the contents of data. This may be removed using BLI_array_store_state_remove, otherwise it will be removed with BLI_array_store_destroy.

Definition at line 1626 of file array_store.cc.

References bchunk_list_fill_from_array(), bchunk_list_from_data_merge(), bchunk_list_new(), BLI_addtail(), BLI_array_store_state_data_get_alloc(), BLI_assert, BLI_findindex(), BArrayState::chunk_list, BArrayInfo::chunk_stride, data, BArrayStore::info, MEM_callocN(), MEM_freeN(), BArrayStore::memory, state, BArrayStore::states, and BChunkList::users.