Tinkex.ByteEstimator (Tinkex v0.3.4)

View Source

Byte size estimation for model inputs and datums.

Provides consistent heuristics across training chunking and sampling dispatch:

  • Image chunks: raw byte size of the data string
  • Image asset pointers: byte size of the location string
  • Encoded text and other length-aware chunks: token count * 10 bytes
  • Loss function inputs (TensorData / tensors / plain maps): element count * 10 bytes

Summary

Functions

estimate_chunk_bytes(chunk)

Estimate byte size of a single ModelInput chunk.

estimate_data_bytes(data)

Estimate byte size for a list of datums.

estimate_datum_bytes(arg1)

Estimate byte size of a Datum.

estimate_loss_fn_inputs_bytes(loss_fn_inputs)

Estimate byte size of loss function inputs map.

estimate_model_input_bytes(arg1)

Estimate byte size of a ModelInput.

Functions

estimate_chunk_bytes(chunk)

@spec estimate_chunk_bytes(struct()) :: non_neg_integer()

Estimate byte size of a single ModelInput chunk.

estimate_data_bytes(data)

@spec estimate_data_bytes([Tinkex.Types.Datum.t()]) :: non_neg_integer()

Estimate byte size for a list of datums.

estimate_datum_bytes(arg1)

@spec estimate_datum_bytes(Tinkex.Types.Datum.t() | map()) :: non_neg_integer()

Estimate byte size of a Datum.

estimate_loss_fn_inputs_bytes(loss_fn_inputs)

@spec estimate_loss_fn_inputs_bytes(map() | any()) :: non_neg_integer()

Estimate byte size of loss function inputs map.

estimate_model_input_bytes(arg1)

@spec estimate_model_input_bytes(Tinkex.Types.ModelInput.t() | any()) ::
  non_neg_integer()

Estimate byte size of a ModelInput.