# Lorax.Config (lorax v0.2.1)

Config for `Lorax.inject/2`

`r`

is the rank in the low-rank matrices used in LoRA.
A higher value of r increases the expressiveness of the adaptation,
However, it also increases the number of parameters and the computational
cost. Conversely, a lower value of r makes the adaptation simpler and less
resource-intensive. Defaults to 1.

`alpha`

is a scaling factor that controls the magnitude of changes introduced
by the low-rank matrices. A higher value of `alpha`

means that the
modifications made by LoRA have a greater impact on the model's original
weights. This can lead to more significant changes in the model's behavior.
A lower value results in more subtle changes. Defaults to 2.

`dropout`

specifies the dropout rate applied to the low-rank matrices.

`dropout_seed`

determines the seed used for `Nx.Random.key/1`

during
dropout application. When defined, it ensures that the LoRA adapter
produces consistent tensor values, assuming that other layers also have
deterministic outputs.

`param_type`

specifies the numerical representation for the A and B
matrices. Defaults to float32

`target_query`

specifies whether to apply LoRA to all query matrices in an
attention block. Defaults to true.

`target_value`

specifies whether to apply LoRA to all value matrices in an
attention block. Defaults to true.

`target_key`

specifies whether to apply LoRA to all key matrices in an
attention block. Defaults to true.