# LLama CPP Nif Wrapper v0.2.1 - Table of Contents

NIF Wrapper around llama.cpp allowing elixir code to load/infer against gguf format models.

## Pages

- [ExLLama: LlammaCpp.rs NIF wrapper for Elixir/Erlang.](readme.md)
- [LICENSE](license.md)

## Modules

- [ExLLama](ExLLama.md)
- [ExLLama.ChatTemplate](ExLLama.ChatTemplate.md)
- [ExLLama.ChatTemplate.Alpaca](ExLLama.ChatTemplate.Alpaca.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/alpaca.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = messages[0]['content'].strip() + '
- [ExLLama.ChatTemplate.AmberChat](ExLLama.ChatTemplate.AmberChat.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/amberchat.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = messages[0]['content'].strip() + '
' %}
{% else %}
  {% set loop_messages = messages %}
  {% set system_message = '' %}
{% endif %}
- [ExLLama.ChatTemplate.ChatML](ExLLama.ChatTemplate.ChatML.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/chatml.jinja]
```jinja
  {% if messages[0]['role'] == 'system' %}
  {% set offset = 1 %}
{% else %}
  {% set offset = 0 %}
{% endif %}
- [ExLLama.ChatTemplate.ChatQA](ExLLama.ChatTemplate.ChatQA.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/chatqa.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
    {% set system_message = 'System: ' + messages[0]['content'] | trim %}
    {% set messages = messages[1:] %}
{% else %}
    {% set system_message = '' %}
{% endif %}
- [ExLLama.ChatTemplate.FalconInstruct](ExLLama.ChatTemplate.FalconInstruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/falcon-instruct.jinja]]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = messages[0]['content'] %}
{% else %}
  {% set loop_messages = messages %}
  {% set system_message = '' %}
{% endif %}
- [ExLLama.ChatTemplate.GemmaInstruct](ExLLama.ChatTemplate.GemmaInstruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/gemma-it.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = messages[0]['content'].strip() + '
- [ExLLama.ChatTemplate.GraniteInstruct](ExLLama.ChatTemplate.GraniteInstruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/granite-3.0-instruct.jinja]
```jinja
{%- if tools %}
    {{- '<|start_of_role|>available_tools<|end_of_role|>
' }}
    {%- for tool in tools %}
        {{- tool | tojson(indent=4) }}
        {%- if not loop.last %}
            {{- '
- [ExLLama.ChatTemplate.LLama2Chat](ExLLama.ChatTemplate.LLama2Chat.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/llama-2-chat.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = '<<SYS>>
' + messages[0]['content'].strip() + '
<</SYS>>
- [ExLLama.ChatTemplate.Llama3Instruct](ExLLama.ChatTemplate.Llama3Instruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/llama-3-instruct.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
    {% set offset = 1 %}
{% else %}
    {% set offset = 0 %}
{% endif %}
- [ExLLama.ChatTemplate.MistralInstruct](ExLLama.ChatTemplate.MistralInstruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/mistral-instruct.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = messages[0]['content'].strip() + '
- [ExLLama.ChatTemplate.OpenChat](ExLLama.ChatTemplate.OpenChat.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/openchat.jinja]]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = messages[0]['content'].strip() + '<|end_of_turn|>' %}
{% else %}
  {% set loop_messages = messages %}
  {% set system_message = '' %}
{% endif %}
- [ExLLama.ChatTemplate.Phi3](ExLLama.ChatTemplate.Phi3.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/phi-3.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
    {% set offset = 1 %}
{% else %}
    {% set offset = 0 %}
{% endif %}
- [ExLLama.ChatTemplate.Phi3Small](ExLLama.ChatTemplate.Phi3Small.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/phi-3-small.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
    {% set offset = 1 %}
{% else %}
    {% set offset = 0 %}
{% endif %}
- [ExLLama.ChatTemplate.QwenInstruct](ExLLama.ChatTemplate.QwenInstruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/qwen2.5-instruct.jinja]
- [ExLLama.ChatTemplate.Saiga](ExLLama.ChatTemplate.Saiga.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/saiga.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set loop_messages = messages[1:] %}
  {% set system_message = bos_token + 'system' + '
' + messages[0]['content'].strip() + eos_token %}
{% else %}
  {% set loop_messages = messages %}
  {% set system_message = '' %}
{% endif %}
- [ExLLama.ChatTemplate.SolarInstruct](ExLLama.ChatTemplate.SolarInstruct.md): based on: [https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/solar-instruct.jinja]
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set offset = 1 %}
{% else %}
  {% set offset = 0 %}
{% endif %}
- [ExLLama.ChatTemplate.Vicuna](ExLLama.ChatTemplate.Vicuna.md): based on: https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/zephyr.jinja
```jinja
  {% if messages[0]['role'] == 'system' %}
    {% set loop_messages = messages[1:] %}
    {% set system_message = messages[0]['content'].strip() + '
- [ExLLama.ChatTemplate.Zephyr](ExLLama.ChatTemplate.Zephyr.md): based on: https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/zephyr.jinja
```jinja
{% if messages[0]['role'] == 'system' %}
  {% set offset = 1 %}
{% else %}
  {% set offset = 0 %}
{% endif %}
- [ExLLama.ContextParams](ExLLama.ContextParams.md)
- [ExLLama.EmbeddingOptions](ExLLama.EmbeddingOptions.md)
- [ExLLama.Model](ExLLama.Model.md)
- [ExLLama.ModelOptions](ExLLama.ModelOptions.md)
- [ExLLama.Nif](ExLLama.Nif.md): C NIF bindings for llama.cpp — bridges old API names to new C NIF functions
and wraps raw references in the expected structs.

- [ExLLama.Session](ExLLama.Session.md)
- [ExLLama.SessionOptions](ExLLama.SessionOptions.md)

- Exceptions
  - [ExLLama.ChatTemplate.Exception](ExLLama.ChatTemplate.Exception.md)