# `IREE.Tokenizers.Model.WordPiece`
[🔗](https://github.com/goodhamgupta/iree_tokenizers/blob/v0.7.0/lib/iree/tokenizers/model/wordpiece.ex#L1)

WordPiece model specification compatible with `IREE.Tokenizers.Tokenizer.init/1`.

# `options`

```elixir
@type options() :: [
  unk_token: String.t(),
  max_input_chars_per_word: number(),
  continuing_subword_prefix: String.t()
]
```

Options for WordPiece model construction.

# `empty`

```elixir
@spec empty() :: {:ok, IREE.Tokenizers.Model.t()}
```

Returns an empty WordPiece model specification.

# `from_file`

```elixir
@spec from_file(String.t(), options()) ::
  {:ok, IREE.Tokenizers.Model.t()} | {:error, term()}
```

Builds a WordPiece model specification from a newline-delimited vocabulary file.

# `init`

```elixir
@spec init(%{required(String.t()) => integer()}, options()) ::
  {:ok, IREE.Tokenizers.Model.t()}
```

Builds a WordPiece model specification from an in-memory vocabulary.

---

*Consult [api-reference.md](api-reference.md) for complete listing*