View Source Unpickler (Unpickler v0.1.0)
Unpickler
is a library for loading data in the Python's
pickle format.
Supports all pickle protocols from 0 to 5.
Link to this section Summary
Functions
Loads the given pickle binary.
Link to this section Functions
Loads the given pickle binary.
Basic literals and data structures are deserialized as corresponding
Elixir terms whenever possible. The pickle data may include arbitrary
Python objects, so in all other cases the Unpickler.Object
struct
is used. This struct holds all the information that would be used for
object reconstruction. You can define a custom :object_resolver
function to recognise certain objects and map them to whatever data
structure you see fit.
object-references
Object references
Note that if the object hierarchy includes circular references, it is inherently impossible to represent in Elixir. One example of such data structure is a list with a reference to itself:
x = []
x.append(x)
On the other hand, multiple references to the same object are restored as expected, without duplicating memory, as in:
x = [1, 2, 3]
y = (x, x)
options
Options
:object_resolver
- a function for constructing a custom term corresponding to a Python object. ReceivesUnpickler.Object
as an argument and should return the term as{:ok, term}
or:error
if not applicable:persistent_id_resolver
- a function returning an object for the given persistent id. This function is required if the data includes a persistent id, otherwise an error is raised
examples
Examples
The scalar 1
would be loaded like so:
iex> Unpickler.load!(<<128, 4, 75, 1, 46>>)
{1, ""}
Next, a more complex data structure:
[1, 2.0, "text", (None, True), {"key": "val"}, b"\x01\x00"]
iex> data =
...> <<128, 4, 149, 47, 0, 0, 0, 0, 0, 0, 0, 93, 148, 40, 75, 1, 71, 64, 0, 0, 0, 0, 0, 0, 0,
...> 140, 4, 116, 101, 120, 116, 148, 78, 136, 134, 148, 125, 148, 140, 3, 107, 101, 121,
...> 148, 140, 3, 118, 97, 108, 148, 115, 67, 2, 1, 0, 148, 101, 46>>
iex> Unpickler.load!(data)
{[1, 2.0, "text", {nil, true}, %{"key" => "val"}, <<1, 0>>], ""}
objects
Objects
Other objects end up as Unpickler.Object
from datetime import date
date.fromisoformat("2022-05-17")
iex> data =
...> <<128, 4, 149, 32, 0, 0, 0, 0, 0, 0, 0, 140, 8, 100, 97, 116, 101, 116, 105, 109, 101,
...> 148, 140, 4, 100, 97, 116, 101, 148, 147, 148, 67, 4, 7, 230, 5, 17, 148, 133, 148, 82,
...> 148, 46>>
iex> Unpickler.load!(data)
{%Unpickler.Object{
append_items: [],
args: [<<7, 230, 5, 17>>],
constructor: "datetime.date",
kwargs: %{},
set_items: [],
state: nil
}, ""}
For those, we can customize construction by specifying an :object_resolver
from datetime import date
date.fromisoformat("2022-05-17")
iex> data =
...> <<128, 4, 149, 32, 0, 0, 0, 0, 0, 0, 0, 140, 8, 100, 97, 116, 101, 116, 105, 109, 101,
...> 148, 140, 4, 100, 97, 116, 101, 148, 147, 148, 67, 4, 7, 230, 5, 17, 148, 133, 148, 82,
...> 148, 46>>
iex> object_resolver = fn
...> # See https://github.com/python/cpython/blob/3.10/Lib/datetime.py#L1094-L1105
...> %{constructor: "datetime.date", args: [<<year_hi, year_lo, month, day>>]} ->
...> {:ok, date} = Date.new(year_hi * 256 + year_lo, month, day)
...> {:ok, date}
...>
...> _ ->
...> :error
...> end
iex> Unpickler.load!(data, object_resolver: object_resolver)
{~D[2022-05-17], ""}