View Source Electric.Utils (electric v1.0.1)
Summary
Functions
Return a list of values from enum
that are the maximal elements as calculated
by the given fun
.
Applies either an anonymous function or a MFA tuple, prepending the given arguments in case of an MFA.
File.copy!()
uses :file.copy()
under the hood. The latter reads the contents from the source
file and writes them into the destination file in chunks of 65536 bytes.
Apply a function to each element of an enumerable, recursively if the element is an enumerable itself.
Undo the obfuscation applied by obfuscate_password/1
.
Encode binary representation of a UUID into a string
Performs external merge sort on a file.
Flat map reduce that marks the last element of the enumerable.
Output a 2-tuple relation (table) reference as pg-style "schema"."table"
.
Map each value of the enumerable using a mapper and reverse the resulting list.
Apply a function to each value of a map.
Map each value of the enumerable using a mapper, unwrapping a result tuple returned by the mapper and stopping on error.
Merge a list of sorted files into a single file.
Merge a list of streams by taking the minimum element from each stream and emitting it and its stream. The streams are compared using the given comparator function.
Given a keyword list of database connection options, obfuscate the password by wrapping it in a zero-arity function.
Open a file, retrying if it doesn't exist yet, up to attempts_left
times, with 20ms delay between
attempts.
Parse a markdown table from a string
Parses quoted names. Lowercases unquoted names to match Postgres' case insensitivity.
Quote a string for use in SQL queries.
Format a relation tuple to be correctly escaped for use in SQL queries.
Transform the stream to call a side-effect function for each element before continuing.
Generate a random UUID v4.
Types
@type item_reader_fn(elem) :: (file :: :file.io_device() -> sortable_binary(elem) | :halt)
@type sortable_binary(key) :: {key :: key, data :: binary()}
Functions
all_max_by(enum, fun, sorter \\ &>=/2, comparator \\ &==/2, empty_fallback \\ fn -> raise Enum.EmptyError end)
View SourceReturn a list of values from enum
that are the maximal elements as calculated
by the given fun
.
Base behaviour is similar to Enum.max_by/4
, but this function returns a list
of all maximal values instead of just the first one.
Examples
iex> all_max_by([4, 1, 1, 3, -4], &abs/1)
[4, -4]
iex> all_max_by([4, 1, -1, 3, 4], &abs/1, &<=/2)
[1, -1]
iex> all_max_by([], &abs/1)
** (Enum.EmptyError) empty error
Applies either an anonymous function or a MFA tuple, prepending the given arguments in case of an MFA.
Examples
iex> apply_fn_or_mfa(&String.contains?(&1, "foo"), ["foobar"])
true
iex> apply_fn_or_mfa({String, :contains?, ["foo"]}, ["foobar"])
true
File.copy!()
uses :file.copy()
under the hood. The latter reads the contents from the source
file and writes them into the destination file in chunks of 65536 bytes.
@spec deep_map(Enumerable.t(elem), (elem -> result)) :: [result] when elem: var, result: var
Apply a function to each element of an enumerable, recursively if the element is an enumerable itself.
Examples
iex> deep_map([1, [2, [3]], 4], &(&1 * 2))
[2, [4, [6]], 8]
Undo the obfuscation applied by obfuscate_password/1
.
This function should be called just before passing connection options to one of
Postgrex
functions. Never store deobfuscated password in any of our process
states.
Encode binary representation of a UUID into a string
Examples
iex> encode_uuid(<<1, 35, 69, 103, 137, 171, 76, 222, 143, 227, 251, 149, 223, 249, 31, 215>>)
"01234567-89ab-4cde-8fe3-fb95dff91fd7"
external_merge_sort(path, reader, sorter \\ &<=/2, chunk_size \\ 52_428_800)
View SourcePerforms external merge sort on a file.
Parameters
path
- Path to the file to sort.reader
- Function that takes a file path and returns a stream of records. Records should be in the form of{key, binary}
, wherebinary
will be written to the file sorted bykey
.sorter
- Function that compares two keys, should return true if first argument is less than or equal to second.chunk_size
- Byte size of each chunk (i.e. how much is sorted in memory at once). Uses 52428800 bytes by default.
The function will:
Split the input file into sorted temporary chunks
Merge the sorted chunks back into the original file
Flat map reduce that marks the last element of the enumerable.
This is equivalent to Enum.flat_map_reduce/3
, but mapping function receives a boolean
indicating if the element is the last one.
Examples
iex> flat_map_reduce_mark_last(
...> [1, 2, 3],
...> 0,
...> fn
...> x, false, acc -> {[x], acc + x}
...> x, true, acc -> {[x * 2], acc + x}
...> end
...> )
{[1, 2, 6], 6}
Output a 2-tuple relation (table) reference as pg-style "schema"."table"
.
Examples
iex> inspect_relation({"schema", "table"})
~S|"schema"."table"|
@spec list_reverse_map(Enumerable.t(elem), (elem -> result), [result]) :: [result] when elem: var, result: var
Map each value of the enumerable using a mapper and reverse the resulting list.
Equivalent to Enum.reverse/1
followed by Enum.map/2
.
Examples
iex> list_reverse_map([1, 2, 3], &(&1 + 1))
[4, 3, 2]
Apply a function to each value of a map.
@spec map_while_ok(Enumerable.t(elem), (elem -> {:ok, result} | {:error, term()})) :: {:ok, [result]} | {:error, term()} when elem: var, result: var
Map each value of the enumerable using a mapper, unwrapping a result tuple returned by the mapper and stopping on error.
Examples
iex> map_while_ok(["2015-01-23 23:50:07.0", "2015-01-23 23:50:08"], &NaiveDateTime.from_iso8601/1)
{:ok, [~N[2015-01-23 23:50:07.0], ~N[2015-01-23 23:50:08]]}
iex> map_while_ok(["2015-01-23 23:50:07A", "2015-01-23 23:50:08"], &NaiveDateTime.from_iso8601/1)
{:error, :invalid_format}
@spec merge_sorted_files( paths :: [String.t()], target_path :: String.t(), reader :: item_reader_fn(elem), sorter :: (elem, elem -> boolean()) | module() ) :: :ok
Merge a list of sorted files into a single file.
Uses a reader function that takes a file descriptor and returns next {key, binary}
tuple,
where binary
will be written to the file as sorted by key
.
merge_sorted_streams(streams, comparator \\ &<=/2, mapper \\ & &1)
View SourceMerge a list of streams by taking the minimum element from each stream and emitting it and its stream. The streams are compared using the given comparator function.
Examples
iex> merge_sorted_streams([[1, 2, 3], [2, 3, 4]]) |> Enum.to_list()
[1, 2, 2, 3, 3, 4]
iex> merge_sorted_streams([[1, 2, 3], [4, 5, 6]]) |> Enum.to_list()
[1, 2, 3, 4, 5, 6]
iex> merge_sorted_streams([[10], [4, 5, 6]]) |> Enum.to_list()
[4, 5, 6, 10]
Given a keyword list of database connection options, obfuscate the password by wrapping it in a zero-arity function.
This should be done as early as possible when parsing connection options from the OS env. The aim of this obfuscation is to avoid accidentally leaking the password when inspecting connection opts or logging them as part of a process state (which is done automatically by OTP when a process that implements an OTP behaviour crashes).
Open a file, retrying if it doesn't exist yet, up to attempts_left
times, with 20ms delay between
attempts.
Parse a markdown table from a string
Options:
after:
- taking a first table that comes right after a given substring.
Example
iex> """
...> Some text
...>
...> ## Known types
...>
...> | type | category | preferred? |
...> | ----------------------- | -------- | ---------- |
...> | bool | boolean | t |
...> | int2 | numeric | |
...> """|> parse_md_table(after: "## Known types")
[["bool", "boolean", "t"], ["int2", "numeric", ""]]
iex> """
...> Some text
...> """|> parse_md_table([])
[]
Parses quoted names. Lowercases unquoted names to match Postgres' case insensitivity.
Examples
iex> parse_quoted_name("foo")
"foo"
iex> parse_quoted_name(~S|"foo"|)
"foo"
iex> parse_quoted_name(~S|"fo""o"|)
~S|fo"o|
iex> parse_quoted_name(~S|"FooBar"|)
~S|FooBar|
iex> parse_quoted_name(~S|FooBar|)
~S|FooBar|
Quote a string for use in SQL queries.
Examples
iex> quote_name("foo")
~S|"foo"|
iex> quote_name(~S|fo"o|)
~S|"fo""o"|
Format a relation tuple to be correctly escaped for use in SQL queries.
Examples
iex> relation_to_sql({"public", "items"})
~S|public.items|
iex> relation_to_sql({"public", "items"}, true)
~S|"public"."items"|
iex> relation_to_sql({"public", "items-again"})
~S|public."items-again"|
iex> relation_to_sql({"public", "99red_balloons"})
~S|public."99red_balloons"|
iex> relation_to_sql({"public", "when"})
~S|public."when"|
iex> relation_to_sql({"with spaces", ~S|and "quoted"!|})
~S|"with spaces"."and ""quoted""!"|
stream_add_side_effect(stream, start_fun, reducer, last_fun \\ & &1, after_fun \\ & &1)
View SourceTransform the stream to call a side-effect function for each element before continuing.
Acts like Stream.each/2
but with an aggregate. start_fun
, last_fun
, after_fun
have the same semantics as in Stream.transform/5
@spec stream_file_items( path :: String.t(), reader :: item_reader_fn(elem) ) :: Enumerable.t(sortable_binary(elem))
Generate a random UUID v4.
Code taken from Ecto: https://github.com/elixir-ecto/ecto/blob/v3.10.2/lib/ecto/uuid.ex#L174
Examples
iex> Regex.match?(~r/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/, uuid4())
true