View Source Electric.Utils (electric v1.0.1)

Summary

Functions

Return a list of values from enum that are the maximal elements as calculated by the given fun.

Applies either an anonymous function or a MFA tuple, prepending the given arguments in case of an MFA.

File.copy!() uses :file.copy() under the hood. The latter reads the contents from the source file and writes them into the destination file in chunks of 65536 bytes.

Apply a function to each element of an enumerable, recursively if the element is an enumerable itself.

Encode binary representation of a UUID into a string

Flat map reduce that marks the last element of the enumerable.

Output a 2-tuple relation (table) reference as pg-style "schema"."table".

Map each value of the enumerable using a mapper and reverse the resulting list.

Apply a function to each value of a map.

Map each value of the enumerable using a mapper, unwrapping a result tuple returned by the mapper and stopping on error.

Merge a list of sorted files into a single file.

Merge a list of streams by taking the minimum element from each stream and emitting it and its stream. The streams are compared using the given comparator function.

Given a keyword list of database connection options, obfuscate the password by wrapping it in a zero-arity function.

Open a file, retrying if it doesn't exist yet, up to attempts_left times, with 20ms delay between attempts.

Parse a markdown table from a string

Parses quoted names. Lowercases unquoted names to match Postgres' case insensitivity.

Quote a string for use in SQL queries.

Format a relation tuple to be correctly escaped for use in SQL queries.

Transform the stream to call a side-effect function for each element before continuing.

Generate a random UUID v4.

Types

@type item_reader_fn(elem) :: (file :: :file.io_device() ->
                           sortable_binary(elem) | :halt)
@type sortable_binary(key) :: {key :: key, data :: binary()}

Functions

Link to this function

all_max_by(enum, fun, sorter \\ &>=/2, comparator \\ &==/2, empty_fallback \\ fn -> raise Enum.EmptyError end)

View Source

Return a list of values from enum that are the maximal elements as calculated by the given fun.

Base behaviour is similar to Enum.max_by/4, but this function returns a list of all maximal values instead of just the first one.

Examples

iex> all_max_by([4, 1, 1, 3, -4], &abs/1)
[4, -4]

iex> all_max_by([4, 1, -1, 3, 4], &abs/1, &<=/2)
[1, -1]

iex> all_max_by([], &abs/1)
** (Enum.EmptyError) empty error
Link to this function

apply_fn_or_mfa(fun, args)

View Source

Applies either an anonymous function or a MFA tuple, prepending the given arguments in case of an MFA.

Examples

iex> apply_fn_or_mfa(&String.contains?(&1, "foo"), ["foobar"])
true

iex> apply_fn_or_mfa({String, :contains?, ["foo"]}, ["foobar"])
true
Link to this function

concat_files(source_paths, dst_path)

View Source
@spec concat_files([Path.t()], Path.t()) :: :ok

File.copy!() uses :file.copy() under the hood. The latter reads the contents from the source file and writes them into the destination file in chunks of 65536 bytes.

@spec deep_map(Enumerable.t(elem), (elem -> result)) :: [result]
when elem: var, result: var

Apply a function to each element of an enumerable, recursively if the element is an enumerable itself.

Examples

iex> deep_map([1, [2, [3]], 4], &(&1 * 2))
[2, [4, [6]], 8]
Link to this function

deobfuscate_password(connection_opts)

View Source
@spec deobfuscate_password(Keyword.t()) :: Keyword.t()

Undo the obfuscation applied by obfuscate_password/1.

This function should be called just before passing connection options to one of Postgrex functions. Never store deobfuscated password in any of our process states.

Encode binary representation of a UUID into a string

Examples

iex> encode_uuid(<<1, 35, 69, 103, 137, 171, 76, 222, 143, 227, 251, 149, 223, 249, 31, 215>>)
"01234567-89ab-4cde-8fe3-fb95dff91fd7"
Link to this function

external_merge_sort(path, reader, sorter \\ &<=/2, chunk_size \\ 52_428_800)

View Source

Performs external merge sort on a file.

Parameters

  • path - Path to the file to sort.

  • reader - Function that takes a file path and returns a stream of records. Records should be in the form of {key, binary}, where binary will be written to the file sorted by key.

  • sorter - Function that compares two keys, should return true if first argument is less than or equal to second.

  • chunk_size - Byte size of each chunk (i.e. how much is sorted in memory at once). Uses 52428800 bytes by default.

The function will:

  1. Split the input file into sorted temporary chunks

  2. Merge the sorted chunks back into the original file

Link to this function

flat_map_reduce_mark_last(enum, acc, fun)

View Source

Flat map reduce that marks the last element of the enumerable.

This is equivalent to Enum.flat_map_reduce/3, but mapping function receives a boolean indicating if the element is the last one.

Examples

iex> flat_map_reduce_mark_last(
...>   [1, 2, 3],
...>   0,
...>   fn
...>     x, false, acc -> {[x], acc + x}
...>     x, true, acc -> {[x * 2], acc + x}
...>   end
...> )
{[1, 2, 6], 6}
@spec inspect_relation({String.t(), String.t()}) :: String.t()

Output a 2-tuple relation (table) reference as pg-style "schema"."table".

Examples

iex> inspect_relation({"schema", "table"})
~S|"schema"."table"|
Link to this function

list_reverse_map(list, mapper, acc \\ [])

View Source
@spec list_reverse_map(Enumerable.t(elem), (elem -> result), [result]) :: [result]
when elem: var, result: var

Map each value of the enumerable using a mapper and reverse the resulting list.

Equivalent to Enum.reverse/1 followed by Enum.map/2.

Examples

iex> list_reverse_map([1, 2, 3], &(&1 + 1))
[4, 3, 2]
@spec map_values(map(), (term() -> term())) :: map()

Apply a function to each value of a map.

Link to this function

map_while_ok(enum, mapper)

View Source
@spec map_while_ok(Enumerable.t(elem), (elem -> {:ok, result} | {:error, term()})) ::
  {:ok, [result]} | {:error, term()}
when elem: var, result: var

Map each value of the enumerable using a mapper, unwrapping a result tuple returned by the mapper and stopping on error.

Examples

iex> map_while_ok(["2015-01-23 23:50:07.0", "2015-01-23 23:50:08"], &NaiveDateTime.from_iso8601/1)
{:ok, [~N[2015-01-23 23:50:07.0], ~N[2015-01-23 23:50:08]]}

iex> map_while_ok(["2015-01-23 23:50:07A", "2015-01-23 23:50:08"], &NaiveDateTime.from_iso8601/1)
{:error, :invalid_format}
Link to this function

merge_sorted_files(paths, target_path, reader, sorter \\ &<=/2)

View Source
@spec merge_sorted_files(
  paths :: [String.t()],
  target_path :: String.t(),
  reader :: item_reader_fn(elem),
  sorter :: (elem, elem -> boolean()) | module()
) :: :ok

Merge a list of sorted files into a single file.

Uses a reader function that takes a file descriptor and returns next {key, binary} tuple, where binary will be written to the file as sorted by key.

Link to this function

merge_sorted_streams(streams, comparator \\ &<=/2, mapper \\ & &1)

View Source

Merge a list of streams by taking the minimum element from each stream and emitting it and its stream. The streams are compared using the given comparator function.

Examples

iex> merge_sorted_streams([[1, 2, 3], [2, 3, 4]]) |> Enum.to_list()
[1, 2, 2, 3, 3, 4]

iex> merge_sorted_streams([[1, 2, 3], [4, 5, 6]]) |> Enum.to_list()
[1, 2, 3, 4, 5, 6]

iex> merge_sorted_streams([[10], [4, 5, 6]]) |> Enum.to_list()
[4, 5, 6, 10]
Link to this function

obfuscate_password(connection_opts)

View Source
@spec obfuscate_password(Keyword.t()) :: Keyword.t()

Given a keyword list of database connection options, obfuscate the password by wrapping it in a zero-arity function.

This should be done as early as possible when parsing connection options from the OS env. The aim of this obfuscation is to avoid accidentally leaking the password when inspecting connection opts or logging them as part of a process state (which is done automatically by OTP when a process that implements an OTP behaviour crashes).

Link to this function

open_with_retry(path, opts, attempts_left \\ 100)

View Source

Open a file, retrying if it doesn't exist yet, up to attempts_left times, with 20ms delay between attempts.

Link to this function

parse_md_table(string, opts)

View Source
@spec parse_md_table(String.t(), [{:after, String.t()}]) :: [[String.t(), ...]]

Parse a markdown table from a string

Options:

  • after: - taking a first table that comes right after a given substring.

Example

iex> """
...> Some text
...>
...> ## Known types
...>
...> | type                    | category | preferred? |
...> | ----------------------- | -------- | ---------- |
...> | bool                    | boolean  | t          |
...> | int2                    | numeric  |            |
...> """|> parse_md_table(after: "## Known types")
[["bool", "boolean", "t"], ["int2", "numeric", ""]]

iex> """
...> Some text
...> """|> parse_md_table([])
[]

Parses quoted names. Lowercases unquoted names to match Postgres' case insensitivity.

Examples

iex> parse_quoted_name("foo")
"foo"

iex> parse_quoted_name(~S|"foo"|)
"foo"

iex> parse_quoted_name(~S|"fo""o"|)
~S|fo"o|

iex> parse_quoted_name(~S|"FooBar"|)
~S|FooBar|

iex> parse_quoted_name(~S|FooBar|)
~S|FooBar|
@spec quote_name(String.t()) :: String.t()

Quote a string for use in SQL queries.

Examples

iex> quote_name("foo")
~S|"foo"|

iex> quote_name(~S|fo"o|)
~S|"fo""o"|
Link to this function

relation_to_sql(relation, force_quote \\ false)

View Source

Format a relation tuple to be correctly escaped for use in SQL queries.

Examples

iex> relation_to_sql({"public", "items"})
~S|public.items|

iex> relation_to_sql({"public", "items"}, true)
~S|"public"."items"|

iex> relation_to_sql({"public", "items-again"})
~S|public."items-again"|

iex> relation_to_sql({"public", "99red_balloons"})
~S|public."99red_balloons"|

iex> relation_to_sql({"public", "when"})
~S|public."when"|

iex> relation_to_sql({"with spaces", ~S|and "quoted"!|})
~S|"with spaces"."and ""quoted""!"|
Link to this function

stream_add_side_effect(stream, start_fun, reducer, last_fun \\ & &1, after_fun \\ & &1)

View Source

Transform the stream to call a side-effect function for each element before continuing.

Acts like Stream.each/2 but with an aggregate. start_fun, last_fun, after_fun have the same semantics as in Stream.transform/5

Link to this function

stream_file_items(path, item_reader)

View Source
@spec stream_file_items(
  path :: String.t(),
  reader :: item_reader_fn(elem)
) :: Enumerable.t(sortable_binary(elem))

Generate a random UUID v4.

Code taken from Ecto: https://github.com/elixir-ecto/ecto/blob/v3.10.2/lib/ecto/uuid.ex#L174

Examples

iex> Regex.match?(~r/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/, uuid4())
true