ex_wire v0.1.1 ExWire.Struct.BlockQueue

A structure to store and process blocks received by peers. The goal of this module is to keep track of partial blocks until we’re ready to add the block to the chain.

There are two reasons we need to keep them stored in a queue:

  1. Block headers are sent separately of block bodies. We need to store the headers until we receive the bodies.
  2. We shouldn’t accept a block as canonical until we’ve heard from several peers that the block is the most canonical block at that number. Thus, we store the block and a number of commitments. Once the number of commitments tips over some threshold, we process the block and add it to our block tree.

Link to this section Summary

Functions

Adds a given block struct received by a peer to a block queue

Adds a given header received by a peer to a block queue. Returns wether or not we should request the block body, as well

Returns the set of blocks which are complete in the block queue, returning a new block queue with those blocks removed. This effective dequeues blocks once they have sufficient data and commitments

Determines if a block is empty. There’s no reason to actually ask for a block body if we know, a priori, that a block is empty

Processes a the block queue, adding any blocks which are complete and pass the number of confirmations to the block tree. Those are then removed from the queue

Link to this section Types

Link to this type block_item()
block_item() :: %{commitments: [binary], block: Blockchain.Block.t, ready: boolean}
Link to this type block_map()
block_map() :: %{optional(EVM.hash) => block_item}
Link to this type t()
t() :: %ExWire.Struct.BlockQueue{do_validation: boolean, queue: %{optional(integer) => block_map}}

Link to this section Functions

Link to this function add_block_struct_to_block_queue(block_queue, block_tree, block_struct, chain, db)

Adds a given block struct received by a peer to a block queue.

Since we don’t really know which block this belongs to, we’re going to just need to look at every block and try and guess.

To guess, we’ll compute the transactions root and ommers hash, and then try and find a header that matches it. For empty blocks (ones with no transactions and no ommers, there may be several matches. Otherwise, each block body should pretty much be unique).

Examples

iex> chain = Blockchain.Test.ropsten_chain()
iex> db = MerklePatriciaTree.Test.random_ets_db(:add_block_struct_to_block_queue)
iex> header = %Block.Header{
...>   transactions_root: <<200, 70, 164, 239, 152, 124, 5, 149, 40, 10, 157, 9, 210, 181, 93, 89, 5, 119, 158, 112, 221, 58, 94, 86, 206, 113, 120, 51, 241, 9, 154, 150>>,
...>   ommers_hash: <<232, 5, 101, 202, 108, 35, 61, 149, 228, 58, 111, 18, 19, 234, 191, 129, 189, 107, 167, 195, 222, 123, 50, 51, 176, 222, 225, 181, 72, 231, 198, 53>>
...> }
iex> block_struct = %ExWire.Struct.Block{
...>   transactions_list: [[1], [2], [3]],
...>   transactions: ["trx"],
...>   ommers: ["ommers"]
...> }
iex> block_queue = %ExWire.Struct.BlockQueue{
...>   queue: %{
...>     1 => %{
...>       <<1::256>> => %{
...>         commitments: MapSet.new([]),
...>         header: header,
...>         block: %Blockchain.Block{header: header, block_hash: <<1::256>>},
...>         ready: false,
...>       }
...>     }
...>   },
...>   do_validation: false
...> }
iex> {block_queue, _block_tree} = ExWire.Struct.BlockQueue.add_block_struct_to_block_queue(
...>   block_queue,
...>   Blockchain.Blocktree.new_tree(),
...>   block_struct,
...>   chain,
...>   db
...> )
iex> block_queue.queue[1][<<1::256>>].block.transactions
["trx"]
iex> block_queue.queue[1][<<1::256>>].block.ommers
["ommers"]
Link to this function add_header_to_block_queue(block_queue, block_tree, header, header_hash, remote_id, chain, db)

Adds a given header received by a peer to a block queue. Returns wether or not we should request the block body, as well.

Note: we will process it if the block is empty (i.e. has no transactions nor ommers).

Examples

iex> chain = Blockchain.Test.ropsten_chain()
iex> db = MerklePatriciaTree.Test.random_ets_db(:proces_block_queue)
iex> header = %Block.Header{number: 5, parent_hash: <<0::256>>, beneficiary: <<2, 3, 4>>, difficulty: 100, timestamp: 11, mix_hash: <<1>>, nonce: <<2>>}
iex> header_hash = <<78, 28, 127, 10, 192, 253, 127, 239, 254, 179, 39, 34, 245, 44, 152, 98, 128, 71, 238, 155, 100, 161, 199, 71, 243, 223, 172, 191, 74, 99, 128, 63>>
iex> {block_queue, block_tree, false} = ExWire.Struct.BlockQueue.add_header_to_block_queue(%ExWire.Struct.BlockQueue{do_validation: false}, Blockchain.Blocktree.new_tree(), header, header_hash, "remote_id", chain, db)
iex> block_queue.queue
%{}
iex> block_tree.parent_map
%{<<109, 191, 166, 180, 1, 44, 85, 48, 107, 43, 51, 4, 81, 128, 110, 188, 130, 1, 5, 255, 21, 204, 250, 214, 105, 55, 182, 104, 0, 94, 102, 6>> => <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0>>}

# TODO: Add a second addition example
Link to this function get_complete_blocks(block_queue)
get_complete_blocks(t) :: {t, [Blockchain.Block.t]}

Returns the set of blocks which are complete in the block queue, returning a new block queue with those blocks removed. This effective dequeues blocks once they have sufficient data and commitments.

Examples

iex> %ExWire.Struct.BlockQueue{
...>   queue: %{
...>     5 => %{
...>       <<1::256>> => %{
...>         commitments: MapSet.new([1, 2]),
...>         header: %Block.Header{number: 5},
...>         block: %Blockchain.Block{block_hash: <<1::256>>},
...>         ready: true,
...>       },
...>       <<2::256>> => %{
...>         commitments: MapSet.new([]),
...>         header: %Block.Header{number: 5},
...>         block: %Blockchain.Block{block_hash: <<2::256>>},
...>         ready: true,
...>       },
...>       <<3::256>> => %{
...>         commitments: MapSet.new([1, 2]),
...>         header: %Block.Header{number: 5, gas_used: 5},
...>         block: %Blockchain.Block{block_hash: <<3::256>>},
...>         ready: false,
...>       },
...>       <<4::256>> => %{
...>         commitments: MapSet.new([1, 2]),
...>         header: %Block.Header{number: 5, ommers_hash: <<5::256>>},
...>         block: %Blockchain.Block{block_hash: <<4::256>>},
...>         ready: false,
...>       }
...>     },
...>     6 => %{
...>       <<5::256>> => %{
...>         commitments: MapSet.new([1, 2]),
...>         header: %Block.Header{number: 6},
...>         block: %Blockchain.Block{block_hash: <<5::256>>},
...>         ready: true,
...>       }
...>     }
...>   }
...> }
...> |> ExWire.Struct.BlockQueue.get_complete_blocks()
{
  %ExWire.Struct.BlockQueue{
    queue: %{
      5 => %{
        <<2::256>> => %{
          commitments: MapSet.new([]),
          header: %Block.Header{number: 5},
          block: %Blockchain.Block{block_hash: <<2::256>>},
          ready: true
        },
        <<3::256>> => %{
          commitments: MapSet.new([1, 2]),
          header: %Block.Header{number: 5, gas_used: 5},
          block: %Blockchain.Block{block_hash: <<3::256>>},
          ready: false
        },
        <<4::256>> => %{
          commitments: MapSet.new([1, 2]),
          header: %Block.Header{number: 5, ommers_hash: <<5::256>>},
          block: %Blockchain.Block{block_hash: <<4::256>>},
          ready: false
        }
      }
    }
  },
  [
    %Blockchain.Block{block_hash: <<1::256>>},
    %Blockchain.Block{block_hash: <<5::256>>}
  ]
}
Link to this function is_block_empty?(header)
is_block_empty?(Block.Header.t) :: boolean

Determines if a block is empty. There’s no reason to actually ask for a block body if we know, a priori, that a block is empty.

Examples

iex> %Block.Header{
...>   transactions_root: MerklePatriciaTree.Trie.empty_trie_root_hash(),
...>   ommers_hash: <<29, 204, 77, 232, 222, 199, 93, 122, 171, 133, 181, 103, 182, 204, 212, 26, 211, 18, 69, 27, 148, 138, 116, 19, 240, 161, 66, 253, 64, 212, 147, 71>>
...> }
...> |> ExWire.Struct.BlockQueue.is_block_empty?
true

iex> %Block.Header{
...>   transactions_root: MerklePatriciaTree.Trie.empty_trie_root_hash(),
...>   ommers_hash: <<1>>
...> }
...> |> ExWire.Struct.BlockQueue.is_block_empty?
false

iex> %Block.Header{
...>   transactions_root: <<1>>,
...>   ommers_hash: <<29, 204, 77, 232, 222, 199, 93, 122, 171, 133, 181, 103, 182, 204, 212, 26, 211, 18, 69, 27, 148, 138, 116, 19, 240, 161, 66, 253, 64, 212, 147, 71>>
...> }
...> |> ExWire.Struct.BlockQueue.is_block_empty?
false
Link to this function process_block_queue(block_queue, block_tree, chain, db)

Processes a the block queue, adding any blocks which are complete and pass the number of confirmations to the block tree. Those are then removed from the queue.

Examples

iex> chain = Blockchain.Test.ropsten_chain()
iex> db = MerklePatriciaTree.Test.random_ets_db(:process_block_queue)
iex> header = %Block.Header{number: 1, parent_hash: <<0::256>>, beneficiary: <<2, 3, 4>>, difficulty: 100, timestamp: 11, mix_hash: <<1>>, nonce: <<2>>}
iex> {block_queue, block_tree} = %ExWire.Struct.BlockQueue{
...>   queue: %{
...>     1 => %{
...>       <<1::256>> => %{
...>         commitments: MapSet.new([1, 2]),
...>         header: header,
...>         block: %Blockchain.Block{header: header, block_hash: <<1::256>>},
...>         ready: true,
...>       }
...>     }
...>   },
...>   do_validation: false
...> }
...> |> ExWire.Struct.BlockQueue.process_block_queue(Blockchain.Blocktree.new_tree(), chain, db)
iex> block_tree.parent_map
%{<<226, 210, 216, 149, 139, 194, 100, 151, 35, 86, 131, 75, 10, 203, 201, 20, 232, 134, 23, 195, 24, 34, 181, 6, 142, 4, 57, 85, 121, 223, 246, 87>> => <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0>>}
iex> block_queue.queue
%{}