Benchmarks
Running
- Quick run:
mix run bench/bench.exs - Full run:
mix run bench/bench.exs full
Benchmarking Summary
Perf-wise, the sweet spot for PersistentVector is scenarios when vector needs to be built by repeatedly appending to the end AND random-access (get/set operations) are also used.
If random-access after building is not required, then building and reversing a List is more efficient.
If building speed is not important, but removal from the end happens often, then Erlang’s :array shows better performance.
get/set operations perform similar to :array:
PersistentVector.get/2is slightly faster for larger collections compared to:array.PersistentVector.set/3is slightly faster for smaller collections compared to:array.
Map is added only for a baseline. In a sense that if Map was to outperform PersistentVector then this library would not be needed.
This comparison is not fair to Map as it has much richer capabilities.
The fact that Map performs worse for bigger collections is not surprising and is not Map’s fault :-).
Raw Benchmarking results for v0.1.4
Using 2 inputs
#
# Build
#
##### With input 1'000 #####
Name ips average deviation median
List Build 41.43 K 24.13 μs ±32.24% 31.00 μs
Vector Build 9.25 K 108.15 μs ±66.78% 150.00 μs
Map Build 4.60 K 217.59 μs ±35.13% 160.00 μs
Array Build 4.42 K 226.07 μs ±34.40% 160.00 μs
Comparison:
List Build 41.43 K
Vector Build 9.25 K - 4.48x slower
Map Build 4.60 K - 9.02x slower
Array Build 4.42 K - 9.37x slower
##### With input 1'000'000 #####
Name ips average deviation median
List Build 14.06 71.15 ms ±10.85% 78.00 ms
Vector Build 5.77 173.22 ms ±5.30% 172.00 ms
Array Build 1.66 602.94 ms ±1.61% 609.00 ms
Map Build 0.99 1007.80 ms ±1.97% 1007.50 ms
Comparison:
List Build 14.06
Vector Build 5.77 - 2.43x slower
Array Build 1.66 - 8.47x slower
Map Build 0.99 - 14.16x slower
#
# Shrink
#
##### With input 1'000 #####
Name ips average deviation median
Array resize 13.96 K 71.65 μs ±11.36% 78.00 μs
Vector remove_last 6.91 K 144.74 μs ±36.61% 160.00 μs
Map resize 4.05 K 246.67 μs ±31.47% 310.00 μs
Comparison:
Array resize 13.96 K
Vector remove_last 6.91 K - 2.02x slower
Map resize 4.05 K - 3.44x slower
##### With input 1'000'000 #####
Name ips average deviation median
Array resize 13.10 76.34 ms ±7.94% 78.00 ms
Vector remove_last 6.14 162.81 ms ±5.01% 157.00 ms
Map resize 1.05 950.27 ms ±3.57% 937.00 ms
Comparison:
Array resize 13.10
Vector remove_last 6.14 - 2.13x slower
Map resize 1.05 - 12.45x slower
#
# Get
#
##### With input 1'000 #####
Name ips average deviation median
Map Get 15.58 K 64.20 μs ±119.83% 0.0 μs
Vector Get 7.68 K 130.25 μs ±44.82% 160.00 μs
Array Get 7.37 K 135.70 μs ±39.04% 160.00 μs
Comparison:
Map Get 15.58 K
Vector Get 7.68 K - 2.03x slower
Array Get 7.37 K - 2.11x slower
##### With input 1'000'000 #####
Name ips average deviation median
Vector Get 5.85 171.08 ms ±2.05% 172.00 ms
Array Get 4.14 241.81 ms ±3.23% 235.00 ms
Map Get 3.14 318.84 ms ±6.58% 312.00 ms
Comparison:
Vector Get 5.85
Array Get 4.14 - 1.41x slower
Map Get 3.14 - 1.86x slower
#
# Set
#
##### With input 1'000 #####
Name ips average deviation median
Map Set 4.58 K 218.36 μs ±35.04% 160.00 μs
Vector Set 4.36 K 229.54 μs ±33.97% 160.00 μs
Array Set 3.35 K 298.54 μs ±15.06% 310.00 μs
Comparison:
Map Set 4.58 K
Vector Set 4.36 K - 1.05x slower
Array Set 3.35 K - 1.37x slower
##### With input 1'000'000 #####
Name ips average deviation median
Array Set 1.41 708.33 ms ±3.22% 703.00 ms
Vector Set 1.18 849.00 ms ±2.85% 844.00 ms
Map Set 0.80 1253.88 ms ±3.92% 1234.00 ms
Comparison:
Array Set 1.41
Vector Set 1.18 - 1.20x slower
Map Set 0.80 - 1.77x slower
#
# Enumerate
#
##### With input 1'000 #####
Name ips average deviation median
Vector Enumerate 17.19 K 58.17 μs ±12.08% 62.00 μs
Map Enumerate 14.01 K 71.36 μs ±10.89% 78.00 μs
Comparison:
Vector Enumerate 17.19 K
Map Enumerate 14.01 K - 1.23x slower
##### With input 1'000'000 #####
Name ips average deviation median
Vector Enumerate 15.60 64.10 ms ±7.92% 63.00 ms
Map Enumerate 8.07 123.84 ms ±15.47% 125.00 ms
Comparison:
Vector Enumerate 15.60
Map Enumerate 8.07 - 1.93x slower
#
# to_list
#
##### With input 1'000 #####
Name ips average deviation median
Vector to_list 190.39 K 5.25 μs ±14.27% 4.70 μs
Array to_list 67.39 K 14.84 μs ±23.25% 16.00 μs
Map to_list 49.62 K 20.15 μs ±35.27% 16.00 μs
Vector into 17.55 K 56.99 μs ±13.19% 62.00 μs
Map into 13.66 K 73.23 μs ±9.91% 78.00 μs
Comparison:
Vector to_list 190.39 K
Array to_list 67.39 K - 2.83x slower
Map to_list 49.62 K - 3.84x slower
Vector into 17.55 K - 10.85x slower
Map into 13.66 K - 13.94x slower
##### With input 1'000'000 #####
Name ips average deviation median
Array to_list 53.20 18.80 ms ±36.19% 16.00 ms
Vector to_list 49.58 20.17 ms ±38.37% 16.00 ms
Map to_list 26.54 37.68 ms ±55.39% 31.00 ms
Vector into 10.98 91.05 ms ±15.50% 93.00 ms
Map into 8.29 120.67 ms ±7.02% 125.00 ms
Comparison:
Array to_list 53.20
Vector to_list 49.58 - 1.07x slower
Map to_list 26.54 - 2.00x slower
Vector into 10.98 - 4.84x slower
Map into 8.29 - 6.42x slower