Tirexs v0.8.15 Tirexs.Bulk

The Bulk API makes it possible to perform many index/delete operations in single call.

This module provides DSL for building Bulk API payload which is ready to use over Tirexs.Resources.bump/1 or Tirexs.HTTP.post/2 conveniences.

The Tirexs.Resources.bump/1 expects payload as a set of JSON documents joined together by newline ( ) characters.

payload = ~S'''
{ "index": { "_index": "website", "_type": "blog", "_id": "1" }}
{ "title": "My second blog post" }
'''
# { :ok, 200, r } = HTTP.post("/_bulk", payload)
# { :ok, 200, r } = Resources.bump(payload)._bulk({ [refresh: true] })
{ :ok, 200, r } = Resources.bump(payload)._bulk()

A bulk macro helps create a single payload from parts (action, metadata, request body) where you Don’t need to Repeat Yourself ;).

For instance:

payload = bulk([ index: "website", type: "blog" ]) do
  index [
    [id: 1, title: "My first blog post"],
    [id: 2, title: "My second blog post"]
  ]
  update [
    [
      doc: [id: 1, _retry_on_conflict: 3, title: "[updated] My first blog post"],
      fields: ["_source"],
    ],
    [
      doc: [id: 2, _retry_on_conflict: 3, title: "[updated] My second blog post"],
      doc_as_upsert: true,
    ]
  ]
end

Find out more details and examples in the bulk/2 macro doc.

Summary

Functions

create(request_body)

Prepares request_body and create action all together

create(metadata, request_body)

delete(request_body)

Prepares request_body and delete action all together

delete(metadata, request_body)

index(request_body)

Prepares request_body and index action all together

index(metadata, request_body)

update(request_body)

Prepares request_body and update action all together

update(metadata, request_body)

Macros

bulk(metadata \\ [], list)

Builds payload from given block and returns the Bulk API JSON structure

Functions

create(request_body)

Prepares request_body and create action all together.

create(metadata, request_body)

delete(request_body)

Prepares request_body and delete action all together.

delete(metadata, request_body)

index(request_body)

Prepares request_body and index action all together.

index(metadata, request_body)

update(request_body)

Prepares request_body and update action all together.

update(metadata, request_body)

Macros

bulk(metadata \\ [], list)

Builds payload from given block and returns the Bulk API JSON structure.

The Bulk request body has the following action, metadata and request body parts.

The bulk to particular _index/_type:

payload = bulk do
  index [
    [id: 1, title: "My second blog post"]
    # ...
  ]
end
Tirexs.bump(payload)._bulk("website/blog", { [refresh: true] })

The same metadata for every document:

payload = bulk([ index: "website", type: "blog" ]) do
  index [
    [id: 1, title: "My second blog post"]
    # ...
  ]
end
Tirexs.bump(payload)._bulk()

Index specific insertion:

payload = bulk do
  index [ index: "website.a", type: "blog" ], [
    [title: "My blog post"]
    # ...
  ]
  index [ index: "website.b", type: "blog" ], [
    [title: "My blog post"]
    # ...
  ]
end
Tirexs.bump(payload)._bulk()

The action could be index, create, update and delete.

Update example:

bulk do
  update [ index: "website", type: "blog"], [
    [
      doc: [id: 1, _retry_on_conflict: 3, title: "[updated] My first blog post"],
      fields: ["_source"],
    ],
    [
      doc: [id: 2, _retry_on_conflict: 3, title: "[updated] My second blog post"],
      doc_as_upsert: true,
    ]
  ]
end

Delete example:

bulk do
  delete [ index: "website.b", type: "blog" ], [
    [id: "1"]
    # ...
  ]
end