Xbase.Parser (Xbase v0.1.0)
View SourceBinary parser for DBF file format components.
This module provides functions to parse DBF file headers and field descriptors from binary data using Elixir's efficient binary pattern matching.
Summary
Functions
Appends a new record to the DBF file.
Appends multiple records to the DBF file in a single batch operation.
Deletes multiple records by their indices in a single operation.
Deletes records in a specified index range (inclusive).
Deletes records that match a given condition function.
Updates multiple records by their indices in a single batch operation.
Updates multiple records with write conflict detection.
Updates records that match a given condition function.
Calculates the byte offset for a specific record in the file.
Closes a DBF file handle.
Counts the number of active (non-deleted) records in the DBF file.
Counts the number of deleted records in the DBF file.
Creates a new DBF file with the specified field structure.
Ensures header consistency for all write operations. Call this after any operation that modifies the DBF structure.
Extracts the deletion flag from record data.
Validates if a record index is within the valid range for the file.
Marks a record as deleted in the DBF file.
Marks a record as deleted with write conflict detection.
Returns current memory usage statistics.
Opens a DBF file and parses its header and field descriptors.
Opens a DBF file and parses its header and field descriptors with specified file modes.
Packs a DBF file by removing all deleted records and creating a compacted file.
Packs a DBF file with write conflict detection.
Parses field descriptors from binary data until field terminator (0x0D).
Parses a 32-byte DBF header from binary data.
Parses record field data according to field descriptors.
Reads records in chunks of specified size.
Reads records in chunks with progress reporting.
Reads a complete record from the DBF file at the specified index.
Reads all records from a DBF file.
Provides comprehensive statistics about records in the DBF file.
Refreshes the DBF structure by re-reading the header and fields from file.
Creates a lazy stream of records from the DBF file.
Creates a stream with progress reporting.
Creates a filtered stream of records from the DBF file.
Executes a transaction function with rollback capability.
Undeletes a previously deleted record in the DBF file.
Updates an existing record in the DBF file.
Updates a record with write conflict detection.
Updates a record with automatic retry on conflict detection.
Validates header consistency after write operations.
Functions
Appends a new record to the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1 or create_dbf/2record_data
- Map of field name => value
Returns
{:ok, updated_dbf}
- Successfully appended with updated DBF structure{:error, reason}
- Error appending record
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, updated_dbf} = Xbase.Parser.append_record(dbf, %{"NAME" => "John", "AGE" => 30})
Appends multiple records to the DBF file in a single batch operation.
Parameters
dbf
- DBF file structure from open_dbf/1records
- List of record data maps to append
Returns
{:ok, updated_dbf}
- Successfully appended all records{:error, reason}
- Error during batch append
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> records = [%{"NAME" => "John"}, %{"NAME" => "Jane"}]
iex> Xbase.Parser.batch_append_records(dbf, records)
{:ok, updated_dbf}
Deletes multiple records by their indices in a single operation.
Parameters
dbf
- DBF file structure from open_dbf/1indices
- List of record indices to delete
Returns
{:ok, updated_dbf}
- Successfully deleted records{:error, reason}
- Error during deletion
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> Xbase.Parser.batch_delete(dbf, [1, 5, 10])
{:ok, updated_dbf}
Deletes records in a specified index range (inclusive).
Parameters
dbf
- DBF file structure from open_dbf/1start_index
- Starting index (inclusive)end_index
- Ending index (inclusive)
Returns
{:ok, updated_dbf}
- Successfully deleted records in range{:error, reason}
- Error during deletion
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> Xbase.Parser.batch_delete_range(dbf, 10, 20)
{:ok, updated_dbf}
Deletes records that match a given condition function.
Parameters
dbf
- DBF file structure from open_dbf/1condition_fn
- Function that takes record data and returns true to delete
Returns
{:ok, updated_dbf}
- Successfully deleted matching records{:error, reason}
- Error during deletion
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> condition = fn record -> record["STATUS"] == "inactive" end
iex> Xbase.Parser.batch_delete_where(dbf, condition)
{:ok, updated_dbf}
Updates multiple records by their indices in a single batch operation.
Parameters
dbf
- DBF file structure from open_dbf/1updates
- List of {index, update_data} tuples
Returns
{:ok, updated_dbf}
- Successfully updated all records{:error, reason}
- Error during batch update
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> updates = [{0, %{"NAME" => "John"}}, {2, %{"STATUS" => "active"}}]
iex> Xbase.Parser.batch_update_records(dbf, updates)
{:ok, updated_dbf}
Updates multiple records with write conflict detection.
Parameters
dbf
- DBF file structure from open_dbf/1updates
- List of {index, update_data} tuples
Returns
{:ok, updated_dbf}
- Successfully updated records{:error, :write_conflict}
- File was modified by another process{:error, reason}
- Other error during batch update
Updates records that match a given condition function.
Parameters
dbf
- DBF file structure from open_dbf/1condition_fn
- Function that takes record data and returns true to updateupdate_data
- Map of field updates to apply
Returns
{:ok, updated_dbf}
- Successfully updated matching records{:error, reason}
- Error during batch update
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> condition = fn record -> record["STATUS"] == "pending" end
iex> Xbase.Parser.batch_update_where(dbf, condition, %{"STATUS" => "active"})
{:ok, updated_dbf}
Calculates the byte offset for a specific record in the file.
Parameters
header
- The DBF header containing file structure informationrecord_index
- Zero-based record index
Returns
- The byte offset where the record starts
Examples
iex> header = %Header{header_length: 97, record_length: 25}
iex> Xbase.Parser.calculate_record_offset(header, 0)
97
iex> Xbase.Parser.calculate_record_offset(header, 1)
122
Closes a DBF file handle.
Parameters
dbf
- DBF structure returned from open_dbf/1
Returns
:ok
- File closed successfully
Counts the number of active (non-deleted) records in the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1
Returns
{:ok, count}
- Number of active records{:error, reason}
- Error reading records
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> Xbase.Parser.count_active_records(dbf)
{:ok, 150}
Counts the number of deleted records in the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1
Returns
{:ok, count}
- Number of deleted records{:error, reason}
- Error reading records
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> Xbase.Parser.count_deleted_records(dbf)
{:ok, 25}
Creates a new DBF file with the specified field structure.
Parameters
path
- Path for the new DBF filefields
- List of field descriptors defining the schemaopts
- Options (version, overwrite)
Returns
{:ok, dbf}
- Successfully created DBF file structure{:error, reason}
- Error creating file
Examples
iex> fields = [%FieldDescriptor{name: "NAME", type: "C", length: 20}]
iex> {:ok, dbf} = Xbase.Parser.create_dbf("new.dbf", fields)
Ensures header consistency for all write operations. Call this after any operation that modifies the DBF structure.
Parameters
dbf
- DBF file structure
Returns
{:ok, dbf}
- Header is consistent{:error, reason}
- Header inconsistency detected
Extracts the deletion flag from record data.
Parameters
record_binary
- The raw record binary data
Returns
{:ok, boolean}
- true if deleted (0x2A), false if active (0x20){:error, reason}
- Error for invalid data
Validates if a record index is within the valid range for the file.
Parameters
header
- The DBF header containing record countrecord_index
- Zero-based record index to validate
Returns
true
if the index is valid,false
otherwise
Marks a record as deleted in the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Zero-based index of the record to mark as deleted
Returns
{:ok, updated_dbf}
- Successfully marked as deleted with updated DBF structure{:error, reason}
- Error marking record as deleted
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, updated_dbf} = Xbase.Parser.mark_deleted(dbf, 2)
Marks a record as deleted with write conflict detection.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Index of record to delete
Returns
{:ok, updated_dbf}
- Successfully marked record as deleted{:error, :write_conflict}
- File was modified by another process{:error, reason}
- Other error during deletion
Returns current memory usage statistics.
Returns
%{total: integer, processes: integer, system: integer}
- Memory usage in bytes
Examples
memory = Xbase.Parser.memory_usage()
# => %{total: 52428800, processes: 12345, system: 9876, ...}
Opens a DBF file and parses its header and field descriptors.
Parameters
path
- Path to the DBF file
Returns
{:ok, %{header: header, fields: fields, file: file}}
- Successfully opened DBF{:error, reason}
- Error opening or parsing file
Examples
iex> Xbase.Parser.open_dbf("data.dbf")
{:ok, %{header: %Header{...}, fields: [...], file: #Port<...>}}
Opens a DBF file and parses its header and field descriptors with specified file modes.
Parameters
path
- Path to the DBF filemodes
- List of file open modes (e.g., [:read], [:read, :write])
Returns
{:ok, %{header: header, fields: fields, file: file}}
- Successfully opened DBF{:error, reason}
- Error opening or parsing file
Packs a DBF file by removing all deleted records and creating a compacted file.
Parameters
dbf
- DBF file structure from open_dbf/1output_path
- Path for the packed output file
Returns
{:ok, packed_dbf}
- Successfully packed DBF file structure{:error, reason}
- Error packing file
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, packed_dbf} = Xbase.Parser.pack(dbf, "data_packed.dbf")
Packs a DBF file with write conflict detection.
Parameters
dbf
- DBF file structure from open_dbf/1output_path
- Path for the packed file
Returns
{:ok, packed_dbf}
- Successfully packed file{:error, :write_conflict}
- File was modified by another process{:error, reason}
- Other error during packing
Parses field descriptors from binary data until field terminator (0x0D).
Parameters
binary
- The binary data containing field descriptorsoffset
- Starting offset in the binary data
Returns
{:ok, [%FieldDescriptor{}]}
- List of parsed field descriptors{:error, reason}
- Parse error with reason
Parses a 32-byte DBF header from binary data.
Parameters
binary
- The binary data containing the DBF header
Returns
{:ok, %Header{}}
- Successfully parsed header{:error, reason}
- Parse error with reason
Examples
iex> header_data = <<0x03, 124, 12, 17, 100::little-32, 161::little-16, 50::little-16, 0::16, 0, 0, 0::12*8, 0, 0, 0::16>>
iex> Xbase.Parser.parse_header(header_data)
{:ok, %Xbase.Types.Header{version: 3, record_count: 100, ...}}
Parses record field data according to field descriptors.
Parameters
record_data
- Binary data for the record fields (without deletion flag)fields
- List of field descriptors
Returns
{:ok, %{field_name => parsed_value}}
- Parsed field data{:error, reason}
- Parse error
Reads records in chunks of specified size.
Parameters
dbf
- DBF file structure from open_dbf/1chunk_size
- Number of records per chunk
Returns
Stream.t()
- Stream of record lists (chunks)
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> chunks = Xbase.Parser.read_in_chunks(dbf, 100)
iex> Enum.each(chunks, fn chunk -> process_chunk(chunk) end)
:ok
Reads records in chunks with progress reporting.
Parameters
dbf
- DBF file structure from open_dbf/1chunk_size
- Number of records per chunkprogress_fn
- Function called with progress info:fn %{current: int, total: int, percentage: float} -> any end
Returns
Stream.t()
- Stream of record lists (chunks) with progress reporting
Examples
progress_fn = fn prog -> IO.puts("Progress: " <> to_string(prog.percentage) <> "%") end
chunks = Xbase.Parser.read_in_chunks_with_progress(dbf, 100, progress_fn)
Enum.each(chunks, fn chunk -> process_chunk(chunk) end)
Reads a complete record from the DBF file at the specified index.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Zero-based record index
Returns
{:ok, %Record{}}
- Successfully parsed record{:error, reason}
- Error reading or parsing record
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, record} = Xbase.Parser.read_record(dbf, 0)
iex> record.data["NAME"]
"John Doe"
Reads all records from a DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1
Returns
{:ok, [record_data]}
- List of record data maps{:error, reason}
- Error reading records
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, records} = Xbase.Parser.read_records(dbf)
iex> length(records)
10
Provides comprehensive statistics about records in the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1
Returns
{:ok, %{total_records: int, active_records: int, deleted_records: int, deletion_percentage: float}}
- Statistics{:error, reason}
- Error reading records
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> Xbase.Parser.record_statistics(dbf)
{:ok, %{total_records: 100, active_records: 85, deleted_records: 15, deletion_percentage: 15.0}}
Refreshes the DBF structure by re-reading the header and fields from file.
Parameters
dbf
- DBF file structure from open_dbf/1
Returns
{:ok, refreshed_dbf}
- DBF with updated header and fields{:error, reason}
- Error refreshing from file
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> Xbase.Parser.refresh_dbf_state(dbf)
{:ok, refreshed_dbf}
Creates a lazy stream of records from the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1
Returns
Stream.t()
- Stream of record data maps (excludes deleted records)
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> stream = Xbase.Parser.stream_records(dbf)
iex> stream |> Enum.take(10) |> length()
10
Creates a stream with progress reporting.
Parameters
dbf
- DBF file structure from open_dbf/1progress_fn
- Function called with progress info
Returns
Stream.t()
- Stream of records with progress reporting
Examples
progress_fn = fn prog -> send(self(), {:progress, prog}) end
records = Xbase.Parser.stream_records_with_progress(dbf, progress_fn) |> Enum.to_list()
Creates a filtered stream of records from the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1filter_fn
- Function that takes record data and returns true/false
Returns
Stream.t()
- Stream of filtered record data maps
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> high_scores = fn record -> record["SCORE"] > 90 end
iex> stream = Xbase.Parser.stream_where(dbf, high_scores)
iex> Enum.to_list(stream)
[%{"SCORE" => 95, ...}, ...]
Executes a transaction function with rollback capability.
Parameters
dbf
- DBF file structure from open_dbf/1transaction_fn
- Function that takes a DBF and returns {:ok, updated_dbf} or {:error, reason}
Returns
{:ok, final_dbf}
- Successfully committed transaction{:error, reason}
- Transaction failed and was rolled back
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, final_dbf} = Xbase.Parser.transaction(dbf, fn dbf ->
...> {:ok, dbf1} = Xbase.Parser.append_record(dbf, %{"NAME" => "John"})
...> {:ok, dbf2} = Xbase.Parser.update_record(dbf1, 0, %{"STATUS" => "active"})
...> {:ok, dbf2}
...> end)
Undeletes a previously deleted record in the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Zero-based index of the record to undelete
Returns
{:ok, updated_dbf}
- Successfully undeleted with updated DBF structure{:error, reason}
- Error undeleting record
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, updated_dbf} = Xbase.Parser.undelete_record(dbf, 2)
Updates an existing record in the DBF file.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Zero-based index of the record to updateupdate_data
- Map of field name => value for fields to update
Returns
{:ok, updated_dbf}
- Successfully updated with updated DBF structure{:error, reason}
- Error updating record
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
iex> {:ok, updated_dbf} = Xbase.Parser.update_record(dbf, 0, %{"NAME" => "Updated", "AGE" => 40})
Updates a record with write conflict detection.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Index of record to updateupdate_data
- Map of field updates
Returns
{:ok, updated_dbf}
- Successfully updated record{:error, :write_conflict}
- File was modified by another process{:error, reason}
- Other error during update
Examples
iex> {:ok, dbf} = Xbase.Parser.open_dbf("data.dbf", [:read, :write])
iex> Xbase.Parser.update_record_with_conflict_check(dbf, 0, %{"NAME" => "John"})
{:ok, updated_dbf}
Updates a record with automatic retry on conflict detection.
This function automatically refreshes the DBF state if a write conflict is detected and retries the operation once.
Parameters
dbf
- DBF file structure from open_dbf/1record_index
- Index of record to updateupdate_data
- Map of field updates
Returns
{:ok, updated_dbf}
- Successfully updated record{:error, reason}
- Error during update (after retry if conflict occurred)
Validates header consistency after write operations.
Parameters
dbf
- DBF file structure
Returns
:ok
- Header is consistent{:error, reason}
- Header inconsistency detected