Quant.Explorer.DataTransformer (quant v0.1.0-alpha.1)
Data normalization utilities for converting raw API responses into standardized Explorer DataFrames.
This module handles the transformation of various data formats (JSON, CSV, maps, lists) from different providers into consistent DataFrame schemas that can be used across the entire library.
Summary
Functions
Filters out rows with invalid or missing critical data.
Converts CSV string to DataFrame with proper column types.
Normalizes numeric values, handling strings, floats, integers, and nil.
Normalizes timestamps from various formats to DateTime.
Normalizes volume values, typically as integers.
Transforms raw historical data into a standardized DataFrame.
Transforms raw quote data into a standardized DataFrame.
Transforms raw search results into a standardized DataFrame.
Types
Functions
@spec clean_dataframe(Explorer.DataFrame.t(), schema()) :: Explorer.DataFrame.t()
Filters out rows with invalid or missing critical data.
@spec csv_to_dataframe( binary(), keyword() ) :: {:ok, Explorer.DataFrame.t()} | {:error, term()}
Converts CSV string to DataFrame with proper column types.
Normalizes numeric values, handling strings, floats, integers, and nil.
@spec normalize_timestamp(term()) :: DateTime.t() | nil
Normalizes timestamps from various formats to DateTime.
Handles:
- Unix timestamps (seconds/milliseconds)
- ISO 8601 strings
- Date strings in various formats
- Already parsed DateTime structs
Normalizes volume values, typically as integers.
@spec transform_history(raw_data(), String.t(), keyword()) :: {:ok, Explorer.DataFrame.t()} | {:error, term()}
Transforms raw historical data into a standardized DataFrame.
Expected columns in output:
- symbol (string): Stock/crypto symbol
- timestamp (datetime): Data timestamp
- open (f64): Opening price
- high (f64): High price
- low (f64): Low price
- close (f64): Closing price
- volume (s64): Trading volume
- adj_close (f64): Adjusted closing price (optional)
@spec transform_quote(raw_data(), String.t() | [String.t()], keyword()) :: {:ok, Explorer.DataFrame.t()} | {:error, term()}
Transforms raw quote data into a standardized DataFrame.
Expected columns in output:
- symbol (string): Stock/crypto symbol
- price (f64): Current price
- change (f64): Price change
- change_percent (f64): Percentage change
- volume (s64): Current volume
- timestamp (datetime): Quote timestamp
@spec transform_search( raw_data(), keyword() ) :: {:ok, Explorer.DataFrame.t()} | {:error, term()}
Transforms raw search results into a standardized DataFrame.
Expected columns in output:
- symbol (string): Trading symbol
- name (string): Company/asset name
- type (string): Asset type (stock, etf, crypto, etc.)
- exchange (string): Trading exchange (optional)