Nous.Tools.WebFetch (nous v0.13.3)

View Source

Tool for fetching and extracting readable content from web pages.

Uses Req for HTTP and Floki for HTML parsing. Strips scripts, styles, and navigation to extract the main content.

Dependencies

Requires the floki package in your mix.exs:

{:floki, "~> 0.36"}

Usage

agent = Agent.new("openai:gpt-4",
  tools: [&WebFetch.fetch_page/2]
)

Summary

Functions

fetch_page(ctx, args)

Fetch a web page and extract its readable content.

Functions

fetch_page(ctx, args)

Fetch a web page and extract its readable content.

Arguments

  • url: The URL to fetch (required)
  • selector: Optional CSS selector to extract specific content

Returns

A map with url, title, content, word_count, and fetched_at.