SpiderMan.Pipeline.Counter (spider_man v0.6.4)
View Sourcemsg counter for component
Usage
settings = [
*_options: [
pipelines: [SpiderMan.Pipeline.Counter | {SpiderMan.Pipeline.Counter, scope}]
],
...
]Support for all component: downloader | spider | item_processor.
Scope
:common|[scope: :common]: save counter tocommon_pipeline_tid.:pipeline|[scope: :pipeline]: save counter topipeline_tid.
common_pipeline_tid use by all component, pipeline_tid only use by one component.