View Source SpiderMan.Component.ItemProcessor (spider_man v0.6.3)
Store items.
Life cycle of request:
- insert requests to ets of downloader.
- downloader's producer get pass out to processes.
- processes handle message.
- handle by pre pipelines.
- call
Requester.request/3
. - handle by post pipelines.
- pass out message.
- if success, pass out to Batchers.
- if failed, maybe try again.
- batcher get enough message and call
handle_batch/4
.- call
Storage.store/3
to save items. - if success, done.
- if failed, maybe try again.
- call