changelog: 要闻流水线v0.4(扩源+RAW分层+行情采集)
This commit is contained in:
25
CHANGELOG.md
25
CHANGELOG.md
@ -94,3 +94,28 @@
|
||||
- 东京 ~/scripts/: fetch_news.py, fetch_policy.py, fetch_research.py, cn_merge.py(新)
|
||||
- 东京 ~/daily-brief/: archive_daily_brief.py(新), archive/2026/06/2026-06-16.md
|
||||
- 本地 Claw/: local_fetch_cn.py(新), local_fetch_daemon.py(新), start_cn_fetch.bat(新)
|
||||
|
||||
### 要闻流水线改造 v0.4 — 扩源 + RAW分层 + 精选器
|
||||
|
||||
**改动:**
|
||||
1. fetch_news.py v2.0 — 新增 5 个国际源(BBC, The Guardian, CNBC, FT中文, Bloomberg),12源/轮约342条
|
||||
2. local_fetch_cn.py v2.0 — 清理失效源(人民网/观察者网/腾讯/澎湃),保留新华网3路+微博
|
||||
3. cn_merge.py v2.0 — 分两路输出:全量→news_raw.json+RAW ndjson,精选→news.json(送AI摘要)
|
||||
4. 新增 selector.py — 多策略评分精选器(来源权重+时效+多样性,16主题分类)
|
||||
5. 行情数据采集 — 新增 fetch_market_daily.py,31品种日线OHLCV,daily-brief/market_data/
|
||||
6. RAW存档机制 — 所有新闻全量写入 daily-brief/raw/YYYY-MM-DD.ndjson(逐行JSON,0 token)
|
||||
|
||||
**架构变更:**
|
||||
- 抓取层从~200条/轮扩张至~500条/轮,RAW全量存档
|
||||
- AI摘要层从全量处理改为精选80条,token成本不变
|
||||
- 新增行情数据采集线(事件线+价格线双轨归档)
|
||||
|
||||
**crontab 更新:**
|
||||
- 50 23 * * * — 行情日线采集(新增)
|
||||
- 10 0 * * * — RAW存档推送到Gitea(新增)
|
||||
- 05 3,9,15,21 — cn_merge v2(含selector精选,更新)
|
||||
|
||||
**涉及文件:**
|
||||
- 东京 ~/scripts/: fetch_news.py, cn_merge.py (更新), selector.py (新)
|
||||
- 东京 ~/daily-brief/: fetch_market_daily.py (新), raw/ (新目录)
|
||||
- 本地 Claw/: local_fetch_cn.py (更新)
|
||||
|
||||
Reference in New Issue
Block a user