refactor(factor): 完全重构因子计算框架 - 引入DSL表达式系统
- 删除旧因子框架:移除 base.py、composite.py、data_loader.py、data_spec.py 及所有子模块(momentum、financial、quality、sentiment等) - 新增DSL表达式系统:实现 factor DSL 编译器和翻译器 - dsl.py: 领域特定语言定义 - compiler.py: AST编译与优化 - translator.py: Polars表达式翻译 - api.py: 统一API接口 - 新增数据路由层:data_router.py 实现字段到表的动态路由 - 新增API封装:api_pro_bar.py 提供pro_bar数据接口 - 更新执行引擎:engine.py 适配新的DSL架构 - 重构测试体系:删除旧测试,新增 test_dsl_promotion.py、 test_factor_integration.py、test_pro_bar.py - 清理文档:删除8个过时文档(factor_design、db_sync_guide等)
This commit is contained in:
@@ -129,7 +129,9 @@ def sync_bak_basic(
|
||||
columns = []
|
||||
for col in sample.columns:
|
||||
dtype = str(sample[col].dtype)
|
||||
if "int" in dtype:
|
||||
if col == "trade_date":
|
||||
col_type = "DATE"
|
||||
elif "int" in dtype:
|
||||
col_type = "INTEGER"
|
||||
elif "float" in dtype:
|
||||
col_type = "DOUBLE"
|
||||
@@ -223,10 +225,16 @@ def sync_bak_basic(
|
||||
|
||||
# Combine and save
|
||||
combined = pd.concat(all_data, ignore_index=True)
|
||||
|
||||
# Convert trade_date to datetime for proper DATE type storage
|
||||
combined["trade_date"] = pd.to_datetime(combined["trade_date"], format="%Y%m%d")
|
||||
|
||||
print(f"[sync_bak_basic] Total records: {len(combined)}")
|
||||
|
||||
# Delete existing data for the date range and append new data
|
||||
storage._connection.execute(f'DELETE FROM "{TABLE_NAME}" WHERE "trade_date" >= ?', [sync_start])
|
||||
# Convert sync_start to date format for comparison with DATE column
|
||||
sync_start_date = pd.to_datetime(sync_start, format="%Y%m%d").date()
|
||||
storage._connection.execute(f'DELETE FROM "{TABLE_NAME}" WHERE "trade_date" >= ?', [sync_start_date])
|
||||
thread_storage.queue_save(TABLE_NAME, combined)
|
||||
thread_storage.flush()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user