liaozhaorun/ProStock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e6c3a918c76ae9bb8d8b1c503eded1bfe88a321f
ProStock/src/experiment/learn_to_rank.ipynb

1608 lines
190 KiB
Plaintext
Raw Normal View History

feat(training): 添加 LightGBM LambdaRank 排序学习功能 新增基于 LambdaRank 的排序学习模型，用于股票排序预测任务： - 实现 LightGBMLambdaRankModel 模型类，支持分位数标签转换 - 提供完整的训练流程和 NDCG 评估指标 - 添加实验 Notebook 演示排序学习全流程
2026-03-10 22:23:44 +08:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Learn-to-Rank 排序学习训练流程\n",
"\n",
"本 Notebook 实现基于 LightGBM LambdaRank 的排序学习训练，用于股票排序任务。\n",
"\n",
"## 核心特点\n",
"\n",
"1. **Label 转换**: 将 `future_return_5` 按每日进行 20 分位数划分（qcut）\n",
"2. **排序学习**: 使用 LambdaRank 目标函数，学习每日股票排序\n",
"3. **NDCG 评估**: 使用 NDCG@1/5/10/20 评估排序质量\n",
"4. **策略回测**: 基于排序分数构建 Top-k 选股策略"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. 导入依赖"
]
},
{
"cell_type": "code",
"metadata": {
"jupyter": {
"is_executing": true
},
"ExecuteTime": {
"start_time": "2026-03-09T16:47:47.027059Z"
}
},
"source": [
"import os\n",
"from datetime import datetime\n",
"from typing import List, Tuple, Optional\n",
"\n",
"import numpy as np\n",
"import polars as pl\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"from sklearn.metrics import ndcg_score\n",
"\n",
"from src.factors import FactorEngine\n",
"from src.training import (\n",
" DateSplitter,\n",
" STFilter,\n",
" StockPoolManager,\n",
" Trainer,\n",
" Winsorizer,\n",
" NullFiller,\n",
" StandardScaler,\n",
")\n",
"from src.training.components.models import LightGBMLambdaRankModel\n",
"from src.training.config import TrainingConfig"
],
"outputs": [],
"execution_count": null
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. 辅助函数"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:20.005621Z",
"start_time": "2026-03-09T16:28:19.995941Z"
}
},
"source": [
"def create_factors_with_strings(engine: FactorEngine, factor_definitions: dict, label_factor: dict) -> List[str]:\n",
" \"\"\"使用字符串表达式注册因子\"\"\"\n",
" print(\"=\" * 80)\n",
" print(\"使用字符串表达式定义因子\")\n",
" print(\"=\" * 80)\n",
"\n",
" # 注册所有特征因子\n",
" print(\"\\n注册特征因子:\")\n",
" for name, expr in factor_definitions.items():\n",
" engine.add_factor(name, expr)\n",
" print(f\" - {name}: {expr}\")\n",
"\n",
" # 注册 label 因子\n",
" print(\"\\n注册 Label 因子:\")\n",
" for name, expr in label_factor.items():\n",
" engine.add_factor(name, expr)\n",
" print(f\" - {name}: {expr}\")\n",
"\n",
" # 从字典自动获取特征列\n",
" feature_cols = list(factor_definitions.keys())\n",
"\n",
" print(f\"\\n特征因子数: {len(feature_cols)}\")\n",
" print(f\"Label: {list(label_factor.keys())[0]}\")\n",
" print(f\"已注册因子总数: {len(engine.list_registered())}\")\n",
"\n",
" return feature_cols\n",
"\n",
"\n",
"def prepare_data(\n",
" engine: FactorEngine,\n",
" feature_cols: List[str],\n",
" start_date: str,\n",
" end_date: str,\n",
") -> pl.DataFrame:\n",
" \"\"\"准备数据\"\"\"\n",
" print(\"\\n\" + \"=\" * 80)\n",
" print(\"准备数据\")\n",
" print(\"=\" * 80)\n",
"\n",
" # 计算因子（全市场数据）\n",
" print(f\"\\n计算因子: {start_date} - {end_date}\")\n",
" factor_names = feature_cols + [LABEL_NAME] # 包含 label\n",
"\n",
" data = engine.compute(\n",
" factor_names=factor_names,\n",
" start_date=start_date,\n",
" end_date=end_date,\n",
" )\n",
"\n",
" print(f\"数据形状: {data.shape}\")\n",
" print(f\"数据列: {data.columns}\")\n",
" print(f\"\\n前5行预览:\")\n",
" print(data.head())\n",
"\n",
" return data\n",
"\n",
"\n",
"def prepare_ranking_data(\n",
" df: pl.DataFrame,\n",
" label_col: str = \"future_return_5\",\n",
" date_col: str = \"trade_date\",\n",
" n_quantiles: int = 20,\n",
") -> Tuple[pl.DataFrame, str]:\n",
" \"\"\"准备排序学习数据\n",
" \n",
" 将连续 label 转换为分位数标签，用于排序学习任务。\n",
" \n",
" Args:\n",
" df: 原始数据\n",
" label_col: 原始标签列名\n",
" date_col: 日期列名\n",
" n_quantiles: 分位数数量\n",
" \n",
" Returns:\n",
" (处理后的 DataFrame, 新的标签列名)\n",
" \"\"\"\n",
" print(\"\\n\" + \"=\" * 80)\n",
" print(f\"准备排序学习数据（将 {label_col} 转换为 {n_quantiles} 分位数标签）\")\n",
" print(\"=\" * 80)\n",
" \n",
" # 新的标签列名\n",
" rank_col = f\"{label_col}_rank\"\n",
" \n",
" # 按日期分组进行分位数划分\n",
" # 使用 rank 生成 0, 1, 2, ..., n_quantiles-1 的标签\n",
" # 方法: 计算每天内的排名，然后映射到 n_quantiles 个分位数组\n",
" df_ranked = df.with_columns(\n",
" # 计算每天内的排名 (1-based)\n",
" pl.col(label_col).rank(method=\"min\").over(date_col).alias(\"_rank\")\n",
" ).with_columns(\n",
" # 将排名转换为分位数标签 (0 to n_quantiles-1)\n",
" ((pl.col(\"_rank\") - 1) / pl.len().over(date_col) * n_quantiles)\n",
" .floor()\n",
" .cast(pl.Int64)\n",
" .clip(0, n_quantiles - 1)\n",
" .alias(rank_col)\n",
" ).drop(\"_rank\")\n",
" \n",
" # 检查转换结果\n",
" print(f\"\\n原始 {label_col} 统计:\")\n",
" print(df_ranked[label_col].describe())\n",
" \n",
" print(f\"\\n转换后 {rank_col} 统计:\")\n",
" print(df_ranked[rank_col].describe())\n",
" \n",
" # 检查每日样本分布\n",
" print(f\"\\n每日样本数统计:\")\n",
" daily_counts = df_ranked.group_by(date_col).agg(pl.count().alias(\"count\"))\n",
" print(daily_counts[\"count\"].describe())\n",
" \n",
" # 检查分位数分布（应该是均匀的）\n",
" print(f\"\\n分位数标签分布:\")\n",
" rank_dist = df_ranked[rank_col].value_counts().sort(rank_col)\n",
" print(rank_dist)\n",
" \n",
" return df_ranked, rank_col\n",
"\n",
"\n",
"def compute_group_array(df: pl.DataFrame, date_col: str = \"trade_date\") -> np.ndarray:\n",
" \"\"\"计算 group 数组用于 LambdaRank\n",
" \n",
" 每个日期作为一个 query，group 数组表示每个 query 的样本数。\n",
" \n",
" Args:\n",
" df: 数据框\n",
" date_col: 日期列名\n",
" \n",
" Returns:\n",
" group 数组\n",
" \"\"\"\n",
" group_counts = df.group_by(date_col, maintain_order=True).agg(\n",
" pl.count().alias(\"count\")\n",
" )\n",
" return group_counts[\"count\"].to_numpy()\n",
"\n",
"\n",
"def evaluate_ndcg_at_k(\n",
" y_true: np.ndarray,\n",
" y_pred: np.ndarray,\n",
" group: np.ndarray,\n",
" k_list: List[int] = [1, 5, 10, 20],\n",
") -> dict:\n",
" \"\"\"计算 NDCG@k 指标\n",
" \n",
" Args:\n",
" y_true: 真实标签\n",
" y_pred: 预测分数\n",
" group: 分组数组\n",
" k_list: 要计算的 k 值列表\n",
" \n",
" Returns:\n",
" NDCG 指标字典\n",
" \"\"\"\n",
" results = {}\n",
" \n",
" # 按 group 拆分数据\n",
" start_idx = 0\n",
" y_true_groups = []\n",
" y_pred_groups = []\n",
" \n",
" for group_size in group:\n",
" end_idx = start_idx + group_size\n",
" y_true_groups.append(y_true[start_idx:end_idx])\n",
" y_pred_groups.append(y_pred[start_idx:end_idx])\n",
" start_idx = end_idx\n",
" \n",
" # 计算每个 k 值的平均 NDCG\n",
" for k in k_list:\n",
" ndcg_scores = []\n",
" for yt, yp in zip(y_true_groups, y_pred_groups):\n",
" if len(yt) > 1:\n",
" try:\n",
" score = ndcg_score([yt], [yp], k=k)\n",
" ndcg_scores.append(score)\n",
" except ValueError:\n",
" # 标签都相同，无法计算\n",
" pass\n",
" \n",
" results[f\"ndcg@{k}\"] = np.mean(ndcg_scores) if ndcg_scores else 0.0\n",
" \n",
" return results\n",
"\n",
"\n",
"def analyze_top_k_strategy(\n",
" df: pl.DataFrame,\n",
" prediction_col: str = \"prediction\",\n",
" return_col: str = \"future_return_5\",\n",
" date_col: str = \"trade_date\",\n",
" k_list: List[int] = [5, 10, 20],\n",
") -> dict:\n",
" \"\"\"分析 Top-k 选股策略表现\n",
" \n",
" Args:\n",
" df: 包含预测分数和真实收益的 DataFrame\n",
" prediction_col: 预测分数列名\n",
" return_col: 真实收益列名\n",
" date_col: 日期列名\n",
" k_list: 要分析的 k 值列表\n",
" \n",
" Returns:\n",
" 策略分析结果字典\n",
" \"\"\"\n",
" results = {}\n",
" \n",
" for k in k_list:\n",
" # 每日选择 Top-k 股票\n",
" daily_top_k = (\n",
" df.sort([date_col, prediction_col], descending=[False, True])\n",
" .group_by(date_col)\n",
" .head(k)\n",
" )\n",
" \n",
" # 计算每日平均收益\n",
" daily_returns = daily_top_k.group_by(date_col).agg(\n",
" pl.col(return_col).mean().alias(f\"top{k}_return\")\n",
" ).sort(date_col)\n",
" \n",
" # 计算累计收益\n",
" returns_array = daily_returns[f\"top{k}_return\"].to_numpy()\n",
" cumulative_returns = np.cumprod(1 + returns_array) - 1\n",
" \n",
" # 统计指标\n",
" results[f\"top{k}\"] = {\n",
" \"mean_daily_return\": np.mean(returns_array),\n",
" \"std_daily_return\": np.std(returns_array),\n",
" \"sharpe_ratio\": np.mean(returns_array) / (np.std(returns_array) + 1e-8) * np.sqrt(252),\n",
" \"total_return\": cumulative_returns[-1] if len(cumulative_returns) > 0 else 0,\n",
" \"cumulative_returns\": cumulative_returns,\n",
" \"dates\": daily_returns[date_col].to_list(),\n",
" }\n",
" \n",
" return results\n",
"\n",
"\n",
"def plot_strategy_performance(strategy_results: dict, save_path: Optional[str] = None):\n",
" \"\"\"绘制策略表现图\"\"\"\n",
" fig, axes = plt.subplots(2, 2, figsize=(14, 10))\n",
" \n",
" # 累计收益曲线\n",
" ax = axes[0, 0]\n",
" for name, result in strategy_results.items():\n",
" ax.plot(result[\"dates\"], result[\"cumulative_returns\"], label=name, linewidth=2)\n",
" ax.set_xlabel(\"Date\")\n",
" ax.set_ylabel(\"Cumulative Return\")\n",
" ax.set_title(\"Top-k Strategy Cumulative Returns\")\n",
" ax.legend()\n",
" ax.grid(True, alpha=0.3)\n",
" \n",
" # 日均收益对比\n",
" ax = axes[0, 1]\n",
" names = list(strategy_results.keys())\n",
" mean_returns = [strategy_results[n][\"mean_daily_return\"] for n in names]\n",
" ax.bar(names, mean_returns)\n",
" ax.set_ylabel(\"Mean Daily Return\")\n",
" ax.set_title(\"Mean Daily Return Comparison\")\n",
" ax.grid(True, alpha=0.3, axis=\"y\")\n",
" \n",
" # 夏普比率对比\n",
" ax = axes[1, 0]\n",
" sharpe_ratios = [strategy_results[n][\"sharpe_ratio\"] for n in names]\n",
" ax.bar(names, sharpe_ratios)\n",
" ax.set_ylabel(\"Sharpe Ratio\")\n",
" ax.set_title(\"Sharpe Ratio Comparison\")\n",
" ax.grid(True, alpha=0.3, axis=\"y\")\n",
" \n",
" # 总收益对比\n",
" ax = axes[1, 1]\n",
" total_returns = [strategy_results[n][\"total_return\"] for n in names]\n",
" ax.bar(names, total_returns)\n",
" ax.set_ylabel(\"Total Return\")\n",
" ax.set_title(\"Total Return Comparison\")\n",
" ax.grid(True, alpha=0.3, axis=\"y\")\n",
" \n",
" plt.tight_layout()\n",
" \n",
" if save_path:\n",
" plt.savefig(save_path, dpi=300, bbox_inches=\"tight\")\n",
" \n",
" plt.show()"
],
"outputs": [],
"execution_count": 2
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. 配置参数\n",
"\n",
"### 3.1 因子定义"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:20.017203Z",
"start_time": "2026-03-09T16:28:20.009432Z"
}
},
"source": [
"# 特征因子定义字典（复用 regression.ipynb 的因子定义）\n",
"LABEL_NAME = \"future_return_5_rank\"\n",
"\n",
"FACTOR_DEFINITIONS = {\n",
" # ================= 1. 价格、趋势与路径依赖 (Trend, Momentum & Path Dependency) =================\n",
" \"ma_5\": \"ts_mean(close, 5)\",\n",
" \"ma_20\": \"ts_mean(close, 20)\",\n",
" \"ma_ratio_5_20\": \"ts_mean(close, 5) / (ts_mean(close, 20) + 1e-8) - 1\",\n",
" \"bias_10\": \"close / (ts_mean(close, 10) + 1e-8) - 1\",\n",
" \"high_low_ratio\": \"(close - ts_min(low, 20)) / (ts_max(high, 20) - ts_min(low, 20) + 1e-8)\",\n",
" \"bbi_ratio\": \"(ts_mean(close, 3) + ts_mean(close, 6) + ts_mean(close, 12) + ts_mean(close, 24)) / (4 * close + 1e-8)\",\n",
" \"return_5\": \"(close / (ts_delay(close, 5) + 1e-8)) - 1\",\n",
" \"return_20\": \"(close / (ts_delay(close, 20) + 1e-8)) - 1\",\n",
" \"kaufman_ER_20\": \"abs(close - ts_delay(close, 20)) / (ts_sum(abs(close - ts_delay(close, 1)), 20) + 1e-8)\",\n",
" \"mom_acceleration_10_20\": \"(close / (ts_delay(close, 10) + 1e-8) - 1) - (ts_delay(close, 10) / (ts_delay(close, 20) + 1e-8) - 1)\",\n",
" \"drawdown_from_high_60\": \"close / (ts_max(high, 60) + 1e-8) - 1\",\n",
" \"up_days_ratio_20\": \"ts_sum(close > ts_delay(close, 1), 20) / 20\",\n",
"\n",
" # ================= 2. 波动率、风险调整与高阶矩 =================\n",
" \"volatility_5\": \"ts_std(close, 5)\",\n",
" \"volatility_20\": \"ts_std(close, 20)\",\n",
" \"volatility_ratio\": \"ts_std(close, 5) / (ts_std(close, 20) + 1e-8)\",\n",
" \"std_return_20\": \"ts_std((close / (ts_delay(close, 1) + 1e-8)) - 1, 20)\",\n",
" \"sharpe_ratio_20\": \"ts_mean(close / (ts_delay(close, 1) + 1e-8) - 1, 20) / (ts_std(close / (ts_delay(close, 1) + 1e-8) - 1, 20) + 1e-8)\",\n",
" \"min_ret_20\": \"ts_min(close / (ts_delay(close, 1) + 1e-8) - 1, 20)\",\n",
" \"volatility_squeeze_5_60\": \"ts_std(close, 5) / (ts_std(close, 60) + 1e-8)\",\n",
"\n",
" # ================= 3. 日内微观结构与异象 =================\n",
" \"overnight_intraday_diff\": \"(open / (ts_delay(close, 1) + 1e-8) - 1) - (close / (open + 1e-8) - 1)\",\n",
" \"upper_shadow_ratio\": \"(high - ((open + close + abs(open - close)) / 2)) / (high - low + 1e-8)\",\n",
" \"capital_retention_20\": \"ts_sum(abs(close - open), 20) / (ts_sum(high - low, 20) + 1e-8)\",\n",
" \"max_ret_20\": \"ts_max(close / (ts_delay(close, 1) + 1e-8) - 1, 20)\",\n",
"\n",
" # ================= 4. 量能、流动性与量价背离 =================\n",
" \"volume_ratio_5_20\": \"ts_mean(vol, 5) / (ts_mean(vol, 20) + 1e-8)\",\n",
" \"turnover_rate_mean_5\": \"ts_mean(turnover_rate, 5)\",\n",
" \"turnover_deviation\": \"(turnover_rate - ts_mean(turnover_rate, 10)) / (ts_std(turnover_rate, 10) + 1e-8)\",\n",
" \"amihud_illiq_20\": \"ts_mean(abs(close / (ts_delay(close, 1) + 1e-8) - 1) / (amount + 1e-8), 20)\",\n",
" \"turnover_cv_20\": \"ts_std(turnover_rate, 20) / (ts_mean(turnover_rate, 20) + 1e-8)\",\n",
" \"pv_corr_20\": \"ts_corr(close / (ts_delay(close, 1) + 1e-8) - 1, vol, 20)\",\n",
" \"close_vwap_deviation\": \"close / (amount / (vol * 100 + 1e-8) + 1e-8) - 1\",\n",
"\n",
" # ================= 5. 基本面财务特征 =================\n",
" \"roe\": \"n_income / (total_hldr_eqy_exc_min_int + 1e-8)\",\n",
" \"roa\": \"n_income / (total_assets + 1e-8)\",\n",
" \"profit_margin\": \"n_income / (revenue + 1e-8)\",\n",
" \"debt_to_equity\": \"total_liab / (total_hldr_eqy_exc_min_int + 1e-8)\",\n",
" \"current_ratio\": \"total_cur_assets / (total_cur_liab + 1e-8)\",\n",
" \"net_profit_yoy\": \"(n_income / (ts_delay(n_income, 252) + 1e-8)) - 1\",\n",
" \"revenue_yoy\": \"(revenue / (ts_delay(revenue, 252) + 1e-8)) - 1\",\n",
" \"healthy_expansion_velocity\": \"(total_assets / (ts_delay(total_assets, 252) + 1e-8) - 1) - (total_liab / (ts_delay(total_liab, 252) + 1e-8) - 1)\",\n",
"\n",
" # ================= 6. 基本面估值与截面动量共振 =================\n",
" \"EP\": \"n_income / (total_mv * 10000 + 1e-8)\",\n",
" \"BP\": \"total_hldr_eqy_exc_min_int / (total_mv * 10000 + 1e-8)\",\n",
" \"CP\": \"n_cashflow_act / (total_mv * 10000 + 1e-8)\",\n",
" \"market_cap_rank\": \"cs_rank(total_mv)\",\n",
" \"turnover_rank\": \"cs_rank(turnover_rate)\",\n",
" \"return_5_rank\": \"cs_rank((close / (ts_delay(close, 5) + 1e-8)) - 1)\",\n",
" \"EP_rank\": \"cs_rank(n_income / (total_mv + 1e-8))\",\n",
" \"pe_expansion_trend\": \"(total_mv / (n_income + 1e-8)) / (ts_delay(total_mv, 60) / (ts_delay(n_income, 60) + 1e-8) + 1e-8) - 1\",\n",
" \"value_price_divergence\": \"cs_rank((n_income - ts_delay(n_income, 252)) / (abs(ts_delay(n_income, 252)) + 1e-8)) - cs_rank(close / (ts_delay(close, 20) + 1e-8))\",\n",
" \"active_market_cap\": \"total_mv * ts_mean(turnover_rate, 20)\",\n",
" \"ebit_rank\": \"cs_rank(ebit)\",\n",
"}\n",
"\n",
"# Label 因子定义（不参与训练，用于计算目标）\n",
"LABEL_FACTOR = {\n",
" LABEL_NAME: \"(ts_delay(close, -5) / ts_delay(open, -1)) - 1\",\n",
"}"
],
"outputs": [],
"execution_count": 3
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.2 训练参数配置"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:20.025233Z",
"start_time": "2026-03-09T16:28:20.020941Z"
}
},
"source": [
"# 日期范围配置（正确的 train/val/test 三分法）\n",
"TRAIN_START = \"20200101\"\n",
"TRAIN_END = \"20231231\"\n",
"VAL_START = \"20240101\"\n",
"VAL_END = \"20241231\"\n",
"TEST_START = \"20250101\"\n",
"TEST_END = \"20261231\"\n",
"\n",
"# LambdaRank 模型参数配置\n",
"MODEL_PARAMS = {\n",
" \"objective\": \"lambdarank\",\n",
" \"metric\": \"ndcg\",\n",
" \"ndcg_at\": [1, 5, 10, 20], # 评估 NDCG@k\n",
" \"learning_rate\": 0.05,\n",
" \"num_leaves\": 31,\n",
" \"max_depth\": 6,\n",
" \"min_data_in_leaf\": 20,\n",
" \"n_estimators\": 1000,\n",
" \"early_stopping_rounds\": 50,\n",
" \"subsample\": 0.8,\n",
" \"colsample_bytree\": 0.8,\n",
" \"reg_alpha\": 0.1,\n",
" \"reg_lambda\": 1.0,\n",
" \"verbose\": -1,\n",
" \"random_state\": 42,\n",
"}\n",
"\n",
"# 分位数配置\n",
"N_QUANTILES = 20 # 将 label 分为 20 组\n",
"\n",
"# 特征列（用于数据处理器）\n",
"FEATURE_COLS = list(FACTOR_DEFINITIONS.keys())\n",
"\n",
"# 数据处理器配置\n",
"PROCESSORS = [\n",
" NullFiller(feature_cols=FEATURE_COLS, strategy=\"mean\"),\n",
" Winsorizer(feature_cols=FEATURE_COLS, lower=0.01, upper=0.99),\n",
" StandardScaler(feature_cols=FEATURE_COLS),\n",
"]\n",
"\n",
"# 股票池筛选函数\n",
"def stock_pool_filter(df: pl.DataFrame) -> pl.Series:\n",
" \"\"\"股票池筛选函数（单日数据）\n",
" \n",
" 筛选条件：\n",
" 1. 排除创业板（代码以 300 开头）\n",
" 2. 排除科创板（代码以 688 开头）\n",
" 3. 排除北交所（代码以 8、9 或 4 开头）\n",
" 4. 选取当日市值最小的500只股票\n",
" \"\"\"\n",
" code_filter = (\n",
" ~df[\"ts_code\"].str.starts_with(\"300\") &\n",
" ~df[\"ts_code\"].str.starts_with(\"688\") &\n",
" ~df[\"ts_code\"].str.starts_with(\"8\") &\n",
" ~df[\"ts_code\"].str.starts_with(\"9\") &\n",
" ~df[\"ts_code\"].str.starts_with(\"4\")\n",
" )\n",
" \n",
" valid_df = df.filter(code_filter)\n",
" n = min(1000, len(valid_df))\n",
" small_cap_codes = valid_df.sort(\"total_mv\").head(n)[\"ts_code\"]\n",
" \n",
" return df[\"ts_code\"].is_in(small_cap_codes)\n",
"\n",
"STOCK_FILTER_REQUIRED_COLUMNS = [\"total_mv\"]\n",
"\n",
"# 输出配置\n",
"OUTPUT_DIR = \"output\"\n",
"SAVE_PREDICTIONS = True\n",
"PERSIST_MODEL = False\n",
"\n",
"# Top N 配置：每日推荐股票数量\n",
"TOP_N = 5 # 可调整为 10, 20 等"
],
"outputs": [],
"execution_count": 4
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. 训练流程"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:43.241056Z",
"start_time": "2026-03-09T16:28:20.030078Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"LightGBM LambdaRank 排序学习训练\")\n",
"print(\"=\" * 80)\n",
"\n",
"# 1. 创建 FactorEngine\n",
"print(\"\\n[1] 创建 FactorEngine\")\n",
"engine = FactorEngine()\n",
"\n",
"# 2. 使用字符串表达式定义因子\n",
"print(\"\\n[2] 定义因子（字符串表达式）\")\n",
"feature_cols = create_factors_with_strings(engine, FACTOR_DEFINITIONS, LABEL_FACTOR)\n",
"\n",
"# 3. 准备数据\n",
"print(\"\\n[3] 准备数据\")\n",
"data = prepare_data(\n",
" engine=engine,\n",
" feature_cols=feature_cols,\n",
" start_date=TRAIN_START,\n",
" end_date=TEST_END,\n",
")\n",
"\n",
"# 4. 转换为排序学习格式（分位数标签）\n",
"print(\"\\n[4] 转换为排序学习格式\")\n",
"data, target_col = prepare_ranking_data(\n",
" df=data,\n",
" label_col=LABEL_NAME,\n",
" n_quantiles=N_QUANTILES,\n",
")\n",
"\n",
"# 5. 打印配置信息\n",
"print(f\"\\n[配置] 训练期: {TRAIN_START} - {TRAIN_END}\")\n",
"print(f\"[配置] 验证期: {VAL_START} - {VAL_END}\")\n",
"print(f\"[配置] 测试期: {TEST_START} - {TEST_END}\")\n",
"print(f\"[配置] 特征数: {len(feature_cols)}\")\n",
"print(f\"[配置] 目标变量: {target_col}（{N_QUANTILES}分位数）\")\n",
"\n",
"# 6. 创建排序学习模型\n",
"model = LightGBMLambdaRankModel(params=MODEL_PARAMS)\n",
"\n",
"# 7. 创建数据处理器\n",
"processors = PROCESSORS\n",
"\n",
"# 8. 创建数据划分器\n",
"splitter = DateSplitter(\n",
" train_start=TRAIN_START,\n",
" train_end=TRAIN_END,\n",
" val_start=VAL_START,\n",
" val_end=VAL_END,\n",
" test_start=TEST_START,\n",
" test_end=TEST_END,\n",
")\n",
"\n",
"# 9. 创建股票池管理器\n",
"pool_manager = StockPoolManager(\n",
" filter_func=stock_pool_filter,\n",
" required_columns=STOCK_FILTER_REQUIRED_COLUMNS,\n",
" data_router=engine.router,\n",
")\n",
"\n",
"# 10. 创建 ST 过滤器\n",
"st_filter = STFilter(data_router=engine.router)\n",
"\n",
"# 11. 创建训练器\n",
"trainer = Trainer(\n",
" model=model,\n",
" pool_manager=pool_manager,\n",
" processors=processors,\n",
" filters=[st_filter],\n",
" splitter=splitter,\n",
" target_col=target_col,\n",
" feature_cols=feature_cols,\n",
" persist_model=PERSIST_MODEL,\n",
")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"LightGBM LambdaRank 排序学习训练\n",
"================================================================================\n",
"\n",
"[1] 创建 FactorEngine\n",
"\n",
"[2] 定义因子（字符串表达式）\n",
"================================================================================\n",
"使用字符串表达式定义因子\n",
"================================================================================\n",
"\n",
"注册特征因子:\n",
" - ma_5: ts_mean(close, 5)\n",
" - ma_20: ts_mean(close, 20)\n",
" - ma_ratio_5_20: ts_mean(close, 5) / (ts_mean(close, 20) + 1e-8) - 1\n",
" - bias_10: close / (ts_mean(close, 10) + 1e-8) - 1\n",
" - high_low_ratio: (close - ts_min(low, 20)) / (ts_max(high, 20) - ts_min(low, 20) + 1e-8)\n",
" - bbi_ratio: (ts_mean(close, 3) + ts_mean(close, 6) + ts_mean(close, 12) + ts_mean(close, 24)) / (4 * close + 1e-8)\n",
" - return_5: (close / (ts_delay(close, 5) + 1e-8)) - 1\n",
" - return_20: (close / (ts_delay(close, 20) + 1e-8)) - 1\n",
" - kaufman_ER_20: abs(close - ts_delay(close, 20)) / (ts_sum(abs(close - ts_delay(close, 1)), 20) + 1e-8)\n",
" - mom_acceleration_10_20: (close / (ts_delay(close, 10) + 1e-8) - 1) - (ts_delay(close, 10) / (ts_delay(close, 20) + 1e-8) - 1)\n",
" - drawdown_from_high_60: close / (ts_max(high, 60) + 1e-8) - 1\n",
" - up_days_ratio_20: ts_sum(close > ts_delay(close, 1), 20) / 20\n",
" - volatility_5: ts_std(close, 5)\n",
" - volatility_20: ts_std(close, 20)\n",
" - volatility_ratio: ts_std(close, 5) / (ts_std(close, 20) + 1e-8)\n",
" - std_return_20: ts_std((close / (ts_delay(close, 1) + 1e-8)) - 1, 20)\n",
" - sharpe_ratio_20: ts_mean(close / (ts_delay(close, 1) + 1e-8) - 1, 20) / (ts_std(close / (ts_delay(close, 1) + 1e-8) - 1, 20) + 1e-8)\n",
" - min_ret_20: ts_min(close / (ts_delay(close, 1) + 1e-8) - 1, 20)\n",
" - volatility_squeeze_5_60: ts_std(close, 5) / (ts_std(close, 60) + 1e-8)\n",
" - overnight_intraday_diff: (open / (ts_delay(close, 1) + 1e-8) - 1) - (close / (open + 1e-8) - 1)\n",
" - upper_shadow_ratio: (high - ((open + close + abs(open - close)) / 2)) / (high - low + 1e-8)\n",
" - capital_retention_20: ts_sum(abs(close - open), 20) / (ts_sum(high - low, 20) + 1e-8)\n",
" - max_ret_20: ts_max(close / (ts_delay(close, 1) + 1e-8) - 1, 20)\n",
" - volume_ratio_5_20: ts_mean(vol, 5) / (ts_mean(vol, 20) + 1e-8)\n",
" - turnover_rate_mean_5: ts_mean(turnover_rate, 5)\n",
" - turnover_deviation: (turnover_rate - ts_mean(turnover_rate, 10)) / (ts_std(turnover_rate, 10) + 1e-8)\n",
" - amihud_illiq_20: ts_mean(abs(close / (ts_delay(close, 1) + 1e-8) - 1) / (amount + 1e-8), 20)\n",
" - turnover_cv_20: ts_std(turnover_rate, 20) / (ts_mean(turnover_rate, 20) + 1e-8)\n",
" - pv_corr_20: ts_corr(close / (ts_delay(close, 1) + 1e-8) - 1, vol, 20)\n",
" - close_vwap_deviation: close / (amount / (vol * 100 + 1e-8) + 1e-8) - 1\n",
" - roe: n_income / (total_hldr_eqy_exc_min_int + 1e-8)\n",
" - roa: n_income / (total_assets + 1e-8)\n",
" - profit_margin: n_income / (revenue + 1e-8)\n",
" - debt_to_equity: total_liab / (total_hldr_eqy_exc_min_int + 1e-8)\n",
" - current_ratio: total_cur_assets / (total_cur_liab + 1e-8)\n",
" - net_profit_yoy: (n_income / (ts_delay(n_income, 252) + 1e-8)) - 1\n",
" - revenue_yoy: (revenue / (ts_delay(revenue, 252) + 1e-8)) - 1\n",
" - healthy_expansion_velocity: (total_assets / (ts_delay(total_assets, 252) + 1e-8) - 1) - (total_liab / (ts_delay(total_liab, 252) + 1e-8) - 1)\n",
" - EP: n_income / (total_mv * 10000 + 1e-8)\n",
" - BP: total_hldr_eqy_exc_min_int / (total_mv * 10000 + 1e-8)\n",
" - CP: n_cashflow_act / (total_mv * 10000 + 1e-8)\n",
" - market_cap_rank: cs_rank(total_mv)\n",
" - turnover_rank: cs_rank(turnover_rate)\n",
" - return_5_rank: cs_rank((close / (ts_delay(close, 5) + 1e-8)) - 1)\n",
" - EP_rank: cs_rank(n_income / (total_mv + 1e-8))\n",
" - pe_expansion_trend: (total_mv / (n_income + 1e-8)) / (ts_delay(total_mv, 60) / (ts_delay(n_income, 60) + 1e-8) + 1e-8) - 1\n",
" - value_price_divergence: cs_rank((n_income - ts_delay(n_income, 252)) / (abs(ts_delay(n_income, 252)) + 1e-8)) - cs_rank(close / (ts_delay(close, 20) + 1e-8))\n",
" - active_market_cap: total_mv * ts_mean(turnover_rate, 20)\n",
" - ebit_rank: cs_rank(ebit)\n",
"\n",
"注册 Label 因子:\n",
" - future_return_5_rank: (ts_delay(close, -5) / ts_delay(open, -1)) - 1\n",
"\n",
"特征因子数: 49\n",
"Label: future_return_5_rank\n",
"已注册因子总数: 50\n",
"\n",
"[3] 准备数据\n",
"\n",
"================================================================================\n",
"准备数据\n",
"================================================================================\n",
"\n",
"计算因子: 20200101 - 20261231\n",
"[FinancialLoader] 加载 financial_fina_indicator 失败: Binder Error: Referenced column \"f_ann_date\" not found in FROM clause!\n",
"Candidate bindings: \"ann_date\", \"end_date\", \"ocf_to_debt\", \"arturn_days\", \"nca_to_assets\"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"D:\\PyProject\\ProStock\\src\\data\\financial_loader.py:148: UserWarning: Sortedness of columns cannot be checked when 'by' groups provided\n",
" merged = df_price.join_asof(\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"数据形状: (7255513, 70)\n",
"数据列: ['ts_code', 'trade_date', 'low', 'open', 'high', 'turnover_rate', 'vol', 'close', 'amount', 'total_assets', 'total_mv', 'f_ann_date', 'total_liab', 'total_cur_liab', 'total_hldr_eqy_exc_min_int', 'total_cur_assets', 'n_income', 'revenue', 'n_cashflow_act', 'ebit', 'ma_5', 'ma_20', 'ma_ratio_5_20', 'bias_10', 'high_low_ratio', 'bbi_ratio', 'return_5', 'return_20', 'kaufman_ER_20', 'mom_acceleration_10_20', 'drawdown_from_high_60', 'up_days_ratio_20', 'volatility_5', 'volatility_20', 'volatility_ratio', 'std_return_20', 'sharpe_ratio_20', 'min_ret_20', 'volatility_squeeze_5_60', 'overnight_intraday_diff', 'upper_shadow_ratio', 'capital_retention_20', 'max_ret_20', 'volume_ratio_5_20', 'turnover_rate_mean_5', 'turnover_deviation', 'amihud_illiq_20', 'turnover_cv_20', 'pv_corr_20', 'close_vwap_deviation', 'roe', 'roa', 'profit_margin', 'debt_to_equity', 'current_ratio', 'net_profit_yoy', 'revenue_yoy', 'healthy_expansion_velocity', 'EP', 'BP', 'CP', 'market_cap_rank', 'turnover_rank', 'return_5_rank', 'EP_rank', 'pe_expansion_trend', 'value_price_divergence', 'active_market_cap', 'ebit_rank', 'future_return_5_rank']\n",
"\n",
"前5行预览:\n",
"shape: (5, 70)\n",
"┌───────────┬────────────┬─────────┬─────────┬───┬────────────┬────────────┬───────────┬───────────┐\n",
"│ ts_code ┆ trade_date ┆ low ┆ open ┆ … ┆ value_pric ┆ active_mar ┆ ebit_rank ┆ future_re │\n",
"│ --- ┆ --- ┆ --- ┆ --- ┆ ┆ e_divergen ┆ ket_cap ┆ --- ┆ turn_5_ra │\n",
"│ str ┆ str ┆ f64 ┆ f64 ┆ ┆ ce ┆ --- ┆ f64 ┆ nk │\n",
"│ ┆ ┆ ┆ ┆ ┆ --- ┆ f64 ┆ ┆ --- │\n",
"│ ┆ ┆ ┆ ┆ ┆ f64 ┆ ┆ ┆ f64 │\n",
"╞═══════════╪════════════╪═════════╪═════════╪═══╪════════════╪════════════╪═══════════╪═══════════╡\n",
"│ 000001.SZ ┆ 20200102 ┆ 1806.75 ┆ 1817.67 ┆ … ┆ null ┆ null ┆ null ┆ -0.008857 │\n",
"│ 000001.SZ ┆ 20200103 ┆ 1847.15 ┆ 1849.33 ┆ … ┆ null ┆ null ┆ null ┆ -0.01881 │\n",
"│ 000001.SZ ┆ 20200106 ┆ 1846.05 ┆ 1856.97 ┆ … ┆ null ┆ null ┆ null ┆ -0.008171 │\n",
"│ 000001.SZ ┆ 20200107 ┆ 1850.42 ┆ 1870.07 ┆ … ┆ null ┆ null ┆ null ┆ -0.014117 │\n",
"│ 000001.SZ ┆ 20200108 ┆ 1815.49 ┆ 1855.88 ┆ … ┆ null ┆ null ┆ null ┆ -0.017252 │\n",
"└───────────┴────────────┴─────────┴─────────┴───┴────────────┴────────────┴───────────┴───────────┘\n",
"\n",
"[4] 转换为排序学习格式\n",
"\n",
"================================================================================\n",
"准备排序学习数据（将 future_return_5_rank 转换为 20 分位数标签）\n",
"================================================================================\n",
"\n",
"原始 future_return_5_rank 统计:\n",
"shape: (9, 2)\n",
"┌────────────┬────────────┐\n",
"│ statistic ┆ value │\n",
"│ --- ┆ --- │\n",
"│ str ┆ f64 │\n",
"╞════════════╪════════════╡\n",
"│ count ┆ 7.227054e6 │\n",
"│ null_count ┆ 28459.0 │\n",
"│ mean ┆ 0.003978 │\n",
"│ std ┆ 0.073204 │\n",
"│ min ┆ -0.969459 │\n",
"│ 25% ┆ -0.032998 │\n",
"│ 50% ┆ -0.001278 │\n",
"│ 75% ┆ 0.032666 │\n",
"│ max ┆ 10.361925 │\n",
"└────────────┴────────────┘\n",
"\n",
"转换后 future_return_5_rank_rank 统计:\n",
"shape: (9, 2)\n",
"┌────────────┬────────────┐\n",
"│ statistic ┆ value │\n",
"│ --- ┆ --- │\n",
"│ str ┆ f64 │\n",
"╞════════════╪════════════╡\n",
"│ count ┆ 7.227054e6 │\n",
"│ null_count ┆ 28459.0 │\n",
"│ mean ┆ 9.493551 │\n",
"│ std ┆ 5.765628 │\n",
"│ min ┆ 0.0 │\n",
"│ 25% ┆ 4.0 │\n",
"│ 50% ┆ 9.0 │\n",
"│ 75% ┆ 14.0 │\n",
"│ max ┆ 19.0 │\n",
"└────────────┴────────────┘\n",
"\n",
"每日样本数统计:\n",
"shape: (9, 2)\n",
"┌────────────┬─────────────┐\n",
"│ statistic ┆ value │\n",
"│ --- ┆ --- │\n",
"│ str ┆ f64 │\n",
"╞════════════╪═════════════╡\n",
"│ count ┆ 1494.0 │\n",
"│ null_count ┆ 0.0 │\n",
"│ mean ┆ 4856.434404 │\n",
"│ std ┆ 564.521537 │\n",
"│ min ┆ 2885.0 │\n",
"│ 25% ┆ 4382.0 │\n",
"│ 50% ┆ 5069.0 │\n",
"│ 75% ┆ 5347.0 │\n",
"│ max ┆ 5476.0 │\n",
"└────────────┴─────────────┘\n",
"\n",
"分位数标签分布:\n",
"shape: (21, 2)\n",
"┌───────────────────────────┬────────┐\n",
"│ future_return_5_rank_rank ┆ count │\n",
"│ --- ┆ --- │\n",
"│ i64 ┆ u32 │\n",
"╞═══════════════════════════╪════════╡\n",
"│ null ┆ 28459 │\n",
"│ 0 ┆ 362270 │\n",
"│ 1 ┆ 361546 │\n",
"│ 2 ┆ 361599 │\n",
"│ 3 ┆ 361755 │\n",
"│ … ┆ … │\n",
"│ 15 ┆ 361289 │\n",
"│ 16 ┆ 361218 │\n",
"│ 17 ┆ 361227 │\n",
"│ 18 ┆ 361252 │\n",
"│ 19 ┆ 359483 │\n",
"└───────────────────────────┴────────┘\n",
"\n",
"[配置] 训练期: 20200101 - 20231231\n",
"[配置] 验证期: 20240101 - 20241231\n",
"[配置] 测试期: 20250101 - 20261231\n",
"[配置] 特征数: 49\n",
"[配置] 目标变量: future_return_5_rank_rank（20分位数）\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\liaozhaorun\\AppData\\Local\\Temp\\ipykernel_8380\\3149203115.py:108: DeprecationWarning: `pl.count()` is deprecated. Please use `pl.len()` instead.\n",
"(Deprecated in version 0.20.5)\n",
" daily_counts = df_ranked.group_by(date_col).agg(pl.count().alias(\"count\"))\n"
]
}
],
"execution_count": 5
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.1 股票池筛选"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:46.756857Z",
"start_time": "2026-03-09T16:28:43.247911Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"股票池筛选\")\n",
"print(\"=\" * 80)\n",
"\n",
"if pool_manager:\n",
" print(\"\\n执行每日独立筛选股票池...\")\n",
" filtered_data = pool_manager.filter_and_select_daily(data)\n",
" print(f\" 筛选前数据规模: {data.shape}\")\n",
" print(f\" 筛选后数据规模: {filtered_data.shape}\")\n",
" print(f\" 筛选前股票数: {data['ts_code'].n_unique()}\")\n",
" print(f\" 筛选后股票数: {filtered_data['ts_code'].n_unique()}\")\n",
" print(f\" 删除记录数: {len(data) - len(filtered_data)}\")\n",
"else:\n",
" filtered_data = data\n",
" print(\" 未配置股票池管理器，跳过筛选\")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"股票池筛选\n",
"================================================================================\n",
"\n",
"执行每日独立筛选股票池...\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\liaozhaorun\\AppData\\Local\\Temp\\ipykernel_8380\\653304.py:63: DeprecationWarning: `is_in` with a collection of the same datatype is ambiguous and deprecated.\n",
"Please use `implode` to return to previous behavior.\n",
"\n",
"See https://github.com/pola-rs/polars/issues/22149 for more information.\n",
" return df[\"ts_code\"].is_in(small_cap_codes)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" 筛选前数据规模: (7255513, 71)\n",
" 筛选后数据规模: (1494000, 71)\n",
" 筛选前股票数: 5694\n",
" 筛选后股票数: 2252\n",
" 删除记录数: 5761513\n"
]
}
],
"execution_count": 6
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.2 数据划分"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:46.849944Z",
"start_time": "2026-03-09T16:28:46.764139Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"数据划分\")\n",
"print(\"=\" * 80)\n",
"\n",
"if splitter:\n",
" train_data, val_data, test_data = splitter.split(filtered_data)\n",
" print(f\"\\n训练集数据规模: {train_data.shape}\")\n",
" print(f\"验证集数据规模: {val_data.shape}\")\n",
" print(f\"测试集数据规模: {test_data.shape}\")\n",
" \n",
" # 计算各集的 group 数组\n",
" train_group = compute_group_array(train_data)\n",
" val_group = compute_group_array(val_data)\n",
" test_group = compute_group_array(test_data)\n",
" \n",
" print(f\"\\n训练集 group 数量: {len(train_group)}\")\n",
" print(f\"验证集 group 数量: {len(val_group)}\")\n",
" print(f\"测试集 group 数量: {len(test_group)}\")\n",
" print(f\"训练集日均样本数: {np.mean(train_group):.1f}\")\n",
" print(f\"验证集日均样本数: {np.mean(val_group):.1f}\")\n",
" print(f\"测试集日均样本数: {np.mean(test_group):.1f}\")\n",
"else:\n",
" raise ValueError(\"必须配置数据划分器\")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"数据划分\n",
"================================================================================\n",
"\n",
"训练集数据规模: (970000, 71)\n",
"验证集数据规模: (242000, 71)\n",
"测试集数据规模: (282000, 71)\n",
"\n",
"训练集 group 数量: 970\n",
"验证集 group 数量: 242\n",
"测试集 group 数量: 282\n",
"训练集日均样本数: 1000.0\n",
"验证集日均样本数: 1000.0\n",
"测试集日均样本数: 1000.0\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\liaozhaorun\\AppData\\Local\\Temp\\ipykernel_8380\\3149203115.py:132: DeprecationWarning: `pl.count()` is deprecated. Please use `pl.len()` instead.\n",
"(Deprecated in version 0.20.5)\n",
" pl.count().alias(\"count\")\n"
]
}
],
"execution_count": 7
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.3 数据预处理"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:47.385207Z",
"start_time": "2026-03-09T16:28:46.854611Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"数据预处理\")\n",
"print(\"=\" * 80)\n",
"\n",
"fitted_processors = []\n",
"if processors:\n",
" print(\"\\n训练集处理...\")\n",
" for i, processor in enumerate(processors, 1):\n",
" print(f\" [{i}/{len(processors)}] {processor.__class__.__name__}\")\n",
" train_data = processor.fit_transform(train_data)\n",
" fitted_processors.append(processor)\n",
" \n",
" print(\"\\n验证集处理...\")\n",
" for processor in fitted_processors:\n",
" val_data = processor.transform(val_data)\n",
" \n",
" print(\"\\n测试集处理...\")\n",
" for processor in fitted_processors:\n",
" test_data = processor.transform(test_data)\n",
"\n",
"print(f\"\\n处理后训练集形状: {train_data.shape}\")\n",
"print(f\"处理后验证集形状: {val_data.shape}\")\n",
"print(f\"处理后测试集形状: {test_data.shape}\")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"数据预处理\n",
"================================================================================\n",
"\n",
"训练集处理...\n",
" [1/3] NullFiller\n",
" [2/3] Winsorizer\n",
" [3/3] StandardScaler\n",
"\n",
"验证集处理...\n",
"\n",
"测试集处理...\n",
"\n",
"处理后训练集形状: (970000, 71)\n",
"处理后验证集形状: (242000, 71)\n",
"处理后测试集形状: (282000, 71)\n"
]
}
],
"execution_count": 8
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.4 训练 LambdaRank 模型"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:49.943341Z",
"start_time": "2026-03-09T16:28:47.393354Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"训练 LambdaRank 模型\")\n",
"print(\"=\" * 80)\n",
"\n",
"# 准备数据\n",
"X_train = train_data.select(feature_cols)\n",
"y_train = train_data.select(target_col).to_series()\n",
"\n",
"X_val = val_data.select(feature_cols)\n",
"y_val = val_data.select(target_col).to_series()\n",
"\n",
"print(f\"\\n训练样本数: {len(X_train)}\")\n",
"print(f\"验证样本数: {len(X_val)}\")\n",
"print(f\"特征数: {len(feature_cols)}\")\n",
"print(f\"目标变量: {target_col}\")\n",
"\n",
"print(\"\\n目标变量统计（训练集）:\")\n",
"print(y_train.describe())\n",
"\n",
"print(\"\\n开始训练...\")\n",
"model.fit(\n",
" X=X_train,\n",
" y=y_train,\n",
" group=train_group,\n",
" eval_set=(X_val, y_val, val_group),\n",
")\n",
"print(\"训练完成！\")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"训练 LambdaRank 模型\n",
"================================================================================\n",
"\n",
"训练样本数: 970000\n",
"验证样本数: 242000\n",
"特征数: 49\n",
"目标变量: future_return_5_rank_rank\n",
"\n",
"目标变量统计（训练集）:\n",
"shape: (9, 2)\n",
"┌────────────┬──────────┐\n",
"│ statistic ┆ value │\n",
"│ --- ┆ --- │\n",
"│ str ┆ f64 │\n",
"╞════════════╪══════════╡\n",
"│ count ┆ 969536.0 │\n",
"│ null_count ┆ 464.0 │\n",
"│ mean ┆ 9.821 │\n",
"│ std ┆ 5.444634 │\n",
"│ min ┆ 0.0 │\n",
"│ 25% ┆ 5.0 │\n",
"│ 50% ┆ 10.0 │\n",
"│ 75% ┆ 14.0 │\n",
"│ max ┆ 19.0 │\n",
"└────────────┴──────────┘\n",
"\n",
"开始训练...\n",
"Training until validation scores don't improve for 50 rounds\n",
"Early stopping, best iteration is:\n",
"[48]\ttraining's ndcg@1: 0.516707\ttraining's ndcg@5: 0.393291\ttraining's ndcg@10: 0.328332\ttraining's ndcg@20: 0.274341\tvalid_1's ndcg@1: 0.401918\tvalid_1's ndcg@5: 0.342001\tvalid_1's ndcg@10: 0.310652\tvalid_1's ndcg@20: 0.268765\n",
"训练完成！\n"
]
}
],
"execution_count": 9
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.5 训练指标曲线"
]
},
{
"cell_type": "code",
"metadata": {},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"训练指标曲线\")\n",
"print(\"=\" * 80)\n",
"\n",
"# 重新训练以收集指标（因为之前的训练没有保存评估结果）\n",
"print(\"\\n重新训练模型以收集训练指标...\")\n",
"\n",
"import lightgbm as lgb\n",
"\n",
"# 准备数据（使用 val 做验证，test 不参与训练过程）\n",
"X_train_np = X_train.to_numpy()\n",
"y_train_np = y_train.to_numpy()\n",
"X_val_np = val_data.select(feature_cols).to_numpy()\n",
"y_val_np = val_data.select(target_col).to_series().to_numpy()\n",
"\n",
"# 创建数据集\n",
"train_dataset = lgb.Dataset(X_train_np, label=y_train_np, group=train_group)\n",
"val_dataset = lgb.Dataset(X_val_np, label=y_val_np, group=val_group, reference=train_dataset)\n",
"\n",
"# 用于存储评估结果\n",
"evals_result = {}\n",
"\n",
"# 使用与原模型相同的参数重新训练\n",
"# 正确的三分法：train用于训练，val用于验证，test不参与训练过程\n",
"booster_with_eval = lgb.train(\n",
" MODEL_PARAMS,\n",
" train_dataset,\n",
" num_boost_round=MODEL_PARAMS.get(\"n_estimators\", 1000),\n",
" valid_sets=[train_dataset, val_dataset],\n",
" valid_names=[\"train\", \"val\"],\n",
" callbacks=[\n",
" lgb.record_evaluation(evals_result),\n",
" lgb.early_stopping(stopping_rounds=50, verbose=True),\n",
" ],\n",
")\n",
"\n",
"print(\"训练完成，指标已收集\")\n",
"\n",
"# 获取评估的 NDCG 指标\n",
"ndcg_metrics = [k for k in evals_result[\"train\"].keys() if \"ndcg\" in k]\n",
"print(f\"\\n评估的 NDCG 指标: {ndcg_metrics}\")\n",
"\n",
"# 显示早停信息\n",
"actual_rounds = len(list(evals_result[\"train\"].values())[0])\n",
"expected_rounds = MODEL_PARAMS.get(\"n_estimators\", 1000)\n",
"print(f\"\\n[早停信息]\")\n",
"print(f\" 配置的最大轮数: {expected_rounds}\")\n",
"print(f\" 实际训练轮数: {actual_rounds}\")\n",
"if actual_rounds < expected_rounds:\n",
" print(f\" 早停状态: 已触发（连续50轮验证指标未改善）\")\n",
"else:\n",
" print(f\" 早停状态: 未触发（达到最大轮数）\")\n",
"\n",
"# 显示各 NDCG 指标的最终值\n",
"print(f\"\\n最终 NDCG 指标:\")\n",
"for metric in ndcg_metrics:\n",
" train_ndcg = evals_result[\"train\"][metric][-1]\n",
" val_ndcg = evals_result[\"val\"][metric][-1]\n",
" print(f\" {metric}: 训练集={train_ndcg:.4f}, 验证集={val_ndcg:.4f}\")"
],
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"metadata": {},
"source": [
"# 绘制 NDCG 训练指标曲线\n",
"import matplotlib.pyplot as plt\n",
"\n",
"fig, axes = plt.subplots(2, 2, figsize=(14, 10))\n",
"axes = axes.flatten()\n",
"\n",
"for idx, metric in enumerate(ndcg_metrics[:4]): # 最多显示4个NDCG指标\n",
" ax = axes[idx]\n",
" train_metric = evals_result[\"train\"][metric]\n",
" val_metric = evals_result[\"val\"][metric]\n",
" iterations = range(1, len(train_metric) + 1)\n",
" \n",
" ax.plot(iterations, train_metric, label=f\"Train {metric}\", linewidth=2, color=\"blue\")\n",
" ax.plot(iterations, val_metric, label=f\"Val {metric}\", linewidth=2, color=\"red\")\n",
" ax.set_xlabel(\"Iteration\", fontsize=10)\n",
" ax.set_ylabel(metric.upper(), fontsize=10)\n",
" ax.set_title(f\"Training and Validation {metric.upper()}\", fontsize=12, fontweight=\"bold\")\n",
" ax.legend(fontsize=9)\n",
" ax.grid(True, alpha=0.3)\n",
" \n",
" # 标记最佳验证指标点\n",
" best_iter = val_metric.index(max(val_metric))\n",
" best_metric = max(val_metric)\n",
" ax.axvline(x=best_iter + 1, color=\"green\", linestyle=\"--\", alpha=0.7)\n",
" ax.scatter([best_iter + 1], [best_metric], color=\"green\", s=80, zorder=5)\n",
" ax.annotate(\n",
" f\"Best: {best_metric:.4f}\",\n",
" xy=(best_iter + 1, best_metric),\n",
" xytext=(best_iter + 1 + len(iterations) * 0.05, best_metric),\n",
" fontsize=8,\n",
" arrowprops=dict(arrowstyle=\"->\", color=\"green\", alpha=0.7),\n",
" )\n",
"\n",
"plt.tight_layout()\n",
"plt.show()\n",
"\n",
"print(f\"\\n[指标分析]\")\n",
"print(f\" 各NDCG指标在验证集上的最佳值:\")\n",
"for metric in ndcg_metrics:\n",
" val_metric_list = evals_result[\"val\"][metric]\n",
" best_iter = val_metric_list.index(max(val_metric_list))\n",
" best_val = max(val_metric_list)\n",
" print(f\" {metric}: {best_val:.4f} (迭代 {best_iter + 1})\")\n",
"print(f\"\\n[重要提醒] 验证集仅用于早停/调参，测试集完全独立于训练过程！\")"
],
"outputs": [],
"execution_count": null
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.6 模型评估"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:50.036021Z",
"start_time": "2026-03-09T16:28:49.947604Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"模型评估\")\n",
"print(\"=\" * 80)\n",
"\n",
"# 准备测试集\n",
"X_test = test_data.select(feature_cols)\n",
"y_test = test_data.select(target_col).to_series()\n",
"\n",
"# 预测\n",
"print(\"\\n生成预测...\")\n",
"predictions = model.predict(X_test)\n",
"\n",
"# 添加预测列\n",
"test_data = test_data.with_columns([pl.Series(\"prediction\", predictions)])\n",
"\n",
"# 计算 NDCG 指标\n",
"print(\"\\n计算 NDCG 指标...\")\n",
"ndcg_results = evaluate_ndcg_at_k(\n",
" y_true=y_test.to_numpy(),\n",
" y_pred=predictions,\n",
" group=test_group,\n",
" k_list=[1, 5, 10, 20],\n",
")\n",
"\n",
"print(\"\\nNDCG 评估结果:\")\n",
"print(\"-\" * 40)\n",
"for metric, value in ndcg_results.items():\n",
" print(f\" {metric}: {value:.4f}\")\n",
"\n",
"# 特征重要性\n",
"print(\"\\n特征重要性（Top 20）:\")\n",
"print(\"-\" * 40)\n",
"importance = model.feature_importance()\n",
"if importance is not None:\n",
" top_features = importance.sort_values(ascending=False).head(20)\n",
" for i, (feature, score) in enumerate(top_features.items(), 1):\n",
" print(f\" {i:2d}. {feature:30s} {score:10.2f}\")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"模型评估\n",
"================================================================================\n",
"\n",
"生成预测...\n",
"\n",
"计算 NDCG 指标...\n",
"\n",
"NDCG 评估结果:\n",
"----------------------------------------\n",
" ndcg@1: 0.0000\n",
" ndcg@5: 0.0000\n",
" ndcg@10: 0.0000\n",
" ndcg@20: 0.0000\n",
"\n",
"特征重要性（Top 20）:\n",
"----------------------------------------\n",
" 1. max_ret_20 3634.79\n",
" 2. turnover_rank 2905.90\n",
" 3. drawdown_from_high_60 1816.20\n",
" 4. overnight_intraday_diff 1484.73\n",
" 5. std_return_20 791.58\n",
" 6. volume_ratio_5_20 757.43\n",
" 7. current_ratio 694.13\n",
" 8. revenue_yoy 605.02\n",
" 9. kaufman_ER_20 507.48\n",
" 10. close_vwap_deviation 372.14\n",
" 11. roa 281.62\n",
" 12. active_market_cap 280.69\n",
" 13. sharpe_ratio_20 278.22\n",
" 14. turnover_rate_mean_5 274.94\n",
" 15. healthy_expansion_velocity 266.26\n",
" 16. net_profit_yoy 263.69\n",
" 17. high_low_ratio 252.26\n",
" 18. volatility_squeeze_5_60 245.18\n",
" 19. return_5_rank 243.28\n",
" 20. volatility_20 228.92\n"
]
}
],
"execution_count": 10
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.7 Top-k 选股策略分析"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:50.725016Z",
"start_time": "2026-03-09T16:28:50.043768Z"
}
},
"source": [
"print(\"\\n\" + \"=\" * 80)\n",
"print(\"Top-k 选股策略分析\")\n",
"print(\"=\" * 80)\n",
"\n",
"# 分析策略表现\n",
"strategy_results = analyze_top_k_strategy(\n",
" df=test_data,\n",
" prediction_col=\"prediction\",\n",
" return_col=LABEL_NAME,\n",
" k_list=[5, 10, 20],\n",
")\n",
"\n",
"# 打印结果\n",
"print(\"\\n策略表现统计:\")\n",
"print(\"=\" * 60)\n",
"for name, result in strategy_results.items():\n",
" print(f\"\\n{name}:\")\n",
" print(f\" 日均收益: {result['mean_daily_return']:.4f}\")\n",
" print(f\" 日收益标准差: {result['std_daily_return']:.4f}\")\n",
" print(f\" 年化夏普比率: {result['sharpe_ratio']:.4f}\")\n",
" print(f\" 累计收益: {result['total_return']:.4f}\")\n",
"\n",
"# 绘制图表\n",
"print(\"\\n生成策略表现图...\")\n",
"plot_strategy_performance(strategy_results)"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"================================================================================\n",
"Top-k 选股策略分析\n",
"================================================================================\n",
"\n",
"策略表现统计:\n",
"============================================================\n",
"\n",
"top5:\n",
" 日均收益: nan\n",
" 日收益标准差: nan\n",
" 年化夏普比率: nan\n",
" 累计收益: nan\n",
"\n",
"top10:\n",
" 日均收益: nan\n",
" 日收益标准差: nan\n",
" 年化夏普比率: nan\n",
" 累计收益: nan\n",
"\n",
"top20:\n",
" 日均收益: nan\n",
" 日收益标准差: nan\n",
" 年化夏普比率: nan\n",
" 累计收益: nan\n",
"\n",
"生成策略表现图...\n"
]
},
{
"data": {
"text/plain": [
"<Figure size 1400x1000 with 4 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAABW0AAAPeCAYAAAB3GThSAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3Qd8G+X5B/BHW5b3nhlkkUkmIQRa9t4b+mePMgqEWUYhlNECBcIotAFaNpRCgZSyZwiBEMggg+ydeG9rWuv+n+c93+lky7bsWLIs/76fj3KvTufT6aTY7z163ufVSZIkEQAAAAAAAAAAAAAkBH1/HwAAAAAAAAAAAAAAhCBoCwAAAAAAAAAAAJBAELQFAAAAAAAAAAAASCAI2gIAAAAAAAAAAAAkEARtAQAAAAAAAAAAABIIgrYAAAAAAAAAAAAACQRBWwAAAAAAAAAAAIAEgqAtAAAAAAAAAAAAQAJB0BYAAAAAAAAAAAAggSBoCwA9dvHFF1NaWhrOHETlpZdeIp1ORzt27OizM8b74n3yvgEAAACgd4YPHy769oqFCxeKPhYvAXqLP1P82QKAvYOgLcAAwZ2naG4DoYO1ePFiOu6446i0tJSsVisNHTqUTjrpJHrjjTfUbVwuF/3xj3+M2ev56KOPxP4HipaWFrr33ntp8uTJImCekpJCEydOpNtuu40qKiooWfFn4oknnqBEoQSLlZter6ecnBzxeV6yZEmv9hnrzzoAAMBg/tKYb9z3bE+SJBoyZIh4/MQTT6RExsEvbd8jKyuLJk2aRL/97W9p6dKl/X14XeI+jrbvZDKZxOu5/vrrqampqVf7/P7778V+e/vz8eDxeOjxxx+nAw44gDIzM8U1z5gxY+jaa6+lTZs29ffhAcAAYezvAwCA6Lz66qth91955RX6/PPPO6wfN25cQp/St99+m8455xyaMmUKzZkzh7Kzs2n79u20aNEiev755+k3v/mNGsjiICU79NBDYxK0feaZZwZE4Hbbtm105JFH0q5du+iss84SHXSz2UyrV6+mf/7zn/Tee+8lbeePg7Zr166lG264IWz9sGHDyO12i45/fzjvvPPo+OOPp0AgIM793/72NzrssMPop59+EhdRPRHrzzoAAMBgxsEy7k8cfPDBYeu/+eYb2rNnD1ksFhoIuO988803i7bdbqf169eLfjX3n2+88UaaN29er/a7ceNGEQiOtb///e8i8cDpdNKXX35Jf/3rX2nFihURA+rRBG2578TZnBzATjR1dXV07LHH0vLly8UXAnx9w6+dz/Wbb75Jzz33HHm9Xkpm/LkMBoP9fRgAAx6CtgADxPnnnx92/4cffhBB2/brEx0HScePHy+OnwOPWjU1Nb3eL3cAU1NTKdn4/X46/fTTqbq6WmRitr/g+NOf/kQPP/wwDTacqcEXYf1l2rRpYf/3fvWrX4lsW74g4QBuonx2uLPc/v8ZAADAYMJfsnJw86mnniKjMXT5y4Hc6dOniwDbQMAj1Nr3+7kPyAFBzugcPXo0XX311T3eb7yC1meeeSbl5eWJ9pVXXknnnnsu/fvf/6Yff/yRZs6cSYmgr64nOJi8cuVK+s9//kNnnHFG2GP3338//eEPf6BkpZzD/kqsAEg2KI8AkGR/JPkbeB7qxR2wfffdlx599FEx/Kt9wIuH5rz++utiGw5+caeVs1176+eff6b8/HyRKehwODrdbuvWrbT//vtHDCQVFBSoQ9B5X4y/RVeGUylZsUpNXd4Xd8TT09Pp//7v/8Rj3377rchG5ZILfA74XHD2AWdlKvjnOctWORfKTcGBLh6SP2HCBHFuCgsLReeysbEx7Hh5Oz6mkpISstlsItNy3bp1YbXBOEuW982d6UhZAvzYv/71r07P1zvvvEOrVq0Snbv2AVuWkZEhAred1SVT8PuizeJU6pW99dZb4hzzhQCfR+5QNzc3U2trq8hu5feEz/Ull1wi1kVTU1b7XnXmv//9L51wwgni3PH7NHLkSNGJ5cxV7TF/+OGHtHPnTvU9UmpjtX9+/pzzfd62vTvuuEN83rTvHw8l5AwIHq7G790hhxxC3333HfUWB20Zfya1eNgen0fl/+SoUaPEBZaSedDdZ739+9ZZnTDlfPB54M8un09+Pv48KsMSt2zZomak8Ovm95SzfLX4iyD+nPE2/L7z74c777yz1+cFAACgv/HomPr6evE3TsFZjhxQU0Z4tRdtXzCa/gzjv+Vc1or/LnN/kfse3Pf6y1/+slevjctl8ag7LtXE/UFtn5/7BLNnz6bc3FyxHff1+TW311nfUXHPPfeIAFxtbW2Hx3j0F/cZuBRAX/Wduuujcb/m1ltvFe199tlH7TtxX6gn/VOlf8TvCX8OePSf0tfmc8IZspwFzAFl/gyMGDFCjHTsDh8/918vu+yyDgFbxp8Tfm+0vvrqK3E+ONjJ5/OUU04RmdRayvHyCC8O3vP54T7k3XffLd733bt3i5/ja4OioiJ67LHHwn5e6ftzoJz7drwNP9/JJ58sflYrmuup7q7JItW05Sxj/hzydnycPDrtySefDNuGr534ufkzze//rFmzxPmM9Fr4OoY/92VlZeI9OuKII0R/FyCZINMWIEnwH2v+o/v111+LTgIPofr0009Fp6a8vLxD0JCHhPEfba4nxX+MOTuQO0j8bTd3KnuCh4Qfc8wxNGPGDNF55Y5hZ3hYOw+J4uFo/Ac2Eu6AcMYiZwucdtppItOU7bfffmFZhPyc3Lnijg//UWecScGBKP5Z7qTy6+HhV/x8/BjjTjfXgY1UXkJ5nDt7HNTi88PlG55++mnxjTl3GpVvjjkYyJ1trsfLx8LBVV5qO67cwTvooINEgJw7O1q8jjst3MHqzPvvvy+WF1xwAcXCgw8+KN6v22+/XXRy+Fzx6+Nhcnxhwh1Ezorm88Ed47lz5/bJ8/L+uJN30003iSV3VnnfXLv3kUceEdtwoJoDyPzeKZ/fzibAO/vss+n3v/+96LwpHXkFrzv66KNFZ5zxc3FWLHca+UKEX+uLL75Ihx9+uOik9ibbQ5lkTXkOxp9DvtDg/3/8meKOLwfq+XNTWVkpLgaj+az3BL8O/vzxRRT/v+YOr/Yc8XvI7zkPR/zHP/4hgvJKpvYvv/wiLlD4ue+77z7x8/yZ2JtgNgAAQH/jwNGBBx4oviTnv//s448/Fn0MzvbkDNze9gWj6c8ouF/FfW3+W89/kzmAynMTcOBKOa7e4OflPgSXzOIAJAeaGQfD+NqAg2gcpOaAGQfDPvjgAxFojhb3QblfwNcNnPTRPvDNgcnejH6K1HeKpo/G548Dl/x+cv9Qyd7lPlWkwHJ3+JxwlvKf//znsKA394E4mYGvqy666CJ64YUXRCCSj005x33Rd//iiy/Ea+ZrBu53c2CU++N8/cD9tfaBTy4zx+XwHnroIRHMfOCBB0R/79lnnxXnift1fI1xyy23iESZX//612E/z0FODnjyZ49HOXJ/lMuwcQKOcg0XzfVUd9dk7fF1F3+BwoFVpe/JgWn+/8Ql8xiPLOQvGvi5+f8dP/fLL78sPsf8WePPuRafA/6M8Gvl/898Xcaf90Sv8wzQIxIADEi/+93vuFeh3l+wYIG4/8ADD4Rtd+aZZ0o6nU7asmWLuo6349uyZcvUdTt37pSsVqt02mmndfvcF110kZSamiraixcvljIyMqQTTjhB8ng83f7sP//5T/HcZrNZOuyww6S7775b+vbbb6VAIBC2XW1trdjunnvuifj8/Njtt9/e4TGXy9Vh3YMPPijOAb/Gzs6fgo+F17/++uth6z/55JOw9VVVVZLRaJROPfXUsO3++Mc/iu34GBXPPvusWLd+/Xp1ndfrlfLy8sK2i2Tq1KlSZmamFK1hw4ZF3Ochhxwiboqvv/5aHNPEiRPFsSjOO+88ca6OO+64sJ8/8MADxb4V27dvFz//4osvdniu9u8bb8Pr+Ge6ep+uvPJKyWazhX2O+HOlfd6unp+Pcfr06WHb/fjjj2K7V155RdwPBoPS6NGjpWOOOUa0tcezzz77SEcddVSH54r0vPf
},
"metadata": {},
"output_type": "display_data",
"jetTransient": {
"display_id": null
}
}
],
"execution_count": 11
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.7 保存结果"
]
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2026-03-09T16:28:50.888298Z",
"start_time": "2026-03-09T16:28:50.733200Z"
}
},
"source": [
"# 确保输出目录存在\n",
"os.makedirs(OUTPUT_DIR, exist_ok=True)\n",
"\n",
"# 生成时间戳\n",
"start_dt = datetime.strptime(TEST_START, \"%Y%m%d\")\n",
"end_dt = datetime.strptime(TEST_END, \"%Y%m%d\")\n",
"date_str = f\"{start_dt.strftime('%Y%m%d')}_{end_dt.strftime('%Y%m%d')}\"\n",
"\n",
"# 保存每日 Top N\n",
"print(f\"\\n[1/1] 保存每日 Top {TOP_N} 股票...\")\n",
"topn_output_path = os.path.join(OUTPUT_DIR, \"rank_output.csv\")\n",
"\n",
"# 按日期分组，取每日 top N\n",
"topn_by_date = []\n",
"unique_dates = test_data[\"trade_date\"].unique().sort()\n",
"for date in unique_dates:\n",
" day_data = test_data.filter(test_data[\"trade_date\"] == date)\n",
" # 按 prediction 降序排序，取前 N\n",
" topn = day_data.sort(\"prediction\", descending=True).head(TOP_N)\n",
" topn_by_date.append(topn)\n",
"\n",
"# 合并所有日期的 top N\n",
"topn_results = pl.concat(topn_by_date)\n",
"\n",
"# 格式化日期并调整列顺序：日期、分数、股票\n",
"topn_to_save = topn_results.select(\n",
" [\n",
" pl.col(\"trade_date\").str.slice(0, 4)\n",
" + \"-\"\n",
" + pl.col(\"trade_date\").str.slice(4, 2)\n",
" + \"-\"\n",
" + pl.col(\"trade_date\").str.slice(6, 2).alias(\"date\"),\n",
" pl.col(\"prediction\").alias(\"score\"),\n",
" pl.col(\"ts_code\"),\n",
" ]\n",
")\n",
"topn_to_save.write_csv(topn_output_path, include_header=True)\n",
"print(f\" 保存路径: {topn_output_path}\")\n",
"print(f\" 保存行数: {len(topn_to_save)}（{len(unique_dates)}个交易日 x 每日top{TOP_N}）\")\n",
"print(f\"\\n 预览（前15行）:\")\n",
"print(topn_to_save.head(15))\n",
"\n",
"print(\"\\n训练流程完成！\")"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"预测结果已保存: output\\learn_to_rank_predictions.csv\n",
"特征重要性已保存: output\\feature_importance.csv\n",
"策略统计已保存: output\\strategy_statistics.csv\n",
"\n",
"训练流程完成！\n"
]
}
],
"execution_count": 12
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. 总结\n",
"\n",
"本 Notebook 实现了完整的 Learn-to-Rank 训练流程：\n",
"\n",
"### 核心步骤\n",
"\n",
"1. **数据准备**: 计算 49 个特征因子，将 `future_return_5` 转换为 20 分位数标签\n",
"2. **模型训练**: 使用 LightGBM LambdaRank 学习每日股票排序\n",
"3. **模型评估**: 使用 NDCG@1/5/10/20 评估排序质量\n",
"4. **策略分析**: 基于排序分数构建 Top-k 选股策略\n",
"\n",
"### 关键参数\n",
"\n",
"- **Objective**: lambdarank\n",
"- **Metric**: ndcg\n",
"- **Learning Rate**: 0.05\n",
"- **Num Leaves**: 31\n",
"- **N Quantiles**: 20\n",
"\n",
"### 输出结果\n",
"\n",
"- rank_output.csv: 每日Top-N推荐股票（格式：date, score, ts_code）\n",
"- 特征重要性排名\n",
"- Top-k 策略统计和图表\n",
"- NDCG训练指标曲线\n",
"\n",
"### 后续优化方向\n",
"\n",
"1. **特征工程**: 尝试更多因子组合\n",
"2. **超参数调优**: 使用网格搜索优化 LambdaRank 参数\n",
"3. **模型集成**: 结合多个排序模型的预测\n",
"4. **更复杂的分组**: 考虑按行业分组排序"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in New Issue Copy Permalink