"""可轉債對股價影響：發行（掛牌）、大量轉換、轉換溢價率、賣回權、到期的事件研究與五個策略回測。

與 https://finlab.finance/blog/convertible-bond-stock-price-impact 同一份程式。
超額報酬 = 標的股還原日報酬 − 同一天全市場普通股等權平均日報酬。
訊號日收盤後才拿得到當天的可轉債報表，所以從訊號日的下一個交易日收盤進場起算。

執行前先安裝 finlab，需要資料時 finlab 會自動引導登入：

    pip install finlab
    python research.py
"""
import numpy as np
import pandas as pd
from finlab import data
from finlab.backtest import sim

START = "2010-02-01"  # FinLab 可轉債每日報表從 2009-12-30 開始

close = data.get("etl:adj_close")
cb = pd.DataFrame(data.get("cb_published_info"))
cb_close = data.get("cb_price:收盤價")
conv = data.get("cb_converted_status:本月轉換張數")

# 普通股：四碼數字、不是 0 開頭（排除 ETF）
stocks = [s for s in close.columns if len(s) == 4 and s.isdigit() and not s.startswith("0")]
ret = close[stocks].loc["2009-06-01":].pct_change(fill_method=None)
excess = ret.sub(ret.mean(axis=1), axis=0).fillna(0)
days = excess.index
col = {s: i for i, s in enumerate(stocks)}
ex = excess.to_numpy()

# 公司自己發行的可轉債：代號前四碼是股票代號；「E + 數字」是交換債，標的是別家公司，排除
cb["symbol"] = cb["symbol"].astype(str)
cb = cb[~cb["債券簡稱"].astype(str).str.contains(r"E\d")].copy()
cb["stock_id"] = cb["symbol"].str[:4]
cb["date"] = pd.to_datetime(cb["date"])
bonds = cb.groupby("symbol").agg(stock_id=("stock_id", "first"), listed=("date", "min"), last=("date", "max"),
                                 issue=("原始發行總額", "max"), conv_end=("轉換迄日", "max"))


def entry_pos(d):
    """訊號日之後第一個交易日（在這天收盤進場）。"""
    return int(days.searchsorted(pd.Timestamp(d), side="right"))


def paths(events, pre, post):
    """每個事件從進場日前 pre 天到進場後 post 天的日超額報酬（第 0 欄 = 進場日）。"""
    out = []
    for s, d in zip(events["stock_id"], events["date"]):
        p = entry_pos(d)
        if s in col and p - pre >= 1 and p + post < len(days):
            out.append(ex[p - pre:p + post + 1, col[s]])
    return np.array(out)


def car(m, pre, a, b):
    """進場後第 a 天收盤到第 b 天收盤的累積超額報酬：平均、t 值、樣本數。"""
    v = m[:, pre + a + 1:pre + b + 1].sum(axis=1)
    return {"mean": float(v.mean()), "t": float(v.mean() / (v.std(ddof=1) / np.sqrt(len(v)))), "n": int(len(v))}


results, curves = {}, {}

# 1. 可轉債掛牌（新債第一次出現在每日報表）：之後 250 個交易日
listing = bonds[bonds["listed"] > START].reset_index()[["stock_id", "listed"]]
listing = listing.rename(columns={"listed": "date"}).drop_duplicates()
results["listing_events_total"] = int(len(listing))
for name, ev in {"listing": listing,
                 "listing_2010_2017": listing[listing["date"] < "2018-01-01"],
                 "listing_2018_": listing[listing["date"] >= "2018-01-01"]}.items():
    m = paths(ev, 0, 250)
    results[name] = {f"car_{h}": car(m, 0, 0, h) for h in (5, 20, 60, 120, 250)}
    curves[name] = m[:, 1:].cumsum(axis=1).mean(axis=0)
    if name == "listing":
        se = m[:, 1:].cumsum(axis=1).std(axis=0, ddof=1) / np.sqrt(len(m))
        curves["listing_lo"] = curves[name] - 1.96 * se
        curves["listing_hi"] = curves[name] + 1.96 * se

# 2. 單月轉換張數占原始發行張數 ≥ 10%／20%／30%（月報公布日當訊號日）
ratio = conv.mul(100_000).div(bonds["issue"].reindex(conv.columns), axis=1)
for th in (0.1, 0.2, 0.3):
    st = ratio.stack()
    st = st[st >= th].reset_index()
    st.columns = ["date", "symbol", "ratio"]
    st["stock_id"] = st["symbol"].str[:4]
    ev = st[st["date"] >= START][["stock_id", "date"]].drop_duplicates()
    m = paths(ev, 0, 250)
    results[f"conversion_{int(th * 100)}"] = {f"car_{h}": car(m, 0, 0, h) for h in (20, 60, 120, 250)}
    curves[f"conversion_{int(th * 100)}"] = m[:, 1:].cumsum(axis=1).mean(axis=0)

# 3. 轉換溢價率：每月底把有可轉債的股票依溢價率分五組，看之後 20 個交易日的超額報酬
conv_price = cb.pivot_table(index="date", columns="symbol", values="轉換價格", aggfunc="last")
stock_price = cb.pivot_table(index="date", columns="symbol", values="轉換標的股票價格", aggfunc="last")
parity = 100 * stock_price.where(stock_price > 0) / conv_price.where(conv_price > 0)  # 轉換價值，100 = 價平
premium = cb_close.reindex(index=parity.index, columns=parity.columns) / parity - 1
prem_stock = premium.ffill(limit=5).T.groupby(premium.columns.str[:4]).min().T  # 同一家取最低
fwd = excess.rolling(20).sum().shift(-21)  # 第 1 天收盤進場，持有 20 個交易日
rows = []
for d in prem_stock.loc[START:].resample("ME").last().index:
    s = prem_stock.loc[:d].iloc[-1].dropna()
    s = s[s.index.isin(stocks)]
    if d not in days:
        d = days[days.searchsorted(d) - 1]
    f = fwd.loc[d].reindex(s.index).dropna()
    s = s.reindex(f.index)
    if len(s) < 50:
        continue
    q = pd.qcut(s.rank(method="first"), 5, labels=[f"Q{i}" for i in range(1, 6)])
    rows.append(f.groupby(q, observed=True).mean().rename(d))
qt = pd.DataFrame(rows)
results["premium_quintiles"] = {k: {"mean": float(qt[k].mean()), "t": float(qt[k].mean() / (qt[k].std() / np.sqrt(len(qt)))),
                                    "months": int(len(qt))} for k in qt.columns}

# 4. 賣回權：報表上第一次出現「最近賣回權起日」的那天進場，抱到賣回權起日
put = cb[cb["最近賣回權起日"].notna()].groupby(["symbol", "最近賣回權起日"]).agg(date=("date", "min")).reset_index()
put = put[(put["最近賣回權起日"] > put["date"]) & (put["date"] >= START)]
put["stock_id"] = put["symbol"].str[:4]
put_lead_days = (put["最近賣回權起日"] - put["date"]).dt.days
put["parity"] = [parity.at[d, s] for s, d in zip(put["symbol"], put["date"])]
put_rows = []
for s, d, p_day, par in zip(put["stock_id"], put["date"], put["最近賣回權起日"], put["parity"]):
    a, b = entry_pos(d), int(days.searchsorted(pd.Timestamp(p_day)))
    if s in col and b > a and b < len(days):
        put_rows.append({"parity": par, "car": ex[a + 1:b + 1, col[s]].sum(), "days": b - a})
put_df = pd.DataFrame(put_rows)
bins = [0, 70, 90, 100, 120, np.inf]
labels = ["<70", "70–90", "90–100", "100–120", "≥120"]
put_df["bucket"] = pd.cut(put_df["parity"], bins, right=False, labels=labels)
results["put_window"] = {
    "all": {"mean": float(put_df["car"].mean()), "n": int(len(put_df)), "median_days": float(put_df["days"].median()),
            "median_lead_calendar_days": float(put_lead_days.median()),
            "t": float(put_df["car"].mean() / (put_df["car"].std() / np.sqrt(len(put_df))))},
    **{k: {"mean": float(v["car"].mean()), "n": int(len(v)), "t": float(v["car"].mean() / (v["car"].std() / np.sqrt(len(v))))}
       for k, v in put_df.groupby("bucket", observed=True)},
}

# 5. 到期：以轉換迄日為第 0 天，看前 120 天到後 20 天。
#    只算撐到到期的可轉債：轉換迄日前 30 天內還在報表上（提早全部轉換、賣回或下櫃的不算）
last_day = cb["date"].max()
in_window = bonds[(bonds["conv_end"] > pd.Timestamp(START) + pd.Timedelta(days=200)) & (bonds["conv_end"] <= last_day)]
mat_bonds = in_window[in_window["last"] >= in_window["conv_end"] - pd.Timedelta(days=30)]
results["maturity_sample"] = {"in_window": int(len(in_window)), "reached_maturity": int(len(mat_bonds))}
mat = mat_bonds.reset_index()
mat = mat.assign(date=mat["conv_end"] - pd.Timedelta(days=1))[["stock_id", "date"]]
m = paths(mat, 120, 20)
results["maturity"] = {"pre_120_60": car(m, 120, -120, -60), "pre_60_0": car(m, 120, -60, 0),
                       "pre_20_0": car(m, 120, -20, 0), "post_0_20": car(m, 120, 0, 20)}
curves["maturity"] = m.cumsum(axis=1).mean(axis=0)  # 從第 -120 天起累積

# 強制贖回欄位的資料起點（樣本太短，本文沒有測）
called = cb[cb["強制贖回起日"].notna()]
results["forced_call"] = {"bonds": int(called["symbol"].nunique()), "first_date": str(called["date"].min().date())}


# ---------------------------------------------------------------------------
# 五個策略（文章表格裡的原參數）。手續費與證交稅用 sim 預設。
# ---------------------------------------------------------------------------
all_days = close.loc["2009-01-01":].index


def holding(rows):
    """rows: (股票, 開始持有的交易日位置, 最後持有的交易日位置)。"""
    a = np.zeros((len(all_days), len(stocks)), dtype=bool)
    for s, lo, hi in rows:
        if s in col and hi >= lo:
            a[max(lo, 0):hi + 1, col[s]] = True
    pos = pd.DataFrame(a, index=all_days, columns=stocks).loc[START:]
    return pos.loc[:, pos.any()]


def at(d):
    return int(all_days.searchsorted(pd.Timestamp(d)))


def strategies():
    reports = {}
    # 掛牌後持有 120 個交易日，每週調整
    rows = [(s, at(d), at(d) + 119) for s, d in zip(listing["stock_id"], listing["date"])]
    reports["掛牌後持有"] = sim(holding(rows), resample="W", upload=False)
    # 轉換溢價率最低 20 檔，每月調整
    ps = prem_stock.reindex(all_days).ffill(limit=5).reindex(columns=stocks).loc[START:]
    pos = ps.rank(axis=1, method="first") <= 20
    reports["溢價率最低"] = sim(pos.loc[:, pos.any()], resample="M", upload=False)
    # 單月轉換 ≥ 發行量 20% 之後持有 60 個交易日，每週調整
    st = ratio.stack()
    st = st[st >= 0.2].reset_index()
    st.columns = ["date", "symbol", "ratio"]
    st = st[st["date"] >= START].assign(stock_id=lambda x: x["symbol"].str[:4])[["stock_id", "date"]].drop_duplicates()
    rows = [(s, at(d), at(d) + 59) for s, d in zip(st["stock_id"], st["date"])]
    reports["大量轉換後"] = sim(holding(rows), resample="W", upload=False)
    # 賣回權前：轉換價值 < 100 時，從賣回權日期出現抱到賣回權起日前一天
    p = put[put["parity"] < 100]
    rows = [(s, at(d), at(e) - 1) for s, d, e in zip(p["stock_id"], p["date"], p["最近賣回權起日"])]
    reports["賣回權前"] = sim(holding(rows), upload=False)
    # 到期前 60 個交易日進場（進場時轉換價值 < 100），抱到轉換迄日前一天
    par = parity.reindex(all_days).ffill(limit=5)
    rows = []
    for sym, s, end in zip(mat_bonds.index, mat_bonds["stock_id"], mat_bonds["conv_end"]):
        hi = at(end) - 1
        lo = hi - 59
        if lo >= 0 and sym in par.columns and par[sym].iloc[lo] < 100:
            rows.append((s, lo, hi))
    reports["到期前"] = sim(holding(rows), upload=False)
    return reports


if __name__ == "__main__":
    import json
    print(json.dumps(results, ensure_ascii=False, indent=1))
    for name, report in strategies().items():
        print(name, report.get_stats()["cagr"])