AlphaWave Data Machine Learning used for Asset Allocation example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine Learning used for Asset Allocation: Multi-task Lasso\n",
    "\n",
    "**Hugh Donnelly, CFA**<br> \n",
    "*AlphaWave Data*\n",
    "\n",
    "**September 2021**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "In this article, we use [machine learning](https://hdonnelly6.medium.com/list/machine-learning-for-investing-7f2690bb1826) to make future returns predictions for equity and fixed income ETFs so that we can create optimized Equity Only, Fixed Income Only, and 60/40 Allocation portfolios versus respective benchmarks.  We show that we are able to outperform the benchmarks using the Multi-task Lasso model.\n",
    "\n",
    "Jupyter Notebooks are available on [Google Colab](https://colab.research.google.com/drive/1HraWoI6I6dHc7YEZuAOgZHMzNgwN8OMZ?usp=sharing) and [Github](https://github.com/AlphaWaveData/Jupyter-Notebooks/blob/master/AlphaWave%20Data%20Machine%20Learning%20used%20for%20Asset%20Allocation%20example.ipynb).\n",
    "\n",
    "For this project, we use several Python-based scientific computing technologies listed below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import requests\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "from itertools import product\n",
    "import plotly.graph_objects as go\n",
    "from IPython.display import display\n",
    "from datetime import datetime, timedelta\n",
    "\n",
    "from scipy import stats\n",
    "from sklearn.linear_model import MultiTaskLasso\n",
    "from sklearn.utils.testing import ignore_warnings\n",
    "from sklearn.exceptions import ConvergenceWarning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Asset Allocation\n",
    "\n",
    "Let's start with a quick overview of asset allocation. Asset owners are concerned with accumulating and maintaining the wealth needed to meet their needs and aspirations. In that endeavor, investment portfolios—including individuals’ portfolios and institutional funds—play important roles. Asset allocation is a strategic—and often a first or early—decision in portfolio construction. Because it holds that position, it is widely accepted as important and meriting careful attention.\n",
    "\n",
    "Generally, investment firms manage a group of portfolios and have particular outcomes or target dates assigned to each of these portfolios. To make sure these portfolios meet their assigned goals, there can be a strategic asset allocation associated with them. The strategic asset allocation decision determines return levels in which allocations are invested, irrespective of the degree of active management.\n",
    "\n",
    "A common example of a portfolio with a strategic asset allocation is a portfolio with defined weightings for equity and fixed income asset classes.  The equity weighting may be allocated between U.S. equities and global equities while the fixed income weighting may be divided between Treasuries, corporate bonds, high-yield credit, and emerging market debt.  These weightings are usually quite fixed and are based on a long time horizon of historical returns and correlations. There may be other asset classes also considered in a portfolio with a strategic asset allocation, like commodities and derivatives. The portfolios are typically rebalanced periodically in order to maintain the same asset class exposure going forward in time.\n",
    "\n",
    "However, we live in a world that changes quickly. As a result, some investment firms often employ a tactical asset allocation. This provides the portfolio with short-term tilts. There can be dynamic weightings, or put differently, migrations slightly away from the strategic asset allocation. The portfolio that uses a tactical asset allocation may focus on allocations within sub-asset classes. If you are working within the fixed income asset class, you may move your allocation away from Treasuries and short-term notes and towards a high-yield or emerging market credit allocation. The belief is that the portfolio is better able to outperform the market with these new asset allocations.\n",
    "\n",
    "To employ a tactical asset allocation, there are a few different approaches that can be taken. One of the simplest approaches is a discretionary one whereby portfolio managers and chief investment officers overweight or underweight particular assets within the portfolio based on their view of the business cycle. They often move within certain risk boundaries when changing portfolio weightings.\n",
    "\n",
    "Another tactical asset allocation technique used is a momentum, trend following approach. With this, you look to capitalize on an asset class outperforming its peers in the next couple of rebalancing periods.\n",
    "\n",
    "The Black-Litterman model is yet another tactical asset allocation approach. The model came out of Goldman Sachs research published in 1992 by Fischer Black and Robert Litterman. This model allows you to look at the historical returns, risk, and correlation of the assets in your portfolio. The user is only required to state how her assumptions about expected returns differ from the markets and to state her degree of confidence in the alternative assumptions. From this, the Black–Litterman method computes the desired (mean-variance efficient) asset allocation.\n",
    "\n",
    "Made famous more recently by the likes of AQR, risk parity has become a popular tactical asset allocation technique. The risk parity approach asserts that when asset allocations are adjusted (leveraged or deleveraged) to the same risk level, the risk parity portfolio can achieve a higher Sharpe ratio and can be more resistant to market downturns than the traditional portfolio. This is a slightly different approach in that you almost forget about forward forecasting returns.  Instead, you take the approach that you better understand where the risks lie in the portfolio and, as such, strive for an equal risk weighting among asset classes in the portfolio.  Essentially, you are focused on allocation of risk, usually defined as volatility, rather than allocation of capital. For example, if you had a 60% allocation to equities and a 40% allocation to fixed income in your portfolio, risk parity will likely force you to increase your allocation to fixed income as it is likely to have less risk than equities. Risk parity is vulnerable to significant shifts in correlation regimes in practice, such as observed in Q1 2020, which led to the significant underperformance of risk-parity funds in the Covid-19 sell-off."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Examine 60/40 Base Case Scenario"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first get total returns for equity and fixed income ETFs that will serve as our benchmarks in this asset allocation analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fetch daily return data for benchmarks: SPY and AGG ETFs\n",
    "global_eq = 'SPY'\n",
    "global_fi = 'AGG'\n",
    "\n",
    "stock_tickers = [global_eq, global_fi]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the [10 Year Historical Monthly Prices](https://rapidapi.com/alphawave/api/stock-prices2) endpoint from the [AlphaWave Data Stock Prices API](https://rapidapi.com/alphawave/api/stock-prices2/endpoints) to pull in the ten year monthly historical prices so that we can calculate the returns.\n",
    "\n",
    "To call this API with Python, you can choose one of the supported Python code snippets provided in the API console. The following is an example of how to invoke the API with Python Requests. You will need to insert your own <b>x-rapidapi-host</b> and <b>x-rapidapi-key</b> information in the code block below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrieving AlphaWave Data Benchmark Info: 100%|██████████| 2/2 [00:01<00:00,  1.55it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SPY</th>\n",
       "      <th>AGG</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-11-01</th>\n",
       "      <td>-0.004064</td>\n",
       "      <td>-0.003926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-12-01</th>\n",
       "      <td>0.004080</td>\n",
       "      <td>0.010590</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-01</th>\n",
       "      <td>0.053011</td>\n",
       "      <td>0.012866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>0.043406</td>\n",
       "      <td>-0.002611</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>0.027660</td>\n",
       "      <td>-0.005755</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>0.019093</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>0.027764</td>\n",
       "      <td>0.011272</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>0.029760</td>\n",
       "      <td>-0.002085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>-0.008150</td>\n",
       "      <td>0.002385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-16</th>\n",
       "      <td>-0.001585</td>\n",
       "      <td>-0.000283</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>120 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 SPY       AGG\n",
       "2011-11-01 -0.004064 -0.003926\n",
       "2011-12-01  0.004080  0.010590\n",
       "2012-01-01  0.053011  0.012866\n",
       "2012-02-01  0.043406 -0.002611\n",
       "2012-03-01  0.027660 -0.005755\n",
       "...              ...       ...\n",
       "2021-06-01  0.019093  0.008333\n",
       "2021-07-01  0.027764  0.011272\n",
       "2021-08-01  0.029760 -0.002085\n",
       "2021-09-01 -0.008150  0.002385\n",
       "2021-09-16 -0.001585 -0.000283\n",
       "\n",
       "[120 rows x 2 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fetch 10 year monthly return data\n",
    "\n",
    "url = \"https://stock-prices2.p.rapidapi.com/api/v1/resources/stock-prices/10y-1mo-interval\"\n",
    "\n",
    "headers = {\n",
    "    'x-rapidapi-host': \"YOUR_X-RAPIDAPI-HOST_WILL_COPY_DIRECTLY_FROM_RAPIDAPI_PYTHON_CODE_SNIPPETS\",\n",
    "    'x-rapidapi-key': \"YOUR_X-RAPIDAPI-KEY_WILL_COPY_DIRECTLY_FROM_RAPIDAPI_PYTHON_CODE_SNIPPETS\"\n",
    "    }\n",
    "\n",
    "stock_frames = []\n",
    "\n",
    "# for ticker in stock_tickers:\n",
    "for ticker in tqdm(stock_tickers, position=0, leave=True, desc = \"Retrieving AlphaWave Data Benchmark Info\"):\n",
    "    \n",
    "    querystring = {\"ticker\":ticker}\n",
    "    stock_daily_price_response = requests.request(\"GET\", url, headers=headers, params=querystring)\n",
    "\n",
    "    # Create Stock Prices DataFrame\n",
    "    stock_daily_price_df = pd.DataFrame.from_dict(stock_daily_price_response.json())\n",
    "    stock_daily_price_df = stock_daily_price_df.transpose()\n",
    "    stock_daily_price_df = stock_daily_price_df.rename(columns={'Close':ticker})\n",
    "    stock_daily_price_df = stock_daily_price_df[{ticker}]\n",
    "    stock_frames.append(stock_daily_price_df)\n",
    "    \n",
    "yf_combined_stock_price_df = pd.concat(stock_frames, axis=1, sort=True)\n",
    "yf_combined_stock_price_df = yf_combined_stock_price_df.dropna(how='all')\n",
    "yf_combined_stock_price_df = yf_combined_stock_price_df.fillna(\"\")\n",
    "\n",
    "periodic_returns = yf_combined_stock_price_df.pct_change().dropna()\n",
    "periodic_returns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add a 60/40 Equity/Fixed Income Allocation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also build a basic benchmark 60/40 Portfolio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SPY</th>\n",
       "      <th>AGG</th>\n",
       "      <th>60/40 Portfolio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-11-01</th>\n",
       "      <td>-0.004064</td>\n",
       "      <td>-0.003926</td>\n",
       "      <td>-0.004009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-12-01</th>\n",
       "      <td>0.004080</td>\n",
       "      <td>0.010590</td>\n",
       "      <td>0.006684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-01</th>\n",
       "      <td>0.053011</td>\n",
       "      <td>0.012866</td>\n",
       "      <td>0.036953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>0.043406</td>\n",
       "      <td>-0.002611</td>\n",
       "      <td>0.024999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>0.027660</td>\n",
       "      <td>-0.005755</td>\n",
       "      <td>0.014294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>0.019093</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.014789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>0.027764</td>\n",
       "      <td>0.011272</td>\n",
       "      <td>0.021167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>0.029760</td>\n",
       "      <td>-0.002085</td>\n",
       "      <td>0.017022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>-0.008150</td>\n",
       "      <td>0.002385</td>\n",
       "      <td>-0.003936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-16</th>\n",
       "      <td>-0.001585</td>\n",
       "      <td>-0.000283</td>\n",
       "      <td>-0.001064</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>120 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 SPY       AGG  60/40 Portfolio\n",
       "2011-11-01 -0.004064 -0.003926        -0.004009\n",
       "2011-12-01  0.004080  0.010590         0.006684\n",
       "2012-01-01  0.053011  0.012866         0.036953\n",
       "2012-02-01  0.043406 -0.002611         0.024999\n",
       "2012-03-01  0.027660 -0.005755         0.014294\n",
       "...              ...       ...              ...\n",
       "2021-06-01  0.019093  0.008333         0.014789\n",
       "2021-07-01  0.027764  0.011272         0.021167\n",
       "2021-08-01  0.029760 -0.002085         0.017022\n",
       "2021-09-01 -0.008150  0.002385        -0.003936\n",
       "2021-09-16 -0.001585 -0.000283        -0.001064\n",
       "\n",
       "[120 rows x 3 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create benchmark 60/40 Portfolio\n",
    "periodic_returns['60/40 Portfolio'] = sum([periodic_returns[global_eq] * 0.6, periodic_returns[global_fi] * 0.4])\n",
    "periodic_returns[[\"SPY\",\n",
    "                 \"AGG\",\n",
    "                 \"60/40 Portfolio\"]] = periodic_returns[[\"SPY\",\n",
    "                                                        \"AGG\",\n",
    "                                                        \"60/40 Portfolio\"]].apply(pd.to_numeric)\n",
    "periodic_returns = periodic_returns.sort_index()\n",
    "periodic_returns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Cumulative Returns\n",
    "\n",
    "Next, we define `make_single_line_chart` and `make_all_line_charts` functions that will help us plot the benchmark returns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to create a single line chart\n",
    "def make_single_line_chart(column, alt_name=None):\n",
    "    data = cumulative_returns[[column]]\n",
    "    name = column\n",
    "    if alt_name is not None:\n",
    "        name = f'{alt_name} ({column})'\n",
    "\n",
    "    return go.Scatter(x=data.index, y=data[column], name=name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to create a multi line chart\n",
    "def make_all_line_charts(emphasize=None):\n",
    "    alt_names = {'SPY': '100% Equities', 'AGG': '100% Bonds'}\n",
    "    data = []\n",
    "    for column in cumulative_returns:\n",
    "        alt_name = None\n",
    "        if column in alt_names:\n",
    "            alt_name = alt_names[column]\n",
    "        chart = make_single_line_chart(column, alt_name)\n",
    "\n",
    "        if emphasize is not None:\n",
    "            if type(emphasize) != list:\n",
    "                emphasize = [emphasize]\n",
    "            if column not in emphasize:\n",
    "                chart.line.width = 1\n",
    "                chart.mode = 'lines'\n",
    "            else:\n",
    "                chart.line.width = 3\n",
    "                chart.mode = 'lines+markers'\n",
    "        data.append(chart)\n",
    "    return data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ee7a17dfbf3a446683fc5ee967a764d6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'line': {'width': 1},\n",
       "              'mode': 'lines',\n",
       "              'name': '100% …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot the cumulative returns\n",
    "cumulative_rtns = (periodic_returns+1).cumprod() - 1\n",
    "cumulative_returns = cumulative_rtns\n",
    "\n",
    "chart_title = '60/40 Base Case'\n",
    "emphasize = '60/40 Portfolio'\n",
    "\n",
    "data = make_all_line_charts(emphasize)\n",
    "\n",
    "layout = {'template': 'plotly_dark',\n",
    "          'title': chart_title,\n",
    "          'xaxis': {'title': {'text': 'Date'}},\n",
    "          'yaxis': {'title': {'text': 'Cumulative Total Return'},\n",
    "                    'tickformat': '.0%'}}\n",
    "\n",
    "figure = go.Figure(data=data, layout=layout)\n",
    "f2 = go.FigureWidget(figure)\n",
    "f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot Returns Chart - Logarithmic Scale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8aaebdc13b6d458191f2a7f1a0830268",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'line': {'width': 1},\n",
       "              'mode': 'lines',\n",
       "              'name': '100% …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot the returns on a logarithmic scale\n",
    "log_cumulative_rtns = (periodic_returns+1).cumprod() * 100\n",
    "cumulative_returns = log_cumulative_rtns\n",
    "\n",
    "chart_title = '60/40 Base Case' \n",
    "emphasize = '60/40 Portfolio'\n",
    "\n",
    "data = make_all_line_charts(emphasize)\n",
    "\n",
    "layout = ({'template': 'plotly_dark',\n",
    "           'xaxis': {'title': {'text': 'Date'}},\n",
    "           'yaxis': {'title': {'text': 'Cumulative Total Return'},\n",
    "                     'type': 'log', 'tickformat': '$.3s'},\n",
    "            'title': f'{chart_title} - Logarithmic Scale'})\n",
    "\n",
    "figure = go.Figure(data=data, layout=layout)\n",
    "f3 = go.FigureWidget(figure)\n",
    "f3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Define Universe of Equity and Fixed Income ETFs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our optimized portfolios will be created using ETFs selected from this universe based on predictions from the Multi-task Lasso model made at a later step."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Equity ETFs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fetch daily return data for equity ETFs\n",
    "equity_etfs = ['QQQ','VUG','VTV','IWF','IJR','IWM','IJH','VIG','IWD','VO','VGT','VB','XLK','XLF']\n",
    "len(equity_etfs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fixed Income ETFs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fetch daily return data for fixed income ETFs\n",
    "fi_etfs = ['VCIT','LQD','VCSH','BSV','TIP','IGSB','MBB','MUB','EMB','HYG','SHY','TLT']\n",
    "len(fi_etfs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Pull Historical Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's get historical returns for our equity ETF universe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the [10 Year Historical Monthly Prices](https://rapidapi.com/alphawave/api/stock-prices2) endpoint from the [AlphaWave Data Stock Prices API](https://rapidapi.com/alphawave/api/stock-prices2/endpoints) to pull in the ten year monthly historical prices so that we can calculate the returns.\n",
    "\n",
    "To call this API with Python, you can choose one of the supported Python code snippets provided in the API console. The following is an example of how to invoke the API with Python Requests. You will need to insert your own <b>x-rapidapi-host</b> and <b>x-rapidapi-key</b> information in the code block below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrieving AlphaWave Data Equity ETF Info: 100%|██████████| 14/14 [00:09<00:00,  1.55it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>QQQ</th>\n",
       "      <th>VUG</th>\n",
       "      <th>VTV</th>\n",
       "      <th>IWF</th>\n",
       "      <th>IJR</th>\n",
       "      <th>IWM</th>\n",
       "      <th>IJH</th>\n",
       "      <th>VIG</th>\n",
       "      <th>IWD</th>\n",
       "      <th>VO</th>\n",
       "      <th>VGT</th>\n",
       "      <th>VB</th>\n",
       "      <th>XLK</th>\n",
       "      <th>XLF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-11-01</th>\n",
       "      <td>-0.026920</td>\n",
       "      <td>-0.002082</td>\n",
       "      <td>-0.005590</td>\n",
       "      <td>-0.001030</td>\n",
       "      <td>0.006243</td>\n",
       "      <td>-0.003783</td>\n",
       "      <td>-0.002368</td>\n",
       "      <td>0.018713</td>\n",
       "      <td>-0.006976</td>\n",
       "      <td>-0.006113</td>\n",
       "      <td>-0.019928</td>\n",
       "      <td>-0.004382</td>\n",
       "      <td>-0.014994</td>\n",
       "      <td>-0.051111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-12-01</th>\n",
       "      <td>-0.009931</td>\n",
       "      <td>-0.008668</td>\n",
       "      <td>0.017445</td>\n",
       "      <td>-0.007045</td>\n",
       "      <td>0.009012</td>\n",
       "      <td>0.000271</td>\n",
       "      <td>-0.009385</td>\n",
       "      <td>0.003857</td>\n",
       "      <td>0.013572</td>\n",
       "      <td>-0.016810</td>\n",
       "      <td>-0.017451</td>\n",
       "      <td>-0.010931</td>\n",
       "      <td>-0.006635</td>\n",
       "      <td>0.014832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-01</th>\n",
       "      <td>0.088333</td>\n",
       "      <td>0.066746</td>\n",
       "      <td>0.042726</td>\n",
       "      <td>0.064839</td>\n",
       "      <td>0.069688</td>\n",
       "      <td>0.076685</td>\n",
       "      <td>0.070519</td>\n",
       "      <td>0.032287</td>\n",
       "      <td>0.045638</td>\n",
       "      <td>0.081027</td>\n",
       "      <td>0.087697</td>\n",
       "      <td>0.083850</td>\n",
       "      <td>0.066729</td>\n",
       "      <td>0.088466</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>0.064101</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.038851</td>\n",
       "      <td>0.047161</td>\n",
       "      <td>0.021703</td>\n",
       "      <td>0.025690</td>\n",
       "      <td>0.045289</td>\n",
       "      <td>0.027644</td>\n",
       "      <td>0.036413</td>\n",
       "      <td>0.043263</td>\n",
       "      <td>0.069756</td>\n",
       "      <td>0.032747</td>\n",
       "      <td>0.071032</td>\n",
       "      <td>0.049787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>0.048750</td>\n",
       "      <td>0.031841</td>\n",
       "      <td>0.022333</td>\n",
       "      <td>0.029765</td>\n",
       "      <td>0.025948</td>\n",
       "      <td>0.021715</td>\n",
       "      <td>0.016286</td>\n",
       "      <td>0.016140</td>\n",
       "      <td>0.025765</td>\n",
       "      <td>0.018986</td>\n",
       "      <td>0.047001</td>\n",
       "      <td>0.023002</td>\n",
       "      <td>0.041796</td>\n",
       "      <td>0.070461</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>0.061390</td>\n",
       "      <td>0.058808</td>\n",
       "      <td>-0.017792</td>\n",
       "      <td>0.059889</td>\n",
       "      <td>0.001152</td>\n",
       "      <td>0.017162</td>\n",
       "      <td>-0.013907</td>\n",
       "      <td>-0.005845</td>\n",
       "      <td>-0.015578</td>\n",
       "      <td>0.014750</td>\n",
       "      <td>0.071156</td>\n",
       "      <td>0.012130</td>\n",
       "      <td>0.066908</td>\n",
       "      <td>-0.034220</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>0.029803</td>\n",
       "      <td>0.033201</td>\n",
       "      <td>0.015624</td>\n",
       "      <td>0.035026</td>\n",
       "      <td>-0.021932</td>\n",
       "      <td>-0.034826</td>\n",
       "      <td>0.006369</td>\n",
       "      <td>0.036500</td>\n",
       "      <td>0.012184</td>\n",
       "      <td>0.016128</td>\n",
       "      <td>0.035255</td>\n",
       "      <td>-0.012336</td>\n",
       "      <td>0.040777</td>\n",
       "      <td>-0.000691</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>0.042187</td>\n",
       "      <td>0.036730</td>\n",
       "      <td>0.020886</td>\n",
       "      <td>0.036485</td>\n",
       "      <td>0.019137</td>\n",
       "      <td>0.022031</td>\n",
       "      <td>0.019873</td>\n",
       "      <td>0.016717</td>\n",
       "      <td>0.019504</td>\n",
       "      <td>0.030232</td>\n",
       "      <td>0.035103</td>\n",
       "      <td>0.019644</td>\n",
       "      <td>0.035593</td>\n",
       "      <td>0.051479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>-0.005001</td>\n",
       "      <td>-0.003422</td>\n",
       "      <td>-0.015450</td>\n",
       "      <td>-0.005191</td>\n",
       "      <td>-0.022070</td>\n",
       "      <td>-0.017218</td>\n",
       "      <td>-0.017850</td>\n",
       "      <td>-0.013609</td>\n",
       "      <td>-0.012386</td>\n",
       "      <td>-0.009526</td>\n",
       "      <td>-0.006609</td>\n",
       "      <td>-0.010031</td>\n",
       "      <td>-0.006358</td>\n",
       "      <td>-0.014063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-16</th>\n",
       "      <td>0.000688</td>\n",
       "      <td>0.001472</td>\n",
       "      <td>-0.004228</td>\n",
       "      <td>0.001279</td>\n",
       "      <td>-0.003731</td>\n",
       "      <td>0.000721</td>\n",
       "      <td>-0.001666</td>\n",
       "      <td>-0.002372</td>\n",
       "      <td>-0.003601</td>\n",
       "      <td>0.001304</td>\n",
       "      <td>0.001675</td>\n",
       "      <td>-0.000223</td>\n",
       "      <td>0.000570</td>\n",
       "      <td>-0.002905</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>120 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 QQQ       VUG       VTV       IWF       IJR       IWM  \\\n",
       "2011-11-01 -0.026920 -0.002082 -0.005590 -0.001030  0.006243 -0.003783   \n",
       "2011-12-01 -0.009931 -0.008668  0.017445 -0.007045  0.009012  0.000271   \n",
       "2012-01-01  0.088333  0.066746  0.042726  0.064839  0.069688  0.076685   \n",
       "2012-02-01  0.064101  0.047836  0.038851  0.047161  0.021703  0.025690   \n",
       "2012-03-01  0.048750  0.031841  0.022333  0.029765  0.025948  0.021715   \n",
       "...              ...       ...       ...       ...       ...       ...   \n",
       "2021-06-01  0.061390  0.058808 -0.017792  0.059889  0.001152  0.017162   \n",
       "2021-07-01  0.029803  0.033201  0.015624  0.035026 -0.021932 -0.034826   \n",
       "2021-08-01  0.042187  0.036730  0.020886  0.036485  0.019137  0.022031   \n",
       "2021-09-01 -0.005001 -0.003422 -0.015450 -0.005191 -0.022070 -0.017218   \n",
       "2021-09-16  0.000688  0.001472 -0.004228  0.001279 -0.003731  0.000721   \n",
       "\n",
       "                 IJH       VIG       IWD        VO       VGT        VB  \\\n",
       "2011-11-01 -0.002368  0.018713 -0.006976 -0.006113 -0.019928 -0.004382   \n",
       "2011-12-01 -0.009385  0.003857  0.013572 -0.016810 -0.017451 -0.010931   \n",
       "2012-01-01  0.070519  0.032287  0.045638  0.081027  0.087697  0.083850   \n",
       "2012-02-01  0.045289  0.027644  0.036413  0.043263  0.069756  0.032747   \n",
       "2012-03-01  0.016286  0.016140  0.025765  0.018986  0.047001  0.023002   \n",
       "...              ...       ...       ...       ...       ...       ...   \n",
       "2021-06-01 -0.013907 -0.005845 -0.015578  0.014750  0.071156  0.012130   \n",
       "2021-07-01  0.006369  0.036500  0.012184  0.016128  0.035255 -0.012336   \n",
       "2021-08-01  0.019873  0.016717  0.019504  0.030232  0.035103  0.019644   \n",
       "2021-09-01 -0.017850 -0.013609 -0.012386 -0.009526 -0.006609 -0.010031   \n",
       "2021-09-16 -0.001666 -0.002372 -0.003601  0.001304  0.001675 -0.000223   \n",
       "\n",
       "                 XLK       XLF  \n",
       "2011-11-01 -0.014994 -0.051111  \n",
       "2011-12-01 -0.006635  0.014832  \n",
       "2012-01-01  0.066729  0.088466  \n",
       "2012-02-01  0.071032  0.049787  \n",
       "2012-03-01  0.041796  0.070461  \n",
       "...              ...       ...  \n",
       "2021-06-01  0.066908 -0.034220  \n",
       "2021-07-01  0.040777 -0.000691  \n",
       "2021-08-01  0.035593  0.051479  \n",
       "2021-09-01 -0.006358 -0.014063  \n",
       "2021-09-16  0.000570 -0.002905  \n",
       "\n",
       "[120 rows x 14 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fetch 10 year monthly return data\n",
    "\n",
    "url = \"https://stock-prices2.p.rapidapi.com/api/v1/resources/stock-prices/10y-1mo-interval\"\n",
    "\n",
    "headers = {\n",
    "    'x-rapidapi-host': \"YOUR_X-RAPIDAPI-HOST_WILL_COPY_DIRECTLY_FROM_RAPIDAPI_PYTHON_CODE_SNIPPETS\",\n",
    "    'x-rapidapi-key': \"YOUR_X-RAPIDAPI-KEY_WILL_COPY_DIRECTLY_FROM_RAPIDAPI_PYTHON_CODE_SNIPPETS\"\n",
    "    }\n",
    "\n",
    "stock_frames = []\n",
    "\n",
    "# for ticker in equity_etfs:\n",
    "for ticker in tqdm(equity_etfs, position=0, leave=True, desc = \"Retrieving AlphaWave Data Equity ETF Info\"):\n",
    "    \n",
    "    querystring = {\"ticker\":ticker}\n",
    "    stock_daily_price_response = requests.request(\"GET\", url, headers=headers, params=querystring)\n",
    "\n",
    "    # Create Stock Prices DataFrame\n",
    "    stock_daily_price_df = pd.DataFrame.from_dict(stock_daily_price_response.json())\n",
    "    stock_daily_price_df = stock_daily_price_df.transpose()\n",
    "    stock_daily_price_df = stock_daily_price_df.rename(columns={'Close':ticker})\n",
    "    stock_daily_price_df = stock_daily_price_df[{ticker}]\n",
    "    stock_frames.append(stock_daily_price_df)\n",
    "    \n",
    "yf_combined_equity_etfs_df = pd.concat(stock_frames, axis=1, sort=True)\n",
    "yf_combined_equity_etfs_df = yf_combined_equity_etfs_df.dropna(how='all')\n",
    "yf_combined_equity_etfs_df = yf_combined_equity_etfs_df.fillna(\"\")\n",
    "\n",
    "equity_returns = yf_combined_equity_etfs_df.pct_change().dropna()\n",
    "equity_returns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We next pull historical returns for our universe of fixed income ETFs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the [10 Year Historical Monthly Prices](https://rapidapi.com/alphawave/api/stock-prices2) endpoint from the [AlphaWave Data Stock Prices API](https://rapidapi.com/alphawave/api/stock-prices2/endpoints) to pull in the ten year monthly historical prices so that we can calculate the returns.\n",
    "\n",
    "To call this API with Python, you can choose one of the supported Python code snippets provided in the API console. The following is an example of how to invoke the API with Python Requests. You will need to insert your own <b>x-rapidapi-host</b> and <b>x-rapidapi-key</b> information in the code block below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrieving AlphaWave Data FI ETF Info: 100%|██████████| 12/12 [00:12<00:00,  1.04s/it]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>VCIT</th>\n",
       "      <th>LQD</th>\n",
       "      <th>VCSH</th>\n",
       "      <th>BSV</th>\n",
       "      <th>TIP</th>\n",
       "      <th>IGSB</th>\n",
       "      <th>MBB</th>\n",
       "      <th>MUB</th>\n",
       "      <th>EMB</th>\n",
       "      <th>HYG</th>\n",
       "      <th>SHY</th>\n",
       "      <th>TLT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-11-01</th>\n",
       "      <td>-0.016619</td>\n",
       "      <td>-0.031356</td>\n",
       "      <td>-0.008432</td>\n",
       "      <td>-0.001437</td>\n",
       "      <td>0.003665</td>\n",
       "      <td>-0.007220</td>\n",
       "      <td>0.001285</td>\n",
       "      <td>0.006404</td>\n",
       "      <td>-0.012718</td>\n",
       "      <td>-0.023676</td>\n",
       "      <td>0.000462</td>\n",
       "      <td>0.019843</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-12-01</th>\n",
       "      <td>0.024755</td>\n",
       "      <td>0.031350</td>\n",
       "      <td>0.009584</td>\n",
       "      <td>-0.003480</td>\n",
       "      <td>0.002671</td>\n",
       "      <td>0.005391</td>\n",
       "      <td>0.005037</td>\n",
       "      <td>0.022422</td>\n",
       "      <td>0.015241</td>\n",
       "      <td>0.038799</td>\n",
       "      <td>0.000095</td>\n",
       "      <td>0.031250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-01</th>\n",
       "      <td>0.027170</td>\n",
       "      <td>0.028552</td>\n",
       "      <td>0.014017</td>\n",
       "      <td>0.013223</td>\n",
       "      <td>0.024997</td>\n",
       "      <td>0.008511</td>\n",
       "      <td>0.007834</td>\n",
       "      <td>0.040789</td>\n",
       "      <td>0.019364</td>\n",
       "      <td>0.027266</td>\n",
       "      <td>0.001823</td>\n",
       "      <td>0.002379</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>0.006693</td>\n",
       "      <td>0.012909</td>\n",
       "      <td>0.008984</td>\n",
       "      <td>-0.001230</td>\n",
       "      <td>-0.005781</td>\n",
       "      <td>0.002862</td>\n",
       "      <td>-0.001660</td>\n",
       "      <td>-0.013206</td>\n",
       "      <td>0.020721</td>\n",
       "      <td>0.015318</td>\n",
       "      <td>-0.001773</td>\n",
       "      <td>-0.028299</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>-0.007371</td>\n",
       "      <td>-0.014270</td>\n",
       "      <td>-0.000974</td>\n",
       "      <td>-0.001789</td>\n",
       "      <td>-0.008595</td>\n",
       "      <td>0.001204</td>\n",
       "      <td>0.000431</td>\n",
       "      <td>-0.008274</td>\n",
       "      <td>-0.001320</td>\n",
       "      <td>-0.009772</td>\n",
       "      <td>-0.000819</td>\n",
       "      <td>-0.042247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>0.011132</td>\n",
       "      <td>0.022107</td>\n",
       "      <td>-0.000518</td>\n",
       "      <td>-0.001894</td>\n",
       "      <td>0.005958</td>\n",
       "      <td>-0.000198</td>\n",
       "      <td>0.000178</td>\n",
       "      <td>0.003531</td>\n",
       "      <td>0.008522</td>\n",
       "      <td>0.013272</td>\n",
       "      <td>-0.001715</td>\n",
       "      <td>0.044219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>0.012610</td>\n",
       "      <td>0.014260</td>\n",
       "      <td>0.003266</td>\n",
       "      <td>0.003810</td>\n",
       "      <td>0.025543</td>\n",
       "      <td>0.003304</td>\n",
       "      <td>0.005438</td>\n",
       "      <td>0.004592</td>\n",
       "      <td>0.005619</td>\n",
       "      <td>0.001070</td>\n",
       "      <td>0.001648</td>\n",
       "      <td>0.037307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>-0.003526</td>\n",
       "      <td>-0.003343</td>\n",
       "      <td>-0.000807</td>\n",
       "      <td>-0.000629</td>\n",
       "      <td>-0.001081</td>\n",
       "      <td>-0.000380</td>\n",
       "      <td>-0.000024</td>\n",
       "      <td>-0.001887</td>\n",
       "      <td>0.009040</td>\n",
       "      <td>0.006183</td>\n",
       "      <td>-0.000313</td>\n",
       "      <td>-0.003317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>0.002495</td>\n",
       "      <td>0.005862</td>\n",
       "      <td>0.000700</td>\n",
       "      <td>-0.000099</td>\n",
       "      <td>0.003579</td>\n",
       "      <td>0.000327</td>\n",
       "      <td>0.001105</td>\n",
       "      <td>-0.000595</td>\n",
       "      <td>0.001876</td>\n",
       "      <td>0.003802</td>\n",
       "      <td>0.000023</td>\n",
       "      <td>0.012991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-16</th>\n",
       "      <td>0.000158</td>\n",
       "      <td>0.000331</td>\n",
       "      <td>0.000920</td>\n",
       "      <td>0.000316</td>\n",
       "      <td>0.005327</td>\n",
       "      <td>0.000840</td>\n",
       "      <td>-0.000525</td>\n",
       "      <td>0.000811</td>\n",
       "      <td>-0.000287</td>\n",
       "      <td>0.002790</td>\n",
       "      <td>-0.000197</td>\n",
       "      <td>-0.003317</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>120 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                VCIT       LQD      VCSH       BSV       TIP      IGSB  \\\n",
       "2011-11-01 -0.016619 -0.031356 -0.008432 -0.001437  0.003665 -0.007220   \n",
       "2011-12-01  0.024755  0.031350  0.009584 -0.003480  0.002671  0.005391   \n",
       "2012-01-01  0.027170  0.028552  0.014017  0.013223  0.024997  0.008511   \n",
       "2012-02-01  0.006693  0.012909  0.008984 -0.001230 -0.005781  0.002862   \n",
       "2012-03-01 -0.007371 -0.014270 -0.000974 -0.001789 -0.008595  0.001204   \n",
       "...              ...       ...       ...       ...       ...       ...   \n",
       "2021-06-01  0.011132  0.022107 -0.000518 -0.001894  0.005958 -0.000198   \n",
       "2021-07-01  0.012610  0.014260  0.003266  0.003810  0.025543  0.003304   \n",
       "2021-08-01 -0.003526 -0.003343 -0.000807 -0.000629 -0.001081 -0.000380   \n",
       "2021-09-01  0.002495  0.005862  0.000700 -0.000099  0.003579  0.000327   \n",
       "2021-09-16  0.000158  0.000331  0.000920  0.000316  0.005327  0.000840   \n",
       "\n",
       "                 MBB       MUB       EMB       HYG       SHY       TLT  \n",
       "2011-11-01  0.001285  0.006404 -0.012718 -0.023676  0.000462  0.019843  \n",
       "2011-12-01  0.005037  0.022422  0.015241  0.038799  0.000095  0.031250  \n",
       "2012-01-01  0.007834  0.040789  0.019364  0.027266  0.001823  0.002379  \n",
       "2012-02-01 -0.001660 -0.013206  0.020721  0.015318 -0.001773 -0.028299  \n",
       "2012-03-01  0.000431 -0.008274 -0.001320 -0.009772 -0.000819 -0.042247  \n",
       "...              ...       ...       ...       ...       ...       ...  \n",
       "2021-06-01  0.000178  0.003531  0.008522  0.013272 -0.001715  0.044219  \n",
       "2021-07-01  0.005438  0.004592  0.005619  0.001070  0.001648  0.037307  \n",
       "2021-08-01 -0.000024 -0.001887  0.009040  0.006183 -0.000313 -0.003317  \n",
       "2021-09-01  0.001105 -0.000595  0.001876  0.003802  0.000023  0.012991  \n",
       "2021-09-16 -0.000525  0.000811 -0.000287  0.002790 -0.000197 -0.003317  \n",
       "\n",
       "[120 rows x 12 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fetch 10 year monthly return data\n",
    "\n",
    "url = \"https://stock-prices2.p.rapidapi.com/api/v1/resources/stock-prices/10y-1mo-interval\"\n",
    "\n",
    "headers = {\n",
    "    'x-rapidapi-host': \"YOUR_X-RAPIDAPI-HOST_WILL_COPY_DIRECTLY_FROM_RAPIDAPI_PYTHON_CODE_SNIPPETS\",\n",
    "    'x-rapidapi-key': \"YOUR_X-RAPIDAPI-KEY_WILL_COPY_DIRECTLY_FROM_RAPIDAPI_PYTHON_CODE_SNIPPETS\"\n",
    "    }\n",
    "\n",
    "stock_frames = []\n",
    "\n",
    "# for ticker in fi_etfs:\n",
    "for ticker in tqdm(fi_etfs, position=0, leave=True, desc = \"Retrieving AlphaWave Data FI ETF Info\"):\n",
    "    \n",
    "    querystring = {\"ticker\":ticker}\n",
    "    stock_daily_price_response = requests.request(\"GET\", url, headers=headers, params=querystring)\n",
    "\n",
    "    # Create Stock Prices DataFrame\n",
    "    stock_daily_price_df = pd.DataFrame.from_dict(stock_daily_price_response.json())\n",
    "    stock_daily_price_df = stock_daily_price_df.transpose()\n",
    "    stock_daily_price_df = stock_daily_price_df.rename(columns={'Close':ticker})\n",
    "    stock_daily_price_df = stock_daily_price_df[{ticker}]\n",
    "    stock_frames.append(stock_daily_price_df)\n",
    "    \n",
    "yf_combined_fi_etfs_df = pd.concat(stock_frames, axis=1, sort=True)\n",
    "yf_combined_fi_etfs_df = yf_combined_fi_etfs_df.dropna(how='all')\n",
    "yf_combined_fi_etfs_df = yf_combined_fi_etfs_df.fillna(\"\")\n",
    "\n",
    "fi_returns = yf_combined_fi_etfs_df.pct_change().dropna()\n",
    "fi_returns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Construct Time Series Model\n",
    "\n",
    "The goal of this model is to predict the returns for each of these equity and fixed income ETFs and pick the best ETFs to place in the portfolio.  We will not be altering the 60/40 allocation split between equities and fixed income ETFs.  We will be rebalancing monthly to target the 60/40 allocation while also changing the composition of equity and fixed income ETFs in the portfolio.  This will give the model the ability to choose which equity and fixed income ETFs to invest in each month.\n",
    "\n",
    "For the equity strategy and fixed income strategy, the model will be trained on past return data only."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Autoregressive Time Series Forecasting\n",
    "\n",
    "We will be using the Multi-task Lasso model in this analysis.  The model looks at return data for equities and fixed income ETFs.  It will train on five periods (monthly in this example) of returns in order to make a prediction of returns one month ahead.  The Multi-task Lasso model allows us to fit multiple regression problems jointly. This means the model will look at all features at the same time to predict all of the future returns.  For example, if we were analyzing stock A and stock B, the Multi-task Lasso model would not only look at the historical returns of stock A to predict the future returns for stock A.  Rather, the model looks at the historical returns for both Stock A and stock B in order to predict the future returns for stock A.\n",
    "\n",
    "The reason we use Lasso is because it has a penalty term, called [regularization](https://medium.datadriveninvestor.com/introduction-to-machine-learning-an-overview-5ed43a37985d), for the betas that tries to shrink the coefficients down toward zero.   If a coefficient is not important, it drops out of the model completely."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/autoregressive_gif.gif\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "@ignore_warnings(category=ConvergenceWarning)\n",
    "def forecast_returns(return_time_series_data, non_return_data=None, window_size=5, num_test_dates=90):\n",
    "    \"\"\"\n",
    "    Use a given dataset and the MultiTaskLasso object from sklearn to \n",
    "    generate a DataFrame of predicted returns\n",
    "    \n",
    "    Args:\n",
    "    ================================\n",
    "    return_time_series_data (pandas.DataFrame):\n",
    "        pandas DataFrame of an actual return time series for a set of given indices.\n",
    "        Must be in the following format:\n",
    "        \n",
    "         Period     |    \n",
    "         Ending     |    Ticker_1    Ticker_2     ...    Ticker_N\n",
    "       -----------  |   ----------  ----------   -----  ----------\n",
    "       YYYY-MM-DD   |      0.01        0.03       ...     -0.05\n",
    "                    |\n",
    "       YYYY-MM-DD   |     -0.05       -0.01       ...      0.04\n",
    "       \n",
    "       \n",
    "    non_return_data (pandas.DataFrame):\n",
    "        pandas DataFrame of an actual time series of non-return data\n",
    "        for a set of given indices. Must be in the same format, same\n",
    "        ticker order, and have the same periodicity as the return_time_series_data above\n",
    "        \n",
    "        \n",
    "    window_size (int):\n",
    "        Number of periods used to predict the next value.\n",
    "        Example: if window_size = 5, look 5 periods back to predict the next value\n",
    "        Default = 5\n",
    "        \n",
    "    \n",
    "    num_test_dates (int):\n",
    "        Number of periods for which to generate forecasts\n",
    "        Example: 120 = 10 years of monthly predictions, or 30 years of quarterly predicitons\n",
    "        depending on the periodicity of the input data in return_time_series_data and non_return_data\n",
    "        Default = 120\n",
    "        \n",
    "        \n",
    "    Returns:\n",
    "    ================================\n",
    "    pandas.DataFrame\n",
    "        Output is a DataFrame of expected returns in the same format as return_time_series_data\n",
    "    \n",
    "    \"\"\"\n",
    "    \n",
    "    # descriptive variables for later use\n",
    "    names = list(return_time_series_data.columns)\n",
    "    dates = [f'{date.year}-{date.month}-{date.day}' for date in list(pd.to_datetime(return_time_series_data.index))]\n",
    "    \n",
    "    # transform pandas to numpy arrays\n",
    "    X_returns = return_time_series_data.to_numpy()\n",
    "    X_input = X_returns\n",
    "    max_iter = 7500\n",
    "    \n",
    "    # concatenate non_return_data if it exists\n",
    "    if non_return_data is not None:\n",
    "        max_iter = 3000\n",
    "        X_non_rtn = non_return_data.to_numpy()\n",
    "        X_input =  np.concatenate((X_returns, X_non_rtn), axis=1)\n",
    "    \n",
    "    # number of time series (tickers) to model\n",
    "    n_series = X_returns.shape[1]\n",
    "    # number of features at each date; equal to n_series * number of features (return, oas_spread, etc.)\n",
    "    n_features_per_time_point = X_input.shape[1]\n",
    "    \n",
    "    num_features = window_size * n_features_per_time_point\n",
    "    num_training_points = X_returns.shape[0] - window_size\n",
    "    X_train = np.zeros((num_training_points, num_features))\n",
    "    Y_train = X_returns[window_size:,:]\n",
    "    \n",
    "    for i in range(num_training_points-1):\n",
    "        X_train[i,:] = np.matrix.flatten(X_input[i : window_size + i,:])\n",
    "    \n",
    "    # establish empty arrays & variables for use in training each model\n",
    "    mtl_list=[]\n",
    "    alpha= 0.001\n",
    "    Y_pred = np.zeros((num_test_dates, n_series))\n",
    "    delta_Y = np.zeros((num_test_dates, n_series))\n",
    "    dY_percent = np.zeros((num_test_dates, n_series))\n",
    "    mse_pred = np.zeros(num_test_dates)\n",
    "    predict_dates=[]    \n",
    "\n",
    "    # loop through dates & predict returns\n",
    "    for i in range(num_test_dates):\n",
    "        X_i = X_train[:num_training_points - num_test_dates + (i-1)]\n",
    "        Y_i = Y_train[:num_training_points - num_test_dates + (i-1)]\n",
    "        print(\"X shape: \", X_i.shape, \"Y shape: \", Y_i.shape)\n",
    "        print(\"number of points in training data:\", X_i.shape[0] )\n",
    "        mtl = MultiTaskLasso(alpha=alpha, max_iter=max_iter, warm_start=True).fit(X_i, Y_i)\n",
    "        mtl_list.append(mtl)\n",
    "        \n",
    "        print(f\"using X from {dates[num_training_points - num_test_dates + (i-1) + window_size]}\\\n",
    "        to predict {dates[num_training_points - num_test_dates + (i-1) + 1 + window_size]}\")\n",
    "        \n",
    "        predict_dates.append(dates[num_training_points - num_test_dates + (i-1) + window_size])\n",
    "        \n",
    "        X_i_plus_1 = X_train[num_training_points - num_test_dates + (i-1) + 1]\n",
    "        \n",
    "        Y_pred[i,:] = mtl.predict([X_i_plus_1])\n",
    "        Y_act =  Y_train[num_training_points - num_test_dates + (i-1) + 1]\n",
    "        delta_Y[i] = (Y_pred[i,:] - Y_act)\n",
    "        mse_pred[i] = np.sqrt(np.sum((Y_pred[i,:] - Y_act)**2))/len(Y_act)\n",
    "        print(\"mse\", mse_pred[i])\n",
    "    \n",
    "    predictions = pd.DataFrame(Y_pred, index=predict_dates, columns=names)\n",
    "    predictions.index = [pd.Timestamp(i).strftime('%Y-%m-%d') for i in predictions.index]\n",
    "    \n",
    "    return predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X shape:  (24, 70) Y shape:  (24, 14)\n",
      "number of points in training data: 24\n",
      "using X from 2014-4-1        to predict 2014-5-1\n",
      "mse 0.005187248493884684\n",
      "X shape:  (25, 70) Y shape:  (25, 14)\n",
      "number of points in training data: 25\n",
      "using X from 2014-5-1        to predict 2014-6-1\n",
      "mse 0.004776947369818568\n",
      "X shape:  (26, 70) Y shape:  (26, 14)\n",
      "number of points in training data: 26\n",
      "using X from 2014-6-1        to predict 2014-7-1\n",
      "mse 0.0118777887186723\n",
      "X shape:  (27, 70) Y shape:  (27, 14)\n",
      "number of points in training data: 27\n",
      "using X from 2014-7-1        to predict 2014-8-1\n",
      "mse 0.008980829661270187\n",
      "X shape:  (28, 70) Y shape:  (28, 14)\n",
      "number of points in training data: 28\n",
      "using X from 2014-8-1        to predict 2014-9-1\n",
      "mse 0.01185961070309144\n"
     ]
    }
   ],
   "source": [
    "# run the model\n",
    "eq_predictions = forecast_returns(equity_returns)\n",
    "fi_predictions = forecast_returns(fi_returns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>QQQ</th>\n",
       "      <th>VUG</th>\n",
       "      <th>VTV</th>\n",
       "      <th>IWF</th>\n",
       "      <th>IJR</th>\n",
       "      <th>IWM</th>\n",
       "      <th>IJH</th>\n",
       "      <th>VIG</th>\n",
       "      <th>IWD</th>\n",
       "      <th>VO</th>\n",
       "      <th>VGT</th>\n",
       "      <th>VB</th>\n",
       "      <th>XLK</th>\n",
       "      <th>XLF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>0.006806</td>\n",
       "      <td>0.007609</td>\n",
       "      <td>0.009984</td>\n",
       "      <td>0.007390</td>\n",
       "      <td>0.009871</td>\n",
       "      <td>0.009004</td>\n",
       "      <td>0.008548</td>\n",
       "      <td>0.007145</td>\n",
       "      <td>0.010242</td>\n",
       "      <td>0.008589</td>\n",
       "      <td>0.003818</td>\n",
       "      <td>0.010137</td>\n",
       "      <td>0.005048</td>\n",
       "      <td>0.008764</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>0.013471</td>\n",
       "      <td>0.014519</td>\n",
       "      <td>0.017437</td>\n",
       "      <td>0.014345</td>\n",
       "      <td>0.017911</td>\n",
       "      <td>0.017049</td>\n",
       "      <td>0.017197</td>\n",
       "      <td>0.014910</td>\n",
       "      <td>0.017597</td>\n",
       "      <td>0.017284</td>\n",
       "      <td>0.010795</td>\n",
       "      <td>0.018168</td>\n",
       "      <td>0.010093</td>\n",
       "      <td>0.019068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>0.015082</td>\n",
       "      <td>0.015284</td>\n",
       "      <td>0.016809</td>\n",
       "      <td>0.014922</td>\n",
       "      <td>0.016982</td>\n",
       "      <td>0.015944</td>\n",
       "      <td>0.016231</td>\n",
       "      <td>0.014615</td>\n",
       "      <td>0.017228</td>\n",
       "      <td>0.016667</td>\n",
       "      <td>0.012430</td>\n",
       "      <td>0.017171</td>\n",
       "      <td>0.012186</td>\n",
       "      <td>0.017483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>0.011082</td>\n",
       "      <td>0.011083</td>\n",
       "      <td>0.012017</td>\n",
       "      <td>0.010521</td>\n",
       "      <td>0.012050</td>\n",
       "      <td>0.011673</td>\n",
       "      <td>0.011589</td>\n",
       "      <td>0.009522</td>\n",
       "      <td>0.012684</td>\n",
       "      <td>0.011689</td>\n",
       "      <td>0.007936</td>\n",
       "      <td>0.012953</td>\n",
       "      <td>0.008450</td>\n",
       "      <td>0.011290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>0.013345</td>\n",
       "      <td>0.012502</td>\n",
       "      <td>0.014117</td>\n",
       "      <td>0.012036</td>\n",
       "      <td>0.012942</td>\n",
       "      <td>0.011758</td>\n",
       "      <td>0.012786</td>\n",
       "      <td>0.011086</td>\n",
       "      <td>0.014274</td>\n",
       "      <td>0.013500</td>\n",
       "      <td>0.010568</td>\n",
       "      <td>0.013404</td>\n",
       "      <td>0.010882</td>\n",
       "      <td>0.014329</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 QQQ       VUG       VTV       IWF       IJR       IWM  \\\n",
       "2014-04-01  0.006806  0.007609  0.009984  0.007390  0.009871  0.009004   \n",
       "2014-05-01  0.013471  0.014519  0.017437  0.014345  0.017911  0.017049   \n",
       "2014-06-01  0.015082  0.015284  0.016809  0.014922  0.016982  0.015944   \n",
       "2014-07-01  0.011082  0.011083  0.012017  0.010521  0.012050  0.011673   \n",
       "2014-08-01  0.013345  0.012502  0.014117  0.012036  0.012942  0.011758   \n",
       "\n",
       "                 IJH       VIG       IWD        VO       VGT        VB  \\\n",
       "2014-04-01  0.008548  0.007145  0.010242  0.008589  0.003818  0.010137   \n",
       "2014-05-01  0.017197  0.014910  0.017597  0.017284  0.010795  0.018168   \n",
       "2014-06-01  0.016231  0.014615  0.017228  0.016667  0.012430  0.017171   \n",
       "2014-07-01  0.011589  0.009522  0.012684  0.011689  0.007936  0.012953   \n",
       "2014-08-01  0.012786  0.011086  0.014274  0.013500  0.010568  0.013404   \n",
       "\n",
       "                 XLK       XLF  \n",
       "2014-04-01  0.005048  0.008764  \n",
       "2014-05-01  0.010093  0.019068  \n",
       "2014-06-01  0.012186  0.017483  \n",
       "2014-07-01  0.008450  0.011290  \n",
       "2014-08-01  0.010882  0.014329  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view predictions\n",
    "eq_predictions.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>QQQ</th>\n",
       "      <th>VUG</th>\n",
       "      <th>VTV</th>\n",
       "      <th>IWF</th>\n",
       "      <th>IJR</th>\n",
       "      <th>IWM</th>\n",
       "      <th>IJH</th>\n",
       "      <th>VIG</th>\n",
       "      <th>IWD</th>\n",
       "      <th>VO</th>\n",
       "      <th>VGT</th>\n",
       "      <th>VB</th>\n",
       "      <th>XLK</th>\n",
       "      <th>XLF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-11-01</th>\n",
       "      <td>-0.026920</td>\n",
       "      <td>-0.002082</td>\n",
       "      <td>-0.005590</td>\n",
       "      <td>-0.001030</td>\n",
       "      <td>0.006243</td>\n",
       "      <td>-0.003783</td>\n",
       "      <td>-0.002368</td>\n",
       "      <td>0.018713</td>\n",
       "      <td>-0.006976</td>\n",
       "      <td>-0.006113</td>\n",
       "      <td>-0.019928</td>\n",
       "      <td>-0.004382</td>\n",
       "      <td>-0.014994</td>\n",
       "      <td>-0.051111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-12-01</th>\n",
       "      <td>-0.009931</td>\n",
       "      <td>-0.008668</td>\n",
       "      <td>0.017445</td>\n",
       "      <td>-0.007045</td>\n",
       "      <td>0.009012</td>\n",
       "      <td>0.000271</td>\n",
       "      <td>-0.009385</td>\n",
       "      <td>0.003857</td>\n",
       "      <td>0.013572</td>\n",
       "      <td>-0.016810</td>\n",
       "      <td>-0.017451</td>\n",
       "      <td>-0.010931</td>\n",
       "      <td>-0.006635</td>\n",
       "      <td>0.014832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-01-01</th>\n",
       "      <td>0.088333</td>\n",
       "      <td>0.066746</td>\n",
       "      <td>0.042726</td>\n",
       "      <td>0.064839</td>\n",
       "      <td>0.069688</td>\n",
       "      <td>0.076685</td>\n",
       "      <td>0.070519</td>\n",
       "      <td>0.032287</td>\n",
       "      <td>0.045638</td>\n",
       "      <td>0.081027</td>\n",
       "      <td>0.087697</td>\n",
       "      <td>0.083850</td>\n",
       "      <td>0.066729</td>\n",
       "      <td>0.088466</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-02-01</th>\n",
       "      <td>0.064101</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.038851</td>\n",
       "      <td>0.047161</td>\n",
       "      <td>0.021703</td>\n",
       "      <td>0.025690</td>\n",
       "      <td>0.045289</td>\n",
       "      <td>0.027644</td>\n",
       "      <td>0.036413</td>\n",
       "      <td>0.043263</td>\n",
       "      <td>0.069756</td>\n",
       "      <td>0.032747</td>\n",
       "      <td>0.071032</td>\n",
       "      <td>0.049787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-03-01</th>\n",
       "      <td>0.048750</td>\n",
       "      <td>0.031841</td>\n",
       "      <td>0.022333</td>\n",
       "      <td>0.029765</td>\n",
       "      <td>0.025948</td>\n",
       "      <td>0.021715</td>\n",
       "      <td>0.016286</td>\n",
       "      <td>0.016140</td>\n",
       "      <td>0.025765</td>\n",
       "      <td>0.018986</td>\n",
       "      <td>0.047001</td>\n",
       "      <td>0.023002</td>\n",
       "      <td>0.041796</td>\n",
       "      <td>0.070461</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 QQQ       VUG       VTV       IWF       IJR       IWM  \\\n",
       "2011-11-01 -0.026920 -0.002082 -0.005590 -0.001030  0.006243 -0.003783   \n",
       "2011-12-01 -0.009931 -0.008668  0.017445 -0.007045  0.009012  0.000271   \n",
       "2012-01-01  0.088333  0.066746  0.042726  0.064839  0.069688  0.076685   \n",
       "2012-02-01  0.064101  0.047836  0.038851  0.047161  0.021703  0.025690   \n",
       "2012-03-01  0.048750  0.031841  0.022333  0.029765  0.025948  0.021715   \n",
       "\n",
       "                 IJH       VIG       IWD        VO       VGT        VB  \\\n",
       "2011-11-01 -0.002368  0.018713 -0.006976 -0.006113 -0.019928 -0.004382   \n",
       "2011-12-01 -0.009385  0.003857  0.013572 -0.016810 -0.017451 -0.010931   \n",
       "2012-01-01  0.070519  0.032287  0.045638  0.081027  0.087697  0.083850   \n",
       "2012-02-01  0.045289  0.027644  0.036413  0.043263  0.069756  0.032747   \n",
       "2012-03-01  0.016286  0.016140  0.025765  0.018986  0.047001  0.023002   \n",
       "\n",
       "                 XLK       XLF  \n",
       "2011-11-01 -0.014994 -0.051111  \n",
       "2011-12-01 -0.006635  0.014832  \n",
       "2012-01-01  0.066729  0.088466  \n",
       "2012-02-01  0.071032  0.049787  \n",
       "2012-03-01  0.041796  0.070461  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view returns\n",
    "equity_returns.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Avg Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>0.013106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>-0.006043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>-0.012869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>0.031823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>-0.031489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-05-01</th>\n",
       "      <td>0.003358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>-0.004362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>-0.003876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>-0.016816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>0.025861</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>90 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Avg Error\n",
       "2014-04-01   0.013106\n",
       "2014-05-01  -0.006043\n",
       "2014-06-01  -0.012869\n",
       "2014-07-01   0.031823\n",
       "2014-08-01  -0.031489\n",
       "...               ...\n",
       "2021-05-01   0.003358\n",
       "2021-06-01  -0.004362\n",
       "2021-07-01  -0.003876\n",
       "2021-08-01  -0.016816\n",
       "2021-09-01   0.025861\n",
       "\n",
       "[90 rows x 1 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate the average equity prediction error\n",
    "average_equity_return_error = eq_predictions.subtract(equity_returns).mean(axis=1).dropna()\n",
    "equity_avg_error_plot_df = pd.DataFrame({'Avg Error': average_equity_return_error}, index=average_equity_return_error.index)\n",
    "equity_avg_error_plot_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Avg Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>-0.006470</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>-0.011109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>0.002353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>0.005313</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>-0.010482</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-05-01</th>\n",
       "      <td>-0.000485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>-0.005837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>-0.006941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>0.003000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>0.000295</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>90 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Avg Error\n",
       "2014-04-01  -0.006470\n",
       "2014-05-01  -0.011109\n",
       "2014-06-01   0.002353\n",
       "2014-07-01   0.005313\n",
       "2014-08-01  -0.010482\n",
       "...               ...\n",
       "2021-05-01  -0.000485\n",
       "2021-06-01  -0.005837\n",
       "2021-07-01  -0.006941\n",
       "2021-08-01   0.003000\n",
       "2021-09-01   0.000295\n",
       "\n",
       "[90 rows x 1 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate the average fixed income prediction error\n",
    "average_fi_return_error = fi_predictions.subtract(fi_returns).mean(axis=1).dropna()\n",
    "fi_avg_error_plot_df = pd.DataFrame({'Avg Error': average_fi_return_error}, index=average_fi_return_error.index)\n",
    "fi_avg_error_plot_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we check if the model introduces any systematic bias by calculating the average prediction error per month.  This means we take the average difference between the actual and estimated returns for each month.  If the error plotted in the below charts were all negative or all positive, then we would know our model has a systematic bias.  According  to the charts below, it appears we do not have a systematic bias which means we are clear to proceed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0553c05fb8524907a5d1fdb91ed50c33",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'marker': {'color': [green, red, red, green, red, green, red, red,\n",
       "              …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check if the model introduces any systematic bias for equity ETFs\n",
    "def SetColor(y):\n",
    "    if(y < 0):\n",
    "        return \"red\"\n",
    "    elif(y >= 0):\n",
    "        return \"green\"\n",
    "\n",
    "layout = ({'template': 'plotly_dark',\n",
    "           'xaxis': {'title': {'text': 'Date'}},\n",
    "           'yaxis': {'title': {'text': 'Avg Error %'}},\n",
    "            'title': f'Average Equity Prediction Error'})\n",
    "\n",
    "fig = go.Figure(layout=layout)\n",
    "\n",
    "fig.add_trace(go.Bar(\n",
    "    x=equity_avg_error_plot_df.index,\n",
    "    y=equity_avg_error_plot_df.iloc[:,0],\n",
    "    marker=dict(color = list(map(SetColor, equity_avg_error_plot_df.iloc[:,0])))\n",
    "    ))\n",
    "\n",
    "f4 = go.FigureWidget(fig)\n",
    "f4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c6971dcde80645a0b4747e4387e37a60",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'marker': {'color': [red, red, green, green, red, green, red, red,\n",
       "              …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check if the model introduces any systematic bias for fixed income ETFs\n",
    "def SetColor(y):\n",
    "    if(y < 0):\n",
    "        return \"red\"\n",
    "    elif(y >= 0):\n",
    "        return \"green\"\n",
    "\n",
    "layout = ({'template': 'plotly_dark',\n",
    "           'xaxis': {'title': {'text': 'Date'}},\n",
    "           'yaxis': {'title': {'text': 'Avg Error %'}},\n",
    "            'title': f'Average Fixed Income Prediction Error'})\n",
    "\n",
    "fig = go.Figure(layout=layout)\n",
    "\n",
    "fig.add_trace(go.Bar(\n",
    "    x=fi_avg_error_plot_df.index,\n",
    "    y=fi_avg_error_plot_df.iloc[:,0],\n",
    "    marker=dict(color = list(map(SetColor, fi_avg_error_plot_df.iloc[:,0])))\n",
    "    ))\n",
    "\n",
    "f5 = go.FigureWidget(fig)\n",
    "f5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Allocate Strategy Portfolio Based on Model Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create three strategies to measure model performance:\n",
    "1. 60/40 Allocation Strategy\n",
    "1. Equity Only Portfolio\n",
    "1. Fixed Income Only Portfolio\n",
    "\n",
    "Below we define `allocate_portfolio` and `get_historical_portfolio_holdings` functions to create these three strategies.  These functions help us identify the equity and fixed income ETFs with the largest expected returns as calculated by our model for each month, which we then place in our optimized portfolios to see if they beat the benchmarks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def allocate_portfolio(expected_eq_returns, \n",
    "                       expected_fi_returns, \n",
    "                       actual_eq_returns,\n",
    "                       actual_fi_returns,\n",
    "                       for_period_ending,\n",
    "                       total_equity_weight=0.6,\n",
    "                       n_equity_funds=5,\n",
    "                       n_bond_funds=5):\n",
    "    \n",
    "    \"\"\"\n",
    "    Allocate a portfolio by picking the top n_equity_funds & top n_bond_funds for the period\n",
    "    ending on for_period_ending\n",
    "    \n",
    "    \"\"\"\n",
    "    \n",
    "    fi_wgt = 1 - total_equity_weight\n",
    "    eq_fund_wgt = total_equity_weight / n_equity_funds\n",
    "    fi_fund_wgt = fi_wgt / n_bond_funds\n",
    "    for_period_ending = pd.Timestamp(for_period_ending).strftime('%Y-%m-%d')\n",
    "    \n",
    "    eq_returns = pd.DataFrame(expected_eq_returns.loc[for_period_ending])\n",
    "    eq_returns.columns = ['Expected Return']\n",
    "    eq_returns['Type'] = ['Equity'] * len(eq_returns)\n",
    "    eq_returns['Weight'] = [eq_fund_wgt] * len(eq_returns)\n",
    "    eq_returns = eq_returns.sort_values(by='Expected Return', ascending=False).head(n_equity_funds)\n",
    "\n",
    "    fi_returns = pd.DataFrame(expected_fi_returns.loc[for_period_ending])\n",
    "    fi_returns.columns = ['Expected Return']\n",
    "    fi_returns['Type'] = ['Fixed Income'] * len(fi_returns)\n",
    "    fi_returns['Weight'] = [fi_fund_wgt] * len(fi_returns)\n",
    "    fi_returns = fi_returns.sort_values(by='Expected Return', ascending=False).head(n_bond_funds)\n",
    "    \n",
    "    holdings_df = pd.concat([eq_returns, fi_returns], axis=0)\n",
    "    holdings_df.index.name = 'Index'\n",
    "    \n",
    "    actual_returns = []\n",
    "    for i in range(len(holdings_df)):\n",
    "        index_type = holdings_df['Type'].iloc[i]\n",
    "        index_name = holdings_df.index[i]\n",
    "        if index_type == 'Equity':\n",
    "            actual_returns.append(actual_eq_returns[index_name].loc[for_period_ending])\n",
    "        elif index_type == 'Fixed Income':\n",
    "            actual_returns.append(actual_fi_returns[index_name].loc[for_period_ending])\n",
    "    holdings_df['Actual Return'] = actual_returns\n",
    "    \n",
    "    holdings_df.index = pd.MultiIndex.from_tuples([(for_period_ending, i) for i in holdings_df.index], names=['For Period Ending', 'Fund Ticker'])\n",
    "    holdings_df = holdings_df[['Type', 'Weight', 'Expected Return', 'Actual Return']]\n",
    "    \n",
    "    return holdings_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_historical_portfolio_holdings(expected_eq_returns, \n",
    "                                      expected_fi_returns, \n",
    "                                      actual_eq_returns, \n",
    "                                      actual_fi_returns, \n",
    "                                      total_equity_weight):\n",
    "    \"\"\"\n",
    "    Loop over the time frame given in expected_fi_returns \n",
    "    and run allocate_portfolio at each date\n",
    "    \n",
    "    \"\"\"\n",
    "\n",
    "    holdings = []\n",
    "    for date in expected_fi_returns.index:\n",
    "        holdings_at_date = allocate_portfolio(expected_eq_returns=expected_eq_returns, \n",
    "                                              expected_fi_returns=expected_fi_returns, \n",
    "                                              actual_eq_returns=actual_eq_returns,\n",
    "                                              actual_fi_returns=actual_fi_returns,\n",
    "                                              for_period_ending=date, \n",
    "                                              total_equity_weight=total_equity_weight)\n",
    "        holdings.append(holdings_at_date)\n",
    "    return pd.concat(holdings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the functions to create our historical ETF holdings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>Type</th>\n",
       "      <th>Weight</th>\n",
       "      <th>Expected Return</th>\n",
       "      <th>Actual Return</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>For Period Ending</th>\n",
       "      <th>Fund Ticker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"10\" valign=\"top\">2021-08-01</th>\n",
       "      <th>VGT</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.016345</td>\n",
       "      <td>0.035103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>QQQ</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.016230</td>\n",
       "      <td>0.042187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>XLK</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.015991</td>\n",
       "      <td>0.035593</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IWF</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.013964</td>\n",
       "      <td>0.036485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VUG</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.013927</td>\n",
       "      <td>0.036730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TLT</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.005219</td>\n",
       "      <td>-0.003317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HYG</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004405</td>\n",
       "      <td>0.006183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LQD</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004375</td>\n",
       "      <td>-0.003343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EMB</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004142</td>\n",
       "      <td>0.009040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VCIT</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004097</td>\n",
       "      <td>-0.003526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"10\" valign=\"top\">2021-09-01</th>\n",
       "      <th>VGT</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.018909</td>\n",
       "      <td>-0.006609</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>XLK</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.018376</td>\n",
       "      <td>-0.006358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>QQQ</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.018122</td>\n",
       "      <td>-0.005001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IWF</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.016159</td>\n",
       "      <td>-0.005191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VUG</th>\n",
       "      <td>Equity</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.015869</td>\n",
       "      <td>-0.003422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TLT</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.005143</td>\n",
       "      <td>0.012991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HYG</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004421</td>\n",
       "      <td>0.003802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LQD</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004307</td>\n",
       "      <td>0.005862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EMB</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004185</td>\n",
       "      <td>0.001876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VCIT</th>\n",
       "      <td>Fixed Income</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.004029</td>\n",
       "      <td>0.002495</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       Type  Weight  Expected Return  \\\n",
       "For Period Ending Fund Ticker                                          \n",
       "2021-08-01        VGT                Equity    0.12         0.016345   \n",
       "                  QQQ                Equity    0.12         0.016230   \n",
       "                  XLK                Equity    0.12         0.015991   \n",
       "                  IWF                Equity    0.12         0.013964   \n",
       "                  VUG                Equity    0.12         0.013927   \n",
       "                  TLT          Fixed Income    0.08         0.005219   \n",
       "                  HYG          Fixed Income    0.08         0.004405   \n",
       "                  LQD          Fixed Income    0.08         0.004375   \n",
       "                  EMB          Fixed Income    0.08         0.004142   \n",
       "                  VCIT         Fixed Income    0.08         0.004097   \n",
       "2021-09-01        VGT                Equity    0.12         0.018909   \n",
       "                  XLK                Equity    0.12         0.018376   \n",
       "                  QQQ                Equity    0.12         0.018122   \n",
       "                  IWF                Equity    0.12         0.016159   \n",
       "                  VUG                Equity    0.12         0.015869   \n",
       "                  TLT          Fixed Income    0.08         0.005143   \n",
       "                  HYG          Fixed Income    0.08         0.004421   \n",
       "                  LQD          Fixed Income    0.08         0.004307   \n",
       "                  EMB          Fixed Income    0.08         0.004185   \n",
       "                  VCIT         Fixed Income    0.08         0.004029   \n",
       "\n",
       "                               Actual Return  \n",
       "For Period Ending Fund Ticker                 \n",
       "2021-08-01        VGT               0.035103  \n",
       "                  QQQ               0.042187  \n",
       "                  XLK               0.035593  \n",
       "                  IWF               0.036485  \n",
       "                  VUG               0.036730  \n",
       "                  TLT              -0.003317  \n",
       "                  HYG               0.006183  \n",
       "                  LQD              -0.003343  \n",
       "                  EMB               0.009040  \n",
       "                  VCIT             -0.003526  \n",
       "2021-09-01        VGT              -0.006609  \n",
       "                  XLK              -0.006358  \n",
       "                  QQQ              -0.005001  \n",
       "                  IWF              -0.005191  \n",
       "                  VUG              -0.003422  \n",
       "                  TLT               0.012991  \n",
       "                  HYG               0.003802  \n",
       "                  LQD               0.005862  \n",
       "                  EMB               0.001876  \n",
       "                  VCIT              0.002495  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'expected_eq_returns': eq_predictions,\n",
    "          'expected_fi_returns': fi_predictions,\n",
    "          'actual_eq_returns': equity_returns,\n",
    "          'actual_fi_returns': fi_returns}\n",
    "\n",
    "portfolio_holdings = get_historical_portfolio_holdings(**params, total_equity_weight=0.6)\n",
    "bond_only_holdings = get_historical_portfolio_holdings(**params, total_equity_weight=0)\n",
    "equity_only_holdings = get_historical_portfolio_holdings(**params, total_equity_weight=1)\n",
    "\n",
    "portfolio_holdings.tail(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Calculate Benchmark & Strategy Portfolio Returns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we begin, let's review the assumptions we apply to this example:\n",
    "* Assume 0% slippage and trading fees\n",
    "* Risk measures not considered\n",
    "\n",
    "Below we define `get_excess_return`, `get_excess_return_string`, and `get_portfolio_returns` functions that will help us calculate and compare the returns of the optimized portfolios to those of the benchmarks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_excess_return(strategy, benchmark):\n",
    "    investment_horizon_years = (datetime.strptime(periodic_returns.index[-1], '%Y-%m-%d') - datetime.strptime(periodic_returns.index[0], '%Y-%m-%d')).days / 365\n",
    "    annualized_excess_return = (cumulative_returns[strategy][-1] / cumulative_returns[benchmark][-1]) ** (1/investment_horizon_years) - 1\n",
    "    return annualized_excess_return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_excess_return_string(strategy, benchmark):\n",
    "    start_date = periodic_returns.index[0]\n",
    "    end_date = periodic_returns.index[-1]\n",
    "    r = get_excess_return(strategy=strategy, benchmark=benchmark)\n",
    "    qualifier = 'UNDERPERFORMED'\n",
    "    if r > 0:\n",
    "        qualifier = 'OUTPERFORMED'\n",
    "    return f'{strategy} {qualifier} {benchmark} by an annualized rate of {r:.2%} per year for the period between {start_date} and {end_date}.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_portfolio_returns(portfolio_holdings_df, port_name='Optimized Portfolio'):\n",
    "    weighted_returns = portfolio_holdings_df['Actual Return'] * portfolio_holdings_df['Weight']\n",
    "    returns_df = pd.DataFrame(weighted_returns.groupby(level=[0]).sum())\n",
    "    returns_df.columns = [port_name]\n",
    "    return returns_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at a dataframe that includes the returns of all the optimized portfolios and benchmarks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SPY</th>\n",
       "      <th>AGG</th>\n",
       "      <th>60/40 Portfolio</th>\n",
       "      <th>Optimized 60/40</th>\n",
       "      <th>Optimized Bond Strategy</th>\n",
       "      <th>Optimized Equity Strategy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>0.011395</td>\n",
       "      <td>0.008164</td>\n",
       "      <td>0.010103</td>\n",
       "      <td>-0.003208</td>\n",
       "      <td>0.008653</td>\n",
       "      <td>-0.011116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>0.023206</td>\n",
       "      <td>0.011868</td>\n",
       "      <td>0.018671</td>\n",
       "      <td>0.015349</td>\n",
       "      <td>0.021233</td>\n",
       "      <td>0.011427</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>0.015778</td>\n",
       "      <td>-0.000640</td>\n",
       "      <td>0.009211</td>\n",
       "      <td>0.018875</td>\n",
       "      <td>0.001132</td>\n",
       "      <td>0.030704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>-0.008710</td>\n",
       "      <td>-0.002413</td>\n",
       "      <td>-0.006191</td>\n",
       "      <td>-0.020813</td>\n",
       "      <td>-0.005099</td>\n",
       "      <td>-0.031289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>0.039463</td>\n",
       "      <td>0.011451</td>\n",
       "      <td>0.028259</td>\n",
       "      <td>0.035377</td>\n",
       "      <td>0.023587</td>\n",
       "      <td>0.043237</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 SPY       AGG  60/40 Portfolio  Optimized 60/40  \\\n",
       "2014-04-01  0.011395  0.008164         0.010103        -0.003208   \n",
       "2014-05-01  0.023206  0.011868         0.018671         0.015349   \n",
       "2014-06-01  0.015778 -0.000640         0.009211         0.018875   \n",
       "2014-07-01 -0.008710 -0.002413        -0.006191        -0.020813   \n",
       "2014-08-01  0.039463  0.011451         0.028259         0.035377   \n",
       "\n",
       "            Optimized Bond Strategy  Optimized Equity Strategy  \n",
       "2014-04-01                 0.008653                  -0.011116  \n",
       "2014-05-01                 0.021233                   0.011427  \n",
       "2014-06-01                 0.001132                   0.030704  \n",
       "2014-07-01                -0.005099                  -0.031289  \n",
       "2014-08-01                 0.023587                   0.043237  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_60_40_returns = get_portfolio_returns(portfolio_holdings, 'Optimized 60/40')\n",
    "bond_strategy_rtns = get_portfolio_returns(bond_only_holdings, 'Optimized Bond Strategy')\n",
    "equity_strategy_rtns = get_portfolio_returns(equity_only_holdings, 'Optimized Equity Strategy')\n",
    "\n",
    "all_returns = pd.concat([periodic_returns, new_60_40_returns, bond_strategy_rtns, equity_strategy_rtns], axis=1).dropna()\n",
    "all_returns.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fixed Income Only Strategy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if the Optimized Fixed Income Only Strategy beats its benchmark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGG</th>\n",
       "      <th>Optimized Bond Strategy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>100.816357</td>\n",
       "      <td>100.865330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>102.012838</td>\n",
       "      <td>103.007026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>101.947577</td>\n",
       "      <td>103.123643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>101.701587</td>\n",
       "      <td>102.597816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>102.866189</td>\n",
       "      <td>105.017747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-05-01</th>\n",
       "      <td>126.653299</td>\n",
       "      <td>141.879606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>127.708688</td>\n",
       "      <td>144.695957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>129.148261</td>\n",
       "      <td>146.746748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>128.878999</td>\n",
       "      <td>146.894588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>129.186401</td>\n",
       "      <td>147.688593</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>90 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   AGG  Optimized Bond Strategy\n",
       "2014-04-01  100.816357               100.865330\n",
       "2014-05-01  102.012838               103.007026\n",
       "2014-06-01  101.947577               103.123643\n",
       "2014-07-01  101.701587               102.597816\n",
       "2014-08-01  102.866189               105.017747\n",
       "...                ...                      ...\n",
       "2021-05-01  126.653299               141.879606\n",
       "2021-06-01  127.708688               144.695957\n",
       "2021-07-01  129.148261               146.746748\n",
       "2021-08-01  128.878999               146.894588\n",
       "2021-09-01  129.186401               147.688593\n",
       "\n",
       "[90 rows x 2 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate the returns\n",
    "periodic_returns = all_returns[['AGG', 'Optimized Bond Strategy']]\n",
    "log_cumulative_rtns = (periodic_returns+1).cumprod() * 100\n",
    "\n",
    "cumulative_returns = log_cumulative_rtns\n",
    "cumulative_returns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6345c8a2a8864cdbb0757a51198a8baf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'line': {'width': 1},\n",
       "              'mode': 'lines',\n",
       "              'name': '100% …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot the returns on a logarithmic scale\n",
    "chart_title = 'Optimized FI Returns vs Bond Index' \n",
    "emphasize = 'Optimized Bond Strategy'\n",
    "\n",
    "data = make_all_line_charts(emphasize)\n",
    "\n",
    "layout = ({'template': 'plotly_dark',\n",
    "           'xaxis': {'title': {'text': 'Date'}},\n",
    "           'yaxis': {'title': {'text': 'Cumulative Total Return'},\n",
    "                     'type': 'log', 'tickformat': '$.3s'},\n",
    "            'title': f'{chart_title} - Logarithmic Scale'})\n",
    "\n",
    "figure = go.Figure(data=data, layout=layout)\n",
    "\n",
    "f6 = go.FigureWidget(figure)\n",
    "f6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimized Bond Strategy OUTPERFORMED AGG by an annualized rate of 1.82% per year for the period between 2014-04-01 and 2021-09-01.\n"
     ]
    }
   ],
   "source": [
    "print(get_excess_return_string(strategy='Optimized Bond Strategy', benchmark='AGG'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Equity Only Strategy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if the Optimized Equity Only Strategy beats its benchmark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SPY</th>\n",
       "      <th>Optimized Equity Strategy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>101.139544</td>\n",
       "      <td>98.888387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>103.486611</td>\n",
       "      <td>100.018346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>105.119398</td>\n",
       "      <td>103.089350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>104.203772</td>\n",
       "      <td>99.863770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>108.316015</td>\n",
       "      <td>104.181605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-05-01</th>\n",
       "      <td>258.264495</td>\n",
       "      <td>302.465158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>263.195640</td>\n",
       "      <td>321.711088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>270.502993</td>\n",
       "      <td>332.910582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>278.553119</td>\n",
       "      <td>345.301462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>276.283048</td>\n",
       "      <td>343.465794</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>90 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   SPY  Optimized Equity Strategy\n",
       "2014-04-01  101.139544                  98.888387\n",
       "2014-05-01  103.486611                 100.018346\n",
       "2014-06-01  105.119398                 103.089350\n",
       "2014-07-01  104.203772                  99.863770\n",
       "2014-08-01  108.316015                 104.181605\n",
       "...                ...                        ...\n",
       "2021-05-01  258.264495                 302.465158\n",
       "2021-06-01  263.195640                 321.711088\n",
       "2021-07-01  270.502993                 332.910582\n",
       "2021-08-01  278.553119                 345.301462\n",
       "2021-09-01  276.283048                 343.465794\n",
       "\n",
       "[90 rows x 2 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate the returns\n",
    "periodic_returns=all_returns[['SPY', 'Optimized Equity Strategy']]\n",
    "log_cumulative_rtns = (periodic_returns+1).cumprod() * 100\n",
    "\n",
    "cumulative_returns = log_cumulative_rtns\n",
    "cumulative_returns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "60755a7442a64478a6c12cad54cc8c9a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'line': {'width': 1},\n",
       "              'mode': 'lines',\n",
       "              'name': '100% …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot the returns on a logarithmic scale\n",
    "chart_title = 'Optimized Equity Returns vs Equity Index' \n",
    "emphasize = 'Optimized Equity Strategy'\n",
    "\n",
    "data = make_all_line_charts(emphasize)\n",
    "\n",
    "layout = ({'template': 'plotly_dark',\n",
    "           'xaxis': {'title': {'text': 'Date'}},\n",
    "           'yaxis': {'title': {'text': 'Cumulative Total Return'},\n",
    "                     'type': 'log', 'tickformat': '$.3s'},\n",
    "            'title': f'{chart_title} - Logarithmic Scale'})\n",
    "\n",
    "figure = go.Figure(data=data, layout=layout)\n",
    "\n",
    "f7 = go.FigureWidget(figure)\n",
    "f7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimized Equity Strategy OUTPERFORMED SPY by an annualized rate of 2.97% per year for the period between 2014-04-01 and 2021-09-01.\n"
     ]
    }
   ],
   "source": [
    "print(get_excess_return_string(strategy='Optimized Equity Strategy', benchmark='SPY'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 60/40 Allocation Strategy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if the Optimized 60/40 Allocation Strategy beats its benchmark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>60/40 Portfolio</th>\n",
       "      <th>Optimized 60/40</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-04-01</th>\n",
       "      <td>101.010270</td>\n",
       "      <td>99.679164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-05-01</th>\n",
       "      <td>102.896223</td>\n",
       "      <td>101.209165</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-06-01</th>\n",
       "      <td>103.843975</td>\n",
       "      <td>103.119538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-07-01</th>\n",
       "      <td>103.201039</td>\n",
       "      <td>100.973301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-08-01</th>\n",
       "      <td>106.117351</td>\n",
       "      <td>104.545432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-05-01</th>\n",
       "      <td>197.956225</td>\n",
       "      <td>227.764032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-06-01</th>\n",
       "      <td>200.883838</td>\n",
       "      <td>238.268112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-07-01</th>\n",
       "      <td>205.136007</td>\n",
       "      <td>244.595704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-08-01</th>\n",
       "      <td>208.627821</td>\n",
       "      <td>250.156560</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-09-01</th>\n",
       "      <td>207.806740</td>\n",
       "      <td>249.899506</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>90 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            60/40 Portfolio  Optimized 60/40\n",
       "2014-04-01       101.010270        99.679164\n",
       "2014-05-01       102.896223       101.209165\n",
       "2014-06-01       103.843975       103.119538\n",
       "2014-07-01       103.201039       100.973301\n",
       "2014-08-01       106.117351       104.545432\n",
       "...                     ...              ...\n",
       "2021-05-01       197.956225       227.764032\n",
       "2021-06-01       200.883838       238.268112\n",
       "2021-07-01       205.136007       244.595704\n",
       "2021-08-01       208.627821       250.156560\n",
       "2021-09-01       207.806740       249.899506\n",
       "\n",
       "[90 rows x 2 columns]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate the returns\n",
    "periodic_returns = all_returns[['60/40 Portfolio', 'Optimized 60/40']]\n",
    "log_cumulative_rtns = (periodic_returns+1).cumprod() * 100\n",
    "\n",
    "cumulative_returns = log_cumulative_rtns\n",
    "cumulative_returns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "616e8fe582324e1095499fc99df9f680",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FigureWidget({\n",
       "    'data': [{'line': {'width': 1},\n",
       "              'mode': 'lines',\n",
       "              'name': '60/40…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot the returns on a logarithmic scale\n",
    "chart_title = 'Optimized 60/40 Returns vs 60/40 Portfolio Index' \n",
    "emphasize = 'Optimized 60/40'\n",
    "\n",
    "data = make_all_line_charts(emphasize)\n",
    "\n",
    "layout = ({'template': 'plotly_dark',\n",
    "           'xaxis': {'title': {'text': 'Date'}},\n",
    "           'yaxis': {'title': {'text': 'Cumulative Total Return'},\n",
    "                     'type': 'log', 'tickformat': '$.3s'},\n",
    "            'title': f'{chart_title} - Logarithmic Scale'})\n",
    "\n",
    "figure = go.Figure(data=data, layout=layout)\n",
    "\n",
    "f8 = go.FigureWidget(figure)\n",
    "f8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimized 60/40 OUTPERFORMED 60/40 Portfolio by an annualized rate of 2.52% per year for the period between 2014-04-01 and 2021-09-01.\n"
     ]
    }
   ],
   "source": [
    "print(get_excess_return_string(strategy='Optimized 60/40', benchmark='60/40 Portfolio'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a framework, we see that the model's optimized portfolios outperform their benchmarks.  We can make this model more complex by adding more data if we wish, but as a start it appears the model is able to make predictions that result in allocations outperforming their benchmarks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Additional Resources\n",
    "[Machine Learning for Investing](https://hdonnelly6.medium.com/list/machine-learning-for-investing-7f2690bb1826)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*This presentation is for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation for any security; nor does it constitute an offer to provide investment advisory or other services by AlphaWave Data, Inc. (\"AlphaWave Data\"). Nothing contained herein constitutes investment advice or offers any opinion with respect to the suitability of any security, and any views expressed herein should not be taken as advice to buy, sell, or hold any security or as an endorsement of any security or company.  In preparing the information contained herein, AlphaWave Data, Inc. has not taken into account the investment needs, objectives, and financial circumstances of any particular investor. Any views expressed and data illustrated herein were prepared based upon information, believed to be reliable, available to AlphaWave Data, Inc. at the time of publication. AlphaWave Data makes no guarantees as to their accuracy or completeness. All information is subject to change and may quickly become unreliable for various reasons, including changes in market conditions or economic circumstances.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}