Chapter_00.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> <h1>Quantitative Risk and Portfolio Management:</h1> </center>\n",
    "<center> <h1>Theory and Practice</h1> </center>\n",
    "\n",
    "&copy; 2022 Kenneth Winston"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction #\n",
    "\n",
    "Humans have far longer planning horizons than other species. We alone have the desire and ability to ponder multiple hypothetical versions of a decades-away future. Dolphins [demonstrate self-awareness, problem-solving, innovation, and teaching skills](https://us.whales.org/whales-dolphins/how-intelligent-are-whales-and-dolphins/) but there aren't any dolphins with [living wills](https://www.mayoclinic.org/healthy-lifestyle/consumer-health/in-depth/living-wills/art-20046303).[$^1$](#zero_footnotes)\n",
    "\n",
    "Some situations admit only a very narrow range of outcomes, so there is little subtlety needed to consider how they might unfold over a planning horizon. Other situations have possible consequences that differ dramatically. A wide spread of potential ways in which the future can unfold is called **`risk`**, which is one of the two central subjects of this book.\n",
    "\n",
    "In particular, we'll explore financial risk. The financial system is a placeholder for human effort; it directs and transports human effort, its inputs, and its outputs across time and space. But no one can know now exactly which efforts will pay off in the future: again, that's risk.\n",
    "\n",
    "There are many mechanisms for reacting to, and planning for, financial risk:\n",
    "- Beneath a veneer of technology, financial markets are as subject to primitive emotional behaviors as any other high-stakes human activity. Hardwired pathways in our brains trigger hot-blooded reactions to short-term risky situations. Panic, for example, can kick in when some of the possible outcomes include great harm. Financial panics occur regularly when market participants lose faith en masse in all or part of the financial system.\n",
    "- But we humans are also capable of cool-headed sorting through multiple possible long-term outcomes. This allows us to form plans &mdash; like living wills &mdash; that can last many years and involve many other individuals.\n",
    "- Quantitative methods even allow the search for risky optimal planning to transcend biology.\n",
    "\n",
    "The word \"transcend\" in the last point is aspirational. Quantitative methods span a spectrum from intuitively approachable closed-form formulas, through complex human-specified algorithms, to fully artificial intelligence. Certainly these methods are <u>not</u> biology: they are implemented on computers. But they show only intermittent ability to <u>transcend</u> biology &mdash; that is, human intuition. No one could write a serious paper titled [\"The Unreasonable Effectiveness of Mathematics in Quantitative Finance.\"](https://doi.org/10.1002%2Fcpa.3160130102)[$^2$](#zero_footnotes)\n",
    "\n",
    "Thus a continuing theme in this book will be the need to mix intuition with quantitative rigor. At the beginning of Chapter 2 we review a litany of cautionary sayings that warn that truth in economics and finance is a moving target. It's a good idea not to fall in love with formulas or algorithms: their constancy is suspect. And it's still better not to fall in love with one's intuitive hunches: they're even more fickle. We'll show the quantitative methods that can work together with intuition to help gain an understanding of the future in a way that we hope does transcend biology alone.\n",
    "\n",
    "Human emotion was assumed away in early economics and finance treatments: people were assumed to act as if they were computers running optimizations. Even the proponents of this view knew it wasn't true, but it was hoped that individual irrationalities canceled out and the overall market was rational, maybe even Gaussian. But the thinking we'll discuss in this book assumes that market moods shift over time. No parameter is fixed forever: ultimately everything is time-varying. We will discuss time-varying estimation methods that have shown good out-of-sample predictive power.\n",
    "\n",
    "The second main subject of this book is the management of **`portfolios`**, which are collections of risky ventures. The mechanisms for dealing with uncertainty change dramatically when multiple risky situations are in play: the range of outcomes can be narrowed by balancing efforts.\n",
    "\n",
    "For example, [Tainter 1990](https://www.cambridge.org/us/academic/subjects/archaeology/archaeological-theory-and-methods/collapse-complex-societies?format=PB&isbn=9780521386739), discussing the Maya &mdash; but more generally all complex societies, noted\n",
    ">In environments characterized by high topographic diversity, where food procurement systems with different productivity cycles exist in close proximity, it is common to alleviate resource fluctuations by developing regional systems of economic symbiosis... a local group can insure itself against lean times by converting temporary surpluses into reciprocal obligations that are called in during times of scarcity.     \n",
    "  \n",
    "The diversification of the food portfolio practiced by the Maya was intended to narrow the range of future outcomes by removing starvation from the list of possibilities.\n",
    "\n",
    "While a portfolio is by definition a collection of risky ventures, it is more than a muddle of individual shiny objects. Modern investors can take chances on ventures without knowing exactly which ones will succeed as long as they are confident that enough ventures will succeed to give them a good overall return. This is more than a financial gimmick: it's a sensible way of organizing cooperative productive behavior. In the absence of knowledge of what exactly will succeed, some need to fail so that others can make progress.\n",
    "\n",
    "The possibility of losing all one's money concentrates the mind almost as wonderfully as the possibility of an interruption in the food supply. That concentration tends to make the required return on a single risky venture high, as less-than-heroic investors demand combat pay to leave their barracks and join the battle. Prices can go up or down as investors get less or more skittish; or they can go up or down as fundamental information improves or degrades an asset's future prospects. Disentangling investor risk preferences from investor assessments of future prospects is difficult. Decades-long academic work in this area shows a glimmer of practical fruit but is still in the early stages in real-world applications.\n",
    "\n",
    "Risk preference is particularly important in a portfolio context, as overall market sentiment tends to shift like human moods. In an pessimistic market, risk premia tend to widen in lockstep, increasing correlations and weakening the diversification benefit of portfolios.\n",
    "\n",
    "In fact the time-varying nature of virtually all financial parameters leads to fat tails: historical patterns work until, violently, they don't. Thus studying tail behavior is an important part of risk and portfolio management. In this book we'll discuss simple Gaussian models, but we'll also discuss distributions and time series approaches that attempt to deal with the one-in-a-billion shocks that actually happen every five years or so in financial markets.\n",
    "\n",
    "The quantitative finance field encompasses two almost wholly disjoint subfields: \"hard quant\" and \"soft quant.\" (Equivalently, \"heavy quant\" and \"light quant.\") The hard/heavy discipline brings to bear techniques from mathematics and physics; practitioners often hold PhDs in quantitative fields and apply their craft to understanding exotic financial instruments. The soft/light discipline uses techniques from social sciences, primarily regressions and machine learning, to find patterns of behavior inductively.\n",
    "\n",
    "Despite the mildly disparaging \"soft/light\" terminology, neither discipline has a monopoly on truth or brilliance. The weakness of hard/heavy is its reliance on axioms that cannot possibly be true; strong results mainly come from strong assumptions. The weakness of soft/light is its reliance on techniques that cannot possibly be proven false as they are designed to fit past data.\n",
    "\n",
    "But over years, a body of elegant work has been done that transcends &mdash; there, I've said it &mdash; the limitations of each subfield. The aim of this book is to present a coherent picture of the progress that has been made on the foundations of quantitative risk and portfolio management."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reader Competencies #\n",
    "\n",
    "In the author's experience as a practitioner and as an educator, a solid majority of people learning about and working in the field of quantitative finance do not have degrees in quantitative finance or economics. Instead, backgrounds in fields like engineering, computer and data science, physics and mathematics are common. This book is aimed at the quantitatively adept student at the late undergraduate or graduate level. No previous knowledge of economics or finance is assumed, but facility with mathematics and statistics is essential.\n",
    "\n",
    "This book gives quantitatively adept students a comprehensive introduction to the body of knowledge required to be a working \"quant,\" i.e. a practitioner of quantitative finance. Both the soft/light and hard/heavy quant subfields described above are covered. Thus the reader should be comfortable with (or able to get comfortable with) both probability theory (heavier) and regression (lighter). We discuss simple stochastic processes, but no previous knowledge is required.\n",
    "\n",
    "Quants can suffer from overspecialization. That's a pitfall for new quants; it's like an actor being typecast. An actor that specializes too soon might be doomed to always be the sympathetic friend who gives wise and funny advice to the star before getting out of the way. As a quant, you may find yourself slotted as the person spending all day thinking about double-knockout inflation-linked catastrophe bonds. You may be called in for a brief stint on the stage when decision makers want a little information about double-knockout inflation-linked catastrophe bonds, but you won't be one of those decision makers if you overspecialize.\n",
    "\n",
    "The frontier of quant knowledge is not a single place: rather is a widely separated group of places. For example, we discuss options in several contexts, but certainly a number of separate full classes in options are required before the student can think of doing professional work with options. We discuss error-limiting portfolio construction techniques like Black-Litterman and resampled efficient frontiers, but further education is required to approach mastery. The aim of this book is to get you to where you can see a number of frontiers from where you stand so that you can decide which one or ones you want to visit.\n",
    "\n",
    "The book was written in Jupyter notebook form, with mixed text and [Python](https://www.python.org/) code segments. Thus previous experience coding in Python is highly recommended, although experience in similar environments like [Mathematica](https://www.wolfram.com/mathematica/) can be ported over without too much trouble.[$^3$](#zero_footnotes)\n",
    "\n",
    "At the time of writing, a popular index puts [Python as the third most popular computer language in the world](https://www.tiobe.com/tiobe-index/).[$^4$](#zero_footnotes) This is puzzling as there is little about the language that would cause a computer scientist to release oxytocin. But it gets the job done and serves as a repository for a vast number of probability, statistics, optimization, machine learning, and other relevant packages. Many students who have taken the classes that formed the basis of this book were more fluent in Python than they were in their native English. If you aren't already fluent, you should want to be in order to use this book.\n",
    "\n",
    "Outside of academia (and even mostly inside of academia), there is no area of quantitative finance that doesn't require diving headfirst into data and algorithms. Data is messy. Some of the code segments contain workarounds where the sources have known problems. If you feel that you should be above such mundane earthly concerns, this book is not for you. \n",
    "\n",
    "Many code segments retrieve data from free sites. I have tried to use government and academic sources where possible. In some cases sites maintained by industry groups like [SIFMA](https://www.sifma.org/)[$^5$](#zero_footnotes) have been used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter Outline #\n",
    "\n",
    "We start **`Chapter 1`** by thinking about the nature of risk. Some people use the word \"risk\" to mean a hazard or a danger, but in a financial context it usually makes more sense to think of risk as an endeavor with a wide range of outcomes, some good and some bad. Portfolio management involves forging ahead when there are chances of bad outcomes, not removing every possibility of bad outcomes.\n",
    "\n",
    "Frank Knight, an early 20th century economist, thought through the various aspects of financial risk in his book *Risk, Uncertainty, and Profit*. His approach still resonates in the early 21st century, and we discuss it. His approach used what we would call today a finite probability space.\n",
    "\n",
    "We then ask readers to record their intuitive responses to some hypothetical risky situations. After reviewing some basic economics, capital markets, and probability theory terminology, we build up the mechanism of Von Neumann-Morgenstern utility theory. In particular the idea of risk preference &mdash; usually but not always risk aversion &mdash; is a key contribution of utility theory. We note that there are many paradoxes and counterexamples to pure utility theory, so it's best used as a way to guide intuition rather than as a natural law.\n",
    "\n",
    "In **`Chapter 2`** we look at metrics for risk. First, we caution the reader with several well-known \"laws of no laws:\" maxims that warn against over-reliance on precise metrics. Mathematical laws that describe the physical world generally don't change the behavior of the physical world: they just describe it. But economic participants do change their behavior because once they know that others are relying on a law, they try to game that law.\n",
    "\n",
    "We proceed nonetheless to describe some precise metrics. Our view is that they are still useful as long as the user is aware of the possibility of behavior shifts. Common risk-adjusted reward measures are also discussed. We show that Artzner, Delbaen, Eber and Heath's definition of coherent risk and its relatives give a good framework for assessing the quality of a prospective risk measure.\n",
    "\n",
    "We then note that risk preferences are embodied in prices. Together with the assumption of no-arbitrage, this brings us to state-price securities and the difference between the risk-neutral probability space and the physical probability space. We conclude Chapter 2 with an examination of stochastic discount factors and the Ross Recovery Theorem, which is an approach to determining market assessments of physical risk from prices. Some trading methods have proposed based on Ross Recovery, but other work shows that the recovered probabilities are unrealistic. Thus practical applications of this heavily trodden academic ground remain elusive.\n",
    "\n",
    "**`Chapter 3`** concerns fixed income modeling. We first note the three different definitions of the time value of money &mdash; nominal, real, and inflation &mdash; and show that any or all of them can be positive or negative. Thus discounting future sure amounts of money to the present can actually involve both increases and decreases.\n",
    "\n",
    "The generic pricing equation for any financial arrangement says that today's price equals the sum of the discounted values of future expected cash flows. Different conventions for discounting are shown.\n",
    "\n",
    "We illustrate the basics of default-free lending and borrowing by working through an example using a US Treasury bond. Pricing, duration, and convexity are shown in general. Closed-form formulas for a bullet bond priced with a flat curve are also worked out. We show that a first-order approximation to price changes using just duration is reasonable in a small neighborhood of the current discount rate, and a second-order approximation bringing in convexity is better. But a large enough move in the discount rate will degrade the quality of even the second-order approximation.\n",
    "\n",
    "Moving away from the assumption of a flat curve, we look at different types of yield curves. We show that level, slope and twist explain the vast majority of yield curve behavior. Key rates and smoothing techniques like Nelson-Siegel also allow important elements of yield curve behavior to be captured in a few variables.\n",
    "\n",
    "After introducing implied forward curves and reviewing stochastic process terminology we look at short-rate models of the term structure of interest rates. We examine some of the features of the original Vasicek model, and then work through a Hull-White model applied to the current US Treasury curve. Finally we show other short-rate models including some that are in current use.\n",
    "\n",
    "**`Chapter 4`** focuses on equity modeling, starting with Markowitz's efficient frontier approach. Formulas for the equality-constrained frontier are developed and an example using currency data is worked out. We also discuss the inequality-constrained frontier, the relationship between the efficient frontier and utility functions, capital market line, and benchmark-relative frontiers. We note that there is a big gap between efficient frontier theory and practical reality so (as with utility functions) this approach is best used as a way to guide intuition rather than as a natural law.\n",
    "\n",
    "After a review of Bayes' Rule, we look at some Bayesian approaches to mitigating some of the implementation problems with efficient frontier theory. Shrinkage estimators like James-Stein and Ledoit-Wolf are covered. We also cover some statistical tests like Hotelling, Levene, and Box's M that can be used to test how well shrinkage estimators are working out of sample.\n",
    "\n",
    "We close Chapter 4 with investigations of two widely used methods &mdash; Jorion's resampling approach and Black-Litterman &mdash; that are intended to alleviate problems of errors in parameter estimation for efficient frontier calculation.\n",
    "\n",
    "**`Chapter 5`** is an overview of convex optimization which is used to form portfolios and to fit model parameters to data. Fortunately the most commonly arising optimization problems in quantitative finance have convex objective functions and convex constraints, so the vastly more efficient procedures that can be brought to bear under convexity can be used. The aim of this chapter is to provide the reader with enough convex optimization expertise that he or she will be able to use some of the many excellent available optimization packages on practical problems, especially portfolio construction problems.\n",
    "\n",
    "After reviewing optimization terminology, we discuss some of the key properties of convex functions that make optimizing them so much easier. We then look at unconstrained optimization methods including finding pseudo-inverses, gradient descent, and Newton's Method.\n",
    "\n",
    "The more realistic approach of constrained optimization is then covered, starting with the familiar Lagrange multipliers approach and expanding to Lagrange duality. That leads to a discussion of the Karush-Kuhn-Tucker conditions and strong duality. We then talk about barrier methods to encode inequality constraints in the objective function, and work an example of a barrier method problem. Finally we redo a portfolio optimization problem from Chapter 4 using a convex optimization package.\n",
    "\n",
    "While Chapter 4 showed the power of portfolio diversification to lower risk without changing the average reward, **`Chapter 6`** shows that there are limits to the benefits of diversification. There are factors that affect all, or many, assets that can't be diversified away just by adding more assets. Determining these factors can help estimate future behaviors of portfolios.\n",
    "\n",
    "At the beginning of Chapter 6, we discuss the Efficient Market Hypothesis (\"EMH\"), the academic theory that all available information is contained in stock market prices. This once-popular theory has been used in practice to justify passive investing in which an investor just participates in a market without seeking to beat it. Current beliefs about market efficiency are more nuanced.\n",
    "\n",
    "The EMH gave rise to simple academic models like the Capital Asset Pricing Model (CAPM) and the Fama-French-Carhart four-factor model. We then develop the Arbitrage Pricing Theory, both in Ross's original exact form and in Huberman's form allowing for idiosyncratic behavior. These models, especially the four-factor model, are widely used in academia but are not used much by practitioners forming investable portfolios.\n",
    "\n",
    "Practitioners do use commercial factor models generally based on company-specific factors such as those originally used by Rosenberg and Marathe in 1976. A number of vendors provide factor model data and software that is heavily used by practitioners forming equity and fixed income portfolios; these data are based on information about assets like the size of the underlying company and its industry group(s). Practitioners also use models based on principal components analysis of covariance matrices, i.e. finding their eigensystems and principal eigenvectors. Some hybrid models combining company-specific information and principal components are also in use.\n",
    "\n",
    "**`Chapter 7`** is an exploration of probability distributions relevant to mathematical finance. It starts with a review of the Central Limit Theorem, which is a powerful force muscling distributions of natural phenomena into becoming Gaussian (normal). But it is only a weak force in mathematical finance, often producing histograms that look roughly bell-shaped but that are actually spectacularly fat-tailed. Various methods of checking normality &mdash; Q-Q and P-P plots and Jarque-Bera &mdash; are shown to fail dramatically when applied to financial data. Theorems don't fail but their assumptions might; we note that it is the assumption of independence that is often violated in finance.\n",
    "\n",
    "If normal distributions don't fit financial data, what does? We look at Student's T distributions which are close to normal but have fatter tails, much like empirical financial data. A better although slightly less closed-form approach is a mixture of normals, which also seems like a plausible first-order model of market regime changes.\n",
    "\n",
    "In the 1960s, Benoit Mandlebrot analyzed cotton prices and concluded that they couldn't possible be normally distributed. He looked for the Central Limit Theorem assumption that failed to apply and concluded that finite variance was the culprit. That's not the mainstream belief now &mdash; as noted, failure of independence is now more widely thought to be the guilty party &mdash; but Mandlebrot's approach still commands a minority following. We explore the family of distributions that Mandlebrot suggested, L&eacute;vy $\\alpha$-stable distributions. The normal distribution is part of the family, but other than distributions that are very close to normal, we find that the stable family is too extreme to represent realistic surviving markets.\n",
    "\n",
    "While the normal distribution is the limiting distribution for averages of suitably well-behaved distributions, the generalized extreme value (\"GEV\") distributions are limits for maxima of suitably well-behaved distributions. We look at GEV distributions and the related GPD (Generalized Pareto Distribution) family that describes tail distributions. These distributions are not as intuitive as normal distributions, but are much better at guiding human intuition through extreme situations.\n",
    "\n",
    "**`Chapter 8`** is about simulation, scenarios, and stress testing. Simulation is used when it is difficult or impossible to develop a closed-form formula for the sample space of a portfolio or instrument's rates of return. For example, the holdings in a portfolio might be priced with complex nonlinear functions of variables like key rates (Chapter 3) or factors (Chapter 6). The interactions between these nonlinear functions could be difficult to predict, so a simulation can be run where multiple values of the input variables are generated and fed through the pricing functions to produce a finite sample space whose characteristics can be computed.\n",
    "\n",
    "Historical simulation is used when there is no need or no desire to impose a model on the distribution of input variables; the historical simulation simply uses the empirical distribution of past values as inputs to the pricing functions. We also show delta-normal simulations where input variables are assumed to follow multivariate normal distributions and portfolios are priced based on first-order dependencies (deltas) on these input variables. While we know from Chapter 7 that multivariate normal distributions are poor descriptors of financial variables, delta-normal is a way to get a quick first-order understanding of outcomes because Euler decomposition of contributions can be used. Higher-order understandings can be obtained with more terms: we show a delta-gamma-theta simulation that adds second-order (gamma) and time dependencies.\n",
    "\n",
    "We discuss Markov Chain Monte Carlo methods that use random sampling in a way that is intended to enrich the predictive validity of the sample. We describe the widely used Metropolis-Hastings Algorithm that is used for efficient sampling of states when there is a known state transition matrix. We further describe the Gibbs Sampler, which among other things is useful for filling in the most likely values of missing observations.\n",
    "\n",
    "Scenarios and stress testing are hybrid approaches that use both qualitative and quantitative insights; these methods can be used to supplement the shortcomings of delta-normal assumptions. We explain that in stress testing, the analyst chooses one or a small number of input variables &mdash; like the average level of interest rates &mdash; and reprices a portfolio after applying an enormous shock to that variable. For example, a $200$bps shock might be applied to interest rates. Scenarios are more elaborate versions of stress testing where an analyst choose a highly unlikely financial story to investigate: for example, a disorderly breakup of the European Union. Reflecting this story in movements in input variables is an art, but then rigorously applying these movements to pricing functions for instruments in a portfolio can give the analyst some intuitive insight.\n",
    "\n",
    "In Chapter 7 we noted that the failure of the Central Limit Theorem to apply to most financial data is due to a failure of the independence assumption. Understanding the nature of dependence, then, might lead to a better understanding of financial distributions. **`Chapter 9`** looks at methods to capture the time-varying nature of volatility, i.e. how volatility in one time period may depend on volatility in previous time periods.\n",
    "\n",
    "The chapter starts by showing that the long-term patterns of volatility in the US stock market can't be explained by sampling variation. There are clearly regimes, like human moods, in the stock market.\n",
    "\n",
    "We then review options terminology because of the tight connection between volatility and options prices. We expect that many readers of this book will have taken a standalone options class; we give a brief background so that all readers can follow our use of options formulas. Options markets are naturally markets for volatility, along with more explicit volatility instruments like variance swaps and VIX futures and options. \n",
    "\n",
    "The shapes of implied volatility skews are clues that the Black-Scholes assumption of lognormality is incorrect. We pursue this idea through the Breeden-Litzenberger risk-neutral recovery of an underlying distribution from options prices. That allows us to complete Chapter 2's description of the Ross Recovery Theorem's real-world density recovery process. Stochastic and local volatility models are shown, along with a brief description of the standard SABR model. We show the replicating portfolio used by the CBOE to compute VIX&reg; in a distribution-agnostic way.\n",
    "\n",
    "We then give a brief review of time series terminology on the way to describing ARCH and GARCH models. We fit a GARCH(1,1) model to US stock market data and show that the de-GARCHed time series has lower kurtosis, moving part of the way to the goal of a \"no surprise\" stationary time series. We conclude the chapter with a review of some variants of ARCH and GARCH, mentioning the Merton model of corporate structure as a brief, and possibly incorrect, justification for asymmetric time series models.\n",
    "\n",
    "In **`Chapter 10`** we focus on relationships between assets. We start with a review of standard Pearson and Spearman correlations. We then discuss conditional versus unconditional correlations. We show that time-varying patterns between correlation pairs of regional stock market indices cannot be due to sampling noise.\n",
    "\n",
    "We discuss the relationship between correlations and the overall economy. We note that highly correlated economies like Saudi Arabia's limit the opportunities for diversification &mdash; a fact the Saudis are most focused on changing. In a diversified economy like the US, lower pairwise correlations are available. However correlations tend to risk in bad times as previously different ventures become more driven by a general slowdown than by their individual properties. We note that in the US, stocks and bonds have been negatively correlated since the late 1990s, but there is no guarantee that this will continue forever.\n",
    "\n",
    "There isn't a general way to get an option-implied correlation between a pair of stocks, but we do show a way to get an average option-implied correlation between a pair of stocks in an index with traded options. This is the methodology used by the CBOE for the ICJ/JCJ/KCJ series which we display. The series clearly shows the effects of the Global Financial Crisis of 2007-2009 in increasing average pairwise correlations.\n",
    "\n",
    "We then turn to copula functions as a more general way to describe relationships than correlations. We give a simplified example of the Li copula approach to CDOs, indicating the problems that ensued when this model was in wide practice.\n",
    "\n",
    "The remainder of the chapter focuses on methods to estimate correlation matrices. The simplest is historical estimation with various types of weighting schemes. We show that constant conditional correlation is not a good model, as even after deGARCHing component time series, correlations are clearly nonstationary. Dynamic conditional correlation models like an integrated model, a mean-reverting model, and an asymmetric model are shown and applied to regional stock market index correlations. We end with a discussion and application of Engel's MacGyver method.\n",
    "\n",
    "**`Chapter 11`** concerns credit modeling. While Chapter 3 analyzed lending arrangement with sure cash flows, Chapter 11 looks at lending where there is a possibility that the borrower will be unable or unwilling to repay the lender. Borrowing and lending is one of the oldest financial activities: we note that in the absence of information networks, the fear of default led lenders to avoid borrowers that they did not know. That generally put a cap on economic growth, but the development of information networks like notaries in France allowed wider interactions and spurred growth.\n",
    "\n",
    "Currently vast amounts of data are available on individuals and businesses. In particular, credit rating agencies collect data on corporate and sovereign bond issuers and produce creditworthiness scores that investors rely on. We examine the categories that ratings agencies use and show how the qualitative assessments that they make lead to excellent ordinal rankings for probabilities of default, but time-varying cardinal measures. We show that investment grade bonds have very low frequencies of default, even during the Great Depression of the 1930s.\n",
    "\n",
    "We note that sovereign (government) borrowers have different criteria and behaviors than corporate borrowers. Some governments, like Argentina, default regularly but are still able to return to the capital markets and borrow again. Overall the default frequencies for governments are not very different by rating than the default frequencies for corporations.\n",
    "\n",
    "We then discuss the credit spread premium puzzle: the fact that compensation for default and the risk of default seems too high. Various explanations have been advanced for this premium, but the puzzle remains. We revisit the Merton model and work examples showing that structural models badly underpredict credit spreads for all but the riskiest bonds. We further explore the commercial KMV model, which addresses some of the practical features missing from the pure Merton model. We describe a power function that balances Type I and Type II error; this can be used to assess the quality of credit rating methods.\n",
    "\n",
    "We show that credit spreads have had reliable negative correlations with risk-free rates for decades (excluding the 1930s), and have positive correlation with volatility (VIX&reg;) as structural models predict.\n",
    "\n",
    "We then upgrade definitions of duration and convexity to cover credit spreads and other variables, defining effective duration, spread duration, and effective convexity. We describe factor models used to form credit portfolios and discuss the DTS (duration times spread) and ASD (adjusted spread duration) approaches.\n",
    "\n",
    "Practical models like Altman's Z-scores, reduced form and hybrid models are covered. We discuss a multi-factor model of correlated defaults that is used to evaluate CDOs. We also show tree models to recover risk-neutral probabilities of default. We finish the chapter with a discussion of credit default swaps.\n",
    "\n",
    "In **`Chapter 12`** we look at five different motivations for hedging, together with associated techniques and relevant instruments.\n",
    "\n",
    "The first motivation is unbundling. We give an example of a French investor whose base currency is euros wanting to invest in the auto company Tesla denominated in dollars. The French investor may want the risks of Tesla, but not of dollar/euro. Hedging with currency forwards is only partly effective if it isn't updated frequently. A quanto option would be more effective but, since quantos are bespoke instruments, possibly not available.\n",
    "\n",
    "The second motivation for hedging is franchise preservation. We give an example of a company importing US wine into France (which we acknowledge is purely hypothetical); the company can go bankrupt on a big enough strengthening of the dollar vs. the euro. Currency hedging could be used, but a structural diversification of the company into a US subsidiary importing French wine (less hypothetical) as well as the original French unit importing US wine is more naturally hedged. However this introduces Siegel's Paradox, which we discuss.\n",
    "\n",
    "We discuss Modigliani-Miller reasoning which in its purest form would argue against franchise preservation hedging. In practice the friction of bankruptcy focuses the mind of most corporations. But we note that franchise perservation hedging involves game theory among competitors. \n",
    "\n",
    "Bank franchise preservation hedging is a matter of necessity. We cite Diamond and Dybvig showing that bank runs can only be prevented by government intervention, like a deposit insurance program. Leaving aside runs, the natural asset-liability mismatch in banks between short-term liabilities and long-term assets can be hedged with interest rate swaps, which we discuss. We note that pension plans are a natural counterparty to banks in interest rate swaps, but that pension plans are subject to longevity risk, which we also discuss.\n",
    "\n",
    "The third type of hedging is for illiquidity. We note that some assets, like commercial real estate, are difficult to transact. Illiquidity hedging might involve transacting in a similar but more liquid instrument to raise or lower exposure. For example, the CMBX credit default swap based on commercial mortgages is easy to buy and sell. An investor wanting to raise or lower general exposure to commercial real estate might use the CMBX index as a liquidity hedge. However this can introduce basis risk, where the illiquid item being hedged could move in the same direction as the hedge, making matters worse.\n",
    "\n",
    "The fourth category is distribution reshaping hedging. The simplest example is adding cash or leverage to a portfolio to decrease or increase exposures linearly, making the outcome distribution less or more volatile. Futures are often used for delta-one (linear) distribution reshaping.\n",
    "\n",
    "Other distribution reshaping techniques involve options, which can be fine-tuned to move around parts of outcome distributions. We review several of the more popular techniques. We then display a table of simple option Greeks, and indicate how option dealers hedge overall portfolios by adjusting overall Greeks. We finish this section with a discussion of fixed income options used to reshape distributions of fixed income portfolios.\n",
    "\n",
    "The fifth reason for hedging is for convexity. We note that in some cases there can be rapid nonlinear movements in markets, and trying to trade through such markets to hedge these movements can be procyclical. We use the example of mortgage convexity hedging. We develop a simplified formula for swaption pricing and show how swaptions can be used to alleviate mortgage convexity exposure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Acknowledgements ##\n",
    "\n",
    "I'd like first to acknowledge my students at Caltech and at NYU. Their curiosity, creativity, and enthusiasm provided endless inspiration. While most were graduate students with strong quantitative backgrounds (in fields like mathematics, physics, computer science, and engineering), some were undergraduates and some came from fields less common to quantitative finance like meteorology and biology.\n",
    "\n",
    "Whatever the background, I was always pleased to see unconventional approaches to material that I though was straightforward being taken by students who weren't burdened by knowing what traditionally had been the right answer. The other students and I benefitted from these unconstrained thinkers.\n",
    "\n",
    "My teaching assistants at Caltech and NYU, mostly former students, were also great to work with. I gradually became aware that many of them were holding sessions where they patiently explained what I really meant, only in clearer language.\n",
    "\n",
    "Many people were kind enough to provide suggestions, assistance, readings, and guidance. I'd especially like to mention (in alphabetic order) Irina Bogacheva, Justin Bois, Kim Border, Peter Bossaerts, Arturo Cifuentes, Jaksa Cvitanic, Peg DiOrio, Charles Fishkin, David Germany, Bob Gingrich, Larry Harris, Thomas Hewett, Mark Kritzman, Tom Kwong, Vadim Martynov, Colm O'Cinneade, Houman Owhadi, Ramesh Pandey, Eugene Park, Daniel Sandberg, Bernd Scherer, Ivor Schucking, Charles Trzcinka, and Xiaotian (Jim) Zhang.\n",
    "\n",
    "Finally in sequence but first in importance I'd like to thank my wife Jackie for her demure and quiet forbearance... sorry, wrong universe. Actually I would like to thank her for firmly, frequently, and forcefully directing me to get on with it and finish this book. But mostly I thank her for the joy she brings to my life."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction Footnotes\n",
    "<a name=\"zero_footnotes\"></a>\n",
    "\n",
    "$^1$ https://us.whales.org/whales-dolphins/how-intelligent-are-whales-and-dolphins/, https://www.mayoclinic.org/healthy-lifestyle/consumer-health/in-depth/living-wills/art-20046303.\n",
    "<br><br>\n",
    "$^2$ [Wigner 1960](https://doi.org/10.1002/cpa.3160130102).\n",
    "<br><br>\n",
    "$^3$ https://www.python.org/; https://www.wolfram.com/mathematica/.\n",
    "<br><br>\n",
    "$^4$ https://www.tiobe.com/tiobe-index/\n",
    "<br><br>\n",
    "$^5$ https://www.sifma.org/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}