-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathcomplex-exponentials.html
2059 lines (1828 loc) · 55 KB
/
complex-exponentials.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>What is e^it</title>
<meta name="description" content="The unit circle!">
<style>
body {
margin: 3em;
line-height: 1.7;
max-width: 45rem;
}
p {
margin-block: 2.25rem;
}
p.footnote {
margin-block: 1em 2.25rem;
margin-inline: 1em;
font-size: small;
}
p:has(+ p.footnote) {
margin-block-end: 0;
}
p:has(+ ol) {
margin-block-end: 1.5rem;
}
ol + p {
margin-block-start: 1.5rem;
}
li {
margin-block: 1rem;
}
hr {
margin-block: 2.75rem;
border: 0;
border-bottom: 1px solid currentColor;
opacity: 0.1;
}
blockquote {
font-size: larger;
}
@media (width <= 600px) {
blockquote {
/* Don't do this always, otherwise I see that sometimes Chrome is adding
vertical scroll bars too. */
overflow-x: scroll;
}
}
@media (width <= 450px) {
blockquote {
margin-inline: 0;
}
}
pre {
overflow-x: scroll;
}
footer {
margin-block: 4rem 5rem;
}
footer p:first-child {
opacity: 0.7;
}
@media (width < 24em) {
body {
margin: 1em;
}
}
a {
text-decoration: none;
}
@media (prefers-color-scheme: dark) {
body {
background-color: hsl(240, 64%, 7%);
color: ghostwhite;
}
a { color: #a2cffe; }
a:visited { color: #ca9bf7; }
}
table {
border-collapse: collapse;
}
th, td {
border: 1px solid rgba(from green r g b / 0.3);
padding: 0.5em 1em;
}
</style>
</head>
<body>
<h1>
What is <math><msup><mi>e</mi><mrow><mi>i</mi><mi>t</mi></mrow></msup></math>
</h1>
<p>
Forget what you have seen - numbers on the number line, complex numbers on the
xy graph.
</p>
<p>
Numbers are things. All that we know about these things are the equations that
relate them to other things (numbers). Numbers don't actually lie on the number
line, that is a convenient visualization but like all analogies it starts to be
harmful when we insist on mistaking the analogy for the territory.
</p>
<p>
All that can be said about a number is the relations that describe it in terms
of other numbers. Everything else is commentary.
</p>
<hr>
<p>
A common pattern in these equations is to find an expression of the sort
</p>
<blockquote>
<math><msup><mi>e</mi><mrow><mi>i</mi><mi>t</mi></mrow></msup></math>
</blockquote>
<p>
This confused me for the longest time. Now I know what it means - it is a number
on the unit circle! These numbers are common since they are a convenient way of
describing a rotation, or a phase shift of a periodic process. Let us arrive at
this conclusion piece by piece.
</p>
<hr>
<p>Firstly, what is <math><msup><mi>e</mi></math>?</p>
<p>Consider the following equation:</p>
<blockquote>
<math>
<mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mn>1</mn>
<mo>+</mo>
<mfrac>
<mrow><mi>x</mi></mrow>
<mrow><mn>1</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>2</mn></msup></mrow>
<mrow><mn>2</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>3</mn></msup></mrow>
<mrow><mn>3</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mo>…</mo>
</math>
</blockquote>
<p>
There are reasons† why a sequence of operations as described by the right hand
side arises commonly in nature's, and in our own, undertakings. So much so that
we have given a name to this function - it is called the exponential function.
</p>
<p class="footnote">
† e.g., it is the fixed point of derivation. That is, <math><mi>exp</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> is the
(only) function whose derivative is equal to itself. That is, the value of
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> at any number <math><mi>x</mi></math> is also
equal to the rate of change of <math><mi>exp</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> at
<math><mi>x</mi></math>.
</p>
<blockquote>
<math>
<mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mn>1</mn>
<mo>+</mo>
<mfrac>
<mrow><mi>x</mi></mrow>
<mrow><mn>1</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>2</mn></msup></mrow>
<mrow><mn>2</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>3</mn></msup></mrow>
<mrow><mn>3</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mo>…</mo>
</math>
</blockquote>
<p>What is the value of this function when we evaluate it at 1?</p>
<pre><code>$ node
> factorial = (n) => n == 0 ? 1 : n * factorial(n - 1)
[Function: factorial]
> exp = (x) => Array(99).fill().reduce((s, _, i) => s + x ** i / factorial(i), 0)
[Function: exp]
> exp(1)
2.7182818284590455
</code></pre>
<p>That looks familiar - it is <math><mi>e</mi></math>.</p>
<p>
So <math><mi>e</mi></math> is the value of the exponential function when it is
evaluated at <math><mn>1</mn></math>. Since <math><mi>e</mi></math>, the
constant, is so ubiquitous, it has somewhat come to overshadow the function it
originates from, and sometimes it just stands in for the entire function itself.
For example, people sometimes write
<math><msup><mi>e</mi><mi>x</mi></msup></math> when what they mean is
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math>.
</p>
<p>
While confusing, this is mathematically correct, because the exponential
function obeys the following identity
</p>
<blockquote>
<math>
<mi>exp</mi>
<mo stretchy="false">(</mo><mi>x</mi><mo>+</mo><mi>y</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>.</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mi>y</mi><mo stretchy="false">)</mo>
</math>
</blockquote>
<p>
This can be seen by plugging in the values into the definition of
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> above and doing some symbolic algebra to convince
ourselves. Assuming we're convinced, then it is easy to see that, for example,
</p>
<blockquote>
<math>
<msup><mi>e</mi><mi>3</mi></msup>
</math>
<br>
<math>
<mo>=</mo>
<msup><mrow><mi>exp</mi><mo stretchy="false">(</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><mi>3</mi></msup>
</math>
<br>
<math>
<mo>=</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mn>1</mn><mo stretchy="false">)</mo><mo>.</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mn>1</mn><mo stretchy="false">)</mo><mo>.</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mn>1</mn><mo stretchy="false">)</mo>
</math>
<br>
<math>
<mo>=</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mn>1</mn><mo>+</mo><mn>1</mn><mo>+</mo><mn>1</mn><mo stretchy="false">)</mo>
</math>
<br>
<math>
<mo>=</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mn>3</mn><mo stretchy="false">)</mo>
</math>
</blockquote>
<p>
So <math><msup><mi>e</mi><mi>x</mi></msup></math> is equivalent to
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math>.
</p>
<p>
This equivalence seems irrelevant, since multiplying <math><mi>e</mi></math> by
itself <math><mi>x</mi></math> times seems a good enough way to arrive at the
same result, and also matches the notation for raising to power that we're
already familiar with, then why remind ourselves that it is a shorthand for an
underlying function evaluation?
</p>
<p>
Because it allows us to see what complex exponentiation means.
</p>
<hr>
<p>
A complex number is a tragedy of nomenclature. They are neither complex, nor
imaginary.
</p>
<p>
This is perhaps an opportune point to repeat the diatribe I started with. Number
are just things defined by their relations, nothing more can be said about them.
We however get misled by labels attached to some of them (e.g. complex number)
or their visual analogies (showing complex numbers on a 2D plane) to think of
them as something they're not.
</p>
<p>
So forget the fact that a complex number has two components, or that it is
different from a real number. Just think of all of them, real or complex, as
just <i>things</i> that we can relate to each other, and all the ways they
relate to other numbers is their very <i>definition</i>, they have no inherent
existence of their own.
</p>
<p>
One such relation is
</p>
<blockquote>
<math>
<mi>x</mi><mo>.</mo><mi>x</mi>
<mo>=</mo>
<mn>-1</mn>
</math>
</blockquote>
<p>
The number <math><mi>x</mi></math> that satisfies this relationship is called
<math><mi>i</mi></math>. It is a complex number; numbers that we're less
familiar with intuitively (but I'm sure if we give it a few hundred years,
children will find complex numbers as intuitive as we find real numbers today),
but it is in no way un<em>real</em>. It is as real as real numbers, and in fact
more so, because complex numbers are <b><i>closed</i></b> while real numbers are
incomplete: what we just found, the square root of minus 1, an eminently
reasonable question, does not have its answer in real numbers, but no matter
what you do to complex numbers, you still get back a complex number.
</p>
<hr>
<p>
Figuratively and literally, complex numbers open another dimension to us. But
when we start putting complex numbers in the relations we had previously used
for with only real numbers, we face new questions. For example, what does it
mean to raise <math><mi>e</mi></math> to the power of a complex number? Or more
specifically, since we've only seen one complex number so far,
<math><mi>i</mi></math>, what does
<math><msup><mi>e</mi><mi>i</mi></msup></math> mean?
</p>
<p>
The repeated multiplication doesn't work. We effortlessly think of
<math><msup><mi>e</mi><mn>3</mn></msup></math> as
<math><mi>e</mi></math> multiplied by itself <math><mn>3</mn></math> times,
<math><mi>e</mi><mo>.</mo><mi>e</mi><mo>.</mo><mi>e</mi></math>, but how do we
multiply <math><mi>e</mi></math> by itself <math><mi>i</mi></math> times?
</p>
<p>
We're asking the wrong question, and getting confused because we're confusing
the notational shorthand for the real thing.
<math><msup><mi>e</mi><mi>x</mi></msup></math> is a shorthand for
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math>, so instead of wondering what is
<math><msup><mi>e</mi><mi>i</mi></msup></math>, what we actually want to know is
what is <math><mi>exp</mi><mo stretchy="false">(</mo><mi>i</mi><mo
stretchy="false">)</mo></math>.
</p>
<p>
This is a simple question with a easy to obtain answer. We can just plug in
<math><mi>i</mi></math> in the definition of <math><mi>exp</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math>
</p>
<blockquote>
<math>
<mi>exp</mi><mo stretchy="false">(</mo><mi>i</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mn>1</mn>
<mo>+</mo>
<mfrac>
<mrow><mi>i</mi></mrow>
<mrow><mn>1</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>i</mi><mn>2</mn></msup></mrow>
<mrow><mn>2</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>i</mi><mn>3</mn></msup></mrow>
<mrow><mn>3</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mo>…</mo>
</math>
</blockquote>
<p>
Hmm. All these symbols Manav, what do they mean?
</p>
<hr>
<p>
Unless you're the sort of person who skips footnotes, you'd remember how we
talked about <math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> being a fixed point of derivation. Let's putter
around with that thought.
</p>
<p>
At the simplest, we can think of a function whose value never changes. The
number of suns in the sky†. No matter at how long after birth (i.e.,
<math><mi>x</mi></math>) I'm trying to evaluate the number of suns in the sky
(i.e., <math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math>), I still get back the same answer
<math><mi>1</mi></math>, which never changes (i.e., it has a rate of change
<math><mn>0</mn></math>, or
<math><mfrac><mrow><mi>d</mi><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>=</mo><mn>0</mn></math>).
</p>
<p class="footnote">
† It is hard to come up with physical examples that are strictly true. One can
come up with exotic examples, say considering <math><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> as the speed
of light in a vaccum at some space-time coordinate <math><mi>x</mi></math>,
something that is generally taken as unchanging, but I have my doubts. I feel
that the more we refine physics, the more we'll find these fundamental constants
as also changing. The only sureshot examples of constant functions that I can
think of are mathematical in nature, say defining <math><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> as the ratio
of the circumference of a circle to its radius.
</p>
<p class="footnote">
As Heraclitus mentioned, perhaps the only constant is change, though I'm not
sure how to formulate that as an equation.
<math><mfrac><mrow><mi>d</mi><mfrac><mrow><mi>d</mi></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>=</mo><mi>a</mi></math>?‡
</p>
<p class="footnote">
‡ There is a mathematical convention (I think?) that I realized after an
embarrasingly long time: constants are represented by <math><mi>a</mi></math>,
<math><mi>b</mi></math>, <math><mi>c</mi></math> etc, while variables are
represented by <math><mi>x</mi></math>, <math><mi>y</mi></math>,
<math><mi>z</mi></math> etc.
</p>
<p>
Next up, we can think of a function whose rate of change is constant - that is,
its rate of change does not depend on the input
<math><mi>x</mi></math>. Everyday my age increases by one day (i.e.,
<math><mfrac><mrow><mi>d</mi><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>=</mo><mi>a</mi></math>,
where <math><mi>a</mi></math> is some constant, in this case
<math><mi>1</mi></math>), independent of my age (i.e., <math><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math>) or how long
after birth I'm trying to evaluate my age (i.e., <math><mi>x</mi></math>).
</p>
<p>
These functions look like lines, which is why they are called
<i>linear</i>. Let us mentally tag this group as functions whose derivative is
a constant.
</p>
<p class="footnote">
While I can't explain what the derivative is in a footnote, what I do want to
re-emphasize is that the derivative is an <i>operator</i> - it takes a function
and returns a function. This is different from regular functions, which take a
number and return a new number. So the derivative of a function
<math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math>,
<math><mfrac><mrow><mi>d</mi></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>)</mo></math>,
is another <i>function</i>, say <math><mi>d′</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math>. This sounds
complicated when put into words, but those of you who have written code would
recognize this sort of a meta-function as quite common in programming, and
they're not all that complicated either.
</p>
<p>
Note that we can think of <math><mn>0</mn></math> as just another constant, so
our previous group of unchanging functions really is part of this current group
of functions that we are considering. Indeed, constant functions are also lines,
just horizontal ones, so both these groups are the same thing that way.
</p>
<p>
If we consider them as distinct groups, we can start building a tower of
changeability. That sounds interesting, let's try that.
</p>
<table>
<th><math><mi>f</mi></math></th>
<th><math><mi>d</mi><mi>f</mi><mo>/</mo><mi>d</mi><mi>x</mi></math></th>
</thead>
<tr>
<td><math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mi>a</mi></math></td>
<td><math><mn>0</mn></math></td>
</tr>
<tr>
<td><math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mi>x</mi></math></td>
<td><math><mi>a</mi></math></td>
</tr>
</table>
<p>
This allows us to find out the rate of change <i>of</i> the rate of
change, i.e., the second derivative. We just go one step up in the tower. So if
we start with a linear function <math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mi>x</mi></math>, like my age, its derivative
would be a constant function <math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mi>a</mi></math>, and we can look up
<i>its</i> derivative one row up in the tower to find out the second derivative
of my age is <math><mn>0</mn></math>.
</p>
<p>
Enough words, I think I'm belabouring the point. Let's move on with our
sequence.
</p>
<p>
Next up, we can think of functions whose rate of change when evaluated for some
number <math><mi>x</mi></math> is in some way related to value of
<math><mi>x</mi></math> itself. These are functions of the form
<math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><msup><mi>x</mi><mn>2</mn></msup></math>. For
example, if I go to a meeting and before the meeting starts, each person present
does a handshake with each other person present there, then the number of
handshakes we'll end up doing, <math><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mi>x</mi><mo>.</mo><mo
stretchy="false">(</mo><mi>x</mi><mo>-</mo><mn>1</mn><mo
stretchy="false">)</mo></mrow><mn>2</mn></mfrac></math>, is (almost) the square
of the number of people <math><mi>x</mi></math>. These numbers grow large very
quickly - for a meeting of 7 people, there will be 49 handshakes - because for
each new person added to the meeting, the number of handshakes that'll increase
is (roughly) equal to the number of people now in the meeting.
</p>
<p>
We can continue this pattern. For example, we can think of functions whose
<i>rate of change</i> when evaluated for some number <math><mi>x</mi></math> is
in some way related to <math><msup><mi>x</mi><mn>2</mn></msup></math>. These
turn out to be functions of the form <math><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><msup><mi>x</mi><mn>3</mn></msup></math>.
Putting these guys in our tower,
</p>
<table>
<th><math><mi>f</mi></math></th>
<th><math><mi>d</mi><mi>f</mi><mo>/</mo><mi>d</mi><mi>x</mi></math></th>
</thead>
<tr>
<td><math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mi>a</mi></math></td>
<td><math><mn>0</mn></math></td>
</tr>
<tr>
<td><math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><mi>x</mi></math></td>
<td><math><mi>a</mi></math></td>
</tr>
<tr>
<td><math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><msup><mi>x</mi><mn>2</mn></msup></math></td>
<td><math><mi>x</mi></math></td>
</tr>
<tr>
<td><math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>=</mo><msup><mi>x</mi><mn>3</mn></msup></math></td>
<td><math><msup><mi>x</mi><mn>2</mn></msup></math></td>
</tr>
<tr>
<td><math><mo>…</mo></math></td>
<td><math><mo>…</mo></math></td>
</tr>
</table>
<p>
Does this tower ever reach a function <math><mi>f</mi></math> whose the rate of
change when evaluated for some value <math><mi>x</mi></math> is in some way
related to <math><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> itself?
</p>
<p>
Yes! Consider
</p>
<blockquote>
<math>
<mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mn>1</mn>
<mo>+</mo>
<mfrac>
<mrow><mi>x</mi></mrow>
<mrow><mn>1</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>2</mn></msup></mrow>
<mrow><mn>2</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>3</mn></msup></mrow>
<mrow><mn>3</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mo>…</mo>
</math>
</blockquote>
<p>
For this function, the rate of change is equal to the function itself. That is,
<math><mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo><mo>)</mo><mo>=</mo><mi>f</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math>.
</p>
<p>
What is this <math><mi>f</mi></math>? Why, it is the exponential function,
<math><mi>exp</mi></math>, that we'd been talking about earlier. And this is
what it means for it to be a <b>fixed point</b> - unlike the other functions
we've seen so far, no matter how many times we take the derivative of
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math>, we get back the exact same function
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> again. Put differently, it forms a (1-)cycle,
like the ouroboros, the snake eating its own tail.
</p>
<p>
So a natural next question to ask would be - is a function that forms a 2-cycle?
That is, if we take its derivative twice, we get back the same function again?
</p>
<p>
Somewhat surprisingly, there is! The pair
<math><mi>sin</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> and <math><mi>cos</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> cycle back
to each other after two steps.
</p>
<blockquote>
<math>
<mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mi>sin</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>)</mo>
<mo>)</mo>
<mo>=</mo>
<mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mi>cos</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>)</mo>
<mo>=</mo>
<mo>-</mo>
<mi>sin</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
</math>
</blockquote>
<p>
There is a slight asymmetry, we get back the negative of where we started with,
and I don't know what to make of it except perhaps that is the reason which
makes this a 2-cycle instead of a 1-cycle.
</p>
<p>
This same 2-cycle works even if we start with the cosine function instead.
</p>
<blockquote>
<math>
<mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mi>cos</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>)</mo>
<mo>)</mo>
<mo>=</mo>
<mfrac><mi>d</mi><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mo>-</mo>
<mi>sin</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>)</mo>
<mo>=</mo>
<mo>-</mo>
<mi>cos</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
</math>
</blockquote>
<p>
So these functions are like yin and yang, each engendering each other again, and
again, ad infinitum. Philosophical glee aside, this behaviour is indeed quite
curious, and one would imagine that there must be some internal similarity
between these two functions and the exponential function, which forms a cycle by
itself, for these two to form a cyclic pair. Or viewed from the other end, it is
natural to wonder if we can somehow combine <math><mi>sin</mi></math> and
<math><mi>cos</mi></math> to get <math><mi>exp</mi></math>?
</p>
<hr>
<p>
Let's look at the formula for the exponential function again
</p>
<blockquote>
<math>
<mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mn>1</mn>
<mo>+</mo>
<mfrac>
<mrow><mi>x</mi></mrow>
<mrow><mn>1</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>2</mn></msup></mrow>
<mrow><mn>2</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow><msup><mi>x</mi><mn>3</mn></msup></mrow>
<mrow><mn>3</mn><mo>!</mo></mrow>
</mfrac>
<mo>+</mo>
<mo>…</mo>
</math>
</blockquote>
<p>
I gave you the formula, and someone gave it to me, but what if I told you we can
derive it from first principles?
</p>
<p>
Alright, here goes. Let us try to come up with a polynomial to approximate
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> without using the definition above. As a
reminder, a polynomial is a function that looks like this:
</p>
<blockquote>
<math>
<mi>polynomial</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<msub><mi>c</mi><mn>0</mn></msub>
<mo>+</mo>
<msub><mi>c</mi><mn>1</mn></msub><mi>x</mi>
<mo>+</mo>
<msub><mi>c</mi><mn>2</mn></msub><msup><mi>x</mi><mn>2</mn></msup>
<mo>+</mo>
<msub><mi>c</mi><mn>3</mn></msub><msup><mi>x</mi><mn>3</mn></msup>
<mo>+</mo>
<mo>…</mo>
</math>
</blockquote>
<p>
That is, it is a sum of successive powers of the input <math><mi>x</mi></math>
to the function. Each power has a constant factor
<math><msub><mi>c</mi><mi>i</mi></msub></math> associated with it, to "scale"
its contribution to the function. This constant can also be zero, in which case
that particular power of <math><mi>x</mi></math> will not be involved at all.
</p>
<p>
The highest power of <math><mi>x</mi></math> with a non-zero constant associated
with it is called the degree of the polynomial. Polynomials of degree 0 are
constants, of degree 1 are lines, and of degree 2 are parabolas.
</p>
<p>
Since polynomials of degree too are quite common, they also have a nickname –
they're called quadratic functions, or quadratic polynomials. Let's start by
approximating <math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> using one of them. It will have the form:
</p>
<blockquote>
<math>
<mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<msub><mi>c</mi><mn>0</mn></msub>
<mo>+</mo>
<msub><mi>c</mi><mn>1</mn></msub><mi>x</mi>
<mo>+</mo>
<msub><mi>c</mi><mn>2</mn></msub><msup><mi>x</mi><mn>2</mn></msup>
</math>
</blockquote>
<p>
We will make use of two facts:
</p>
<ol>
<li>
<math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> is a fixed point of derivation, that is, the
derivative of <math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> is <math><mi>exp</mi><mo
stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> itself.
</li>
<li><math><mi>exp</mi><mo stretchy="false">(</mo><mi>0</mi><mo
stretchy="false">)</mo><mo>=</mo><mn>1</mn></math> (this
fact can either be given to us, or we could guess at it by considering the
interpretation of <math><mi>exp</mi><mo stretchy="false">(</mo><mi>x</mi><mo
stretchy="false">)</mo></math> as
<math><msup><mi>e</mi><mi>x</mi></msup></math>, and then recalling that the
anything raised to the power of 0 is 1)
</li>
</ol>
<p>
Where do we start? Well, since <math><mi>exp</mi><mo
stretchy="false">(</mo><mi>0</mi><mo
stretchy="false">)</mo><mo>=</mo><mn>1</mn></math>, we can start by making our
approximation <math><mi>f</mi><mo stretchy="false">(</mo><mi>0</mi><mo
stretchy="false">)</mo></math> also equal to 1 at 0. This lets us deduce the
value of the constant <math><msub><mi>c</mi><mn>0</mn></msub></math>.
</p>
<blockquote>
<math>
<mtable>
<mtr>
<mtd>
<mi>f</mi><mo stretchy="false">(</mo><mi>0</mi><mo stretchy="false">)</mo>
</mtd>
<mtd>
<mo>=</mo>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msub><mi>c</mi><mn>0</mn></msub>
<mo>+</mo>
<msub><mi>c</mi><mn>1</mn></msub><mn>0</mn>
<mo>+</mo>
<msub><mi>c</mi><mn>2</mn></msub><msup><mn>0</mn><mn>2</mn></msup>
</mtd>
<mtd>
<mo>=</mo>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msub><mi>c</mi><mn>0</mn></msub>
</mtd>
<mtd>
<mo>=</mo>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</math>
</blockquote>
<p>
Alright. Next up, we can try to imagine that <i>around</i> the input 0, if our
approximation should have the same "shape" as <math><mn>exp</mn></math>, then it
should have the same derivative as of <math><mn>exp</mn></math> at 0. Let us
find the derivative of f.
</p>
<blockquote>
<math>
<mfrac><mrow><mi>d</mi></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo>
<mo>)</mo>
<mo>=</mo>
<mfrac><mrow><mi>d</mi></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<msub><mi>c</mi><mn>0</mn></msub>
<mo>+</mo>
<msub><mi>c</mi><mn>1</mn></msub><mi>x</mi>
<mo>+</mo>
<msub><mi>c</mi><mn>2</mn></msub><msup><mi>x</mi><mn>2</mn></msup>
<mo>)</mo>
<mo>=</mo>
<msub><mi>c</mi><mn>1</mn></msub>
<mo>+</mo>
<mn>2</mn><mo>.</mo><msub>2<mi>c</mi><mn>2</mn></msub><mi>x</mi>
</math>
</blockquote>
<p>
And we can easily deduce that the derivative of <math><mn>exp</mn></math> at 0
is 1.
</p>
<blockquote>
<math>
<mfrac><mrow><mi>d</mi></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mi>0</mi><mo stretchy="false">)</mo>
<mo>)</mo>
<mo>=</mo>
<mi>exp</mi><mo stretchy="false">(</mo><mi>0</mi><mo stretchy="false">)</mo>
<mo>=</mo>
<mn>1</mn>
</math>
</blockquote>
<p>
This lets us deduce the value of the constant
<math><msub><mi>c</mi><mn>1</mn></msub></math>, since we're seting its
derivative at 0 to be equal to the derivative of <math><mi>exp</mi></math> at 0
to give our approximation the same slope.
</p>
<blockquote>
<math>
<mtable>
<mtr>
<mtd>
<mfrac><mrow><mi>d</mi></mrow><mrow><mi>d</mi><mi>x</mi></mrow></mfrac><mo>(</mo>
<mi>f</mi><mo stretchy="false">(</mo><mi>0</mi><mo stretchy="false">)</mo>
<mo>)</mo>
</mtd>
<mtd>
<mo>=</mo>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<msub><mi>c</mi><mn>1</mn></msub>
<mo>+</mo>
<mn>2</mn><mo>.</mo><msub>2<mi>c</mi><mn>2</mn></msub><mn>0</mn>
</mtd>
<mtd>
<mo>=</mo>