Skip to content

Commit 3f406f6

Browse files
behricabehrica
committed
added interactions with OLS
1 parent fbbba26 commit 3f406f6

File tree

4 files changed

+340
-1
lines changed

4 files changed

+340
-1
lines changed

‎README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ The Clojure machine learning library scicloj.ml is documented here:
1414
* [kmeans in Python vs Clojure](https://scicloj.github.io/scicloj.ml-tutorials/polyglot_kmeans.html)
1515
* [Experiment tracking](https://scicloj.github.io/scicloj.ml-tutorials/userguide-experiment-tracking.html)
1616
* [Unsupervised learning](https://scicloj.github.io/scicloj.ml-tutorials/userguide-unsupervised.html)
17+
* [Variable interaction in linear regression](https://scicloj.github.io/scicloj.ml-tutorials/interations_ols.html)
1718

1819

19-
The source files for this documenation using [notespace](https://github.com/scicloj/notespace) are in this repository.
20+
The source files for this documentation using [notespace](https://github.com/scicloj/notespace)
21+
and [Clerk](https://github.com/nextjournal/clerk) are in this repository.

‎data/marketing.csv

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
youtube,facebook,newspaper,sales
2+
276.12,45.35999999999999,83.04,26.52
3+
53.4,47.16,54.12,12.48
4+
20.639999999999997,55.08,83.16,11.16
5+
181.79999999999998,49.559999999999995,70.2,22.2
6+
216.96,12.96,70.08,15.48
7+
10.44,58.67999999999999,90,8.64
8+
69,39.35999999999999,28.2,14.16
9+
144.24,23.52,13.92,15.839999999999998
10+
10.319999999999999,2.52,1.2,5.76
11+
239.76,3.12,25.439999999999998,12.719999999999999
12+
79.32,6.96,29.04,10.319999999999999
13+
257.64,28.799999999999997,4.8,20.88
14+
28.56,42.12,79.08,11.04
15+
117,9.12,8.64,11.639999999999999
16+
244.92,39.48,55.199999999999996,22.8
17+
234.48,57.24,63.48,26.88
18+
81.36,43.92,136.79999999999998,15
19+
337.67999999999995,47.52,66.96,29.279999999999998
20+
83.04,24.599999999999998,21.96,13.56
21+
176.76000000000002,28.679999999999996,22.92,17.52
22+
262.08,33.239999999999995,64.08,21.599999999999998
23+
284.88,6.119999999999999,28.2,15
24+
15.839999999999998,19.08,59.519999999999996,6.72
25+
273.96,20.279999999999998,31.439999999999998,18.599999999999998
26+
74.75999999999999,15.12,21.96,11.639999999999999
27+
315.47999999999996,4.2,23.4,14.399999999999999
28+
171.48,35.16,15.12,18
29+
288.12,20.04,27.479999999999997,19.08
30+
298.56,32.52,27.479999999999997,22.679999999999996
31+
84.71999999999998,19.2,48.959999999999994,12.6
32+
351.47999999999996,33.96,51.84,25.679999999999996
33+
135.48,20.88,46.32,14.28
34+
116.64,1.7999999999999998,36,11.52
35+
318.72,24,0.36,20.88
36+
114.84,1.68,8.88,11.4
37+
348.84,4.919999999999999,10.2,15.36
38+
320.28,52.559999999999995,6,30.479999999999997
39+
89.64,59.279999999999994,54.84,17.639999999999997
40+
51.72,32.04,42.12,12.12
41+
273.59999999999997,45.24,38.4,25.8
42+
243,26.76,37.92,19.92
43+
212.4,40.08,46.440000000000005,20.52
44+
352.32,33.239999999999995,2.16,24.84
45+
248.28,10.08,31.679999999999996,15.48
46+
30.12,30.839999999999996,51.959999999999994,10.2
47+
210.11999999999998,27,37.8,17.88
48+
107.64,11.88,42.84,12.719999999999999
49+
287.88,49.8,22.2,27.84
50+
272.64,18.96,59.879999999999995,17.76
51+
80.28,14.04,44.16,11.639999999999999
52+
239.76,3.7199999999999998,41.52,13.68
53+
120.48,11.52,4.32,12.839999999999998
54+
259.68,50.04,47.52,27.12
55+
219.11999999999998,55.440000000000005,70.44,25.439999999999998
56+
315.23999999999995,34.56,19.08,24.24
57+
238.68,59.279999999999994,72,28.439999999999998
58+
8.76,33.72,49.68,6.6
59+
163.43999999999997,23.04,19.92,15.839999999999998
60+
252.96,59.519999999999996,45.24,28.56
61+
252.83999999999997,35.4,11.16,22.08
62+
64.2,2.4,25.679999999999996,9.719999999999999
63+
313.56,51.24,65.64,29.04
64+
287.16,18.599999999999998,32.76,18.84
65+
123.24,35.52,10.08,16.8
66+
157.32,51.35999999999999,34.68,21.599999999999998
67+
82.8,11.16,1.08,11.16
68+
37.8,29.52,2.64,11.4
69+
167.16,17.4,12.239999999999998,16.08
70+
284.88,33,13.2,22.679999999999996
71+
260.16,52.68,32.64,26.76
72+
238.92,36.72,46.440000000000005,21.96
73+
131.76,17.16,38.04,14.879999999999999
74+
32.16,39.6,23.16,10.56
75+
155.28,6.84,37.56,13.2
76+
256.08,29.52,15.719999999999999,20.4
77+
20.279999999999998,52.440000000000005,107.28,10.44
78+
33,1.92,24.84,8.28
79+
144.6,34.199999999999996,17.04,17.04
80+
6.48,35.879999999999995,11.28,6.359999999999999
81+
139.2,9.24,27.720000000000002,13.2
82+
91.68,32.04,26.76,14.16
83+
287.76,4.919999999999999,44.279999999999994,14.76
84+
90.36,24.36,39,13.56
85+
82.08,53.4,42.72,16.32
86+
256.2,51.6,40.559999999999995,26.04
87+
231.83999999999997,22.08,78.84,18.24
88+
91.55999999999999,33,19.2,14.399999999999999
89+
132.84,48.72,75.84,19.2
90+
105.96,30.599999999999998,88.08,15.48
91+
131.76,57.35999999999999,61.67999999999999,20.04
92+
161.16,5.88,11.16,13.44
93+
34.32,1.7999999999999998,39.6,8.76
94+
261.23999999999995,40.199999999999996,70.8,23.279999999999998
95+
301.08,43.8,86.75999999999999,26.639999999999997
96+
128.88,16.8,13.08,13.799999999999999
97+
195.96,37.92,63.48,20.279999999999998
98+
237.11999999999998,4.2,7.08,14.04
99+
221.88,25.2,26.4,18.599999999999998
100+
347.64,50.76,61.44,30.479999999999997
101+
162.23999999999998,50.04,55.08,20.639999999999997
102+
266.88,5.159999999999999,59.75999999999999,14.04
103+
355.67999999999995,43.559999999999995,121.08,28.56
104+
336.23999999999995,12.12,25.679999999999996,17.76
105+
225.48,20.639999999999997,21.479999999999997,17.639999999999997
106+
285.84,41.16,6.359999999999999,24.84
107+
165.48,55.68,70.8,23.04
108+
30,13.2,35.64,8.64
109+
108.48,0.36,27.84,10.44
110+
15.719999999999999,0.48,30.72,6.359999999999999
111+
306.48,32.279999999999994,6.6,23.76
112+
270.96,9.839999999999998,67.8,16.08
113+
290.03999999999996,45.6,27.84,26.16
114+
210.83999999999997,18.48,2.88,16.919999999999998
115+
251.51999999999998,24.720000000000002,12.839999999999998,19.08
116+
93.84,56.16,41.4,17.52
117+
90.11999999999999,42,63.24,15.12
118+
167.04,17.16,30.72,14.639999999999999
119+
91.68,0.96,17.76,11.28
120+
150.84,44.279999999999994,95.04,19.08
121+
23.279999999999998,19.2,26.76,7.919999999999999
122+
169.56,32.16,55.440000000000005,18.599999999999998
123+
22.56,26.04,60.48,8.4
124+
268.8,2.88,18.72,13.92
125+
147.72,41.52,14.879999999999999,18.24
126+
275.4,38.76,89.04,23.639999999999997
127+
104.64,14.16,31.08,12.719999999999999
128+
9.36,46.68,60.72,7.919999999999999
129+
96.24,0,11.04,10.56
130+
264.36,58.8,3.84,29.639999999999997
131+
71.52,14.399999999999999,51.72,11.639999999999999
132+
0.84,47.52,10.44,1.92
133+
318.23999999999995,3.48,51.6,15.239999999999998
134+
10.08,32.64,2.52,6.84
135+
263.76,40.199999999999996,54.12,23.52
136+
44.279999999999994,46.32,78.71999999999998,12.96
137+
57.959999999999994,56.4,10.2,13.92
138+
30.72,46.8,11.16,11.4
139+
328.44,34.68,71.64,24.96
140+
51.6,31.08,24.599999999999998,11.52
141+
221.88,52.68,2.04,24.84
142+
88.08,20.4,15.48,13.08
143+
232.43999999999997,42.48,90.71999999999998,23.04
144+
264.59999999999997,39.84,45.48,24.12
145+
125.51999999999998,6.84,41.279999999999994,12.48
146+
115.44,17.76,46.68,13.68
147+
168.36,2.28,10.799999999999999,12.360000000000001
148+
288.12,8.76,10.44,15.839999999999998
149+
291.84,58.8,53.16,30.479999999999997
150+
45.6,48.35999999999999,14.28,13.08
151+
53.64,30.96,24.720000000000002,12.12
152+
336.84,16.68,44.4,19.32
153+
145.2,10.08,58.44,13.92
154+
237.11999999999998,27.96,17.04,19.92
155+
205.56,47.64,45.24,22.8
156+
225.36,25.32,11.4,18.72
157+
4.919999999999999,13.92,6.84,3.84
158+
112.68,52.199999999999996,60.599999999999994,18.36
159+
179.76000000000002,1.56,29.16,12.12
160+
14.04,44.279999999999994,54.24,8.76
161+
158.04,22.08,41.52,15.48
162+
207,21.720000000000002,36.839999999999996,17.28
163+
102.84,42.959999999999994,59.16,15.96
164+
226.08,21.720000000000002,30.72,17.88
165+
196.2,44.16,8.88,21.599999999999998
166+
140.64,17.639999999999997,6.48,14.28
167+
281.4,4.08,101.75999999999999,14.28
168+
21.479999999999997,45.12,25.92,9.6
169+
248.16,6.24,23.279999999999998,14.639999999999999
170+
258.48,28.32,69.12,20.52
171+
341.16,12.719999999999999,7.68,18
172+
60,13.92,22.08,10.08
173+
197.4,25.08,56.879999999999995,17.4
174+
23.52,24.12,20.4,9.12
175+
202.08,8.52,15.36,14.04
176+
266.88,4.08,15.719999999999999,13.799999999999999
177+
332.28,58.67999999999999,50.16,32.4
178+
298.08,36.239999999999995,24.36,24.24
179+
204.23999999999998,9.36,42.24,14.04
180+
332.03999999999996,2.76,28.439999999999998,14.16
181+
198.72,12,21.12,15.12
182+
187.92,3.12,9.96,12.6
183+
262.2,6.48,32.879999999999995,14.639999999999999
184+
67.44,6.84,35.64,10.44
185+
345.12,51.6,86.16,31.439999999999998
186+
304.56,25.56,36,21.12
187+
246,54.12,23.52,27.12
188+
167.4,2.52,31.92,12.360000000000001
189+
229.32,34.44,21.84,20.76
190+
343.2,16.68,4.44,19.08
191+
22.439999999999998,14.52,28.08,8.04
192+
47.4,49.32,6.96,12.96
193+
90.6,12.96,7.199999999999999,11.88
194+
20.639999999999997,4.919999999999999,37.92,7.08
195+
200.16,50.4,4.32,23.52
196+
179.64,42.72,7.199999999999999,20.76
197+
45.84,4.44,16.56,9.12
198+
113.04,5.88,9.719999999999999,11.639999999999999
199+
212.4,11.16,7.68,15.36
200+
340.32,50.4,79.44,30.599999999999998
201+
278.52,10.319999999999999,10.44,16.08

‎docs/interactions_ols.html

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
<!DOCTYPE html>
2+
<html lang="en-US">
3+
<head>
4+
<meta charset="UTF-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1">
6+
<link href="https://stackpath.bootstrapcdn.com/bootswatch/4.5.0/sandstone/bootstrap.min.css" rel="stylesheet" type="text/css">
7+
<link href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.6.0/styles/solarized-light.min.css" rel="stylesheet" type="text/css">
8+
<link href="https://cdnjs.cloudflare.com/ajax/libs/ag-grid/24.0.0/styles/ag-grid.min.css" rel="stylesheet" type="text/css">
9+
<link href="https://cdnjs.cloudflare.com/ajax/libs/ag-grid/24.0.0/styles/ag-theme-balham.min.css" rel="stylesheet" type="text/css">
10+
<link href="https://unpkg.com/[email protected]/dist/leaflet.css" rel="stylesheet" type="text/css">
11+
12+
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css" integrity="sha384-AfEj0r4/OFrOo5t7NnNe46zW/tFgW6x/bCJG8FqQCEo3+Aro6EYUG4+cU+KJWu/X" crossorigin="anonymous">
13+
<!-- The loading of KaTeX is deferred to speed up page rendering -->
14+
<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.js" integrity="sha384-g7c+Jr9ZivxKLnZTDUhnkOnsh30B4H0rpLUpJ4jAIKs4fnJI+sEnkvrMWph2EDg4" crossorigin="anonymous"></script>
15+
</head>
16+
<body>
17+
<p id="loading">Loading ...</p>
18+
<div id="app"></div>
19+
</body>
20+
<script id="state" type="text">"{:options {:reverse-notes? false, :header? false, :notes-in-cards? false, :initially-collapse? false, :auto-scroll? false, :port 1903, :custom-header [:div {:style {:font-style \"italic\", :font-family \"\\\"Lucida Console\\\", Courier, monospace\"}} \"(notespace)\" [:p \"Mon Jun 20 14:23:40 CEST 2022\"] nil [:hr]], :custom-footer [:div [:hr] [:hr]]}, :ids [\"132\" \"159\" \"90\" \"134\" \"135\" \"93\" \"160\" \"95\" \"96\" \"97\" \"161\" \"162\" \"163\" \"164\" \"165\" \"166\" \"167\" \"168\" \"169\" \"170\" \"171\" \"172\" \"148\" \"173\" \"174\" \"175\" \"176\" \"177\" \"178\" \"155\" \"179\" \"157\"], :id->content {\"93\" [:div [:p] nil nil [:p/markdown \"First we load the data:\"]], \"157\" [:div [:p] nil nil [:p/markdown \"RMSE and R2 of the intercation model are sligtly better.\\nThese results suggest that the model with the interaction term is better than the model that contains only main effects.\\nSo, for this specific data, we should go for the model with the interaction model.\\n\"]], \"175\" [:div [:p] [:div [:p/code {:code \"^kind/hiccup\\n(text->hiccup\\n (str\\n (-> evaluations flatten first :fit-ctx :model ml/thaw-model str)))\", :bg-class \"bg-light\"}]] nil (\"Linear Model:\" [:br] \"\" [:br] \"Residuals:\" [:br] \" Min 1Q Median 3Q Max\" [:br] \" -4.7198 -0.5571 0.2279 0.7265 1.7631\" [:br] \"\" [:br] \"Coefficients:\" [:br] \" Estimate Std. Error t value Pr(>|t|)\" [:br] \"Intercept 7.8323 0.3194 24.5216 0.0000 ***\" [:br] \"youtube 0.0197 0.0016 12.3197 0.0000 ***\" [:br] \"facebook 0.0410 0.0097 4.2202 0.0000 ***\" [:br] \"youtube*facebook 0.0009 0.0000 17.8680 0.0000 ***\" [:br] \"---------------------------------------------------------------------\" [:br] \"Significance codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\" [:br] \"\" [:br] \"Residual standard error: 1.0545 on 129 degrees of freedom\" [:br] \"Multiple R-squared: 0.9689, Adjusted R-squared: 0.9682\" [:br] \"F-statistic: 1340.7554 on 4 and 129 DF, p-value: 5.169e-97\")], \"155\" [:div [:p] nil nil [:p/markdown \"R2\"]], \"135\" [:div [:p] [:div [:p/code {:code \"(defn pp-str [x]\\n (with-out-str (clojure.pprint/pprint x)))\", :bg-class \"bg-light\"}]] nil nil], \"169\" [:div [:p] [:div [:p/code {:code \"(-> evaluations flatten first :test-transform :other-metrices first :metric)\", :bg-class \"bg-light\"}]] nil [:p/code {:code \"0.8955176888694834\\n\"}]], \"179\" [:div [:p] [:div [:p/code {:code \"(-> evaluations flatten first :test-transform :other-metrices first :metric)\", :bg-class \"bg-light\"}]] nil [:p/code {:code \"0.9630498727360517\\n\"}]], \"96\" [:div [:p] nil nil [:p/markdown \"Firts we build an additiv model, which model equatio is 'sales = b0 + b1*youtube + b2*facebook'\"]], \"159\" [:div [:p] [:div [:p/code {:code \"(comment\\n (note/init-with-browser)\\n (note/eval-this-notespace)\\n (note/render-static-html \\\"docs/userguide-models.html\\\"))\", :bg-class \"bg-light\"}]] nil [:p/code {:code \"nil\\n\"}]], \"162\" [:div [:p] [:div [:p/code {:code \"(def evaluations\\n (ml/evaluate-pipelines\\n [additive-pipeline]\\n (ds/split->seq marketing :holdout)\\n ml/rmse\\n :loss\\n {:other-metrices [{:name :r2\\n :metric-fn fmstats/r2-determination}]}))\", :bg-class \"bg-light\"}]] nil nil], \"167\" [:div [:p] [:div [:p/code {:code \"(-> evaluations flatten first :test-transform :metric)\", :bg-class \"bg-light\"}]] nil [:p/code {:code \"2.203614042165717\\n\"}]], \"172\" [:div [:p] [:div [:p/code {:code \"(def pipe-interaction\\n (ml/pipeline\\n (mm/drop-columns [:newspaper])\\n (mm/add-column :youtube*facebook (fn [ds] (dtf/* (ds :youtube) (ds :facebook))))\\n (mm/set-inference-target :sales)\\n {:metamorph/id :model}(mm/model {:model-type :smile.regression/ordinary-least-square})))\", :bg-class \"bg-light\"}]] nil nil], \"176\" [:div [:p] nil nil [:p/markdown \"As the multiplcation of 'youtube*facebook' is as well statistically relevant, it\\nsuggests that there is indeed an interaction between these 2 predictor variables youtube and facebook.\"]], \"173\" [:div [:p] [:div [:p/code {:code \"(def evaluations\\n (ml/evaluate-pipelines\\n [pipe-interaction]\\n (ds/split->seq marketing :holdout)\\n ml/rmse\\n :loss\\n {:other-metrices [{:name :r2\\n :metric-fn fmstats/r2-determination}]}))\", :bg-class \"bg-light\"}]] nil nil], \"90\" [:div [:p] nil nil [:p/markdown \"This examples how, how to do interactions in linear regression with `scicloj.ml`\"]], \"148\" [:div [:p] nil nil [:p/markdown \"Agin we evaluate the model,\"]], \"166\" [:div [:p] nil nil [:p/markdown \"RMSE\"]], \"168\" [:div [:p] nil nil [:p/markdown \"R2\"]], \"97\" [:div [:p] [:div [:p/code {:code \"(def additive-pipeline\\n (ml/pipeline\\n (mm/set-inference-target :sales)\\n (mm/drop-columns [:newspaper])\\n {:metamorph/id :model}\\n (mm/model {:model-type :smile.regression/ordinary-least-square})))\", :bg-class \"bg-light\"}]] nil nil], \"174\" [:div [:p] nil nil [:p/markdown \"and print it and the performance metrices:\"]], \"171\" [:div [:p] nil nil [:p/markdown \"Know we add interaction effects to it, resulting in this model equation: 'sales = b0 + b1*youtube + b2*facebook + b3*(youtube*facebook)'\"]], \"134\" [:div [:p] nil nil [:p/markdown \"Taking ideas from: \\nhttp://www.sthda.com/english/articles/40-regression-analysis/164-interaction-effect-in-multiple-regression-essentials/#comments-list\"]], \"177\" [:div [:p] nil nil [:p/markdown \"RMSE\"]], \"95\" [:div [:p] nil nil [:p/markdown \"## Additive model\"]], \"165\" [:div [:p] nil nil [:p/markdown \"We have the following metrices:\"]], \"178\" [:div [:p] [:div [:p/code {:code \"(-> evaluations flatten first :test-transform :metric)\", :bg-class \"bg-light\"}]] nil [:p/code {:code \"1.2915463808935799\\n\"}]], \"170\" [:div [:p] nil nil [:p/markdown \"## Interaction effects\"]], \"161\" [:div [:p] nil nil [:p/markdown \"We evaluate it, \"]], \"164\" [:div [:p] [:div [:p/code {:code \"^kind/hiccup\\n(text->hiccup\\n (str\\n (-> evaluations flatten first :fit-ctx :model ml/thaw-model str)))\", :bg-class \"bg-light\"}]] nil (\"Linear Model:\" [:br] \"\" [:br] \"Residuals:\" [:br] \" Min 1Q Median 3Q Max\" [:br] \" -6.4828 -0.9805 0.3129 1.4686 3.2026\" [:br] \"\" [:br] \"Coefficients:\" [:br] \" Estimate Std. Error t value Pr(>|t|)\" [:br] \"Intercept 3.9658 0.4036 9.8264 0.0000 ***\" [:br] \"youtube 0.0438 0.0016 27.8044 0.0000 ***\" [:br] \"facebook 0.1823 0.0095 19.1115 0.0000 ***\" [:br] \"---------------------------------------------------------------------\" [:br] \"Significance codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\" [:br] \"\" [:br] \"Residual standard error: 1.9357 on 130 degrees of freedom\" [:br] \"Multiple R-squared: 0.9021, Adjusted R-squared: 0.9006\" [:br] \"F-statistic: 599.0831 on 3 and 130 DF, p-value: 2.483e-66\")], \"163\" [:div [:p] nil nil [:p/markdown \"and print the result:\"]], \"160\" [:div [:p] [:div [:p/code {:code \"(def marketing (tc/dataset \\\"data/marketing.csv\\\" {:key-fn keyword}))\", :bg-class \"bg-light\"}]] nil nil], \"132\" [:div [:p] [:div [:p/code {:code \"(require '[scicloj.ml.core :as ml]\\n '[scicloj.ml.metamorph :as mm]\\n '[scicloj.ml.dataset :refer [dataset add-column]]\\n '[scicloj.ml.dataset :as ds]\\n '[tech.v3.dataset.math :as std-math]\\n '[tech.v3.datatype.functional :as dtf]\\n '[scicloj.metamorph.ml.toydata :as datasets])\", :bg-class \"bg-light\"}]] nil nil]}}"</script>
21+
<script src="gorilla-notes/js/compiled/main.js"></script>
22+
</html>

0 commit comments

Comments
 (0)