Try StableRNG

Author: penelopeysm

Manifest.toml

@@ -2,7 +2,7 @@
 
 julia_version = "1.11.7"
 manifest_format = "2.0"
-project_hash = "271c58c55ced391f9e88f814e2e4b8b4b7d5724d"
+project_hash = "2b3993e6e60ba9c1c456523b8f2caaae65279013"
 
 [[deps.ADTypes]]
 git-tree-sha1 = "27cecae79e5cc9935255f90c53bb831cc3c870d7"

Project.toml

@@ -41,6 +41,7 @@ RDatasets = "ce6b1742-4840-55fa-b093-852dadbb1d8b"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267"
 SciMLSensitivity = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
+StableRNGs = "860ef19b-820b-49d6-a774-d7a799459cd3"
 Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
 StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
 StatsFuns = "4c63d2b9-4356-54db-8cca-17b64c39e42c"

tutorials/bayesian-linear-regression/index.qmd

@@ -42,7 +42,7 @@ using StatsBase
 using LinearAlgebra
 
 # For ensuring reproducibility.
-using Random
+using StableRNGs: StableRNG
 ```
 
 ```{julia}
@@ -75,7 +75,7 @@ The next step is to get our data ready for testing. We'll split the `mtcars` dat
 select!(data, Not(:Model))
 
 # Split our dataset 70%/30% into training/test sets.
-trainset, testset = map(DataFrame, splitobs(Xoshiro(472), data; at=0.7, shuffle=true))
+trainset, testset = map(DataFrame, splitobs(StableRNG(468), data; at=0.7, shuffle=true))
 
 # Turing requires data in matrix form.
 target = :MPG
@@ -142,7 +142,7 @@ With our model specified, we can call the sampler. We will use the No U-Turn Sam
 
 ```{julia}
 model = linear_regression(train, train_target)
-chain = sample(Xoshiro(468), model, NUTS(), 20_000)
+chain = sample(StableRNG(468), model, NUTS(), 20_000)
 ```
 
 We can also check the densities and traces of the parameters visually using the `plot` functionality.
@@ -242,9 +242,11 @@ let
     ols_test_loss = msd(test_prediction_ols, testset[!, target])
     @assert bayes_train_loss < bayes_test_loss "Bayesian training loss ($bayes_train_loss) >= Bayesian test loss ($bayes_test_loss)"
     @assert ols_train_loss < ols_test_loss "OLS training loss ($ols_train_loss) >= OLS test loss ($ols_test_loss)"
-    @assert isapprox(bayes_train_loss, ols_train_loss; rtol=0.01) "Difference between Bayesian training loss ($bayes_train_loss) and OLS training loss ($ols_train_loss) unexpectedly large!"
-    @assert isapprox(bayes_test_loss, ols_test_loss; rtol=0.05) "Difference between Bayesian test loss ($bayes_test_loss) and OLS test loss ($ols_test_loss) unexpectedly large!"
+    @assert bayes_train_loss > ols_train_loss "Bayesian training loss ($bayes_train_loss) <= OLS training loss ($bayes_train_loss)"
+    @assert bayes_test_loss < ols_test_loss "Bayesian test loss ($bayes_test_loss) >= OLS test loss ($ols_test_loss)"
 end
 ```
 
-As we can see above, OLS and our Bayesian model fit our training and test data set about the same.
+We can see from this that both linear regression techniques perform fairly similarly.
+The Bayesian linear regression approach performs worse on the training set, but better on the test set.
+This indicates that the Bayesian approach is more able to generalise to unseen data, i.e., it is not overfitting the training data as much.