1.0.0

Author: matthewwardrop

README.md

@@ -10,8 +10,6 @@
 
 Formulaic is a high-performance implementation of Wilkinson formulas for Python.
 
-**Note:** This project, while largely complete, is still a work in progress, and the API is subject to change between major versions (0.<major>.<minor>).
-
 - **Documentation**: https://matthewwardrop.github.io/formulaic
 - **Source Code**: https://github.com/matthewwardrop/formulaic
 - **Issue tracker**: https://github.com/matthewwardrop/formulaic/issues
@@ -31,6 +29,7 @@ It provides:
     - `numpy.ndarray`
     - `scipy.sparse.CSCMatrix`
 - support for symbolic differentiation of formulas (and hence model matrices).
+- and much more.
 
 ## Example code
 
@@ -107,6 +106,13 @@ y, X = Formula('y ~ x + z').get_model_matrix(df)
   </tbody>
 </table>
 
+Note that the above can be short-handed to:
+
+```
+from formulaic import model_matrix
+model_matrix('y ~ x + z', df)
+```
+
 ## Benchmarks
 
 Formulaic typically outperforms R for both dense and sparse model matrices, and vastly outperforms `patsy` (the existing implementation for Python) for dense matrices (`patsy` does not support sparse model matrix output).

benchmarks/benchmarks.csv

@@ -1,46 +1,46 @@
 ,formula,tooling,mean,stderr
-0,a,patsy,0.06235763004847935,0.005395749279517771
-1,a,formulaic,0.01614434378487723,0.0032546966398573733
-2,a,formulaic_sparse,0.3261915956224714,0.016261649503701136
-3,a,R,0.28679302760532926,0.04140188245157612
-4,a,R_sparse,0.3757050037384033,0.10653194222854354
-5,A,patsy,5.07630558013916,0.21714776785515827
-6,A,formulaic,0.20960685185023717,0.00647732136737535
-7,A,formulaic_sparse,0.49657981736319406,0.014160553634556148
-8,A,R,0.270564215523856,0.048446057523137825
-9,A,R_sparse,0.6199769633156913,0.046671131192277546
-10,a+A,patsy,5.372520089149475,0.24907024334140557
-11,a+A,formulaic,0.21435914720807756,0.005022992135378462
-12,a+A,formulaic_sparse,0.5922877447945731,0.01141291396766895
-13,a+A,R,0.3385444368634905,0.050949061524224834
-14,a+A,R_sparse,0.8427209513528007,0.05420930370296844
-15,a:A,patsy,5.41690331697464,0.20045329513286106
-16,a:A,formulaic,0.2447828565325056,0.009831368693974841
-17,a:A,formulaic_sparse,0.5952485970088414,0.015557371582484582
-18,a:A,R,0.325153112411499,0.0525960780057241
-19,a:A,R_sparse,0.6293824059622628,0.05176575091323222
-20,A+B,patsy,10.592723488807678,0.36380136013031006
-21,A+B,formulaic,0.39785667828151156,0.004189211004718843
-22,A+B,formulaic_sparse,0.7370290756225586,0.00559989056050753
-23,A+B,R,0.45774364471435547,0.04577860217478202
-24,A+B,R_sparse,1.128925051007952,0.0730276871543806
-25,a:A:B,patsy,13.139377474784851,0.735141396522522
-26,a:A:B,formulaic,0.5296461582183838,0.02897198866398977
-27,a:A:B,formulaic_sparse,0.9496099608285087,0.017482256914701913
-28,a:A:B,R,0.5121760368347168,0.05913549901764887
-29,a:A:B,R_sparse,2.4410063539232527,0.15718149749347077
-30,A:B:C:D,patsy,33.971909284591675,0.0
-31,A:B:C:D,formulaic,1.4003467900412423,0.013122149254960603
-32,A:B:C:D,formulaic_sparse,2.6644029957907542,0.0594315471815126
-33,A:B:C:D,R,1.5739161627633231,0.043335739540618624
-34,A:B:C:D,R_sparse,11.206892251968384,0.07203364372253418
-35,a*b*A*B,patsy,14.135663151741028,0.023609280586242676
-36,a*b*A*B,formulaic,0.7015061037881034,0.015836408587630867
-37,a*b*A*B,formulaic_sparse,1.2936896937234061,0.008783658171925213
-38,a*b*A*B,R,0.7440026828220913,0.0779464030983464
-39,a*b*A*B,R_sparse,8.046716928482056,0.09924621730008089
-40,a*b*c*A*B*C,patsy,52.30743145942688,0.0
-41,a*b*c*A*B*C,formulaic,3.124175344194685,0.015513429204320773
-42,a*b*c*A*B*C,formulaic_sparse,4.722880220413208,0.05794530543951235
-43,a*b*c*A*B*C,R,3.261254208428519,0.03376348572979368
-44,a*b*c*A*B*C,R_sparse,96.12985253334045,0.0
+0,a,patsy,0.05834197998046875,0.00803367432263398
+1,a,formulaic,0.02302394594464983,0.005276772701941135
+2,a,formulaic_sparse,0.21061321667262486,0.010992980906779363
+3,a,R,0.20319366455078125,0.04054977688850774
+4,a,R_sparse,0.25407181467328754,0.06840892083032614
+5,A,patsy,4.188197422027588,0.06544489467164907
+6,A,formulaic,0.1523878574371338,0.0036558939106885466
+7,A,formulaic_sparse,0.3251234803880964,0.011790060964279047
+8,A,R,0.17913893290928432,0.024771923314745647
+9,A,R_sparse,0.2776027406964983,0.030147102033801613
+10,a+A,patsy,4.570634412765503,0.15648980716104596
+11,a+A,formulaic,0.17730648177010672,0.011166617158905944
+12,a+A,formulaic_sparse,0.4082690307072231,0.019076310496142206
+13,a+A,R,0.38174584933689665,0.07049519867629231
+14,a+A,R_sparse,0.5332009451729911,0.21147057077093623
+15,a:A,patsy,4.846947574615479,0.17439756289836647
+16,a:A,formulaic,0.18150435175214494,0.0029803910808661733
+17,a:A,formulaic_sparse,0.40479908670697895,0.02179493572840004
+18,a:A,R,0.209270749773298,0.02890730110968228
+19,a:A,R_sparse,0.3095934050423758,0.03366518302136102
+20,A+B,patsy,8.886903127034506,0.12687320827416307
+21,A+B,formulaic,0.37893104553222656,0.07142932738422411
+22,A+B,formulaic_sparse,0.6603872776031494,0.12048599251620122
+23,A+B,R,0.3503831795283726,0.08556777381884671
+24,A+B,R_sparse,0.6867697579520089,0.1680831720230895
+25,a:A:B,patsy,10.59350836277008,0.006071925163269043
+26,a:A:B,formulaic,0.38779779842921663,0.005652063736152758
+27,a:A:B,formulaic_sparse,0.6174772126334054,0.006656982815848345
+28,a:A:B,R,0.41255525180271696,0.005872361057027324
+29,a:A:B,R_sparse,1.3681020736694336,0.13250905747410396
+30,A:B:C:D,patsy,27.812817335128784,0.0
+31,A:B:C:D,formulaic,1.7389381953648158,0.1128489022833264
+32,A:B:C:D,formulaic_sparse,1.821084805897304,0.04040580196899275
+33,A:B:C:D,R,1.1703059673309326,0.01475477228204255
+34,A:B:C:D,R_sparse,6.603186547756195,0.10862060432577084
+35,a*b*A*B,patsy,14.305930256843567,1.4570282697677612
+36,a*b*A*B,formulaic,0.849949870790754,0.12062261319473745
+37,a*b*A*B,formulaic_sparse,1.194093908582415,0.27659465274967987
+38,a*b*A*B,R,0.633225509098598,0.09868026112633763
+39,a*b*A*B,R_sparse,7.428930600484212,1.4652407668448042
+40,a*b*c*A*B*C,patsy,48.66431951522827,0.0
+41,a*b*c*A*B*C,formulaic,4.352833080291748,0.31885221781014655
+42,a*b*c*A*B*C,formulaic_sparse,4.8097954273223875,0.7055727752553242
+43,a*b*c*A*B*C,R,2.6774498394557407,0.06813521510330559
+44,a*b*c*A*B*C,R_sparse,72.07087659835815,0.0

benchmarks/benchmarks.png

@@ -3,6 +3,40 @@ For changes since the latest tagged release, please refer to the
 
 ---
 
+## 1.0.0 (24 December 2023)
+
+**Breaking changes:**
+
+* Python tokens are now canonically formatted (see below).
+* Methods deprecated during the 0.x series have been removed: `Formula.terms`,
+  `ModelSpec.feature_names`, and `ModelSpec.feature_indices`.
+
+**New features and enhancements:**
+
+* Python tokens are now sanitized and canonically formatted to prevent
+  ambiguities and better align with `patsy`.
+* Added official support for Python 3.12 (no code changes were necessary).
+* Added the `hashed` transform for categorically encoding deterministically
+  hashed representations of a dataset.
+
+**Bugfixes and cleanups:**
+
+* Fixed transform state not propagating correctly when Python code tokens were
+  not canonically formatted.
+* Literals in formulae will no longer be silently ignored, and feature scaling
+  is now fully supported.
+* Improved code parsing and formatting utilities and dropped the requirement for
+  `astor` for Python 3.9 and newer.
+* Fixed all warnings emitted during unit tests.
+
+**Documentation:**
+
+* Removed incompleteness warnings.
+* Added some lightweight developer documents.
+* Fixed some broken links.
+
+---
+
 ## 0.6.6 (4 October 2023)
 
 This is minor release with one important bugfix.