Update docs (#986)

* fix readme link

* add new references

* update readme for polynomials

* add readmes and links

* add readme for cFFT

* improve readme

* add fiat-shamir

* add info for hash

* add example starks

* polish readme

* example folder

* explain more starks

* add example

* fix typo

* fix links

* change to relative path

* add application

* add reference

* add readme for msm

* Update crates/math/src/polynomial/README.md

Co-authored-by: Mario Rugiero <[email protected]>

---------

Co-authored-by: jotabulacios <[email protected]>
Co-authored-by: Mario Rugiero <[email protected]>

Author: 3 people

README.md

@@ -24,11 +24,11 @@ Below is a list of examples to understand lambdaworks and learn what you can bui
 - [Shamir's secret sharing](./examples/shamir_secret_sharing/)
 - [BabySNARK](./examples/baby-snark/)
 - [Pinocchio](./examples/pinocchio/)
-- [Using Circom with lambdaworks's Groth16](./provers/groth16/circom-adapter/src/README.md)
+- [Using Circom with lambdaworks's Groth16](./examples/prove-verify-circom/circom_lambdaworks_tutorial.md)
 - [Proving Fibonacci using Circom and lambdaworks](./examples/prove-verify-circom/circom_lambdaworks_tutorial.md)
 
-- You can use Circom to generate circuits and use lambdaworks's capabilities to prove the execution with [Groth16](./provers/groth16/README.md).
-- You can use the [Stark prover](./provers/stark/src/) to define an algebraic intermediate representation (AIR) and prove the execution of a program
+- You can use Circom to generate circuits and use lambdaworks's capabilities to prove the execution with [Groth16](./crates/provers/groth16/README.md).
+- You can use the [Stark prover](./crates/provers/stark/README.md) to define an algebraic intermediate representation (AIR) and prove the execution of a program
 
 ## Why we built lambdaworks
 
@@ -215,6 +215,10 @@ The following links, repos, companies and projects have been important in the de
 - [Gnark](https://github.com/Consensys/gnark)
 - [Constantine](https://github.com/mratsim/constantine)
 - [Plonky3](https://github.com/Plonky3/Plonky3)
+- [Stwo](https://github.com/starkware-libs/stwo/tree/dev)
+- [Binius](https://gitlab.com/IrreducibleOSS/binius)
+- [Zorch](https://github.com/vbuterin/zorch/tree/main)
+- [Jolt](https://github.com/a16z/jolt)
 
 # Security
 

crates/crypto/README.md

@@ -18,3 +18,10 @@ This crate contains different cryptographic primitives needed for proof systems.
 - [Hash functions](./src/hash/)
 - [Fiat Shamir transformation](./src/fiat_shamir/)
 - [Polynomial commitment schemes](./src/commitments/)
+
+For examples on:
+- How do Merkle trees work, refer to [Merkle CLI](../../examples/merkle-tree-cli/README.md)
+- Hash functions, refer to [Hash functions' readme](./src/hash/README.md)
+- Fiat-Shamir heuristic, refer to [Fiat-Shamir's readme](./src/fiat_shamir/README.md)
+- Polynomial commitment schemes, refer to [PCS's readme](./src/commitments/README.md)
+

crates/crypto/src/fiat_shamir/README.md

@@ -0,0 +1,5 @@
+# Fiat-Shamir
+
+This contains the basic functionality to implement the [Fiat-Shamir heuristic](https://en.wikipedia.org/wiki/Fiat%E2%80%93Shamir_heuristic) ([see also](https://link.springer.com/chapter/10.1007/3-540-47721-7_12)).
+
+The transcript should be able to accept bytes or field elements and should output random challenges (field elements) from a uniform distribution.

crates/crypto/src/hash/README.md

@@ -0,0 +1,11 @@
+# Hash functions
+
+This folder contains hash functions that are typically used in non-interactive proof systems. The hash functions we have implemented are:
+- [Monolith](./monolith/mod.rs)
+- [Poseidon](./poseidon/)
+- [Pedersen](./pedersen/)
+- [Rescue Prime](./rescue_prime/)
+
+Pedersen is based on elliptic curves, while [Monolith](https://eprint.iacr.org/2023/1025), [Poseidon](https://eprint.iacr.org/2019/458.pdf) and [Rescue Prime](https://eprint.iacr.org/2020/1143) are algebraic hash functions.
+
+For an introduction to hash functions, see [this intro](https://blog.alignedlayer.com/introduction-hash-functions-in-cryptography/) and [its follow-up](https://blog.alignedlayer.com/design-strategies-how-to-construct-a-hashing-mode-2/).

crates/math/README.md

@@ -16,6 +16,8 @@ This crate contains all the relevant mathematical building blocks needed for pro
 - [Finite Fields](./src/field/README.md)
 - [Elliptic curves](./src/elliptic_curve/README.md)
 - [Polynomials - univariate and multivariate](./src/polynomial/README.md)
+- [CircleFFT](./src/circle/README.md)
 - [Large unsigned integers](./src/unsigned_integer/)
 - [Fast Fourier Transform](./src/fft/README.md)
 - [Optimized Multiscalar Multiplication](./src/msm/)
+- [Cyclic Group](./src/cyclic_group.rs)

crates/math/src/circle/README.md

@@ -0,0 +1,5 @@
+# Circle Fast-Fourier Transform (CircleFFT)
+
+This folder contains all the necessary tools to work with the [circle FFT](https://eprint.iacr.org/2024/278), which is a suitable way of performing an analogue of the [radix-2 FFT algorithm](../fft/README.md) over fields which are not smooth. We say a finite field is smooth if the size of multiplicative group of the field is divisible by a sufficiently high power of 2. In the case of $\mathbb{Z}_p$, the previous sentence indicates that $p - 1 = 2^m c$, where $m$ is sufficiently large (for example, $2^{25}$), ensuring we can use the radix-2 Cooley-Tuckey algorithm for the FFT with vectors of size up to $2^{25}$.
+
+For an introduction to circle STARKs, we recommend [our blog](https://blog.lambdaclass.com/an-introduction-to-circle-starks/) or [Vitalik's explanation](https://vitalik.eth.limo/general/2024/07/23/circlestarks.html).

crates/math/src/elliptic_curve/README.md

@@ -118,7 +118,7 @@ let y = g2_affine.y();
 
 One common operation for different proof systems is the Mutiscalar Multiplication (MSM), which is given by a set of points $P_0 , P_1 , P_2 , ... , P_n$ and scalars $a_0 , a_1 , a_2 ... n_n$ (the scalars belong to the scalar field of the elliptic curve, which is the field whose size matches the size of the elliptic curve's group):
 $$R = \sum_k a_k P_k$$
-This operation could be implemented by using `operate_with_self` with each point and scalar and then add the results using `operate_with`, but this is not efficient. lambdaworks provides an optimized [MSM using Pippenger's algorithm](https://github.com/lambdaclass/lambdaworks/blob/main/math/src/msm/pippenger.rs). A naïve version is given [here](https://github.com/lambdaclass/lambdaworks/blob/main/math/src/msm/naive.rs). Below we show how to use MSM in the context of a polynomial commitment scheme: the scalars are the coefficients of the polynomials and the points are provided by an SRS.
+This operation could be implemented by using `operate_with_self` with each point and scalar and then add the results using `operate_with`, but this is not efficient. lambdaworks provides an optimized [MSM using Pippenger's algorithm](../msm/pippenger.rs). A naïve version is given [here](../msm/naive.rs). Below we show how to use MSM in the context of a polynomial commitment scheme: the scalars are the coefficients of the polynomials and the points are provided by an SRS.
 ```rust
 fn commit(&self, p: &Polynomial<FieldElement<F>>) -> Self::Commitment {
         let coefficients: Vec<_> = p
@@ -158,6 +158,8 @@ This allows us to reduce the amount of information we send to define a unique po
 
 In many curves, the base field contains some spare bits (as is the case of BLS12-381 or BN254, but not secp256k1), which allows us to codify the extra bit into the free bits of the element. Depending on the number of spare bits, we could compress points in different ways.
 
+Pairings and elliptic curve point compression and decompression are needed for [BLS signatures](https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signature-05.html) and in [EIP-4844](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4844.md). 
+
 ## References
 
 - [HyperElliptic - formulae for EC addition and doubling](https://hyperelliptic.org/EFD/g1p/index.html)

crates/math/src/fft/README.md

@@ -1,6 +1,6 @@
 # lambdaworks Fast Fourier Transform
 
-This folder contains the [fast Fourier transform](https://en.wikipedia.org/wiki/Fast_Fourier_transform) (FFT) over finite fields (also known as number theoretic transform, NTT). If you are unfamiliar with how lambdaworks handles fields, see [examples](https://github.com/lambdaclass/lambdaworks/blob/main/examples/README.md). Currently, the following algorithms are supported:
+This folder contains the [fast Fourier transform](https://en.wikipedia.org/wiki/Fast_Fourier_transform) (FFT) over finite fields (also known as number theoretic transform, NTT). If you are unfamiliar with how lambdaworks handles fields, see [examples](../../../../examples/README.md). Currently, the following algorithms are supported:
 - Cooley-Tukey Radix-2
 - Cooley-Tukey Radix-4
 

crates/math/src/field/README.md

@@ -368,4 +368,5 @@ This defines a 6th degree extension over the scalar field of BLS12-381. We only
 - [High-Speed Algorithms & Architectures For Number-Theoretic Cryptosystems](https://www.microsoft.com/en-us/research/wp-content/uploads/1998/06/97Acar.pdf)
 - [Developer math survival kit](https://blog.lambdaclass.com/math-survival-kit-for-developers/)
 - [Montgomery Arithmetic from a Software Perspective](https://eprint.iacr.org/2017/1057.pdf)
-- [Guajardo, Kumar, Paar, Perzl - Efficient software implementation of finite fields with applications to Cryptography](https://www.sandeep.de/my/papers/2006_ActaApplMath_EfficientSoftFiniteF.pdf)
+- [Guajardo, Kumar, Paar, Perzl - Efficient software implementation of finite fields with applications to Cryptography](https://www.sandeep.de/my/papers/2006_ActaApplMath_EfficientSoftFiniteF.pdf)
+- [Ingonyama - Montgomery-Barrett duality](https://hackmd.io/@Ingonyama/Barret-Montgomery)

crates/math/src/msm/README.md

@@ -0,0 +1,9 @@
+# lambdaworks MultiScalar Multiplication (MSM)
+
+This contains implementations for the MultiScalar Multiplication (MSM):
+- Naïve
+- Pippenger
+
+[Multiscalar multiplication](https://blog.lambdaclass.com/multiscalar-multiplication-strategies-and-challenges/) is an important primitive that appears in some polynomial commitment schemes and proof systems. It is also at the core of [EIP-4844](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4844.md). Given a set of scalars in a [finite field](../field/README.md) $a_0, a_1, ..., a_n$ and [elliptic curve points](../elliptic_curve/README.md) $P_0, P_1, ... , P_n$, the MSM computes
+$$P = \sum_k a_k P_k$$
+where $a_k P_k$ is understood as applying the group operation with $P_k$ a number of $a_k$ times. For its application in a protocol, see [KZG](../../../crypto/src/commitments/README.md) 

crates/math/src/polynomial/README.md

@@ -66,3 +66,66 @@ let p = Polynomial::interpolate(&[FE::new(0), FE::new(1)], &[FE::new(2), FE::new
 ```
 
 Many polynomial operations can go faster by using the [Fast Fourier Transform](../fft/polynomial.rs).
+
+## Multilinear polynomials
+
+Multilinear polynomials are useful to define multilinear extensions of functions, which then play an important role in proof systems involving the [sumcheck protocol](../../../provers/sumcheck/README.md). There are two ways to define multilinear polynomials:
+- Dense
+- Sparse
+
+Sparse is more convenient whenever the number of non-zero coefficients in the polynomial is small (compared to the length of the polynomial), avoiding the storage of unnecessary zeros. For dense multilinear polynomials we have the following structure, working over some field $F$:
+```rust
+pub struct DenseMultilinearPolynomial<F: IsField>
+where
+    <F as IsField>::BaseType: Send + Sync,
+{
+    evals: Vec<FieldElement<F>>,
+    n_vars: usize,
+    len: usize,
+}
+```
+The polynomial is assumed to be given in evaluation form over the binary strings of length $\{0 , 1 \}^{n_{vars}}$. We can also interpret this as the coefficients of the polynomial with respect to the Lagrange basis polynomials over $\{0 , 1 \}^{n_{vars}}$. There are $2^{n_{vars}}$ Lagrange polynomials, given by the formula:
+$L_k (x_0 , x_1 , ... , x_{n_{vars} - 1}) = \prod (x_j b_{kj} + (1 - x_j ) (1 - b_{kj} ))$
+where $b_{kj}$ are given by the binary decomposition of $k$, that is $k = \sum_j b_{kj} 2^j$. We can see that each such polynomial is equal to one over $\{b_{k0}, b_{k1} , ... b_{k (n_{vars} - 1)}}$ and zero for any other element in $\{0 , 1 \}^{n_{vars}}$. The polynomial is thus defined as
+$p (x_0 , x_1, ... , x_{n_{vars} - 1} ) = \sum_k p(b_{k0}, b_{k1} , ... , b_{k (n_{vars} - 1)}) L_k (x_0 , x_1, ... , x_{n_{vars} - 1} )$
+Sometimes, we will use $L_k (j)$ to refer to the evaluation of $L_k$ at the binary decomposition of $j$, that is $j = \sum_k b_{k}2^k$.
+
+An advantage of Lagrange basis polynomials is that we can evaluate all $2^{n_{vars}}$ polynomials at a point $(r_0 , r_1 ... , r_{n_{vars} - 1})$ in $\mathcal{O}(2^{n_{vars}})$ operations (linear in the size of the number of polynomials). Refer to [Thaler's book](https://people.cs.georgetown.edu/jthaler/ProofsArgsAndZK.pdf) for more information.
+
+To create a new polynomial, provide a list of evaluations of $p$; the length of this list should be a power of 2.
+```rust
+pub fn new(mut evals: Vec<FieldElement<F>>) -> Self {
+    while !evals.len().is_power_of_two() {
+        evals.push(FieldElement::zero());
+    }
+    let len = evals.len();
+    DenseMultilinearPolynomial {
+        n_vars: log_2(len),
+        evals,
+        len,
+    }
+}
+```
+
+Dense multilinear polynomials allow you to access the fields `n_vars`, `len` and `evals` with the methods `pub fn num_vars(&self) -> usize`, `pub fn len(&self) -> usize` and `pub fn evals(&self) -> &Vec<FieldElement<F>>`.
+
+If you want to evaluate outside $\{0 , 1 \}^{n_{vars}}$, you can use the functions `pub fn evaluate(&self, r: Vec<FieldElement<F>>)` and `pub fn evaluate_with(evals: &[FieldElement<F>], r: &[FieldElement<F>])`, providing the point $r$ whose length must be $n_{vars}$. For evaluations over $\{0 , 1 \}^{n_{vars}}$, you can get the value directly from the list of evaluations defining the polynomial. For example, 
+```rust
+// Example: Z = [1, 2, 1, 4]
+let z = vec![FE::one(), FE::from(2u64), FE::one(), FE::from(4u64)];
+// r = [4, 3]
+let r = vec![FE::from(4u64), FE::from(3u64)];
+let eval_with_lr = evaluate_with_lr(&z, &r);
+let poly = DenseMultilinearPolynomial::new(z);
+let eval = poly.evaluate(r).unwrap();
+assert_eq!(eval, FE::from(28u64));
+```
+
+An important functionality is `pub fn to_univariate(&self) -> Polynomial<FieldElement<F>>`, which converts a multilinear polynomial into a univariate polynomial, by summing over all variables over $\{0 , 1 \}^{n_{vars} - 1}$, leaving $x_{n_{vars} - 1}$ as the only variable, 
+$$f(x) = \sum_{(x_0 , x_1, ... , x_{n_{vars} - 2} ) \in \{0 , 1 \}^{n_{vars} - 1}} p(x_0 , x_1, ... , x_{n_{vars} - 2} , x)$$. For example,
+```rust
+let univar0 = prover.poly.to_univariate();
+```
+is used in the sumcheck protocol.
+
+Multilinear polynomials can be added and multiplied by scalars.