WIP: Preparsed cache

[b1ek]

scratch implementation of caching parsed blocks to speed up performance with libraries

[Mte90]

I can't wait for this :-D

[Ph0enixKM]

I think I understand what this PR adds. @b1ek could you at least create an issue for it so that we have a clear definition of what this PR solves?

[b1ek]

create an issue for it

its #524, but needs to be edited a little bit

[Mte90]

@b1ek requires some updates

[Copilot]

Pull Request Overview

This pull request implements a preparsed cache system to improve compilation performance by caching both tokenized and parsed representations of Amber source files. The caching system supports multiple file sources (File, Stdlib, Stream) with appropriate invalidation based on file modification times and Git hash changes.

• Adds comprehensive serialization support across the entire AST and metadata structures
• Implements tokenization and parsing cache with file-based storage
• Introduces file metadata system to differentiate between file sources and control cache behavior

Reviewed Changes

Copilot reviewed 80 out of 84 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/compiler/cache/tokenize.rs	Implements tokenization cache with file validation and storage
src/compiler/cache/parse.rs	Implements parsing cache with metadata serialization
src/compiler/cache.rs	Cache infrastructure and home directory management
src/compiler/file_source.rs	File metadata system for cache control
src/compiler.rs	Integration of caching into the compilation pipeline
src/stdlib.rs	Precompilation functionality for standard library
Multiple module files	Addition of Serialize/Deserialize traits to AST structures

Comments suppressed due to low confidence (3)

src/stdlib.rs:62

• The variable name 'i' is ambiguous. Consider renaming it to 'completed_count' or 'processed_files' to better reflect its purpose as a counter for completed thread operations.

    let mut i = 0;

src/compiler/cache/tokenize.rs:16

• [nitpick] The file extension 'ab.tk.c' could be more descriptive. Consider using 'amber_tokens_cache' or 'atk' to make the purpose clearer.

pub const FILE_EXT: &'static str = "ab.tk.c";

src/compiler/cache/parse.rs:8

• [nitpick] The file extension 'ab.pr.c' could be more descriptive. Consider using 'amber_parse_cache' or 'apc' to make the purpose clearer.

pub const FILE_EXT: &'static str = "ab.pr.c";

[Copilot]

The code is setting permissions on the original filename instead of the cache_file. This should be 'fs::set_permissions(&cache_file, ...)' to properly secure the cache file.

Suggested change

	fs::set_permissions(&filename, Permissions::from_mode(0o700)).map_err(|x| format!("Cannot set perms to {cache_file:?}: {x}"))?;
	fs::set_permissions(&cache_file, Permissions::from_mode(0o700)).map_err(|x| format!("Cannot set perms to {cache_file:?}: {x}"))?;

[Copilot]

Similar to the permissions issue, this is hiding the original filename instead of the cache_file. This should be 'hf::hide(&cache_file)' and 'hf' appears to be undefined.

[Copilot]

The identifier 'hf' is not defined or imported. This will cause a compilation error.

[Copilot]

Consider logging or handling the panic information from 'caught' to help with debugging. The current implementation silently converts all panics to "error".

[Copilot]

This import appears to be unused in the file. Consider removing it to keep the imports clean.

Suggested change

use crate::modules::expression::unop::not;