High collision rate in default company-name anonymizer

[kirienko]

Summary:
When anonymising client/company names with the built-in DefaultCompanyNames (148 items, see here) and DefaultCompanySuffixes ({"Ltd.", "Inc.", "LLC.", "LLP.", "P.C.", "Corp."} = 6 total), the output contains many duplicates. The attached screenshot shows several repeats (e.g., “Blue Harbor P.C.”, “Blue Horizon Corp.”) in a dataset of < 1,000 rows.

Here is what happens with my clients table:

Expected

• Option to generate unique company names within a run (or within a column), or at least a much lower collision rate by default.

Actual

• Numerous duplicates are produced.
• This makes anonymised data less realistic, can violate uniqueness constraints, and reduces test coverage quality.

Why this likely happens:

• The current name space size is ~148 × 6 = 888 combinations.
• If values are sampled with replacement, collisions are statistically inevitable (birthday paradox).
• The expected number of unique values after drawing n names is
  E[unique] = m · (1 − (1 − 1/m)^n) ≈ m · (1 − e^{−n/m}).
  For this generator with m = 888:
  • n = 600 → ~436 unique / ~164 duplicates
  • n = 800 → ~527 unique / ~273 duplicates
  • n = 1,000 → ~600 unique / ~ 400 duplicates
    Hence many duplicates are mathematically expected on datasets approaching ~1k rows.

Steps to reproduce:

1. Anonymise a column of ~800–1,000 company names using the default company name generator (default DefaultCompanyNames + DefaultCompanySuffixes).
2. Inspect the output: repeated names occur frequently.

Impact

• Breaks tests that expect distinct companies.
• Makes data look synthetic/low-quality to analysts and auditors.
• Forces consumers to add post-processing to de-duplicate, which is brittle.

Requested change (any of the following would solve it)

1. unique: true option for this generator that:
   • Samples without replacement from the available pool for the current batch/column.
   • When the pool is exhausted, either:
     • emits a clear error, or
     • automatically expands the name space (see 3. / 4. below), or
     • appends a stable numeric/alpha suffix to guarantee uniqueness.
2. Documented knob to increase pool size, with guidance on expected collision rates vs. dataset size.
3. Expand the default name space (e.g., Adjective + Noun + Suffix, large adjective/noun lists) to produce millions of combinations.
4. Deterministic mapping mode: map each original value (or row id) to a unique anonymised name via hashing + collision resolution (stable across runs).

Workarounds today:

• Provide a custom dictionary large enough for the dataset and implement a de-dup pass (costly and fragile).
• Pre-generate a pool and join by a deterministic key (adds plumbing in pipelines).

[wwoytenko]

Hi! Thbak you for reporting. I will take a look soon

[GreatBahram]

@wwoytenko I wonder if there is a faker function for company name? I'm asking since I couldn't find it in the documentation and if there is such a function then one can use something like this: {{- .fakerCompanyName -}} {{- someRandomChars -}} with the Template

[wwoytenko]

I wonder if there is a faker function for company name?

@GreatBahram It’s not implemented yet, but I’m working on it in V1 so that built-in transformations will be accessible in the Template scope. If you’d like to join the discussion or help with this implementation, let me know.