-
Notifications
You must be signed in to change notification settings - Fork 229
feat(compass-collection): Mongosh Script Generation Engine - CLOUDP-333860 #7313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements a mongosh script generation engine that creates executable scripts for generating and uploading mock data based on faker schemas. The engine supports complex nested structures, arrays, probabilistic fields, and configurable array lengths.
- Core script generation functionality with support for nested objects, arrays, and multi-dimensional arrays
- Faker.js integration with argument handling and fallback methods for unrecognized field types
- Probabilistic field generation for optional data modeling
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts | Implements the complete script generation engine with type definitions, path parsing, document structure building, and code generation |
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.spec.ts | Comprehensive test suite covering all functionality including edge cases, array handling, probability features, and faker argument processing |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Fixed
Show fixed
Hide fixed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Fixed
Show fixed
Hide fixed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Fixed
Show fixed
Hide fixed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Fixed
Show fixed
Hide fixed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Fixed
Show fixed
Hide fixed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Show resolved
Hide resolved
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Outdated
Show resolved
Hide resolved
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Outdated
Show resolved
Hide resolved
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Show resolved
Hide resolved
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Outdated
Show resolved
Hide resolved
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Outdated
Show resolved
Hide resolved
.../compass-collection/src/components/mock-data-generator-modal/script-generation-utils.spec.ts
Show resolved
Hide resolved
.../compass-collection/src/components/mock-data-generator-modal/script-generation-utils.spec.ts
Show resolved
Hide resolved
.../compass-collection/src/components/mock-data-generator-modal/script-generation-utils.spec.ts
Outdated
Show resolved
Hide resolved
if (probability < 1.0) { | ||
// Use Math.random for conditional field inclusion | ||
rootLevelFields.push( | ||
`${fieldIndent}...(Math.random() < ${probability} ? { ${fieldName}: ${fakerCall} } : {})` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why include the empty object? vs simply leaving rootLevelFields as is
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I realize this isn't the most readable but it's conditional rendering using the spread operator ...
, so if the probability condition is not met, nothing gets rendered (i.e no empty object). I can add a comment, or potentially refactor this to make it clearer. The advantage of this notation is that it's one line, and there may be many of these
/** | ||
* Gets default faker method for unrecognized fields based on MongoDB type | ||
*/ | ||
export function getDefaultFakerMethod(mongoType: string): string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to cover any more of the allowed datatypes?
some of these would be easy like null, Long, timestamp
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Also changed to explicitly handle null
and undefined
by passing through those literal values
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jcobis This doesn't actually generate code to create instances of these types, though ... for example, for Decimal128, you want something like Decimal128.fromStringWithRounding(`${faker.number.float()}`)
, not just faker.number.float()
It's also honestly a bit unclear here what mongoType
actually is – e.g. in what situation would int64
be passed vs. in what situation long
? Can we re-use an existing enum here, e.g. TypeCastTypes
found elsewhere in the Compass source?
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Dismissed
Show dismissed
Hide dismissed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Dismissed
Show dismissed
Hide dismissed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Fixed
Show fixed
Hide fixed
packages/compass-collection/src/components/mock-data-generator-modal/script-generation-utils.ts
Dismissed
Show dismissed
Hide dismissed
// This is equivalent to: function(faker) { return <returnExpression>; } | ||
// The 'faker' parameter will receive the real faker.js library when we pass it in on call | ||
// eslint-disable-next-line @typescript-eslint/no-implied-eval | ||
const generateDocument = new Function( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I never knew the Function constructor could be used this way. Very cool ✨
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see more codeql+copilot suggestions, but otherwise latest diffs lgtm
// The "{ ... }" part is the document structure | ||
|
||
// Extract the return statement from the generateDocument function | ||
const returnMatch = script.match(/return ([\s\S]*?);[\s]*\}/); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit, completely optional, but this can be simplified a bit:
const returnMatch = script.match(/return ([\s\S]*?);[\s]*\}/); | |
const returnMatch = script.match(/return (.*?);\s*\}/s); |
console.log(\`Successfully inserted \${documents.length} documents into ${ | ||
options.databaseName | ||
}.${options.collectionName}\`);`; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^ This still needs to be escaped, '
is a valid character in database and collection names
/** | ||
* Gets default faker method for unrecognized fields based on MongoDB type | ||
*/ | ||
export function getDefaultFakerMethod(mongoType: string): string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jcobis This doesn't actually generate code to create instances of these types, though ... for example, for Decimal128, you want something like Decimal128.fromStringWithRounding(`${faker.number.float()}`)
, not just faker.number.float()
It's also honestly a bit unclear here what mongoType
actually is – e.g. in what situation would int64
be passed vs. in what situation long
? Can we re-use an existing enum here, e.g. TypeCastTypes
found elsewhere in the Compass source?
Description
Implement the mongosh script generation engine that creates an executable script based on the faker schema. The script will generate and upload mock data.
Checklist
Types of changes