-
Notifications
You must be signed in to change notification settings - Fork 36
feat: Add full-featured code generator #1016
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| val gitRepo = config.expectStringMember(GIT_REPO).value | ||
| val swiftVersion = config.expectStringMember(SWIFT_VERSION).value | ||
| val sdkId = sanitizeSdkId(config.getStringMemberOrDefault(SDK_ID, serviceId.name)) | ||
| val sdkId = config.getStringMemberOrDefault(SDK_ID, service?.sdkId ?: serviceId.name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What we currently call sdkId here is actually the service's sdkId with some transformations applied to it. To avoid confusion, this settings field is now the sdkId taken directly from the service.
|
|
||
| val clientBaseName: String | ||
| get() = sdkIdStrippingService.clientName() | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The 4 methods above, extended onto SwiftSettings, provide names of Swift symbols that are needed throughout the codebase, but provide them with more meaningful descriptions than they have right now.
|
|
||
| override fun serviceShape(shape: ServiceShape): Symbol { | ||
| val name = sdkId.clientName() | ||
| val name = swiftSettings.clientBaseName |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, clientName isn't actually the client name - it's the client name minus the word Client attached to the end.
The new name makes it more clear that it is the base for the client name.
| let value: Int64 | ||
|
|
||
| public init(value: Int) { | ||
| public init(value: Int64) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This type was incorrect in our Smithy document implementation.
This is a @spi() interface so no harm in changing it.
|
|
||
| public static var documentSchema: Schema { | ||
| Schema(id: .init("smithy.api", "PrimitiveDocument"), type: .document) | ||
| Schema(id: .init("smithy.api", "Document"), type: .document) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This name was incorrect, fixed.
| /// Typically, the Schema contains only modeled info & properties that are relevant to | ||
| /// serialization, transport bindings, and other functions performed by the SDK. | ||
| public final class Schema: Sendable { | ||
| public struct Schema: Sendable { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed this to a struct. There is no advantage to typing it as class.
| id.member | ||
| /// Only access this property on a schema of type `.map`. | ||
| public var value: Schema { | ||
| members[1] // `value` will be the second / last member in a map schema, after `key` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some convenience accessors to Schema above.
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
||
| public protocol Trait { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a protocol for type-safe traits. See TraitCollection below for how it's used.
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
||
| public struct TraitCollection: Sendable, Hashable { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TraitCollection allows for access to traits by trait types which provide safe, easy access to traits. Internally, TraitCollection still stores traits by Shape ID & Node, but Trait-conforming structures are passed in & out of this type.
| // | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a very simple trait. It has an ID, and its node value is not used or important (Smithy uses empty object as placeholder for this type of trait.)
| /// | ||
| /// This list can be expanded as features are added to Smithy/SDK that use them. | ||
| public let allSupportedTraits = Set([ | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This list of traits is the only ones that will be copied into a schema; other traits will be used at codegen time only but since they don't appear in a schema, can't be used at runtime.
| // | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an example of a trait that has content in its node.
Code is provided to safely unwrap the data in the node and provide it to callers in simple properties.
| // | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a "synthetic trait" (i.e. it doesn't exist in the Smithy spec, only in smithy-swift.)
It is used to signify inputs & outputs that were synthesized because either no shape or the Unit shape was targeted by an operation input/output.
| self.name = id.name | ||
| self.member = member | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below, clarified names around string conversions of Shape ID.
| @@ -12,34 +12,44 @@ import struct SmithyCodegenCore.CodeGenerator | |||
| @main | |||
| struct SmithyCodegenCLI: AsyncParsableCommand { | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this type below, added a service param for the Shape ID of the service to be generated, and got rid of a lot of logging we don't need.
Only thing that is now logged by the codegen CLI is what model was generated and the elapsed time.
|
|
||
| // Initialize the properties of self | ||
| self.init(version: astModel.smithy, metadata: astModel.metadata, shapes: shapes) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The functions below are moved elsewhere.
| @@ -56,14 +51,30 @@ extension Model { | |||
| baseMembers["value"] = value | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The conversion below is needed because several AWS services still use the @enum trait to define enums, even though it is deprecated in Smithy 2.0.
| resourceIDs: try astShape.resources?.map { try $0.id } ?? [], | ||
| errorIDs: try astShape.errors?.map { try $0.id } ?? [] | ||
| ) | ||
| return (shapeID, shape) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below, resource shape is now added.
| init(version: String, metadata: Node?, shapes: [ShapeID: Shape]) { | ||
| self.version = version | ||
| self.metadata = metadata | ||
| self.shapes = shapes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shapes need a reference back to the model that contains them so that they can resolve their relations to other shapes.
Because of this relation, Shape must have a mutable property, and therefore can't be Sendable.
This is not a problem because the code generator is not asynchronous.
| } | ||
| return shape | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below are several convenience accessors for getting shapes of a certain type by ID.
| import struct Smithy.ShapeID | ||
|
|
||
| extension Model { | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This model transform is needed to generate code with the same optionality as the existing code generator.
It reproduces the logic in BoxServices.kt.
| @@ -43,31 +43,40 @@ struct SmithyCodeGeneratorPlugin: BuildToolPlugin { | |||
|
|
|||
| let currentWorkingDirectoryFileURL = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file adds a service param, to specify the service in the model that should be built.
|
|
||
| import Foundation | ||
|
|
||
| extension Model { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This extension removes shapes that were deprecated prior to GA.
It duplicates logic in AWSDeprecatedShapeRemover.kt.
| import struct Smithy.TargetsUnitTrait | ||
| import struct Smithy.TraitCollection | ||
|
|
||
| extension Model { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This model transformer:
- adds empty structures to the model wherever an operation has no input/output, or the input/output targets Unit.
- adds the
UsedAsInputTrait&UsedAsInputTraitto inputs/outputs so codegen can reliably tell which shapes are inputs/outputs.
|
|
||
| import struct Smithy.ShapeID | ||
|
|
||
| extension Model { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This model transform deletes all shapes from the model that are not referenced directly or indirectly by the service being built.
| import enum Smithy.Prelude | ||
| import struct Smithy.ShapeID | ||
|
|
||
| extension Model { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This transform creates an associated empty structure for union cases that target Unit. (Our current codegen creates an empty structure for these cases, so we reproduce that behavior here.)
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
||
| // Code generated by SmithyCodegenCLI. DO NOT EDIT! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The contents above are the header that gets put at the top of every Swift source file.
| import struct Smithy.ShapeID | ||
| import enum Smithy.ShapeType | ||
|
|
||
| package struct SchemasCodegen { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This type generates the schemas for the service.
Schemas are generated for:
- the service
- all operations
- any model shape referenced by an operation's input, output, and errors.
Note that both shapes that generate to a custom Swift type (like Smithy structures, unions & enums) and shapes that don't (i.e. a map/list of some shape type, or a String, Integer, etc. with traits applied to it) will get schemas.
|
|
||
| import struct Smithy.ShapeID | ||
|
|
||
| extension Shape { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This extension provides the variable name for a schema.
The schema variable is constructed from the shape's ID, i.e. the schema var for com.amazonaws.ec2#AmazonEC2 would be:
schema__com_amazonaws_ec2__AmazonEC2
These schema vars are internal only, and are only referenced by generated code.
| try errorIDs.compactMap { try model.expectStructureShape(id: $0) } | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below,, there are methods for generating service-related names that closely follow those that are now used in Kotlin codegen.
| } | ||
|
|
||
| override func immediateDescendants(includeInput: Bool, includeOutput: Bool) throws -> Set<Shape> { | ||
| try Set(members) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each shape also gets an immediateDescendants method that is used to access the shapes referenced by a shape.
| new.model = model | ||
| return new | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below, public Shape properties descendants, inputDescendants, and outputDescendants are used to get a set of the shapes referenced by this shape.
These properties use the immediateDescendants() method on every shape subclass to get the exact shapes referenced by a particular shape.
| /// - Parameters: | ||
| /// - includeInput: Whether to include shapes that are associated with input | ||
| /// - includeOutput: Whether to include shapes that are associated with input | ||
| /// - Returns: A set of shapes that this shape refers to. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shape subclasses customize the method below to return their referenced shapes. This is used to build a set of shapes that are used in a services' inputs, outputs, or both.
|
|
||
| import enum Smithy.Node | ||
|
|
||
| extension Node { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renders a Node into generated Swift. The node is generated onto a single line, i.e. it's not "pretty printed".
| import struct Foundation.Data | ||
| import struct Foundation.URL | ||
|
|
||
| class SwiftWriter { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This type reproduces some of the functionality of the Kotlin codegen's SwiftWriter to write structured Swift code.
| // | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
| // | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This trait is used by the model transformer above to remove deprecated shapes prior to codegen.
This trait is not used at runtime, so it does not appear in schemas.
| import struct Smithy.ShapeID | ||
| import protocol Smithy.Trait | ||
| import struct Smithy.TraitError | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This trait is the now-deprecated method of defining a Smithy enum.
This trait is converted to an enum with members when the model is loaded, so it is not referenced at runtime & doesn't appear in schemas.
| import struct Foundation.URL | ||
| import struct Smithy.ShapeID | ||
|
|
||
| public struct CodeGenerator { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The CodeGenerator now takes a service as a param, and actually generates schemas to be compiled.
| self.model = model | ||
| self.symbolProvider = SymbolProvider(service: service, model: model) | ||
| init(serviceID: ShapeID, model: Model) throws { | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below, several transforms (code is up above) are applied to the model after it is loaded, and before the model is used to generate code.
| var id: ShapeID { get } | ||
| } | ||
|
|
||
| extension Array where Element: HasShapeID { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This extension implements sorting with an algorithm equivalent to what is used by Kotlin, to ensure that sorted lists in this code generator match sorted lists in the Kotlin code generator.
| // | ||
|
|
||
| import struct Smithy.ShapeID | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The purpose of the HasShapeID protocol is to extend collections with a custom sorting algorithm. (See the Array extension immediately below.)
| import Smithy | ||
| @testable import SmithyCodegenCore | ||
|
|
||
| class SmithyCodegenCoreTests: XCTestCase { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the code generator runs as part of the build process, the debugger does not attach to it, which makes it hard to debug code generator failures.
This test case runs the code generator in the context of an XCTest so that it may be debugged. The CBOR protocol test model is included here but any model may be copied into this test bundle's Resources and the test below may be modified to build its service.
|
|
||
| /// Target schema is passed as an autoclosure so that schemas with self-referencing targets will not cause | ||
| /// an infinite loop when accessed. | ||
| private let _target: @Sendable () -> Schema? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Schema target property, which references another Schema, has been changed to a closure because some models are self-referencing, and this can cause an infinite loop when creating or accessing schemas.
The Swift @autoclosure feature is used on the initializer to simplify generated code.
sichanyoo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one comment on module name for the plugin
Package.swift
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this isn't part of this PR but I think better name for this might be something like SmithyCodegenPlugin since it's just a build tool plugin that uses CLI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An informal survey of plugins shows that about 50% use Plugin as part of their name:
https://github.com/BarredEwe/awesome-swift-plugins
If you want to add Plugin to the name of ours, I'll add it. (I'd prefer to do it in a separate PR though, for the purpose of managing size/conflicts in this one)
Description of changes
This PR merges to the schema-based serialization epic branch
epic/sbs.This PR includes the entire native-Swift code generator. It generates error- and warning-free code for all AWS services when used in conjunction with its companion aws-sdk-swift PR awslabs/aws-sdk-swift#2094.
Specific changes:
Also:
sdkIdand client naming.Scope
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.