Fix: Add comprehensive configuration setup documentation #14
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
Problem
The WRAVAL application was missing critical configuration documentation, making it impossible for users to set up and run the tool successfully. Users encountered various errors including:
AttributeError: 'Settings' object has no attribute 'ENDPOINT_TYPE'
AWS credential errors
Bedrock model access issues
Missing SageMaker execution roles
Undefined S3 bucket references
The README only provided basic installation instructions (uv pip install . && wraval generate) without any guidance on the extensive AWS configuration required.
Root Cause
Through code analysis, I discovered that WRAVAL heavily depends on AWS services:
Amazon Bedrock for LLM inference
Amazon SageMaker for model deployment and custom endpoints
Amazon S3 for dataset storage
AWS STS for account ID retrieval
However, the configuration file (config/settings.toml) contained placeholder values like {aws_account} that needed to be replaced, and there was no documentation explaining:
Solution
I added comprehensive configuration documentation to address all setup requirements:
Enhanced README.md
Added a detailed "Configuration" section after "Quick start"
Documented three methods for AWS credential configuration
Listed specific AWS permissions required for each service
Provided step-by-step instructions for enabling Bedrock models
Added AWS CLI commands for SageMaker role creation
Included S3 bucket setup instructions
Added troubleshooting section for common configuration errors
Provided verification steps to test the setup
Benefits
Transforms WRAVAL from a broken out-of-the-box experience to a properly documented tool
Provides clear setup path for different use cases (Bedrock-only, SageMaker, local)
Reduces support burden by anticipating and addressing common configuration issues
Makes the tool accessible to users without deep AWS expertise
This fix enables users to successfully configure and run WRAVAL for LLM writing assistant evaluation tasks.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.