Skip to content

Commit babc51e

Browse files
authored
Merge pull request #40 from Invoca/TECH-4996_support_collation_and_character_set
Tech 4996 support collation and character set
2 parents 278305c + ca5dc5b commit babc51e

File tree

17 files changed

+482
-77
lines changed

17 files changed

+482
-77
lines changed

CHANGELOG.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ Inspired by [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
44

55
Note: this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## [0.5.0] - Unreleased
8+
### Added
9+
- Added support for configuring the character set and collation for MySQL databases
10+
at the global, table, and field level
11+
712
## [0.4.2] - 2020-12-05
813
### Fixed
914
- Generalize the fix below to sqlite || Rails 4.
@@ -15,7 +20,7 @@ Note: this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0
1520
## [0.4.0] - 2020-11-20
1621
### Added
1722
- Fields may be declared with `serialize: true` (any value with a valid `.to_yaml` stored as YAML),
18-
or `serialize: <serializeable-class>`, where `<serializeable-class>`
23+
or `serialize: <serializeable-class>`, where `<serializeable-class>`
1924
may be `Array` (`Array` stored as YAML) or `Hash` (`Hash` stored as YAML) or `JSON` (any value with a valid `.to_json`, stored as JSON)
2025
or any custom serializable class.
2126
This invokes `ActiveSupport`'s `serialize` macro for that field, passing the serializable class, if given.
@@ -68,6 +73,7 @@ using the appropriate Rails configuration attributes.
6873
### Added
6974
- Initial version from https://github.com/Invoca/hobo_fields v4.1.0.
7075

76+
[0.5.0]: https://github.com/Invoca/declare_schema/compare/v0.4.2...v0.5.0
7177
[0.4.2]: https://github.com/Invoca/declare_schema/compare/v0.4.1...v0.4.2
7278
[0.4.1]: https://github.com/Invoca/declare_schema/compare/v0.4.0...v0.4.1
7379
[0.4.0]: https://github.com/Invoca/declare_schema/compare/v0.3.1...v0.4.0

Gemfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ gem 'bundler', '< 2'
1414
gem "climate_control", '~> 0.2'
1515
gem 'pry'
1616
gem 'pry-byebug'
17+
gem 'mysql2'
1718
gem 'rails', '~> 5.2', '>= 5.2.4.3'
1819
gem 'responders'
1920
gem 'rspec'

Gemfile.lock

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
PATH
22
remote: .
33
specs:
4-
declare_schema (0.4.2)
4+
declare_schema (0.5.0.pre.3)
55
rails (>= 4.2)
66

77
GEM
@@ -85,6 +85,7 @@ GEM
8585
mini_portile2 (2.4.0)
8686
minitest (5.14.2)
8787
msgpack (1.3.3)
88+
mysql2 (0.5.3)
8889
nio4r (2.5.4)
8990
nokogiri (1.10.10)
9091
mini_portile2 (~> 2.4.0)
@@ -187,6 +188,7 @@ DEPENDENCIES
187188
climate_control (~> 0.2)
188189
declare_schema!
189190
listen
191+
mysql2
190192
pry
191193
pry-byebug
192194
rails (~> 5.2, >= 5.2.4.3)

README.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,72 @@ DeclareSchema::Migration::Migrator.before_generating_migration do
7070
end
7171
```
7272

73+
## Declaring Character Set and Collation
74+
_Note: This feature currently only works for MySQL database configurations._
75+
76+
MySQL originally supported UTF-8 in the range of 1-3 bytes (`mb3` or "multi-byte 3")
77+
which covered the full set of Unicode code points at the time: U+0000 - U+FFFF.
78+
But later, Unicode was extended beyond U+FFFF to make room for emojis, and with that
79+
UTF-8 require 1-4 bytes (`mb4` or "multi-byte 4"). With this addition, there has
80+
come a need to dynamically define the character set and collation for individual
81+
tables and columns in the database. With `declare_schema` this can be configured
82+
at three separate levels
83+
84+
### Global Configuration
85+
The character set and collation for all tables and fields can be set at the global level
86+
using the `Generators::DeclareSchema::Migrator.default_charset=` and
87+
`Generators::DeclareSchema::Migrator.default_collation=` configuration methods.
88+
89+
For example, adding the following to your `config/initializers` directory will
90+
turn all tables into `utf8mb4` supporting tables:
91+
92+
**declare_schema.rb**
93+
```ruby
94+
# frozen_string_literal: true
95+
96+
Generators::DeclareSchema::Migration::Migrator.default_charset = "utf8mb4"
97+
Generators::DeclareSchema::Migration::Migrator.default_collation = "utf8mb4_general"
98+
```
99+
100+
### Table Configuration
101+
In order to configure a table's default character set and collation, the `charset` and
102+
`collation` arguments can be added to the `fields` block.
103+
104+
For example, if you have a comments model that needs `utf8mb4` support, it would look
105+
like the following:
106+
107+
**app/models/comment.rb**
108+
```ruby
109+
# frozen_string_literal: true
110+
111+
class Comment < ActiveRecord::Base
112+
fields charset: "utf8mb4", collation: "utf8mb4_general" do
113+
subject :string, limit: 255
114+
content :text, limit: 0xffff_ffff
115+
end
116+
end
117+
```
118+
119+
### Field Configuration
120+
If you're looking to only change the character set and collation for a single field
121+
in the table, simply set the `charset` and `collation` configuration options on the
122+
field definition itself.
123+
124+
For example, if you only want to support `utf8mb4` for the content of a comment, it would
125+
look like the following:
126+
127+
**app/models/comment.rb**
128+
```ruby
129+
# frozen_string_literal: true
130+
131+
class Comment < ActiveRecord::Base
132+
fields do
133+
subject :string, limit: 255
134+
context :text, limit: 0xffff_ffff, charset: "utf8mb4", collation: "utf8mb4_general"
135+
end
136+
end
137+
```
138+
73139
## Installing
74140

75141
Install the `DeclareSchema` gem directly:

gemfiles/rails_4.gemfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ gem "bundler", "< 2"
77
gem "climate_control", "~> 0.2"
88
gem "pry"
99
gem "pry-byebug"
10+
gem "mysql2"
1011
gem "rails", "~> 4.2"
1112
gem "responders"
1213
gem "rspec"

gemfiles/rails_5.gemfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ gem "bundler", "< 2"
77
gem "climate_control", "~> 0.2"
88
gem "pry"
99
gem "pry-byebug"
10+
gem "mysql2"
1011
gem "rails", "~> 5.2"
1112
gem "responders"
1213
gem "rspec"

gemfiles/rails_6.gemfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ gem "bundler", "< 2"
77
gem "climate_control", "~> 0.2"
88
gem "pry"
99
gem "pry-byebug"
10+
gem "mysql2"
1011
gem "rails", "~> 6.0"
1112
gem "responders"
1213
gem "rspec"

lib/declare_schema.rb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,5 +41,6 @@ def to_class(type)
4141
require 'declare_schema/model/field_spec'
4242
require 'declare_schema/model/index_definition'
4343
require 'declare_schema/model/foreign_key_definition'
44+
require 'declare_schema/model/table_options_definition'
4445

4546
require 'declare_schema/railtie' if defined?(Rails)

lib/declare_schema/extensions/active_record/fields_declaration.rb

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,13 @@
66

77
module DeclareSchema
88
module FieldsDsl
9-
def fields(&block)
9+
def fields(table_options = {}, &block)
1010
# Any model that calls 'fields' gets DeclareSchema::Model behavior
1111
DeclareSchema::Model.mix_in(self)
1212

1313
# @include_in_migration = false #||= options.fetch(:include_in_migration, true); options.delete(:include_in_migration)
1414
@include_in_migration = true
15+
@table_options = table_options
1516

1617
if block
1718
dsl = DeclareSchema::FieldDeclarationDsl.new(self, null: false)

lib/declare_schema/model.rb

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ def mix_in(base)
3030
inheriting_cattr_reader ignore_indexes: []
3131
inheriting_cattr_reader constraint_specs: []
3232

33+
# table_options holds optional configuration for the create_table statement
34+
# supported options include :charset and :collation
35+
inheriting_cattr_reader table_options: HashWithIndifferentAccess.new
36+
3337
# eval avoids the ruby 1.9.2 "super from singleton method ..." error
3438

3539
eval %(

0 commit comments

Comments
 (0)