Advanced ORM between postgreSQL and Crystal
#v0.9
I'm pleased to announce the version 0.9 of Crystal Clear ORM ! This version is probably the biggest version released since Clear is born.
Under the hood, it simplifies a lot of code, push testing to another level with a great amount of new specs.
On the feature part, it add full support for serializing to and from json, with mass assignment secure check, big decimal type, PostgreSQL view management at migration, new callbacks methods, support for postgres interval object and so on...
Finally, the code has been tweaked to be compatible with release of Crystal 1.0.
With that in mind, Clear starts to mature, with only CLI, polymorphic relations and model inheritance still lacking.
Note of warning: some changes will break your code. However everything can be fixed in matter of minutes (hopefully)
Special thanks to all contributors of this version:
@007lva @anykeyh @GabFitzgerald @dukeraphaelng @mamantoha @watzon @yujiri8
(hopefully I did not forget someone)
Clear::SQL::ConnectionPool now returns DB::Connection instead of DB::Database (fix #177)
Clear::Migration::Direction is now an enum instead of a struct.
where and having clauses use splat and named tuple always. This is breaking change.
where("a = ?", [1])
Now you can do much more easy:
where("a = ?", 1)
Same apply for the named parameters version:
# Instead of
where("a = :a", { a: 1 } )
# Do
where("a = :a", a: 1)
json, with mass_assignment security
(thanks @dukeraphaelng and Caspian Baska for this awesome work!)Collection#add_operation has been renamed to Collection#append_operation
Clear::SQL.after_commit methodRegister a callback function which will be fired once when SQL COMMIT
operation is called
This can be used for example to send email, or perform others tasks when you want to be sure the data is secured in the database.
transaction do
@user = User.find(1)
@user.subscribe!
Clear::SQL.after_commit{ Email.deliver(ConfirmationMail.new(@user)) }
end
In case the transaction fail and eventually rollback, the code won't be called.
Same method exists now on the model level, using before and after hooks:
class User
include Clear::Model
after(:commit){ |mdl| WelcomeEmail.new(mdl.as(User)).deliver_now }
end
Note: before(:commit) and after(:commit) are both called after the transaction has been commited.
Before hook always call before after hook.
Clear::SQL.with_savepoint("a") do
Clear::SQL.with_savepoint("b") do
Clear::SQL.rollback("a") # < Exit to "a"
end
puts "This won't be called"
end
puts "This will be called"
Clear.json_serializable_converter(CustomType)
This macro help setting a converter transparently for any CustomType.
Your CustomType must be JSON::Serializable, and the database column
must be of type jsonb, json or text.
class Color
include JSON::Serializable
@[JSON::Field]; property r: Int8
@[JSON::Field]; property g: Int8
@[JSON::Field]; property b: Int8
@[JSON::Field]; property a: Int8
end
Clear.json_serializable_converter(Color)
# Now you can use Color in your models:
class MyModel
include Clear::Model
column color : Color
end
jsonb().contains?(...) methodThis allow usage of Postgres ? operator over jsonb fields:
# SELECT * FROM actors WHERE "jsonb_column"->'movies' ? 'Top Gun' LIMIT 1;
Actor.query.where{ var("jsonb_column").jsonb("movies").contains?("Top Gun") }.first!.name # << Tom Cruise
SelectQuery#reverse_order_by methodA convenient method to reverse all the order by clauses,
turning each ASC to DESC direction, and each NULLS FIRST to NULLS LAST
Void to Nil
first and last on collection object does not change the collection anymore (previously would add limit/offset and change order_by clauses)This version fix a lot of small things:
or_where featureorder_by methodClear::Interval typeThis type is related to the type Clear::Interval of PostgreSQL. It stores month, days and microseconds and can be used
with Time (Postgres' datetime) by adding or substracting it.
Usage in Expression engine:
interval = Clear::Interval.new(months: 1, days: 1)
MyModel.query.where{ created_at - interval > updated_at }.each do |model|
# ...
end
It might be used as column definition, and added / removed to crystal Time object
class MyModel
include Clear::Model
column i : Clear::Interval
end
puts "Expected time: #{Time.now + MyModel.first!.i}"
Clear::TimeInDay columns type, which stands for the time object in PostgreSQL.Usage as stand alone:
time = Clear::TimeInDay.parse("12:33")
puts time.hour # 12
puts time.minutes # 0
Time.now.at(time) # Today at 12:33:00
time.to_s # 12:33:00
time.to_s(false) # don't show seconds => 12:33
time = time + 2.minutes #12:35
As with Interval, you might wanna use it as a column (use underlying time type in PostgreSQL):
class MyModel
include Clear::Model
column i : Clear::TimeInDay
end
v0.6 should have shipped polymorphic relations, spec rework and improvement in documentation. That's a lot of work (honestly the biggest improvement since v0) and since already a lot of stuff have been integrated, I think it's better to ship now and prepare it for the next release.
Since few weeks I'm using Clear in a full-time project, so I can see and correct many bugs. Clear should now be more stable in term of compilation and should not crash the compiler (which happened in some borderline cases).
Release of a guide and documentation to use Clear: https://clear.gitbook.io/project/
json = JSON.parse(%({"first_name": "John", "last_name": "Doe", "tags": ["customer", "medical"] }))
User.new(json)
Add of pluck and pluck_col methods to retrieve one or multiple column in a Tuple,
which are super super fast and convenient!
Add Clear.with_cli method to allow to use the CLI in your project. Check the documentation !
Additional comments in the source code
SelectQuery now inherits from Enumerable(Hash(String, Clear::SQL::Any))
Add optional block on Enum definition. This allow you to define custom methods for the enum:
Clear.enum ClientType, "company", "non_profit", "personnal" do
def pay_vat?
self == Personnal
end
end
Add ? support in raw method:
a = 1
b = 1000
c = 2
where{ raw("generate_series(?, ?, ?)", a, b, c) }
[EXPERIMENTAL] Add << operation on collection which comes from has_many and has_many through:
[EXPERIMENTAL] Add unlink method on collection which comes from has_many through:
[EXPERIMENTAL] Add possibility to create model from JSON:
table.column instead of table.${type} (remove the method missing method); this was causing headache
in some case where the syntax wasn't exactly followed, as the error output from the compiler was unclear.with_serial_pkey to primary_key; refactoring of the macro-code allowing to add other type of keys.text, int and bigint primary key, with still the uuid, serial and bigserial primary keys available.Clear::Model::InvalidModelError to Clear::Model::InvalidError and Clear::Model::ReadOnlyError to
Clear::Model::ReadOnly to simplify as those classes are already in the Clear::Model namespaceModel#set methods has been transfered to Model#reset, and Model#set now change the status of the column to dirty. (see #81)Clear wasn't fiber-proof since it lacks of connection pool system. It's now fixed, the connection pooling is done completely transparently without any boilerplate on your application side.
Each fiber may require a specific connection; then the connection is binded to the fiber. In the case of transaction
and with_savepoint, the connection is kept until the end of the block happens.
On the case of normal execution or cursors, we store the connection until the execution is done.
The connection pool is using Channel so in case of pool shortage, the fiber requesting the connection is put in waiting state.
This is a huge step forward:
spawn-based server and other event machine system.lateral join feature:
Model.query.left_join("other_model", lateral: true){ model.id == other_model.model_id }
import over collection, to be able to insert multiple objects with one query:
user_array = 10.times.map{ |x| User.new(name: "user#{x}") }
Model.import(user_array)
ON CONFLICT both in Insert and Model#save
u = User.new(id: 1, first_name: "me")
u.save! &.on_conflict("(id)").do_update(&.set("first_name = excluded.first_name").where { model_users.id == excluded.id })
Model#import explained above can use the same syntax to resolve conflict.
This will helps to use Clear for import, CSV and batch processing.to_json supports to model. Please note that some core lib and shards pg objects got
extended to allow this support:
full: true. For example: User.query.first!.to_json # => {"id":1, "first_name":"Louis", "last_name": "XVI"}
User.query.select("id, first_name").first!.to_json # => {"id":1, "first_name":"Louis"}
User.query.select("id, first_name").first!.to_json(full: true) # => {"id":1, "first_name":"Louis", "last_name": null}
insert method on InsertQuery to values, making API more elegant.var in Expression engine has been changed and is now different from raw:
var provide simple way to construct [schema].table.field structure,
with escaped table, field and schema keywords.raw works as usual, printing the raw string fragment to you condition. where{ var("a.b") == 1 } # Wrong now! => WHERE "a.b" = 1
# Must be changed by:
where{ var("a", "b") == 1 } # OR
where{ raw("a.b") }
var function, please use raw instead from now.Clear.seed(&block)
Clear.seed goes in pair with bin/clear migrate seed which will call the seed blocks.has_many through: and selectDISTINCT ON feature.Model#save on read only model do not throw exception anymore but return false (save! still throw error)with_serial_pkey use Int32 (type :serial) and Int64 (type :longserial) pkey instead of UInt32 and UInt64. This would prevent issue with default belongs_to behavior and simplify static number assignation.