Group of paper airplanes, orange one is the first place, can be used leadership/individuality concepts. ( 3d render )
Confusingly, bundler can have more than one version of Rails installed at once. if you had many versions, when you ran rails new, it probably used the default one, which could have been a very old one for you. This often confuses new developers, and especially if you installed Rails years ago and then come back to pick it up again.
To see which versions of Rails you have installed in bundler, use
gem list |grep rails
(here you are grepping, or searching against, the output for the string “rails”; without grep you would see all of your gems)
You’ll see some other gems with the name “rails” in them too, fortunately, all the rails gems are numbered concurrently.
TO install a different version of Rails in your bundler (remember, this just installs the gem code in your bundler’s system ready for use)
gem install rails -v 5.2.4.3
Finally, if you want to force rails new to use a specific version, use underscores before the “new” (that is, between “rails” and “new”)
Common Core JS am pleased to announce a rapid application development tool for Rails: It’s called “Common Core JS.”
The source code is on Github. When you install it, the gem is pulled from RubyGems.
The finished result of the example app we will build today can be seen here. (I recommend you type out each step and only refer to this if you need to.)
It’s based on the idea that you will embrace a common naming convention for your Rails app, will use AJAX rendered-everything (that means all the forms are remote: true), and want a common set of tools for rolling out a quick, dashboard-like app in Rails.
It comes with poor-man’s authentication built-in. Poor man’s auth is safe and works fine, but it isn’t designed for you to grow your entire app on and you should graduate out of if you have any granularity to your access control.
Common Core is a fantastic tool for rapidly building prototypes. It can also be used to create API back-ends for Javascript-heavy apps.
It is a blunt instrument and you should wield it carefully. In particular, you’ll want to remove specific code that it creates because it can create situations where users may have the access they shouldn’t.
Quick Setup
Let’s build the fastest app you can possibly build in Rails.
I’ll call it MyGreatApp, but yours can be named anything of coruse.
rails new MyGreatApp
Once you have the app generated (and even on faster processors Rails new still takes a hot minute or two), go to modify the Gemfile
Add the Gem
gem 'common_core_js'
To your Gemfile.
Then run: bundle install
Setup The Common Core
Then run the Common core install generator (which is implemented as a generator, not a rake task)
bundle exec rails generate common_core:install
Now you’ll create a User objeect, with a migration and model.
do this using. Note that you should not add any fields that will conflict with Devise fields, like email (I’ll add those in the next step).
bundle exec rails generate model User name:string joined_date:date timezone:integer
Add Devise
Add devise to your Gemfile
gem 'devise'
run bundle install, then install devise on the User model:
rails generate devise:install
rails generate devise User
Take a look at the fields on your User database table now. (Here I’m using DB Browser for SQLite)
Notice the fields added name, joined_date, and timezone I’ve put a red box around and the devise fields, including email and encrypted_password, I’ve put a blue box around.
Now you have the most bare-bones Rails app with Devise gem installed. This is great because you get free sign up, login and forgot password fundionality immediately.
Go to
http://127.0.0.1:3000/users/sign_up
Add jQuery
Add jQuery to your package.json via yarn
yarn add jquery
Go to config/webpack/environment.js and add this code in between the existing two lines
Now go ahead and start your rails server with bundle exec rails server
Go to /users/sign_up and create a new user account (be sure to enter the same password twice)
You will then be redirected to the Home page, which is now the Ruby on Rails “Yay you’re on Rails.” That’s fine— go ahead and customize your root URL.
Today’s Example App
I’ll make a super-simple system today where Users have many Events An Event belongs to a Format, which in our fictional world is only two choices: Zoom or Outdoor.
Events have a name, a starting datetime, and an ending datetime, and a “publicize” date on which should be before the starting time.
The two datetime fields (starting_at and ending_at) and the date field (publicize_on) can be empty (nil), but if set they are enforced to be: starting_at must be before ending_at. publicize_on must be before starting_at.
An Event will belong_to a Format (class & table). Formats will have a name field and only two records: Zoom and Outdoor. (We can assume they will be id 1 and id 2 in the formats table.)
All Events must belong_to a format, and when we create or edit an Event we can switch its format, but the format cannot be blank (null).
We already have the User object, and we also already have all the login, logout forgot password, and more provided by Devise.
We want the users to log-in and go to a dashboard of their own events.
They should be able to create, edit, & delete their own events with only the validations discussed above.
They should not be able to edit or create events belonging to other users, even by hacking the query parameters.
Finally, the user should be able to edit their own name and timezone, but not any other user’s name or timezone.
Because we want to name our routes from the perspective of the context-interaction, we’ll namespace the controller to Dashboard:: in our Ruby code and /dashboard in the URL. The controller will be at controllers/dashboard and the views will be at views/dashboard
Make the models
Since you already made the User model in the devise setup, let’s go ahead and create the Events and Formats tables.
Then open up the migration file and ed the description line, adding a larger than 256 limit, like 400
Next run bundle exec rake db:migrate to create the table.
Before I go further, let’s edit our models just a bit.
Open models/user.rb and add has_many :events
also add validates_presence_of :name
class User < ApplicationRecord
# Include default devise modules. Others available are:
# :confirmable, :lockable, :timeoutable, :trackable and :omniauthable
devise :database_authenticatable, :registerable,
:recoverable, :rememberable, :validatable
has_many :events
validates_presence_of :name
end
Likewise, on the Event object, defined models/event.rb, you’ll need to add the reflexive relationship for belongs_to :user and belongs_to :format
class Event < ApplicationRecord
belongs_to :user
belongs_to :format
end
Now make a formats table
bundle exec rails generate model Format name:string
class CreateFormats < ActiveRecord::Migration[6.0]
def change
create_table :formats do |t|
t.string :name
t.timestamps
end
Format.create(name: "Outdoor")
Format.create(name: "Zoom")
end
end
Then run the migration itself, which will now make the table & the two format records.
In order to show dates, we use a localized date display: the date & time is always shown to the user in their own date. To do this you have a few choices: (1) You can save the timezone to the user’s table, and let them set it for themselves, (2) You can show everybody the server’s date
Option #1 – Store timezone On the User object
We already took care of this by adding timezone to our User object.
Option #2 – Use the Server’s Timezone
If the auth object (current_user) does not respond to timezone, the Rails “system clock” will be used. The system clock’s timezone as set by the Rails app is used. This is often the timezone of the headquarter’s of the company that owns the application. (That is, if you do not know the user’s context, you simply use your own company’s context instead.)
Make the Controller, Views & Specs
Next we’re going to do the thing. We’ll make two controllers: Dashboard::EventsController for editing events and Dashboard::UsersController for the user editing their own name.
When passing the model name, pass it in the singular form
Here I’ve provided the namespace= with a value of dashboard. You will this is important to how our code comes out.
Here is the heart & soul of the common core: 5 .js.erb files, 5 .haml files, a controller, and a controller spec. Also along for the ride came controllers/dashboard/base_controller.rb as well as views/dashboard/_errors.haml, and layouts/_flash_notices.haml
Take a peak through all the generated code now.
Pay particular attention to _line.haml and _form.haml. You will note _form.haml conveniently is used for both the new/create actions and also for the update action, unifying the layout of your record across CRUD.
You can use both to customize your app quickly and easily.
One more quick step, add this to your routes.rbfile.
Make sure to nest the :events route within the :dashboard namespace, as shown here. If you aren’t familiar with namespacing in Rails, check out this blog post on my other blog, The Rails Coach.
Rails.application.routes.draw do
devise_for :users
namespace :dashboard do
resources :events
end
end
Start your server with
bundle exec rails server
If the Common Core finds null for timezone on your User object, it will default to either (1) whatever is set for your Rails app in either application.rb or an environment file, or, if don’t have this set (2) the clock timezone of the server that is running your Ruby application.
It’s generally a good idea to set your Rails app timezone to the same timezone of your company’s headquarters, and then don’t change it, because that way if your server happens to move from one timezone to another (for example, you migrate from a server on the East coast to the West coast), your app will be unaffected. If your company changes timezones, you can either leave the Rails app as-is or change it, but be sure to note any place where your default timezone comes through.
config.time_zone = 'Eastern Time (US & Canada)'
Done!
We can now do all of these fancy thigns.
Create an event. Leave name or format blank, get an error.
When you create a new event, you must give it a name and Format. Notice how if you don’t, the Rails-side logic will return the form shown with the erroneous fields marked in red.
Edit an Event
Your model-level validations — name and format as required — are enforced in the update action as well.
Deleting An Event
Adding Validation
Add this to your Event class in app/models/event.rb
validate :start_at_before_end_at, if: -> {!start_at.nil? && !end_at.nil?}
def start_at_before_end_at
if end_at < start_at
errors.add(:start_at, "can't be after end at")
errors.add(:end_at, "can't be before start at")
end
end
*Validation magic*
Finally, to add validation on all the date fields, here’s our completed Event model
class Event < ApplicationRecord
belongs_to :user
belongs_to :format
validates_presence_of :name
validate :start_at_before_end_at, if: -> {!start_at.nil? && !end_at.nil?}
validate :promote_on_before_start_at, if: -> {!promote_on.nil? && !start_at.nil?}
def start_at_before_end_at
if end_at < start_at
errors.add(:start_at, "can't be after end at")
errors.add(:end_at, "can't be before start at")
end
end
def promote_on_before_start_at
if start_at < promote_on
errors.add(:promote_on, "can't be after start at")
end
end
end
Account Dashboard
Next we’re going to create the very simplest of Account Dashboards. Remember that Devise already handles log in, log out, and forgot password, meaning most of the heavy lifting of user authentication has been taken care of.
In this simple app, we want an Account dashboard that let’s us edit only two fields: name and timezone.
First let’s add the route to routes.rb
Rails.application.routes.draw do
devise_for :users
# For details on the DSL available within this file, see https://guides.rubyonrails.org/routing.html
namespace :dashboard do
resources :events
resources :users
end
end
Next let’s generate some scaffold
rails generate common_core:scaffold User namespace=dashboard
We now instantaly have a very basic dashboard for the User to edit their own details
The final finishing touch here will be to make the Timezone into a drop-down.
Remember, make your models first: Add limits and defaults to your database fields by modifying your migrations. Then add the relasionships between the tables using standard ActiveRercord has_many, belongs_to, and has_one.
Then build the common core scaffolding & customize the views and controllers it produces.
With these powerful tools, you can build a dashboard-like app in minutes, complete with simple interface buttons that let your users accomplish most of what they’ll need. The philosophy is that you will want this dashboard as you initially introduce people to your product. The main logic of your application will likely live more in the models, service objects, and domain layer (business logic) parts of your Rails app. For this reason, you are encouraged to customize the files only lightly. (Add some verbiage or change the CSS to customize the look & feel.)
The code you build with common core is cheap and disposable. It is not very modern, but it gets the job done. It is just “good enough” to launch a sophisticated app on, but it isn’t good enough to impress your users with a really good UI.
For that, you’ll want to throw away the front-end code and replace it with a modern JS UI like React, Vue, Ember, or Angular.
By that time, the Common core will have already helped you build (1) a prototype, (2) your business logic, and (3) possibly even some controllers you can re-use (for example, remove the format.js responses and replace them with format.json to change the controllers into a JSON-responding API controller.)
Fat Models are unavoidable when you are learning Rails. That’s because when they teach you MVC— that is model, view, controller— they tell you: Don’t put your business logic into your view. And then they tell you don’t put your business logic into your controllers. So where do you put your business logic?
Unfortunately, you’re sometimes taught or encouraged to put them into the models. Thus, the models get very large. And when I say large I mean hundreds and hundreds of lines of code.
This is not something you should do. Instead, you should understand your models as the bare-bones operations that read or write to your database only. Then you should learn and use the other patterns I’m going to talk about today to build what we call a business domain.
This lesson can be considered an Introduction to Domain Driven Design, which will discuss explicitly later on.
Today we’ll look is meant when we say “prefer composition over inheritance.”
Perhaps you have heard the term: “prefer composition over inheritance.” But what does it really mean understand what that means?
Well, give me a few minutes of your time and I’m going to take you in this lesson into a broad overview of 9 different patterns you can use to break out of the MVC cycle.
Fat Models Skinny Controllers
In the early days of Rails people said “fat models, skinny controllers.”
In the really earlydays when the apps themselves were much smaller. People only actually said “fat models skinny controllers” for a short time because then the models got unmanageably large.
Rails was pioneering in the basic concept of splitting Models (persistency logic), Views (view logic) and Controllers (display and response logic).
Notice that the descriptions I used were very intentional and specific. Most people teach the “M” in MVC as “business logic” and the “V” as “view logic or display logic.” Then they try to describe what a controller is. What’s a controller, exactly?
Hmm. Well, for Rails, in the context of a GET request, it:
1) typically prepares a query in anticipation of view being rendered (like setting up a query, may be based on search criteria)
2) Handles authentication, authorization and access control if appropriate. (Arguably, the implemented class Contrtoller doesn’t typically handle this, but the Controller object, as a concept, is where this is responsible)
3) Interacts with the Rails request layer for params, headers, etc.
That’s pretty much it. In other languages, Controllers are sometimes called ViewControllers. You shouldn’t think a Controller as really anything more than a set of code that deals with how to present stuff and respond to things.
That’s why I call it “display logic and response logic”
Breaking out of the MVC Antipattern
If you’re new to Rails or programming, the first thing you learn is MVC (model, view, controller). Then, you should unlearn it.
In domain-driven design, which really I won’t even get to until pattern #6 below, we separate out Rails models from the business domain. That is, we think the ‘M’ in the traditional Rails sense as only what is necessary to read to and write from the database. This way, we still get to use all of the good parts of ActiveRecord while working towards what is fundamentally a domain layer that doesn’t live in any of the models, views, or controllers.
That, in a nutshell, is the bird’s eye view of what I’m going to talk about today.
We might do this by creating simple methods that will instantiate these service objects when needed.
Service objects have the unique property of coming into memory when needed and then disappearing when your code finishes running on each request.
The name of the game here is to compose the other objects of multiple smaller objects that will keep the business logic in the business domain layer.
Today is a broad overview of some answers to the question “How do I deal with fat models?” We’re gonna look at
1. Classic Delegation
2. Inheritance (bad)
Then I’ll talk about the classic composition patterns, including:
3. Composition with Modules (& Using Helpers in Rails View) and Composition with Rails Concerns
(And I’ll cover what a Rails cocern is and how it adds to normal Ruby modules.)
Then I’ll cover patterns found in larger Rails apps:
4. Service Objects
Then we have the aforementioned Introduction to domain-driven design, and other “out-of-the-box” architecture concepts, like:
5. Domain Context Interaction as per James Coplien in Lean Architecture
A look the Trailblazer gem – a complete domain driven design system that separates business logic from persistence logic.
At this point, you will understand what I mean when I say we are “separating business logic from persistence logic.”
Then we’ll take a quick look at 3 more patterns:
6. The “pub-sub” pattern: Publish-Subscribe and we’ll have a quick look at a Gem called Whisper
7. The “Interactor” Pattern: Interactor and ActiveInteraction, and U Case gems – to perform complex business operations together
8. The “mutation” pattern: Mutations using a gem called (unsurprisingly) Mutations
It’s a big lesson so be sure to take your time with it. As well, I will return to some of the higher-level concepts in future courses to get more hands-on with these ideas.
Again, these nine types of abstraction are presented here without strong bias (except #2 which is generally considered bad). You should learn them all and learn how to reach for the right abstraction at the right time. I’ll have another lesson in this course about too early abstractions. For now, I’m presenting this as a very high-level overview so that new and intermediate developers understand the landscape of possibilities.
Well, in truth the sky is the limit! But in the real world, these patterns as outlined here are based decades of industry -wide working and re-working of what are commonly known as “design patterns.”
Although the classic “design patterns” in programming languages that predate Ruby are more numerous than these 9, I’ve chosen these 9 to focus on because they are a great way to teach someone who has only learned about MVC what the alternatives are.
1. Classic Delegation
Ok so the first topic is called delegation.
Delegation is simply we move logic out of a class and delegate it to another class. It is a common pattern and one of the first ones you learn. Consider for example a Thing object that can export itself to XML, JSON, or CSV
class Thing
# no delegation — all export methods are here
def as_csv
#...
end
def as_xml
#...
end
def as_json
#...
end
end
As our model gets “fat,” we’ll want to move those specialized methods out of it. Delegation is our first strategy.
Examine our new Thing object, and another object called Converter::Thing
class Thing
# delegate to a converter, passing self as the object
def converter
Converter::Thing.new(self)
end
end
class Converter::Thing
attr_reader :thing
def initialize(thing)
@thing = thing
end
def as_xml
# ...
end
def as_json
# ...
end
def as_csv
# ...
end
end
Here we’re simply moving the methods out of the original object and into another object. It is important to note it is a “simple” move.
In other words, we’re just fundamentally moving code around and splitting it out into new objects. We aren’t actually changing anything fundamental about how we think about functionality and objects — we’re just changing where we think about functionality and objects.
As the complexity of your app grows, the more basic solutions (like this one, “delegation”) will only be building blocks. This is fundamentally abstraction.
Let’s move on to pattern #2.
2. Inheritance
Classical inheritance is what they teach you when learning computer science as, well, classical inheritance. Its name offically means “inherticance using classes,” but as a tounge-in-cheek joke the double entdre is now that it is “classic” as in outdated. The easiest way to describe classical inheritance is this way:
models/animal.rb
class Animal def blood_temperture raise "superclass must implement" end end
models/mammal.rb
class Mammal < Animal
def blood_temperature
"warm"
end
end
models/bear.rb
class Bear < Mammal
end
Ok so what have we achieved? We can ask questions about the animal and, for example, if we want to implement a different species or genus, we would know where to implement things like: Does the animal have hair? Does it have skin or fur? How does it reproduce?
Species of animals lend themselves particularly well to the teaching about classical inheritance. It’s a great use case for teaching, but unfortunately, classical inheritance isn’t often as useful or practical in the real world.
Some may think that categorization and graphing of this complex hierarchy is the stuff of OO developers. In some ways it is and in some ways, it isn’t. In some ways, an obsession with over-categorization is what gets OO a bad rap.
Think about a developer who learns a pattern— like inheritance— and then everything they implement is done with inheritance. It’s like they keep repeating the same solution for every problem. Why? Because our brains operate in the mechanism that our brains were just operating.
That is— once you start doing something one way, you are cognitively biased to repeat the same solution to every new problem that same way. This doesn’t actually make sense and you shouldn’t be that developer.
Because more often than categorization and graphing of a complex hierarchy you as the developer are considering how and why external users— that is, an end user— come into play with the data.
Experienced OO developers say “prefer composition over inheritance” so let’s a take a look at composition.
Rails Autoloading is what happens in the background that most developers don’t even think about. The Rails Autoload is something that is important for you to know about. Rails will automatically load certain files your Rails app application when it boots up.
When you first learn Rails you learn about its basic structure:
App/
Models/
Views/
controllers/
We call this “MVC” for “model-view-controller.”
Then you get a little more experience and you learn about domain architecture and services. Perhaps you make a new Rails app and you notice an empty folder
Controllers/concerns/
And you look at this folder and you think, “What do I put there?”
Sometimes you must use the ‘require’ keyword to load files—- as you see in specs often— but sometimes you don’t. When do you use ‘require’ and when don’t you?
Rails has an important piece of magic that’s significant for every new Rails dev to know: the autoload with same-name, or correctly named, files. That is, Rails will autoload files for you if you meet the three conditions below.
Before letting you in on the secret, lets review the concepts of underscores, titlecase and namespaces.
Underscore
An underscore is a character on your keyboard (probably to the right of the 0 on your number line). On most keyboards you press SHIFT-(-). Press SHIFT and the dash key to produce an underscore, which looks like this: _ . The underscore appears where the underline would just below where a letter would be, but denotes that there is no letter or number. Do not confuse the underscore with the dash, which appears in the middle of the line. On your keyboard, you don’t hold SHIFT to create a dash character, and you do hold shift to create an underscore.
What does the underscore mean? The underscore means the space of “no character” when in fact it does represent a character in the computer’s implementation. Because loading from operating systems is finicky with spaces, we don’t use spaces in our file names. Instead we use underscores.
Here are some examples of Ruby files with underscores in them. You always use underscores when naming the actual files on your disk. (Even though your operating system might actually allow you to put a space in a file name, don’t do this when naming Ruby files.)
payment_adjustor.rb
line_item.rb
subscriptions_controller.rb
Titlecase:
Titlecase refers to a special syntax used by Ruby and some other programming languages to represent objects. In short, if you have several words, you make each of the first letters of each word uppercase and then you string the words together. So, if you have a phrase like “something wicked this way comes” to make it title case would look like:
SomethingWickedThisWayComes
Note how we capitalize (uppercase) the first letter of each word but the other letters are not capitalized. Also notice how there are no spaces or underscore separating the words. We call this titlecase and it is how we write all objects in Ruby. (Do not confuse titlecase with camelcase, often found in other programming languages, which is very similar to titlecase but the very first letter of the whole object name is lower case. In our example, the camelcase of what we have above would be somethingWicketThisWayComes with lowercase s in something)
Namespace or Namespacing
A namespace, or namespacing (the act of applying a namespace), is really just when we want to group things together and give them a common element in their name, typically prepended (or put before). In Rails, we have a special way to namespace (explained below) using subfolders. But, if, for example, we wanted all parts of our application involving subscription management to be under the name ‘subscriptions,’ we might (actually, as I will explain below, we can and should) move our Ruby objects into a subfolder called ‘subscriptions’ and also namespace our objects by renaming them Subscriptions:: plus what we first called them.
A namespace is really just a way for programmers to group ideas together (and truly, it is probably a relatively poor way to think about object abstraction. The examples provided in this post provide examples of what I would call good use of namespacing. If you find a large system that is highly namespaced or is overly convoluted, you may have some antipatterns. This is because developers overused namespacing, or relied on it when they should have created more extensible encapsulations. This advanced subject is beyond the scope fo this post. Just keep in mind that namespacing, while a quick way to clean up small messes, should not be overused when what you really need are better abstractions.)
OK, now the 3 magic rules of the Rails autoload:
Your Rails files will be loaded by Rails if (and only if):
1) they are in the folders models, views, controllers, or test, or spec, and
2) The name of the file matches, using the correct underscore-to-titlecase conversion when writing your Ruby class names. Put another way, your file names must be in underscores (and lowercase), and your Ruby classes that each file defines must match exactly (in titlecase) the name of its file.
3) When a file appears in a subfolder, which is encouraged, you must namespace it –– the folder’s name (or namespace) is titlecased and prepended to the class name with ::
Those 3 rules are key. If you get them, you will master the the Rails autoload.
Rails has all kinds of reasons why you would want to create sub folders INSIDE of other folders. For example, let’s say you have a polymorphic set of Products. That is, you have a Product object with a type column that Rails will use to instantiate the objects.
A Basic Polymorphic Example
class Product < ApplicationRecord
end
class Shirt < Product
end
class Purse < Product
end
In the type column, you will see the name of the subclass directly. I will start by making a new Purse called “Little Clutch”
In the example above, the three Ruby files (Product, Shirt, Purse) all appear in the root of the models/ folder
Polymorphic Autoload Example
Now let’s say we have many products and we want to move them into a sub folder called products/
We’ll want to do this with a little change to our class objects. Here’s what our new files & folders will look like:
Notice we did some things here: One, the superclassProduct (was product.rb) has been renamed to products/base.rb
The class names now have namespaces— that’s because they are in the products/ folder. You must always match the folder name to the namespace, but you will titleize the namespace and keep the folder names lowercase.
So our new Purse becomes
class Producs::Purse < Products::Base
end
And now, our Shirt becomes
class Products::Shirt < Products::Base
end
One last thing, in our Products::Base we’ll need to set the table name:
class Products::Base < ApplicationRecord self.table_name = 'products' end
Now, Rails knows how to magically load your files. Never do anything else– like put non-namespaced files into subfolders. That is a Very Bad Thing and Very Bad Things will happen to you if you do.
Remember, Rails will magically load all of your objects in models/ controllers/ but only when the name of the object matches exactly— including the namespace to it titleized form— with the name of your Ruby class. THE RAILS COACH
So you’re inside of a Rake task and you invoke an object in your app. In a classic blunder, your Rails app tells you
Uninitialized Constant: Thingy
What?! “Where’s Thingy!” you proclaim. It can’t be. You check the spelling. You double-check Rails’ object-file name to make sure it’s the same name as the object. (This way, Rails will autoload your file — there’s a post dedicated to that coming up in this series.) In other words, it checks out.
The Rake task: the Rails’ developer’s best friend. A utility knife of utility knives— the Rake task is where you get stuff done. But unfortunately not when it can’t find your Rails objects.
You can do all kinds of things in Rake tasks.
Background jobs are the primary function of Rake tasks.
For instance, queues are run in Rake tasks.
In other words, lots of Rails operates on Rake tasks. They are the underpinning of most SAAS (Software-As-A-Service) back-ends that are written in Ruby on Rails.
Remember you must always specify to the task the symbol :environment . That’s the key, like so:
task my_task: :environment do
# do some stuff here
end
If you fail to do this, Rails will load your task, but won’t load your environment, so it won’t have access to your models or application objects. You’ll get funny errors, like Uninitialized Constant and the like.
Always remember, load your environment in Rake tasks.
Ahh, the elusive N+1 query. Perhaps you’ve heard of him from your engineering pit, cursed under the breath of engineers or beshrewed out load when such an engineer encounters an endpoint (or web request) that’s slow, slow, slow. The N+1 query is very often the culprit.
See, in web development, we like things fast, fast, fast. That is, we want our webpages to load really fast. (And typically that means 200ms or faster).
Let’s say you have a database of teachers and students. Your teachers have many students.
In Rails, this might be expressed as a Rails class Teacher that has_many :students.
Let’s load the students in the HAML via the association.
When viewed in the browser, it might look something like so:
Finally, if you examine the Rails console, you’ll see Rails makes 3 queries: 1) to load up the teacher, 2) to count the number of students (this is because I explicitly asked for the count of students) , and 3) it the loads all of the associated students in the final query.
Let me propose making this slightly more complex to demonstrate how to create an N+1 query. Remember, N+1 queries are not something we want. I’m going to show you how to create an N+1 query and then how to fix it.
Students have many ReportCards. In this simple example, we’re going to be querying for the last report card for each student.
We have some HAML that looks like this:
Here I want to show an unordered list of the name of the students and the last report card for each student.
When we run this, we’ll see this in our browser:
In our console, we will see:
Take a look at that nasty N+1 query. See how repetitive it is? For these seven records, it might seem fast now. But as your app grows, your database server will strain and struggle to keep up.
Won’t Fly With N+1 Queries
While it might look trivial now, those “ReportCard Load” lines are telling you something important: Your app won’t scale with N+1 queries.
As a result of the object relational mapping (ORM) we are doing with Active Record, we will do something called pre-fetching. That’s when we tell Rails to fetch all of the ReportCards we will need to display quickly in one single query.
As you can see from the query, without telling Rails to side-load the related data, Rails must loop through the Student results as it fetches each of the “last” report cards for the student, causing it to query 8 times, 7 times for the report cards and once for the student (thus, “n + 1”). In fact, it does it the other way around, but we call the phenomenon “n+1” and not “1+n”.
As the developer you must know N+1 Queries
Using an ActiveRecord helper method called:includes(:____) we will chain this instruction to load the associated objects onto our ActiveRecord query.
(Do not confuse this with the ruby command include [singular], which is to load a Ruby file other than the one you are executing—they are unrelated: include is Ruby and looks like this
include ‘byebug’
.includes, on the other hand (notice the dot ‘.’) comes after an Active-Relation (also known as an “AREL”), like so:
@teacher.students.includes(:report_cards)
This directive tells Active Record to smartly fetch the student data in “as few queries as possible.” You will note I’ve put this in quotation marks because Rails actually does some advanced stuff under the hood to choose one of two loading strategies: JOINing the table in the original query or pre-fetching when the data is needed in a single, composite query for all of the associated Students.
Notice in this example that Rails makes precisely two queries to the database (and remember, each query has overhead, or a time that it takes for the webserver to communicate with the database server and for the database server to process the request): once for the students and then again for all of the report cards.
SELECT `students`.* FROM `students` WHERE `students`.`teacher_id` = 4 [["teacher_id", 4]]
ReportCard Load (0.6ms)SELECT `report_cards`.* FROM `report_cards` WHERE `report_cards`.`student_id` IN (22, 23, 24, 25, 26, 27, 28)
Boom! Your N+1 query is gone. Take special note that Rails has queried for the students associated with teacher_id4 and found that they all have ids of 22, 23, 24, 25, 26, 27, and 28. Since we only need those report cards records for the seven associated students, Rails has concatenated the ids of the students in the second query, to form the WHERE clause. As in: WHERE … IN (22, 23, 24, 25, 26, 27, 28)
This technique is called pre-fetching or eager loading. We’re telling Rails that we’ll need the associated report_cards on each student when we are looping through the students.
Prefetching fixes N+1 queries
The key to fixing your N+1 queries is to realize that Active Record relations are instantiated in one place and then invoked “just-in-time,” that is, typically when the view is rendered.
Now, finally, here there one more thing to note: AR has three different methods for loading your associations: includes, eager_load, andpreload. A fourth method, joins, allows you to join a table in your SQL without necessarily loading the associated records.
Includes will load all of related the records referenced as arguments in the includes method. Typically includes will load the records like you see above (in a separate query using WHERE … IN (…) syntax)
However, if you have a where or order clause that references a relationship, includes will instead us a left outer join.
Rails has two ways of dealing with N+1s: 1) “big join” and 2) “separate queries.”
.includes — uses either strategy #1 or #2 depending on whether or not there is a where or order clause that references a relationship. .eager_load always uses strategy #1 and .preload uses #2.
I recommend that you always start with .includes. When your app gets large and your queries are slow, then you will be forced to optimize, which means measure and tweak your queries to make them faster. During this process is when you will experiment with .eager_load vs. .preload and determine on a case-by-case basis which one is most optimized for you.
I hope you’ve enjoyed this brief introduction to N+1 queries.
What does that mean? Upon inspection you realize that someone has typed into your website the all-too-favorite Millenial communication tool: the emoji. ?✨
MySQL needs a little massaging. For steps 1 & 2 in Rails, create a new database migration and use execute to run these commands directly onto your database.
Make sure you actually alter the database’s character set
ALTER DATABASE <DATABASE NAME> CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
Where <DATABASE NAME>is your database. in Rails, because we have different database names for different environments, try this inside of your migration…
execute "ALTER DATABASE #{ActiveRecord::Base.connection.current_database} CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;"
Make sure you ALTER the TABLE AND FIELD where you want to insert the character set
ALTER TABLE CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; ALTER TABLE modify TEXT charset utf8mb4;
(If you made a Rails migration, you can do this in the same migration using another execute.)
IMPORTANT: I have assumed above that your field is of type TEXT. If it is not, change your field’s type definition before running this. If you don’t, you’ll convert your existing field to a TEXT field even if you didn’t want to.
Restart your MySQL instance (DO NOT FORGET)
In Rails, you must also specify the encoding & collation in the MySQL adapter which you will find in your config/database.yml file (where you have specified mysql2 as the adapter).
It’s a plug-and-play Rails engine that you install into your Ruby on Rails application with just three simple steps (see the README). You can then immediately pick up your visitors’:
IP address
Ad campaign where they came from
the browser they are using
In my next version, I’ll add support for http referrer and more too. Give it a try today.
It’s an great little utility project that will surely have appeal to many websites and developers.
Visitors come to your site every day. Along with their visits, four key pieces of information come along for the ride:
— IP address
— browser name (which lets you infer operating system and sometimes device)
— UTM parameters (if they clicked from another site or an ad campaign)
— HTTP referrer, (which shows if they clicked directly from another site to your site, even when no UTMs are set)
Universal Track Manager, a play-on-words that shares an acronym with “UTM Parameters,” is a plug & play gem to automatically scoop up this information and stash it into your database. You can think of it like a built-in Google Analytics (without the fancy dashboard).
With a tiny bit of trickery, support for Viewport size too is possible (width X height of the users’s window), which can let you determine if the user is on a desktop or mobile browser.
Today I’ve bumped the version up to 0.4. (I realize I made a 10x version change but this Gem is nearing its ‘version 1.0’ release so I am anticipating that when it is feature complete.)
This is the second version I’m dubbing as ‘public beta.’ Although this is production-quality code, it should be used with caution until it is no longer in BETA status. Please give it a try on your Rails projects today. With an easy 3-step installation into any Rails 4+ app and you can sit back and sweep up tracking info on your visitors.
*MOST* of the core functionality now works! This version 0.4 implements fully support for timestamping your visits, the user’s IP address, browser, and UTMs.
Support for HTTP referrer and more coming soon! Kindly submit feedback via Github.
This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.