Categories
Programming

How to Get Emojis in MySQL (fixing Incorrect String Value)

So your Rails application with a MySQL database was humming along and all of a sudden it hit this error:

Mysql2::Error: Incorrect string value: '\xC5\x99\xC3\xA1k

What does that mean? Upon inspection you realize that someone has typed into your website the all-too-favorite Millenial communication tool: the emoji. 💛✨

MySQL needs a little massaging. For steps 1 & 2 in Rails, create a new database migration and use execute to run these commands directly onto your database.

  1. Make sure you actually alter the database’s character set

ALTER DATABASE <DATABASE NAME> CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

Where <DATABASE NAME> is your database. in Rails, because we have different database names for different environments, try this inside of your migration…

execute "ALTER DATABASE #{ActiveRecord::Base.connection.current_database} CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;"

  1. Make sure you ALTER the TABLE AND FIELD where you want to insert the character set

ALTER TABLE CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE modify TEXT charset utf8mb4;

(If you made a Rails migration, you can do this in the same migration using another execute.)

IMPORTANT: I have assumed above that your field is of type TEXT. If it is not, change your field’s type definition before running this. If you don’t, you’ll convert your existing field to a TEXT field even if you didn’t want to.

  1. Restart your MySQL instance (DO NOT FORGET)
  2. In Rails, you must also specify the encoding & collation in the MySQL adapter which you will find in your config/database.yml file (where you have specified mysql2 as the adapter).

For example:

production:
encoding: utf8mb4
collation: utf8mb4_general_ci

or if you want it in your default configs

default: &default
encoding: utf8mb4
collation: utf8mb4_general_ci

See also…

https://stackoverflow.com/questions/22464011/mysql2error-incorrect-string-value#22464749

https://stackoverflow.com/questions/16350310/mysql-mysql2error-incorrect-string-value/18498210#18498210

https://stackoverflow.com/questions/16350310/mysql-mysql2error-incorrect-string-value/16934647#16934647

Categories
Programming

Halfway to One Point Oh: UTM Version 0.5

Today I’ve finished version 0.5 of my new Gem, Universal Track Manager.

It’s a plug-and-play Rails engine that you install into your Ruby on Rails application with just three simple steps (see the README). You can then immediately pick up your visitors’:

IP address
Ad campaign where they came from
the browser they are using

In my next version, I’ll add support for http referrer and more too. Give it a try today.

If you like the Gem, please ‘star’ it on Github or download it from RubyGems (you do that just by running bundle install). Also, consider supporting it today with a small contribution today through the Github sponsors program. Sponsors levels start at just $1/month.

Categories
Programming

Universal Track Manager Version 0.4

Today I’m announcing Version 0.4 of my new Gem: Universal Track Manager.

It’s an great little utility project that will surely have appeal to many websites and developers.

Visitors come to your site every day. Along with their visits, four key pieces of information come along for the ride:

— IP address
— browser name (which lets you infer operating system and sometimes device)
— UTM parameters (if they clicked from another site or an ad campaign)
— HTTP referrer, (which shows if they clicked directly from another site to your site, even when no UTMs are set)

Universal Track Manager, a play-on-words that shares an acronym with “UTM Parameters,” is a plug & play gem to automatically scoop up this information and stash it into your database. You can think of it like a built-in Google Analytics (without the fancy dashboard).

With a tiny bit of trickery, support for Viewport size too is possible (width X height of the users’s window), which can let you determine if the user is on a desktop or mobile browser.

Today I’ve bumped the version up to 0.4. (I realize I made a 10x version change but this Gem is nearing its ‘version 1.0’ release so I am anticipating that when it is feature complete.)

This is the second version I’m dubbing as ‘public beta.’ Although this is production-quality code, it should be used with caution until it is no longer in BETA status. Please give it a try on your Rails projects today. With an easy 3-step installation into any Rails 4+ app and you can sit back and sweep up tracking info on your visitors.

*MOST* of the core functionality now works! This version 0.4 implements fully support for timestamping your visits, the user’s IP address, browser, and UTMs.

Support for HTTP referrer and more coming soon! Kindly submit feedback via Github.

Links:

Github Repo

Rubygems page

Categories
Programming

Announcing Nonschema Migrations — NOW FOR RAILS 6!

Nonschema migrations version 5.0.0: RubyGems Page, GitHub page

Now compatible with Rails 6, nonschema_migrations is the best way to separate your schema changes from your data changes.

Want to run no-downtime deploys with data-only migrations? No problem.

Install it in your Rails app today to see what a difference data migrations can make.

Shout out to Mikls Fazekas who hails all the way from Gyenesdis, Hungary for the pull request for this release!

Categories
Programming

Jason FB’s 10 Magical Ruby Developer Tools

1. deivid-rodriguez/byebug
Byebug is a fantastic debugger available for Ruby 2 (and presumably above). Drop

gem ‘byebug’

into your Rails app Gemfile and bundle install. In either your test run or your development run, write

byebug

on a single line of your app and voila. When you hit that line, you will drop into the debugger.

If you’re not developing a Rails app, you can include ‘byebug’ at the top of your Ruby file.

Full docs here.

2. pretty print (pp)

Pretty print is one of my favorite introspection weapons to help see variables more clearly.

pretty print, which you write as pp, prints out your object with each attribute on its own line. Take for example this Spree::Country object, shown here on the Rails console without pretty print

2.4.6 :010 > x
=> #<Spree::Country id: 232, iso_name: “UNITED STATES”, iso: “US”, iso3: “USA”, name: “United States”, numcode: 840, states_required: true, updated_at: “2019-05-19 17:16:07”, zipcode_required: true>

Now, with pretty print, the same object is conveniently displayed with each attribute as its own line. This is invaluably helpful when you have deep nesting of objects.

2.4.6 :009 > pp x
#<Spree::Country:0x00007fd8507ea358
id: 232,
iso_name: “UNITED STATES”,
iso: “US”,
iso3: “USA”,
name: “United States”,
numcode: 840,
states_required: true,
updated_at: Sun, 19 May 2019 17:16:07 UTC +00:00,
zipcode_required: true>

3. puts, .to_s, and inspect

OK, so we get a 3-in-1 here: When you call puts on an object, .to_s will be called and then output to your screen. So you should make your objects have a .to_s that is human readable, possibly even for use in, say, a drop-down menu or label. 

def class Person
 attr_accessor :first_name, :last_name

 def to_s
   “#{first_name} #{last_name}”
 end
end

inspect, on the other hand, is specifically for developers. In this method, you would write out as much information as you the developer (or next developer) want to see, including the keys (ids) of your objects if those will be helpful:

def class Person
 attr_accessor :first_name, :last_name

 def inspect
   “Person id: #{id} – first: #{first_name}; last: #{last_name}”
 end
end

Your objects should have both .to_s and .inspect on them, and you can try these universally named Ruby methods on other people’s objects to examine them. A well-formed codebase implements them or has helpful output for both of these.

4. .to_yaml
Pretty print’s cousin is the .to_yaml method, which will take your object and convert it into yaml. Take for example this arbitrary object, which you will notice contains a :ghi key that has a nested object as its value:

2.4.6 :023 > x= {abc: 1, def: 4, ghi: {ye: 6, nm: 3}}
=> {:abc=>1, :def=>4, :ghi=>{:ye=>6, :nm=>3}}
2.4.6 :024 > x
=> {:abc=>1, :def=>4, :ghi=>{:ye=>6, :nm=>3}}

.to_yaml on its own will produce a string that will output with newline characters, like so:

2.4.6 :025 > x.to_yaml
=> “-\n:abc: 1\n:def: 4\n:ghi:\n :ye: 6\n :nm: 3\n”

To make this more useful, try puts along with .to_yaml

2.4.6 :026 > puts x.to_yaml

:abc: 1
:def: 4
:ghi:
 :ye: 6
 :nm: 3

5. x.method(:_____).source_location

(where :_____ is name of the method — as a symbol — you are trying to search for)

OK, so the ultimate secret weapon of Ruby debugging is this little-known method that will magically — and I mean magically — tell you where a method was defined. That’s right, I mean the actual line number itself.

2.2.5 :002 > a.method(:hello)
=> #<Method: Apple(Thingy)#hello>
2.2.5 :003 > a.method(:hello).source_location
=> [“/Users/jason/Projects/nokogiri-playground/app/models/thingy.rb”, 2]

Look, ma, take a peek into my hard drive and you would find that the hello method is actually defined on the file at the full path /Users/jason/Projects/nokogiri-playground/app/models/thingy.rb on line 2.

Like magic it works for Rails and Gem code too, and is invaluable when you are ready to dive into the APIs you are working with.

6. x.methods
By default this method will return a list of all of the methods on an object. Watch out because you’ll get all the methods on the ancestor chain too.

In older versions of Ruby, you could use this method to examine the instance methods that were defined on this class only (excluding the ancestors), but unfortunately this no longer works.

If you pass this method false, like so:

x.methods(false)

…things get more interesting: then you get only the class methods defined on this object’s class itself. (Remember in Ruby class methods are defined with self.)

7. brunofacca/active-record-query-trace

An excellent gem that’s still a non-optional workhorse in my development practice – especially when debugging a legacy codebase. This gem will display for ALL of your SQL queries where in your Ruby or Gem code the active record update commands are coming from.

Follow the instructions in the Gem to create an initialize file and set it up. My only tip here that adds to the docs is that you’ll want to set the number of lines:

ActiveRecordQueryTrace.lines = 10

I find that when debugging a problem in my own Rails app I want this set to a lower number (like 5 or 10) and when debugging a problem in Gem code or in Rails I need this at a much higher number (like 50 or 100).

8. flyerhzm/bullet

Understanding N+1 queries is a significant litmus test that sets amateurs from the professionals. Bullet is like a magic bullet – literally named so – for finding your N+1 queries.

Bullet is a great gem that you should install in either development or test, not in production. Often because it does add overhead to your speed, I install it but leave it configured so that it is turned off by default and any developer on the team who needs it can turn it on.

You CAN and SHOULD turn it on periodically too, to examine where your app is producing N+1 queries.

Here’s the catch with Bullet: Remember that Active Relation objects are created as chains of conditions before they get translated and executed as SQL. That’s why when you do this you must carefully consider

query = Country.where(name: “United States”)

When you do this in your Rails console, you will see it run the SQL right away.

2.4.6 :039 > Spree::Country.where(name: “United States”)
 Spree::Country Load (0.7ms) SELECT `spree_countries`.* FROM `spree_countries` WHERE `spree_countries`.`name` = ‘United States’ LIMIT 11

It only does this because of the ‘print’ effect the Rails console has on your objects. If you string another .where onto this object, when translated into SQL, ActiveRecord will combine the queries:

query = Spree::Country.where(name: “United States”); query.where(iso: “US”);
 Spree::Country Load (1.0ms) SELECT `spree_countries`.* FROM `spree_countries` WHERE `spree_countries`.`name` = ‘United States’ AND `spree_countries`.`iso` = ‘US’ LIMIT 11

The reason this is important is that to understand where your N+1 queries are coming from you need to understand when you are creating your Active Relation objects and when they are invoked. They are not the same place, although on the Rails console it makes you think it is one and the same. When you grok this, you will see why Active Record’s side loading (which loads a related set of objects in a single optimized query, taking the number of queries from N+1 to 1+1=2) is both efficient and can be tricky to work with, especially with objects that have many relationships.

Don’t be fooled: Side-loading is nearly always more efficient than N+1 queries. 

Bullet tells you where those pesky N+1 queries are invoked, but not where you are creating them. What you then need to do is trace your code (manually) to figure out where the queries are being created, which hopefully should be near in the code to where they are being invoked ( but in the case of complex filtering logic might not be).

Here you need to add the appropriate .joins(:____) to your code anywhere between when the objects are set up and when they are invoked by Active Record. Note that you’ll only want to join in those additional tables if they are actually used. If not, you don’t need the over head.

You’ll know you’ve solved your N+1 queries because you won’t see them output in your Rails log, and they’ll disappear from the Bullet code.

9. better_errors

Since Rails 4 adopted a near identical default, this used to be more interesting. For a Rails 3 app it can bring your error crash console up to Rails 4 standards.

10. Introspect, Introspect, Introspect but remember Ruby’s last-line quirk

Always look at what you’re doing. Drop into your debugger, look at your variables, clone & freeze them, look for race conditions, look for flip-floppers, watch out for your own confirmation bias. Remember that when in the debugger or on the Rails console and you type a SINGLE VARIABLE and HIT RETURN, the console will interpret that action as-if you had called .inspect

2.2.5 :007 > a
=> #<Apple id: nil, created_at: nil, updated_at: nil>
2.2.5 :008 > a.inspect
=> “#<Apple id: nil, created_at: nil, updated_at: nil>”

In some cases, the act of inspection actually changes the object (like in the case of an Active Relation, in which case it invokes the query), so keep that in mind (we might call this the “observer effect” in software development.)

Categories
Programming

Amazon RDS on Heroku Still Works

Although Heroku has put a lot of attention into their own Postgres-based datbase architecture, Heroku Data, this article still works for rebels who want to use Amazon RDS on Heroku:

https://devcenter.heroku.com/articles/amazon-rds

Note that I can’t seem to get “sslca” to work as a config option in my database.yml file (in Rails), I get this I get an error after deploying, upon connection

SSL_CTX_set_default_verify_paths failed

This happens when configuring by database.yml, which I shouldn’t be doing anyway.

Nonetheless, the configuration by DATABASE_URL (which is preferred for security anyway), does actually work.

Although it continues to work in 2018, this may not be good long-term strategy as my last conversations with Heroku support led me to conclude that their failover policies for catastrophic failures aren’t set up for an RDS- backed solution unless you go into Heroku Private Spaces, which you need to be an enterprise client for.

Categories
Programming

How to Install an Older Version of Rails

My post from four years ago with this same title is Google’s #2 result for a search for “How to Install an Older Version of Rails,” so I thought I’d make a repost with some updates.

You can install an older version of rails using this syntax:

rails _5.1.6_ new my-test-app

(Where “5.1.6” is the version you want and “my-test-app” is the name of your new rails app.)

Yes, you actually type those underscores around the version number.

If you get …

can’t find gem railties (>= 0.a) (Gem::GemNotFoundException)

You probably don’t have the right version of Rails itself installed as a gem; try this:

rails install -v=5.1.6

or…

gem install rails -v 5.1.6

Then try again with…

rails _5.1.6_ new my-test-app
Categories
Programming

Announcing Nondestructive Migrations Version 1.3 for Rails 5.1

I am pleased to announce Version 1.3 of my gem nondestructive_migrations. With this update, nondestructive_migrations is ready for Rails 5.1.

The Gem’s Github page can be found here:

https://github.com/jasonfb/nondestructive_migrations and you’ll find a sample app for Rails 5.1 here.

Version 1.3 is officially pushed to Rubygems as of this morning.

I know I’m slightly behind the Rails release schedule itself. (We just had Rails 5.2 officially released last month.) Thanks to some help from Mikls Fazekas from Hungary hopefully this gem should be ready for Rails 5.2 soon. Watch this space for updates!

Categories
Programming

Custom Error Handling in Rails

1. Create an ErrorsController in app/controllers

class ErrorsController < ApplicationController
 def not_found
  respond_to do |format|
   format.html { render template: “errors/not_found”,
              layout: “layouts/application”,
              status: 404 }
  end
 end

 def server_error
  respond_to do |format|
   format.html { render template: “errors/server_error”,
              layout: “layouts/application”,
              status: 500 }
  end
 end
end

2. Add these to your routes.rb file

match “/404”, :to => “errors#not_found”, :via => :all
match “/500”, :to => “errors#internal_server_error”, :via => :all

3. Add this to your application.rb file

config.exceptions_app = self.routes

4. Delete public/404.html, public/422.html, and public/500.html

5. Remember while developing you should change this to false in config/environments/development.rb

config.consider_all_requests_local = false

If you fail to perform this step, Rails will show you full stacktraces instead of your error page.

Sample app can be found here

Categories
Programming

The Great Rails Cache Lie

Today I’ll take a moment to expound on how web development has changed over the last two decades. Long ago, when we started back in the 90s, connections were slow and web pages didn’t change much.

In the design of the internet itself is something you should be familiar with if you are reading this post: browser caching.