Colby.so

Searching and filtering with Turbo 8

2024-03-27T00:00:00+00:00

Today we are going to build on top of an article I published last week to expand our knowledge of Turbo 8’s page refreshes and, as a treat, we will write some beautiful Ruby code along the way.

In this article we are going to add searching and filtering to an existing sortable table interface. We will use Turbo 8 page refreshing, a tiny Stimulus controller, and regular old Rails code to build a solution that maintains the user’s page state and scroll position while they search and filter the results shown in a table, no Turbo Frames or Streams required.

When we are finished, the project will look like this:

This article assumes you are comfortable writing Rails code. You do not need any prior experience with Turbo or Stimulus to follow along. Reading the previous article first will help clarify some of the concepts that we move over quickly in this article. You really should read the previous article before you read this one, but I’m not the boss, live life on the edge if you want.

When you’re ready, dive in.

Setup

This article picks up where the previous article left off and you can pick up your code where you left it at the end of that article or clone this GitHub repo as a starting point.

If you choose to clone the repo instead of working from the previous project, you will need to work from the finished-project branch of that repo. After you clone the repo to your local machine, start in your terminal:

cd turbo_8_refresh_sorting
git fetch origin finished-project
git checkout finished-project
bin/setup

If you did not follow along with the previous article, you will also need to populate some data in the Players table. Run this simple script from the Rails console:

['Dallas Mavericks', 'Denver Nuggets', 'Phoenix Suns', 'San Antonio Spurs', 'Boston Celtics', 'Miami Heat', 'New Orleans Pelicans'].each do |name|
  Team.create(name: name)
end

100.times do
  Player.create(name: Faker::Name.name, team: Team.find(Team.pluck(:id).sample), seasons: rand(25))
end

Finally, whether you followed along with the previous article or not, head to app/views/players/index.html.erb and update it to fix an issue that causes the table to resize in a jarring way when the list of players changes:

<%= turbo_refreshes_with method: :morph, scroll: :preserve %>
<div class="max-w-7xl mx-auto mt-12 w-full">
  <div class="shadow rounded border-b border-gray-200">
    <table class="bg-white min-w-full">
      <thead class="bg-gray-800 text-white">
        <tr>
          <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
            <%= show_sort_indicator_for("name") %>
            <%= sort_link(column: "name", label: "Name") %>
          </th>
          <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
            <%= show_sort_indicator_for("teams.name") %>
            <%= sort_link(column: "teams.name", label: "Team") %>
          </th>
          <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
            <%= show_sort_indicator_for("seasons") %>
            <%= sort_link(column: "seasons", label: "Seasons") %>
          </th>
        </tr>
      </thead>

      <tbody class="text-gray-700">
        <% @players.each do |player| %>
          <tr>
            <td class="text-left py-3 px-6 w-1/2"><%= player.name %></td>
            <td class="text-left py-3 px-6 w-1/4"><%= player.team.name %></td>
            <td class="text-left py-3 px-6 w-1/4"><%= player.seasons %></td>
          </tr>
        <% end %>
      </tbody>
    </table>
  </div>
</div>

Here we set fixed widths on the table columns and made the table wider by default. These changes are not core to the function of the project, but they make it nicer to look at as we work.

If all has gone well you can start the rails server with bin/dev and go to http://localhost:3000/players and see that you have a list of players that can be sorted by clicking on the column headers.

Searching

Our goal is to be able to search players by name and to filter the list of players by their Team while maintaining any existing sorting applied to the table.

We will take it one step at a time, starting by adding a simple search form to the players index that queries the Players table by player name. Add the search form to the players index at app/views/players/index.html.erb, just above the <table> element:

<div class="flex justify-end mb-1">
  <%= form_with url: players_path, method: :get do |form| %>
    <%= form.text_field :name, placeholder: "Search by name", value: params[:name], class: "border border-blue-500 rounded p-2" %>
    <%= form.button "Search", class: "bg-blue-500 text-white p-2 rounded-sm" %>
  <% end %>
</div>

This is a standard Rails form. Type in a name, click search, and a GET request fires off to /players. We will make it fancier soon, but first let’s make it work. This form submission will just reload the index page without changing the list of players because we are not doing anything with the form submission on the server. Let’s change that now.

Go to app/controllers/players/players_controller.rb and update the index action:

def index
  @players = Player.includes(:team)
  @players = @players.where("players.name like ?", "%#{params[:name]}%") if params[:name].present?
  @players = @players.order(params.slice('column', 'direction').values.join(' '))
end

This code is clunky, but it gets the job done. It will be beautiful Ruby code before we finish this article, but it works and working is good enough for now.

Try another search on the players index page and you will see that the list of players filters down to match your search term. We are not using Turbo yet, but we have a working search form. Before we add page refreshing, clicking to submit a search form is very 2010s, results should update as the user types, right?

Let’s solve that next with a small Stimulus controller.

The Stimulus controller’s job will be to listen to changes to the form inputs (for now just the name field) and submit the form in response to those changes. This allows us to build a type-as-you-search experience instead of asking the user to click submit and see what comes back.

Get started by generating a new Stimulus controller from your terminal:

bin/rails g stimulus autosubmit
bin/rails stimulus:manifest:update

And then fill in the new controller at app/javascript/controllers/autosubmit_controller.js:

import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  submit() {
    clearTimeout(this.timeout)
    this.timeout = setTimeout(() => {
      this.element.requestSubmit()
    }, 250)
  }
}

The important line in this controller is this.element.requestSubmit(). this.element is a reference to the DOM element that the controller is attached to. In this project, this.element will be the <form> we added in the last section. Calling requestSubmit on the form element, naturally, submits it.

A very important note here is that we are calling requestSubmit on the form element, not submit. submit is an entirely different thing that submits the form but does not trigger the form’s submit event. Turbo listens for the form’s submit event to process the request, and using submit will completely bypass Turbo. There is a long, worthwhile discussion on the Hotwire forums about requestSubmit and submit if you want to get into the weeds.

The clearTimeout and setTimeout calls are a simple implementation of a debounce function so that requestSubmit does not fire every single time the user types a key. Instead, this.element.requestSubmit() will not fire until 250ms after the user stops typing. This slight delay avoids hammering the server with a request for every keypress without adding a noticeable delay for the user. If debouncing is a new concept for you, this is a great article on the topic.

Now that we have our Stimulus controller, we need to attach it to the search form. Update the form in app/views/players/index.html.erb:

<%= form_with url: players_path, method: :get, data: { controller: "autosubmit" } do |form| %>
  <%= form.text_field :name, placeholder: "Search by name", value: params[:name], class: "border border-blue-500 rounded p-2", data: { action: "autosubmit#submit" } %>
<% end %>

Here we added the data-controller attribute to the <form> element and added data-action="autosubmit#submit" to the text_field. In Stimulus, input elements assume that the action to fire on is an input event, so you can leave the event out of the action descriptor. data-action="input->autosubmit#submit" would work fine, if you really like being explicit.

Since the form submits as the user types, we also removed the unnecessary submit button. This means that users with JavaScript disabled will not be able to submit a form, but I am going to assume you trust me enough to enable JavaScript as you follow along with this tutorial.

Now that we have the Stimulus controller connected to the form, refresh the players index page and try out the new search experience.

Oh. That’s not what we wanted. Type-as-you-search is not very useful if we lose focus each time the search form submits!

What’s going on here? Right now, the search form is a standard Rails form. Each time we submit the form, a full page load occurs. The page content is replaced, scroll position is lost, and ephemeral state (like the user’s current focus) is lost.

This is where Turbo 8’s morphing page refreshes come in. Instead of a normal page turn, we can use morphing to replace only the content that’s changed, leaving page state and unchanged content untouched. Just like in the previous article, we will write very little code to make this happen.

Head to app/views/players/index.html.erb and update the form again:

<%= form_with url: players_path, method: :get, data: { turbo_action: "replace", controller: "autosubmit", turbo_permanent: "" } do |form| %>
  <%= form.text_field :name,
    placeholder: "Search by name",
    class: "border border-blue-500 rounded p-2",
    data: {
      action: "autosubmit#submit"
    }
  %>
<% end %>

Here we added two new data attributes to the <form>. data-turbo-action="replace" tells Turbo to treat these form submissions as replace visits instead of advance visits. This change enables morphing refresh behavior for our form — as discussed in detail in the previous article, without setting data-turbo-action on the visit, page refreshing will not be used.

As mentioned in the previous article, the use of replace instead of the default advance visit means that submitting the search form will not add a new entry into the user’s browser history, so they will not be able to use the back button to move between different searches. When a form is submitted as the user types, not preserving history for each search is probably the best call, but it is important to note that if preserving each form submission in the history is important, morphing page refreshes are not the right tool.

The other new data attribute is data-turbo-permanent="". We use this attribute to inform Turbo that the element (and the element’s children) should not be morphed on refresh visits. Form elements without data-turbo-permanent will lose focus and be reset to their original (empty) state each time new content is morphed in during a refresh.

With both data attributes in place (and the turbo-refresh meta tags we added in the previous article), we have everything we need for our live search form to work as expected. Refresh the page and see that as you type, the list of players is updated without losing focus.

Great work so far!

We have written all of the Turbo 8 code we need to write. Two data attributes, two meta tags, and some standard Rails code to deliver search-as-you-type is a delight. In previous versions of Turbo we could achieve the same experience, but we needed to use Turbo Frames, which bring more complexity and a learning curve. We can stay closer to standard Rails, simplify our code, and still deliver a great user experience with page refreshes.

Let’s continue this article by improving the existing search and sort functionality and adding the ability to filter the table by team.

Search and sort together

As you tested out the code on your machine you may have noticed a problem with our search and sort functionality. You cannot use both at the same time.

When you apply a search, the sort order is removed, and when you apply a sort order, any search is cleared away. This happens because we are using params to query and sort @players in the PlayersController index action, like this:

def index
  @players = Player.includes(:team)
  @players = @players.where("players.name LIKE ?", "%{params[:name]}%") if params[:name].present?
  @players.order(params.slice('column', 'direction').values.join(' '))
end

The search form does not submit with the sort params, and the links to sort in the table headers do not include params from the search form and params are not permanent, if they are not present in the current request, they do not exist.

There are a few ways to handle this situation. We could try to keep the current params in sync, making sure the search form always included existing sort params in the form submission and pushing the current state of the form params into each sort link, but that is a lot of work.

Instead, we will move from using params to query for Players to storing search and sort values in the session. Storing search and filter data in the session allows us to persist that data across requests, and is a great solution to support more complex sorting and filtering UIs like ours.

To move from params to session, first update app/controllers/players_controller.rb like this:

def index
  session['filters'] ||= {}
  session['filters'].merge!(filter_params)

  @players = Player.includes(:team)
  @players = @players.where("players.name LIKE ?", "%#{session['filters']['name']}%") if session['filters']['name'].present?
  @players = @players.order(session['filters'].slice('column', 'direction').values.join(' '))
end

The queries are the same, but instead of using params for values, we use session['filters']. Before querying for the players we set session['filters'] based on any new inbound params with merge!(filter_params).

filter_params is a method that we have not defined yet. Define it next, staying in app/controllers/players_controller.rb:

private

# Snip existing private methods

def filter_params
  params.permit(:name, :column, :direction)
end

We use filter_params to whitelist the attributes that can be used for filtering, any param not in this list will be discarded instead of being stored in the session.

With these changes in place, sorting and searching at the same time will work, but the sort indicators will not work as expected. This is because the indicators rely on params, not session values. Head to app/helpers/players_helper.rb to fix that:

module PlayersHelper  
  def sort_link(column:, label:)
    direction = column == session['filters']['column'] ? next_direction : 'asc'
    link_to(label, players_path(column: column, direction: direction), data: { turbo_action: 'replace' })
  end

  def next_direction
    session['filters']['direction'] == 'asc' ? 'desc' : 'asc'
  end

  def sort_indicator
    tag.span(class: "sort sort-#{session['filters']['direction']}")
  end
  
  def show_sort_indicator_for(column)
    sort_indicator if session['filters']['column'] == column
  end
end

Here we replaced references to params with session['filters'], everything else is the same.

Finally, we need to read the value of the search field from session['filters'] instead of params to, which we can do by updating the form in app/views/players/index.html.erb:

<%= form_with url: players_path, method: :get, data: { turbo_action: "replace", controller: "autosubmit", turbo_permanent: "" } do |form| %>
  <%= form.text_field :name,
    placeholder: "Search by name",
    value: session.dig('filters', 'name'),
    class: "border border-blue-500 rounded p-2",
    data: {
      action: "autosubmit#submit"
    }
  %>
<% end %>

Here we dig for the value of session['filters']['name'] and use it to set the value of the search field, plain old Ruby here.

Now reload http://localhost:3000/players and see that you can search by name and apply sorting in both directions and everything works as expected.

Next up, we are going to add the ability to filter player’s by their Team. In the process, we will clean up the query logic in the index method. Let’s dig in.

Filter by team

Before we add a new filter for Team, we should address the clunky code in PlayersController#index. Each new filter we add means another conditional in the method, and it is already hard to understand at a glance.

Fortunately, we can make this code easier to read and maintain with a few small adjustments. Head to app/controllers/players_controller.rb and update the index method:

def index
  session['filters'] ||= {}
  session['filters'].merge!(filter_params)

  @players = Player.includes(:team)
                   .then { search_by_name _1 }
                   .then { apply_order _1 }
end

And then define the search_by_name and apply_order methods, staying in the PlayersController:

private

# Snip existing private methods

def search_by_name(scope)
  session['filters']['name'].present? ? scope.where('players.name like ?', "%#{session['filters']['name']}%") : scope
end

def apply_order(scope)
  scope.order(session['filters'].slice('column', 'direction').values.join(' '))
end

Let’s pause here and look at what might be some surprising syntax. In the index method we are using then and numbered parameters, like this:

.then { search_by_name _1 }

then is an alias for yield_self. then (and yield_self) yields self to a block and then returns the result of the block, and is helpful for exactly our use case; passing an object through a chain of method calls.

Numbered parameters (_1) were introduced in Ruby 2.7 to simplify writing short blocks by allowing us to skip naming the parameters passed to the block. Alternatively, we could write the same line as:

yield_self { |scope| search_by_name(scope) }

But why write Ruby if we don’t care about aesthetics?

The use of then with numbered parameters in this article came from these excellent articles from the folks at ThoughtBot:

Our refactored index method still does the same thing, but it is now easier to scan the index method to understand what’s going on. The refactor also gives us a pattern we can use to support filtering by Team. Let’s do that now, staying in app/controllers/players_controller.rb:

def index
  session['filters'] ||= {}
  session['filters'].merge!(filter_params)

  @players = Player.includes(:team)
                    .then { search_by_name _1 }
                    .then { filter_by_team _1 }
                    .then { apply_order _1 }
end

One new line here, .then { filter_by_team _1 }. filter_by_team is not defined yet, define it next in the PlayersController private methods:

def filter_by_team(scope)
  session['filters']['team_id'].present? ? scope.where(team_id: session['filters']['team_id']) : scope
end

And then update filter_params to whitelist the new team_id attribute:

def filter_params
  params.permit(:name, :team_id, :column, :direction)
end

Now that we support filtering by team in the controller, we can update the form in app/views/players/index.html.erb:

<%= form_with url: players_path, method: :get, data: { turbo_action: "replace", controller: "autosubmit", turbo_permanent: "" } do |form| %>
  <%= form.select :team_id,
    options_for_select(
      Team.all.pluck(:name, :id),
      session.dig('filters', 'team_id')
    ),
    { include_blank: 'All Teams' },
    class: "border-blue-500 rounded",
    data: {
      action: "autosubmit#submit"
    } 
  %>
  <%= form.text_field :name,
    placeholder: "Search by name",
    value: session.dig('filters', 'name'),
    class: "border border-blue-500 rounded p-2",
    data: {
      action: "autosubmit#submit"
    }
  %>
<% end %>

Here we added a team_id select that displays all of the Team records in the database. Note the data-action attribute that submits the form automatically.

Load up http://localhost:3000/players one last time and see the new Team filter option. Test it out and see that you can apply any combination of sorting, searching, and filtering and it will all work as expected. With a couple lines of Turbo, and some regular old Rails and Ruby code we now have an interactive table UI that feels great to use and will look familiar to any Rails developer, even if they are new to Turbo.

That’s all for this article, great job following along and thank you for reading! As always, you can find the full code for this article on Github.

Wrapping up

Today we enhanced our sortable table by adding live searching and filtering, powered by a little bit of Stimulus and a splash of Turbo.

Turbo 8 plus Rails gives us the tools to write simple, Rails-y code without sacrificing user experience. To me, Turbo 8 feels peaceful in a way that previous versions of Turbo did not.

Peaceful is an odd term, maybe, to describe writing code, but that’s what it feels like after a few years of watching Turbo evolve, and seeing folks struggle with the conceptual shifts required to use Turbo. As we saw in this article and the first article in this series, page refreshes are vanilla Rails with a dash of magic on top. Things work like you expect them to in a way that feels, to me, peaceful, calm, and a tiny bit magical.

If you want to dive deeper into the topics we covered today:

The articles discussing uses for yield_self and then that I linked above are really good reads, even if they don’t have anything to do with Hotwire directly
In the previous article in this series, I linked to a Github PR with an important discussion about the difference between advance and replace visits and how they relate to page refreshes.
An article always inspires me, not related to Turbo directly is “Vanilla Rails is enough” by Jorge Manrubia. When I think about simple, peaceful Rails code, I think about vanilla Rails.

Building a sortable table with Turbo 8's page refreshes

2024-03-21T00:00:00+00:00

Scroll-restoring, morphing page refreshes are one of the headline features of Turbo 8. Page refreshes make it easier than ever to build modern, SPA-like experiences with Turbo and Ruby on Rails.

One of the most immediate wins for page refreshes is making it simpler to build search and filter UIs, so today we are going to get practical with Turbo 8 by building a sortable table view with page refreshes. I wrote about building sortable tables with Turbo several years ago; back when that article was written we needed to lean heavily on Turbo Frames. With the new tools in Turbo 8 at our disposal, we can simplify our code to the point that we will be writing code that will look familiar to anyone that’s worked in Rails before. Turbo’s page refreshes blend into the background and let us write simple, efficient controllers and views without sacrificing the smooth feel of a single page application.

The application will allow users to sort a list of Players by their attributes. The players will be displayed in a table, and users will click on the table headers to apply their desired sort order. When we are finished, our project will work like this:

To follow along with this article, you should have be comfortable with Ruby on Rails and have Ruby 3 and Rails 7 ready to run locally. You do not need any experience with Turbo, I will fill you in as we work.

Let’s dive in.

Setup

If you prefer to skip past the setup, you can clone this Github repo and skip ahead to the next section.

To follow along locally, create a new Rails application with Tailwind added from your terminal:

rails new turbo_8_refresh_sorting --css=tailwind
cd turbo_8_refresh_sorting
bin/setup

The core of our application is a list of Players that users can sort by name, team, and the number of seasons they played. Staying in your terminal, generate the models and scaffold up a basic Player resource to build on. Staying in your terminal:

rails g model Team name:string
rails g scaffold Player name:string team:references seasons:integer
rails db:migrate

Next use faker to generate test data so we can see our sort mechanism working. From your terminal:

bundle add faker

And then hop into the rails console with bin/rails c and run this code to generate test data in your local database:

['Dallas Mavericks', 'Denver Nuggets', 'Phoenix Suns', 'San Antonio Spurs', 'Boston Celtics', 'Miami Heat', 'New Orleans Pelicans'].each do |name|
  Team.create(name: name)
end

100.times do
  Player.create(name: Faker::Name.name, team: Team.find(Team.pluck(:id).sample), seasons: rand(25))
end

Finish setup by creating a basic table structure on the players index page at app/views/players/index.html.erb:

<div class="max-w-7xl mx-auto mt-12">
  <div class="shadow overflow-hidden rounded border-b border-gray-200">
    <table class="min-w-full bg-white">
      <thead class="bg-gray-800 text-white">
        <tr>
          <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
            <span>Name</span>
          </th>
          <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
            <span>Team</span>
          </th>
          <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
            <span>Seasons</span>
          </th>
        </tr>
      </thead>

      <tbody class="text-gray-700">
        <% @players.each do |player| %>
          <tr>
            <td class="text-left py-3 px-6"><%= player.name %></td>
            <td class="text-left py-3 px-6"><%= player.team.name %></td>
            <td class="text-left py-3 px-6"><%= player.seasons %></td>
          </tr>
        <% end %>
      </tbody>
    </table>
  </div>
</div>

Here we are rendering an HTML table with columns for player name, team name, and number of seasons played, and then rendering each player in the list of @players set by the PlayersController#index action. At this point you can boot up the Rails app with bin/dev and head to http://localhost:3000/players. If all has gone well you should a static list of the Players that we generated with faker.

A static table is not very exciting. Next let’s make it possible to sort the table when a user clicks on the column headers.

Sorting the table

In previous versions of Turbo, the best method for building sorting or filtering features was Turbo Frames. Without too much code or complexity you could get a SPA-like experience that maintained the user’s scroll position and updated the user’s results with data from the server very quickly. Turbo Frames still work fine in Turbo 8 (you do not need to go rip them all out before upgrading), but we can simplify our code with the addition of morphing, scroll preserving page refreshes. With a Turbo-powered refresh, we can write standard Rails code that provides exactly the same experience without any extra views, routes, or controller code.

First, make the table headers clickable links. When a user clicks on any of the table headers, we will sort the table by that column in ascending order.

Head to app/views/players/index.html.erb and update it:

<thead class="bg-gray-800 text-white">
  <tr>
    <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
      <%= sort_link(column: "name", label: "Name") %>
    </th>
    <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
      <%= sort_link(column: "teams.name", label: "Team") %>
    </th>
    <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
      <%= sort_link(column: "seasons", label: "Seasons") %>
    </th>
  </tr>
</thead>

Here we replaced the text of the <th> elements in the table with calls to a sort_link method that takes the column to sort by and label text to display in the UI.

sort_link has not been defined yet, do that next in app/helpers/players_helper.rb:

module PlayersHelper
  def sort_link(column:, label:)
    link_to(label, players_path(column: column))
  end
end

Each call to sort_link returns a link_to pointing to Players#index with column in the URL parameters so that we can use the column from the link to apply the user’s desired sort value to the list of players.

Head to the PlayersController at app/controllers/players_controller.rb and update the index action to use the column param to order the list of players:

def index
  if params[:column].present?
    @players = Player.includes(:team).order("#{params[:column]}")
  else
    @players = Player.includes(:team).all
  end
end

At this point you can refresh the players index page and see that clicking the header links sorts the table by the column you clicked. But you will notice that each click performs a full page turn, replacing the entire body of the page and resetting the user’s scroll position back to the top of the page. That’s not the experience we are looking for!

Let’s use a Turbo page refresh, with morphing and scroll preservation to make the sort feel like a seamless, on-page update.

We need to add two meta tags to the <head> of the players index page to enable Turbo refreshes. These meta tags tell Turbo to morph page refreshes with idiomorph and to preserve the user’s scroll position during the refresh.

The meta tags we need are:

<meta name="turbo-refresh-method" content="morph">
<meta name="turbo-refresh-scroll" content="preserve">

turbo-rails provides a handy view helper we can use to generate both meta tags at once. Head to app/views/players/index.html.erb and add the helper:

<%= turbo_refreshes_with method: :morph, scroll: :preserve %>

And then update app/views/layouts/application.html.erb to yield the head contentprovided by turbo_refreshes_with:

<% if content_for? :head %>
  <%= yield :head %>
<% end %>

You can add turbo_refreshes_with directly into the <head> in app/views/layouts/application.html.erb if you prefer:

<head>
  <%= turbo_refreshes_with method: :morph, scroll: :preserve  %>
  <%= yield :head %>
</head>

Adding these meta tags only on pages we need them (by calling turbo_refreshes_with in the players index view) is the safest approach, especially if you are migrating an existing application to Turbo 8 and want to avoid any surprising application behavior with refreshes and scroll preservation in your existing code.

You can also add the meta tags individually with the dedicated helpers for each tag:

<%= turbo_refresh_method_tag :morph %>
<%= turbo_refresh_scroll_tag :preserve %>

Whichever route you choose, once you have the required meta tags in the <head> of the players index page, head to app/helpers/players_helper.rb to update the sort_link helper:

module PlayersHelper
  def sort_link(column:, label:)
    link_to(label, players_path(column: column), data: { turbo_action: 'replace' })
  end
end

Here we added data-turbo-action to the link_to. The addition of this data attribute tells Turbo to clicks on the link as a replace visit instead of an advance visit. There are some subtleties with the implementation of Turbo refreshes when you want a refresh to be triggered by a direct visit to a URL instead of by a redirect initiated by a form submission. It is worth a brief break from coding to talk through these subtleties.

Turbo only triggers a refresh when a visit is a replace visit to the same pathname as the previous page; however, the default visit type for a visit in Turbo is advance. In our case, we are rendering links back to the same URL with different URL parameters to change the sort options. Turbo will treat each of those clicks as an advance visit and will not morph the page. Instead an advance visit treat will be treated as a normal navigation with the full body replaced and scroll position set to the top of the page.

That is not the behavior we are looking for, so we override the turbo-action to specify a replace visit. Doing this will cause Turbo to morph in the new content and maintain scroll position.

This can be a bit of a tricky concept. You do not always need to specify data-turbo-action to get refresh behavior. Redirecting after a POST is automatically treated by Turbo as a replace visit, for example. The intricacies of this can be traced through the code and are best captured in a currently (as of March 2024) open PR to the Turbo documentation. It is worth reading through the discussion on this PR to understand the current behavior in more depth.

It is also important to understand that using a replace visit means that the browser’s back button will not navigate through each change to the sort order. Because a replace visit replaces the last entry in the history stack each time, if you apply a few different sort directions and then click the back button in your browser, you will be taken back to the last page you were on before your initial visit to /players. This is a tradeoff of using page refreshes in this manner — if maintaining a complete navigation history as sorting options change is important for your use case, you can use Turbo Frames to implement sorting rather than using morphing refreshes. There will be more code to write, but you will get complete browser history as compensation for the extra work.

Once we have this change in place, we have written all the code we need to implement Turbo refreshing with page morphing and scroll preservation. Reload http://localhost:3000/players, scroll down the page a bit, and then click on the page header and see that the table updates while scroll position is maintained.

Magical.

Sorting only in one direction is not very useful. Let’s finish up the sortable table implementation by swapping between ascending and descending sort order. No extra Turbo required.

Sort both directions

First, we will update the sort_link helper to include a direction parameter that toggles between ascending and descending order. Head to app/helpers/players_helper.rb and update it:

module PlayersHelper  
  def sort_link(column:, label:)
    direction = column == params[:column] ? next_direction : 'asc'
    link_to(label, players_path(column: column, direction: direction), data: { turbo_action: 'replace' })
  end

  def next_direction
    params[:direction] == 'asc' ? 'desc' : 'asc'
  end
end

Here we add a direction parameter to the link_to. If the column is currently sorted in ascending order, the next click on that column will sort in descending order, otherwise the next click on the column will sort in ascending order.

Now head to app/controllers/players_controller.rb and update it to apply the new direction parameter to the order query:

@players = Player.includes(:team).order("#{params[:column]} #{params[:direction]}")

Note that we are using user supplied parameters directly in an ActiveRecord query without sanitizing them. There is nothing stopping a malicious user from attempting an attack by passing in dangerous values to the column and direction params. Fortunately, Rails automatically sanitizes parameters passed to order to keep us safe.

Since users can sort by any column and in ascending or descending order, we should tell them which column they are currently sorting by, and in which direction. Let’s finish up this tutorial by adding a sort indicator to the table header. Head back to app/views/players/index.html.erb and update the <thead> element:

<thead class="bg-gray-800 text-white">
  <tr>
    <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
      <%= show_sort_indicator_for("name") %>
      <%= sort_link(column: "name", label: "Name") %>
    </th>
    <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
      <%= show_sort_indicator_for("teams.name") %>
      <%= sort_link(column: "teams.name", label: "Team") %>
    </th>
    <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
      <%= show_sort_indicator_for("seasons") %>
      <%= sort_link(column: "seasons", label: "Seasons") %>
    </th>
  </tr>
</thead>

Here we added calls to show_sort_indicator_for to each column header. This helper method will render a sort indicator next to the header text if the table is currently sorted by that column. That helper method does not exist yet, so head to app/helpers/players_helper.rb to define it:

def sort_indicator
  tag.span(class: "sort sort-#{params[:direction]}")
end

def show_sort_indicator_for(column)
  sort_indicator if params[:column] == column
end

show_sort_indicator_for renders the sort_indicator helper when appropriate, otherwise it renders nothing.

Finally, we need a little CSS for the sort indicator tag. Add the CSS in app/assets/stylesheets/application.tailwind.css:

.sort {
  position: absolute;
  top: 1rem;
  left: 0.5rem;
  width: 0;
  height: 0;
  border-left: 6px solid transparent;
  border-right: 6px solid transparent;
}

.sort-desc {
  border-top: 8px solid #fff;
}

.sort-asc {
  border-bottom: 8px solid #fff;
}

With this last change in place, refresh the players index page and see that you can sort columns in both directions. As you apply sorting, you will see that your scroll position is preserved and a nice little arrow appears to show the user which column they are sorting by, and in which direction.

That is all for this tutorial, thank you for following along! As always, you can find the complete code for this tutorial on Github. Great work!

Wrap up

Today we learned how we can use page morphing and scroll preservation in Turbo 8 to build a simple table filtering mechanism in Ruby on Rails. This implementation can replace the more complex versions of this experience that could be built in previous versions of Turbo. Implementations without page refreshing typically relied on Turbo Frames or Turbo Streams to maintain scroll position and provide the snappy-feel that we can now get out of the box with Turbo 8 and plain old Rails code.

If you want to learn more about Turbo 8 and page refreshing, you may find these resources useful:

I wrote this piece to dive into more detail about morphing page refreshes on Rails
The announcement post from Jorge Manrubia provides more details about the motivations and real world use cases for refreshing
Also linked above, this PR on the Turbo docs is useful for tracing through some of the code that powers refreshes
Finally, reading the PR that added refreshes to Turbo is worthwhile if you want to understand how the underlying tools are implemented in JavaScript to make the magic happen

Turbo 8 morphing refreshes on Rails

2024-03-17T00:00:00+00:00

Turbo 8 was released in February of 2024, to much excitement in the Rails world, and the broader HTML-over-the-wire community. At the top of the list of new features was the combination of tooling that added support for morphing page refreshes and scroll position preservation, powered by idiomorph and the ability to broadcast page refreshes with a new Turbo Stream action.

This combination offers developers a “happier path” to partial page updates. Turbo Stream DOM manipulation actions (like prepend and replace) work well, but they require maintaining separate code paths and have limitations when broadcasting data in applications that need to care about authorization and access levels inside of a team or company setting.

Morphing refreshes are a powerful new tool in the Turbo (and turbo-rails) toolbox but they require a bit of a mental shift to understand and they are not without their drawbacks. Today we are going to explore Turbo 8’s new refresh feature in the context of a Rails application, learning how to use page refreshes, how to use broadcasted refreshes to push updates to all interested users at once, and dig into the pros and cons of refreshes compared to Turbo Stream actions that manipulate the DOM.

Before we begin, this article assumes you have written Rails code before and that you have a basic level of understanding with Turbo and how to use it within a Rails application. If you are not a Rails developer, you may have a hard time following the code at some points.

Let’s dive in.

Application Setup

We are going to build a simple blog + post data model to try out page refreshing, starting from a new Rails 7 application with Tailwind CSS. Tailwind is not required for any part of this article, but we will add it to have nicer looking pages as we work.

If you prefer to skip past the setup and go right to refreshing, you can clone this repository and skip ahead to the next section. We will move rapidly through the setup code, which should all be familiar if you are comfortaable working in Turbo-powered Rails applications.

Get started from your terminal:

rails new refreshable-blog --css=tailwind
cd refreshable-blog
bin/setup

After creating the new application, generate the Blog and Post models:

bin/rails g scaffold Blog name:string
bin/rails g model Post name:string blog:references
bin/rails db:migrate

And then generate a controller for Posts and the views we will need to create and render posts from a Blog show page. From your terminal:

bin/rails g controller Posts
touch app/views/posts/_form.html.erb app/views/posts/create.turbo_stream.erb app/views/posts/_post.html.erb

Posts belong to a Blog, so update the Blog model to reflect that association:

class Blog < ApplicationRecord
  has_many :posts
end

Posts are only visible from their parent Blog, so we can update config/routes.rb to define the proper nested routes for our application:

Rails.application.routes.draw do
  resources :blogs do
    resources :posts
  end
  # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html

  # Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500.
  # Can be used by load balancers and uptime monitors to verify that the app is live.
  get "up" => "rails/health#show", as: :rails_health_check

  # Defines the root path route ("/")
  root "blogs#index"
end

Now we’ll build our views, starting with the posts form at app/views/posts/_form.html.erb:

<%= form_with(id: "post_form", model: [blog, post], class: "contents") do |form| %>
  <% if post.errors.any? %>
    <div id="error_explanation" class="bg-red-50 text-red-500 px-3 py-2 font-medium rounded-lg mt-3">
      <h2><%= pluralize(post.errors.count, "error") %> prohibited this post from being saved:</h2>

      <ul>
        <% post.errors.each do |error| %>
          <li><%= error.full_message %></li>
        <% end %>
      </ul>
    </div>
  <% end %>

  <div class="my-5">
    <%= form.label :name %>
    <%= form.text_field :name, class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full" %>
  </div>

  <div class="inline">
    <%= form.submit class: "rounded-lg py-3 px-5 bg-blue-600 text-white inline-block font-medium cursor-pointer" %>
  </div>
<% end %>

Just regular old ERB and Tailwind here, with an id on the form to identify it in the DOM, which we need for targeting a Turbo stream replace call when the form is submitted. Next up, app/views/posts/_post.html.erb:

<div>
  <%= post.name %>
</div>

Complicated stuff there.

Next fill in our Turbo Stream erb template at app/views/posts/create.turbo-stream.erb:

<%= turbo_stream.replace('post_form', partial: 'posts/form', locals: { post: Post.new, blog: @post.blog }) %>
<%= turbo_stream.append('posts', partial: 'posts/post', locals: { post: @post}) %>

This view will be rendered when a new post is created. In it, the two Turbo Stream actions replace the post_form form to clear whatever data the user entered on the form and then append the new post to the list of posts.

Next, we want to render posts and the new post form on the blog show page. Let’s head there next and update it:

<div class="mx-auto md:w-2/3 w-full flex">
  <div class="mx-auto">
    <% if notice.present? %>
      <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
    <% end %>

    <%= render @blog %>

    <%= link_to "Back to blogs", blogs_path, class: "ml-2 rounded-lg py-3 px-5 bg-gray-100 inline-block font-medium" %>
  </div>
</div>
<div class="mx-auto md:w-2/3 w-full w-full">
  <h2>Blog posts</h2>
  <div id="posts">
    <% @blog.posts.each do |post| %>
      <%= render "posts/post", post: post %>
    <% end %>
  </div>
  <%= render "posts/form", post: @post.presence || Post.new, blog: @blog %>
</div>

More standard ERB here, with some Tailwind classes provided by bin/rails scaffold. Before this will work, we need to implement the create action in our PostsController. Head to app/controller/posts_controller.rb and update it:

class PostsController < ApplicationController
  def create
    @post = Post.new(post_params)
    @post.blog_id = params[:blog_id]

    respond_to do |format|
      if @post.save
        format.turbo_stream
      else
        format.turbo_stream { render turbo_stream: turbo_stream.replace('post_form', partial: 'posts/form') }
      end
    end
  end

  private

  def post_params
    params.require(:post).permit(:name)
  end
end

When the create action runs, successful form submissions render a turbo_stream view (that’s the create.turbo_stream.erb file we created earlier). When errors are present, a turbo_stream.replace action runs to replace the form content with any relevant errors.

This implementation of creating and inserting new objects using Turbo and turbo-rails is a common way to leverage Turbo in a Rails application today and will be our starting point for exploring how Turbo refreshes can be leveraged to simplify our code without sacrificing user experience.

Before ending the setup section, head over to app/views/layouts/application.html.erb to adjust the base page layout:

<!DOCTYPE html>
<html>
  <head>
    <title>RefreshableBlog</title>
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>
    <%= stylesheet_link_tag "tailwind", "inter-font", "data-turbo-track": "reload" %>

    <%= stylesheet_link_tag "application", "data-turbo-track": "reload" %>
    <%= javascript_importmap_tags %>
  </head>

  <body>
    <main class="container mx-auto mt-28 px-5">
      <%= yield %>
    </main>
  </body>
</html>

Here we dropped a flex class from the main element so that the form shows up under the list of posts instead of beside the list of posts. Why that flex is there in the default Tailwind + Rails layout is anyone’s guess, but it is gone from our application now.

If all has gone well with setup, you should be able to start up your Rails server with bin/dev and visit http://localhost:3000. Create at least one blog, head to that blog’s show page, and add a few posts to see that inserting new posts with Turbo Streams works as expected and then move on to the next section where we will start adding Turbo refreshes to our application.

Refreshing posts

Our goal in this section is to replace the current method of inserting new posts on the blog show page with a Turbo-powered morphing refresh. This will require much less code than you might expect.

Start by updating the PostsController at app/controllers/posts_controller.rb:

class PostsController < ApplicationController
  def create
    @blog = Blog.find(params[:blog_id])
    @post = Post.new(post_params)
    @post.blog = @blog

    if @post.save
      redirect_to @blog
    else
      render 'blogs/show', status: :unprocessable_entity
    end
  end

  private

  def post_params
    params.require(:post).permit(:name)
  end
end

Here we redirect to the blog show page if the new post is created. If the post cannot be saved we render the blog show page again with a 422 status code. This is standard Rails, there is no new magic required in the controller to use page refreshing, we are redirecting and rendering as we have been for more than a decade.

If you go to a blog show page now and submit the post form, you will notice that everything works except that the redirect is a full, normal page turn, meaning you lose your scroll position. Any ephemeral on-page changes (like opening a dropdown menu or interacting with a form element) would also be lost. We have done a full page turn and redirected the browser to a new page, so that’s all expected behavior but the user experience of jumping to the top of the page when they submitted a form to add a new post is not great.

Let’s fix that by telling Turbo to using morphing for refreshes and to maintain scroll position when a page refresh occurs. Head to app/views/layouts/application.html.erb and update it:

<!DOCTYPE html>
<html>
  <head>
    <title>RefreshableBlog</title>
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>
    <%= stylesheet_link_tag "tailwind", "inter-font", "data-turbo-track": "reload" %>

    <%= stylesheet_link_tag "application", "data-turbo-track": "reload" %>
    <%= javascript_importmap_tags %>
    <%= turbo_refreshes_with(method: :morph, scroll: :preserve) %>
    <%= yield :head %>
  </head>

  <body>
    <main class="container mx-auto mt-28 px-5">
      <%= yield %>
    </main>
  </body>
</html>

Here we added two lines to the head:

<%= turbo_refreshes_with(method: :morph, scroll: :preserve) %>
<%= yield :head %>

This helper method adds the meta tags Turbo looks for to enable morphing and scroll preservation on a page. In the source, you will note that turbo_refreshes_with uses provide to add content to head, which will not be rendered until yield :head is called. If you have encountered issues using turbo_refreshes_with without the following yield call and gotten frustrated at the difference in this helper and the other meta tag helpers in turbo-rails which do not require a yield call after, the discussion on this (later reverted) PR may be interesting. You can also use turbo_refresh_method_tag and turbo_refresh_scroll_tag to set the refresh method and scroll handling meta tags individually, or just add the <meta> tags directly without using helper methods, if that’s your style.

You should also note that we added the meta tags into the application layout, which means they will be there on every page in our application. In a real application, you may find that setting these values on every page can have unintended consequences. If you are adopting morphing refreshes and the accompanying scroll preservation mechanic in an existing application, it is wise to add these meta tags into the head only on pages that require morphing and scroll preservation to avoid unexpected behavior on pages that should not attempt to morph content or preserve scroll position.

Refresh the blog show page, add a new post and see that submitting the form and following the redirect does not lose scroll position. That’s the all of the work that it takes to use morphing page refreshes with Turbo and Rails. There are no more steps, there is no more code to write. Magical.

Let’s pause here and talk through what’s happening. In particular, what triggers a morphing refresh before we move on to fancier use cases, including broadcasting refreshes to other users of the application.

Users of our application visit a blog show page, at a url like /blogs/12. On the page, we render a form to create a new post. When that form is submitted, two outcomes are possible:

If the post is saved, we redirect to the blog show page again, so the original and the destination URL match.
If the post is not saved, we render the blog show page again without redirecting anywhere, so the original and the destination URL match.

In both cases, the original URL and the destination URL match. This matching is the key that unlocks Turbo refreshes. A morphing refresh triggered by a normal response/request cycle will only occur when the user starts and ends on the same page (with the same URL). If we are navigating between different pages, at different URLs, Turbo Drive won’t attempt to morph anything. This makes sense, I think, but it is worth stating it. Turbo refreshes are refreshing the same content, with changes that are morphed in using idiomorph.

You can use morphing refreshes for pages like our blog show page that allow users to manage child content or otherwise make changes that a user would expect to keep them in the current context of the page. If you have a more traditional CRUD application with full new and edit pages, those pages will not benefit from refreshing.

For a thoughtful, more in-depth discussion of what page refreshes are and how they work, I found this piece by Jon Sully to be a valuable read.

So far, we have seen how we can use Turbo refreshes and morphing to replace Turbo Streams rendered from a controller without sacrificing user experience. Our implementation of refreshes so far is still within the normal request/response cycle for a single user. The user submits a form, and we send back an HTML payload that Turbo processes with idiomorph. This is valuable, but we can unlock more power by broadcasting refresh actions to all users with Turbo Streams.

Let’s see how it works.

Broadcast refreshes

Broadcasting Turbo Stream actions via a WebSocket connection allows us to push real-time updates out to all interested users as data in our application changes. In our application, we want to broadcast changes to posts each time a post is created, updated, or deleted. Any user on the post’s blog show page should see those updates as they happen.

Start by updating the Post model at app/models/post.rb:

class Post < ApplicationRecord
  belongs_to :blog

  broadcasts_refreshes_to :blog
end

broadcasts_refreshes_to adds an after_commit callback that broadcasts a refresh Turbo Stream action. The refresh action in turn performs a replace visit to fetch updated content for the same URL from the server and update the page with the new content on each message recipient’s browser. Thanks to the magic of morphing, this all happens without the recipient losing their scroll position or the current state of the page — updated content just appears seamlessly.

To make this work, we have one more piece to add. Head to app/views/blogs/show.html.erb and update the posts section of the page like this:

<%= turbo_stream_from @blog %>
<div>
  <% @blog.posts.each do |post| %>
    <%= render "posts/post", post: post %>
  <% end %>
</div>

Here we added the turbo_stream_from call, subscribing to a channel specific to the @blog we are viewing. This channel matches the channel we are broadcasting on in the model — the :blog symbol translates into a call to post.blog. Since we are viewing the show page for the blog that the post belongs to, post.blog equals @blog, and so our users will be subscribed to the right channel to see updates for the blog they are viewing.

It is important to understand that broadcasted refreshes do not care about what page the user is on. To use morphing refreshes in the normal request/response cycle, like we did in the last section, the original and destination URL need to match. When we are broadcasting refreshes to users, the Turbo Stream refresh action prompts Turbo to go fetch the user’s current URL again, as long as they are on a page that subscribes to the relevant stream channel, they will get updated content for that page. Again, this makes sense I believe, but it is important to understand that refresh Stream Actions are different than morphing page refreshes.

Let’s pause here and see that broadcasting works. Open up the same blog show page in two browsers. In one browser, create a new post. If all has gone well, the post will appear in the list of posts on the second browser without changing scroll position.

Neat. We have now seen how to use Turbo 8’s morphing refreshes in the context of a regular request/response cycle and using Turbo Stream broadcasts in turbo-rails. Broadcasted refreshes are a powerful tool, but they can have some surprising behavior. Let’s demonstrate one of the most common gotchas now, and discuss how to work around this behavior.

Head to app/controllers/posts_controller.rb and update the create action to add a sleep, to make it easier to see the problem:

def create
  @blog = Blog.find(params[:blog_id])
  @post = Post.new(post_params)
  @post.blog = @blog
  sleep 5
  if @post.save
    redirect_to @blog
  else
    render 'blogs/show', status: :unprocessable_entity
  end
end

The sleep here adds 5 seconds of wait time before a form submission processes, which gives us enough time to submit the form in one window and then switch over and start working on the form in the other window. Try it out now — submit the form in one window and then move over to the other window and start typing in the post name field in the other window.

This is not a great experience for your users! Saving scroll position is great, but if you wipe out a user’s work every time a broadcasted refresh runs, they are not going to be happy.

Why does this happen? Idiomorph is smart and only updates the parts of the page that are different when a refresh runs; however, form inputs are tricky. When the new page content comes back, the form will always be a clean, default version of the form because that is what the server knows about; the server does not know that the user is currently halfway through writing a novel in a form input. Idiomorph sees the clean version of the form, sees that the current version of the page is different, and replaces the old, active version of the form with the new, empty version. Woof.

There are ways to fix this behavior, most commonly reaching for the data-turbo-permanent attribute. This attribute skips morphing of the element and any child elements. In our particular use case, since we are relying on Turbo refreshing to update the browser of the user that submitted the form and broadcasting those changes as refreshes to every other user, we need more than data-turbo-permanent to make it work. Adding data-turbo-permanent to a wrapped div around the form inputs will fix the issue of lost focus that we saw in the above demonstration. However, that change alone will also cause the form not to reset for the user who submitted the form in the first place, requiring them to delete whatever they typed into the form or making it look like something broke, which is not an acceptable user experience.

A complete fix for this behavior in our use case could be both a data-turbo-permanent element and the introduction of a Stimulus controller to clear the form for the user that made the original submission.

A separate article will dive further into the details of handling forms in refreshes, but it is important to think about morphing and how it will affect users as you begin to build more complex user interfaces. Modals, drawers, drop down menus, forms, and third party JavaScript libraries that dynamically add elements to the DOM can have surprising behavior when combined with morphing refreshes.

That’s all of the code we are going to write today, nice work following along!

We will wrap up today by talking through some of the pros and cons of using morphing refreshes, and broadcasted refreshes in your Turbo-powered Rails applications. You can see the full code for today’s article on Github.

Pros and cons of Turbo Refreshes

From my perspective, the two biggest wins from Turbo morphing refreshes, and the accompanying broadcasted refresh in turbo-rails are:

Simpler happy paths in the code. With refreshes we do not need to worry about setting DOM ids (and hoping they aren’t removed or duplicated in the future), maintaining separate paths in our controllers, or creating and maintaining turbo_stream.erb view files. Write regular old Rails code and shove a couple of meta tags into the head on pages that should use morphing and you are good to go.
Broadcasted refreshes solve the session context problem that is one of the most common stumbling blocks for developers learning to use Turbo Stream broadcasts. Because a refresh broadcast is a ping to tell the recipient browser to go fetch new content for the page, we can broadcast updates without worrying about exposing data or functions that should be hidden. While there were ways to work around the lack of session context in stream broadcasts that deliver rendered content in the original message, those workarounds are more complex and provide a worse user and developer experience than than the simple, clean implementation we get with broadcasted refreshes.

As we saw with the challenge of keeping form state intact, there are tradeoffs and challenges that come with morphing refreshes:

Form inputs and dynamic page content can be tricky to deal with, requiring more code and a carefully planned architecture to get the most benefit from refreshes. Needing to write extra code to support refreshes gets off of the happy path sold as a key benefit of morphing refreshes.
Refreshes are more resource heavy than other Turbo Stream actions. In a standard Turbo Stream action, the server renders only the content that needs to be updated and nothing more, and much less work is required by the user’s browser to update the DOM. Rendering the entire page and then asking the browser to diff the entire page and update it in real-time is not free for you or for your users, especially if they are on lower end devices without the processing power that our development machines have. While we may not need to care about speed and costs on small personal projects, if we are building real applications, we should consider what the cost is for us (rendering lots of full pages when 95% of the content is identical) and to our users (who suddenly have more work for their browsers to do) for the cleaner code paths that morphing refreshes provide.
There are also speed differences here with broadcasted refreshes. A broadcasted refresh requires the original WebSocket message to be sent to tell the recipients browser to fetch new content and then a round trip to the server to fetch that content and replace the current page. With a traditional Turbo Stream broadcast, the updated DOM content is contained in the original WebSocket message. Cutting out that round trip means that a Turbo Stream append broadcast will get updated content to the user faster than a Turbo Stream refresh broadcast. In many applications this speed different will not matter. The user is getting updates in the background and does not know an update is coming, but for some use cases refreshes will feel slower than other broadcast actions.

Turbo 8’s morphing refreshes are an exciting new tool in our toolbox, and as they mature and the community coalesces around best practices and expands the Turbo feature set around morphing, we can expect to see real value in production applications from adopting refreshes.

As always with shiny new developer tools, I encourage you to think about your use case and what value refreshes add to your application. For many existing applications that rely on Turbo Streams, the incremental value added by replacing all of that code with refreshes is not likely to be worth the effort; however, you may find that in new applications, or as you add new features to your existing application, relying on morphing refreshes lets you move faster without sacrificing the single page application feel that Turbo Streams provide.

For further reading on Turbo 8’s changes, the motivation behind the changes, and a deeper dive into how refreshes and morphing work, you may find these resources valuable:

A happier path in Turbo with morphing
Jon Sully’s deep dive into Turbo page refreshes and morphing
The Basecamp turbo-8-morphing-demo repo (the main branch is empty here, check the page-refreshes and stream-actions branches for comparisons between approaches)

That’s all for this article, thank you for reading! Turbo morphing and refreshing are very new and some functionality is still in flux. If I have missed something or you see a better approach, reach out and share your thoughts.

Building a modal form with Turbo Stream GET requests and custom stream actions

2022-09-24T00:00:00+00:00

Turbo 7.2 brought major changes to what you can accomplish with Turbo Streams. To demonstrate those changes, today we are going to build a Turbo Stream-powered modal form without a single Turbo Frame. We will also create our own custom Turbo Stream action that emits a custom event on the browser.

The key technique we will use in this article is an update to the standard method of using Turbo to open and populate modals. Prior to Turbo 7.2, Turbo would not allow GET requests to respond directly with Turbo Streams. Smart folks devised a workaround to this limitation which involved using empty Turbo Frames to sneak in Turbo Stream actions. That workaround still works fine, but it is no longer necessary.

As part of our modal implementation, we will also have a chance to build a custom Turbo Stream action. Custom actions are an entirely new feature added in Turbo 7.2, and we will cover enough to get you started with this new tool in Turbo.

When we are finished our application work like this:

To follow along with this article, you will need to be comfortable with Ruby on Rails and you should have some experience using Stimulus. We will move quickly through the Turbo content, so previous exposure to using Turbo Streams will be helpful but is not required. As usual, you can review the complete code for this tutorial on Github.

Let’s get started!

Application setup

We will start with a new Rails 7 application with esbuild and Tailwind along with Turbo 7.2 and Stimulus. From your terminal:

rails new turbo-stream-modals --javascript=esbuild --css=tailwind
cd turbo-stream-modals

Note that neither Tailwind or esbuild are hard requirements, just personal preferences. The Turbo Stream and Stimulus techniques will work fine here with import maps, Vite, and any CSS framework you like.

For this exercise, users will be creating “cards”, which we will scaffold up using the Rails generator. From your terminal again:

rails g scaffold Card name:string
rails db:migrate

Start up the server with bin/dev from your terminal.

Open the code up in your favorite editor and head to app/views/cards/index.html.erb for some non-essential layout updates:

<div class="max-w-3xl mx-auto mt-8">
  <div class="flex justify-between items-baseline mb-6">
    <h1 class="text-3xl text-gray-900">Cards</h1>
    <%= link_to 'New Card', new_card_path, class: "text-blue-600" %>
  </div>
  <div id="card-list" class="flex flex-col items-baseline space-y-6 p-4 shadow-lg rounded">
    <% @cards.each do |card| %>
      <%= render "card", card: card %>
    <% end %>
  </div>
</div>

More small layout updates in app/views/cards/_card.html.erb:

<div class="text-gray-700 border-b border-gray-200 w-full pb-2">
  <%= card.name %>
</div>

With cards scaffolded up and the views cleaned up a bit, we are ready to start building the modal.

To open and close the card form modal, we are going to use a Stimulus controller.

Generate that controller from your terminal:

rails g stimulus modal

Fill that new Stimulus controller in at app/javascript/controllers/modal_controller.js:

// This controller is an edited-to-the-essentials version of the modal component created by @excid3 as part of the essential tailwind-stimulus-components package found here:
// https://github.com/excid3/tailwindcss-stimulus-components

import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
static targets = ['container'];

  connect() {
    this.toggleClass = 'hidden';
    this.backgroundId = 'modal-background';
    this.backgroundHtml = this._backgroundHTML();
  }

  disconnect() {
    this.close();
  }

  open() {
    document.body.classList.add('fixed', 'inset-x-0', 'overflow-hidden');
    this.containerTarget.classList.remove(this.toggleClass);
    document.body.insertAdjacentHTML('beforeend', this.backgroundHtml);
    this.background = document.querySelector(`#${this.backgroundId}`);
  }

  close() {
    if (typeof event !== 'undefined') {
      event.preventDefault()
    }
    this.containerTarget.classList.add(this.toggleClass);
    if (this.background) { this.background.remove() }
  }

  _backgroundHTML() {
    return `<div id="${this.backgroundId}" class="fixed top-0 left-0 w-full h-full" style="background-color: rgba(0, 0, 0, 0.7); z-index: 9998;"></div>`;
  }
}

As the comment notes, this code is a stripped-down version of the production-ready modal provided in Chris Oliver’s tailwindcss-stimulus-components library. I modified it to remove the extras that we don’t need for our project and to save us a few chunks of copy/pasted code that we would need if we were using the full library.

The JavaScript in this controller is fairly standard stuff, open opens the modal and close, shockingly, closes the modal by applying the right Tailwind classes and toggling in and out a background overlay.

To use this new Stimulus controller, we need to connect it to the DOM. Head over to app/views/layouts/application.html.erb:

<!DOCTYPE html>
<html>
  <head>
    <title>TurboStreamModals</title>
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>

    <%= stylesheet_link_tag "application", "data-turbo-track": "reload" %>
    <%= javascript_include_tag "application", "data-turbo-track": "reload", defer: true %>
  </head>

  <body data-controller="modal">
    <%= yield %>
    <%= render "shared/modal" %>
  </body>
</html>

Here, we connected the modal controller to the body. Inside of the body, we render a modal partial that does not exist yet.

Create that partial next, from your terminal:

mkdir app/views/shared
touch app/views/shared/_modal.html.erb

Fill in the modal partial:

<div data-modal-target="container"
     class="hidden fixed inset-0 overflow-y-auto flex items-center justify-center"
     style="z-index: 9999;">
  <div class="max-w-lg max-h-screen w-full relative">
    <div class="m-1 bg-white rounded shadow">
      <div class="px-4 py-5 border-b border-gray-200 sm:px-6">
        <h3 id="modal-title" class="text-lg leading-6 font-medium text-gray-900"></h3>
      </div>
      <div id="modal-body"></div>
    </div>
  </div>
</div>

The key pieces in this partial are:

The modal-target data attribute on the container div. The Stimulus controller uses this attribute to know which element to apply classes to when the modal is opened and closed.
The modal-title header and the modal-body div, both of which are empty when the modal partial is rendered.

The empty modal-title and modal-body elements will be filled in with content from the server when we open the modal using the magic of Turbo Streams.

Next update app/views/cards/index.html.erb to adjust the New Card link to open a modal when clicked:

<div class="max-w-3xl mx-auto mt-8">
  <div class="flex justify-between items-baseline mb-6">
    <h1 class="text-3xl text-gray-900">Cards</h1>
    <%= link_to 'New Card', new_card_path, class: "text-blue-600", data: { action: "click->modal#open", turbo_stream: "" } %>
  </div>
  <div class="flex flex-col items-baseline space-y-6 p-4 shadow-lg rounded">
    <% @cards.each do |card| %>
      <%= render "card", card: card %>
    <% end %>
  </div>
</div>

Note the addition here of two data attributes on the “New Card” link_to. The data-action attribute tells Stimulus to fire the modal#open method in the modal controller, making the modal visible to the user. The other attribute, data-turbo-stream tells Turbo to send the request to the server as a Turbo Stream request, instead of a standard HTML request.

data-turbo-streams is new in Turbo 7.2. This addition opens up the ability to respond to GET requests with Turbo Stream actions directly, instead of trojan horsing actions inside of empty Turbo Frames. This change allows us to write a little less code which is nice. Even better, it makes our intentions as developers more clear. The empty Turbo Frame work around was a confusing piece of indirection that we no longer need to rely on.

With the data-turbo-stream attribute added, Turbo will expect a Turbo Stream response from the server when the new card link is processed. To make this work, we need to tell Rails what we want to render in response to a Turbo Stream request to cards/new.

From your terminal:

touch app/views/cards/new.turbo_stream.erb

Fill in the new view:

<%= turbo_stream.update "modal-title", "Add a card" %>
<%= turbo_stream.update "modal-body", partial: "form", locals: { card: @card } %>

Here our “view” is really two Turbo Stream actions. These actions will be picked up by Turbo’s JavaScript on the frontend and processed to update the DOM.

In this case, that means we will send over content for the empty header with an id of modal-title and the empty modal-body div. Turbo makes those updates, and our modal magically gets the updated, server-rendered content.

Update the cards form partial at app/views/cards/_form.html.erb before checking to see if our modal works:

<%= form_with(model: @card, id: "card-form") do |form| %>
  <div class="px-4 pt-5 pb-4 sm:p-6 sm:pb-4">
    <% if @card.errors.any? %>
      <div class="p-4 border border-red-600">
        <ul>
          <% @card.errors.full_messages.each do |message| %>
            <li><%= message %></li>
          <% end %>
        </ul>
      </div>
    <% end %>

    <div class="form-group">
      <%= form.label :name %>
      <%= form.text_field :name %>
    </div>

  </div>
  <div class="rounded-b mt-6 px-4 sm:grid sm:grid-cols-2 sm:gap-3 sm:grid-flow-row-dense">
    <%= form.button class: "w-full sm:col-start-2 bg-blue-600 px-4 py-2 mb-4 text-white rounded-sm hover:bg-blue-700" %>
    <button data-action="click->modal#close" class="mt-3 w-full sm:mt-0 sm:col-start-1 mb-4 bg-gray-100 hover:bg-gray-200 rounded-sm px-4 py-2">
      Cancel
    </button>
  </div>
<% end %>

Refresh the cards index page at http://localhost:3000/cards, click on the “New Card” link and, if all has gone well so far, a modal should open with the new card header text and the card form populated.

At this point submitting the card form will create a new card in the database but, after saving the card you will be redirected to the card show page. Redirecting defeats the purpose of opening the card form in the modal. Let’s fix this issue next.

First, create another Turbo Stream view. From your terminal:

touch app/views/cards/create.turbo_stream.erb

And fill that view in:

<%= turbo_stream.prepend "card-list", @card %>

Easy enough. We use a Turbo Stream prepend action to insert the newly created card in the list.

Head over to app/controllers/cards_controller.rb and update the create method:

def create
  @card = Card.new(card_params)

  respond_to do |format|
    if @card.save
      format.html { redirect_to card_url(@card), notice: "Card was successfully created." }
      format.json { render :show, status: :created, location: @card }
      format.turbo_stream
    else
      format.html { render :new, status: :unprocessable_entity }
      format.json { render json: @card.errors, status: :unprocessable_entity }
    end
  end
end

The addition of format.turbo_stream when a card saves ensures that inbound Turbo Stream requests render the create.turbo_stream.erb view that we added.

By default, Turbo assumes all form submissions are Turbo Stream requests, so we don’t need to make any changes to our form for this to work, just having a Turbo Stream view is enough.

Head back to the cards index, open the modal and save a card. You’ll see the newly created card added to the top of the list of cards but the modal stays open.

Users will expect the modal to close when they submit the form, so let’s address that next.

Creating a custom Turbo Stream action

We are going to use a custom Turbo Stream action to dispatch an event when the card form is submitted successfully. This event will be generated on the server, sent over the wire to the frontend, processed by Turbo, and picked up by a waiting Stimulus controller to close the modal.

It’ll be pretty fancy. It will also be completely unnecessary. Turbo emits an event after a form submission that would work just fine to close our modal. Building a custom Turbo Stream action to submit an event is purely for demonstration purposes.

In the real world, if you need to close a modal after a Turbo Stream submission, the built-in event is just fine. It just isn’t as fun as rolling our own custom Turbo Stream action with the new tools provided for us in Turbo 7.2.

To implement the custom action we need a new helper on the Rails side and a little bit of JavaScript to tell Turbo what to do when it receives our custom action in the browser. Let’s start with the JavaScript.

From your terminal, create a new directory and file where we will define our custom event:

mkdir app/javascript/custom_actions
touch app/javascript/custom_actions/dispatch_event.js

Fill in dispatch_event.js:

import { StreamActions } from '@hotwired/turbo'

StreamActions.dispatch_event = function() {
  const name = this.getAttribute('name')
  const event = new Event(name)
  window.dispatchEvent(event)
  // If you want to send the event somewhere besides the window
  // document.getElementById(this.target).dispatchEvent(event) 
}

Here, we are importing StreamActions from Turbo and extending it to add a new dispatch_event action to StreamActions.

dispatch_event takes a name attribute that we then use to define and dispatch an Event on the window.

While dispatching the event on the window works fine for this tutorial, in the real world, you can scope the event to a particular element instead of blasting it out on the window. I’ve noted a simple approach to that problem as a comment.

You might also want to define a new CustomEvent instead of an Event. Using a CustomEvent allows you to include additional data in the event payload. Again, we don’t need that extra data in this tutorial, but the possibility is there as you explore.

Head over to app/javascript/application.js to import the new custom action:

import "@hotwired/turbo-rails"
import "./controllers"
import "./custom_actions/dispatch_event"

That’s the JavaScript-side completed.

Next we need to define a matching Turbo Stream helper tag on the server. From your terminal:

touch app/helpers/dispatch_event_helper.rb

And fill in the new dispatch_event_helper file:

module DispatchEventHelper
  def dispatch_event(name)
    turbo_stream_action_tag :dispatch_event, name: name
  end
end

Turbo::Streams::TagBuilder.prepend(DispatchEventHelper)

Nothing fancy. We add a new dispatch_event helper and add it to the list of available Turbo Stream action tags.

Both the JavaScript and Rails side of the equation here are derived from Marco Roth’s example in the PR that made these custom helper tags possible in turbo-rails. This work was instrumental in getting full-featured support for custom Turbo Stream actions into our hands.

To use this new helper tag, update app/views/create.turbo_stream.erb:

<%= turbo_stream.prepend "card-list", @card %>
<%= turbo_stream.dispatch_event "modalClose" %>

With this change, the Turbo Stream payload will now include both a prepend action and our custom dispatch_event action, which will emit an event on the client side named “modalClose”.

The last step to make this work is to fire the close method in the modal Stimulus controller when the modalClose event is received.

Back in app/views/application.html.erb:

<!DOCTYPE html>
<html>
  <head>
    <title>TurboStreamModals</title>
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>

    <%= stylesheet_link_tag "application", "data-turbo-track": "reload" %>
    <%= javascript_include_tag "application", "data-turbo-track": "reload", defer: true %>
  </head>

  <body data-controller="modal" data-action="modalClose@window->modal#close">
    <%= yield %>
    <%= render "shared/modal" %>
  </body>
</html>

Note the new data-action attribute on the body listening for the modalClose custom event.

Head back to the cards index page, open the modal, create a new card, and see that the new card is prepended to the list and the modal closes. Incredible stuff.

Let’s look closer at what is happening here by looking at the payload that the server sends over the wire when it renders create.turbo_stream.erb.

That payload will look something like this:

<turbo-stream action="prepend" target="card-list">
  <template>
    <div class="text-gray-700 border-b border-gray-200 w-full pb-2">
      Grim Patron
    </div>
  </template>
</turbo-stream>
<turbo-stream name="modalClose" action="dispatch_event">
  <template></template>
</turbo-stream>

The payload contains turbo-stream custom elements with an action set. Each element wraps a template tag, although the template for our dispatch_event action is empty since that action does not need to render any HTML.

Turbo’s JavaScript processes the payload. The custom action we defined in app/javascript/custom_actions/dispatch_event.js is used to process the dispatch_event action in the payload.

This technique is incredibly powerful and opens the door for Turbo users to begin to experiment with more complex DOM manipulation in Turbo Stream actions. Our example dispatches an event but we could just as easily play a sound, morph in new content with morphdom, manipulate data attributes, or add or remove CSS classes by defining new custom Turbo Stream actions.

As you experiment with your own custom actions, you’ll find the Turbo source helpful for understanding how the existing Turbo Stream actions are implemented, and the turbo-rails source helpful for learning how to build new action helpers.

Let’s finish up this article by handling invalid form submissions with Turbo Streams.

First, update the create method in app/controllers/cards_controller.rb:

def create
  @card = Card.new(card_params)

  respond_to do |format|
    if @card.save
      format.html { redirect_to card_url(@card), notice: "Card was successfully created." }
      format.json { render :show, status: :created, location: @card }
      format.turbo_stream
    else
      format.html { render :new, status: :unprocessable_entity }
      format.json { render json: @card.errors, status: :unprocessable_entity }
      format.turbo_stream { render turbo_stream: turbo_stream.replace('card-form', partial: 'form'), status: :unprocessable_entity }
    end
  end
end

Here, we updated the unhappy path to render an inline turbo_stream.replace action. When the card isn’t saved, we re-render the form and replace it inline, leaving the modal open with the errors displayed to the user.

Before we can test this, we need to add validation to the card model. Update app/models/card.rb like this:

class Card < ApplicationRecord
  validates_presence_of :name
end

Open the card modal and submit the form with a blank name input. See that the modal stays open with the errors rendered on the form. Beautiful.

Great work making it to the end of this tutorial — I hope it has helped to see the changes in Turbo 7.2 in action, and I hope it gets the wheels turning on what you can do with custom Turbo Stream actions.

Wrapping up

Today we built a modal form with Rails, Turbo, and Stimulus, using a Turbo Stream GET request and a custom Turbo Stream action, both of which are new tools unlocked by the release of Turbo 7.2.

Our implementation of a modal has some room to grow. If you want to stretch these muscles a bit, try adding functionality to clear the modal content each time it closes. Without clearing the content on close, the user will see a brief flash of old content each time the modal is opened. To implement this, you might consider emptying the modal-title and modal-body elements each time close is called in the Stimulus controller. Adding the ability to edit existing cards in the same modal would also be a useful exercise.

If the world of highly targeted DOM manipulation initiated from the server is new to you, you might find inspiration in CableReady. CableReady provides 30+ operations out of the box, everything from simple HTML replacement to adding CSS classes, logging to the console, and dispatching events.

Custom Turbo Stream actions are a powerful tool, but if you find you need more, CableReady is the place to look.

That’s all for today — as always, thanks for reading!

Everyone GET in here! Infinite scroll with Rails, Turbo Streams, and Stimulus

2022-09-21T00:00:00+00:00

If you have worked with Turbo Streams, you have probably run into a frustrating limitation. From the beginning, Turbo Streams were designed exclusively for handling form submissions, and that was the way it worked. If you wanted to respond to a GET request with a Turbo Stream, you couldn’t, without a clunky workaround.

Until now.

In Turbo 7.2, you can respond to GET requests with Turbo Streams. No more empty Turbo Frame hacks.

To demonstrate, we are going to build a simple Rails application that supports infinite scrolling through a paginated resource using Pagy, Turbo Streams, and a tiny Stimulus controller.

When we are finished, our application will work like this, and we won’t use a single Turbo Frame:

Before we begin, you should be comfortable working with Rails and have at least a passing familiarity with Turbo. If you have never used Turbo before, this article will not be the best starting place. This article is an update to an article I published earlier this year. Reviewing the previous article will help you understand why GET Turbo Streams are such an exciting change.

As usual, you can find the completed code for this tutorial on Github.

Let’s get started!

Application setup

We will start with a new Rails 7 application, using esbuild and Tailwind. Tailwind is entirely optional in this article, but it will make things look a bit nicer.

From your terminal:

rails new turbo-stream-pagination --javascript=esbuild --css=tailwind
cd turbo-stream-pagination

Make sure that you are using Turbo 7.2 or greater, and turbo-rails 1.3 or greater.

To finish application setup, scaffold up a new resource to paginate. From your terminal:

rails g scaffold card name:string
rails db:migrate

Since we are paginating the Cards resource, jump into the Rails console with rails c from your terminal and populate some database rows with a script like this:

50.times do |n|
  Card.create(name: "Grim Patron #{n}")
end

Setup complete, let’s start writing some code!

Paginating a resource with Pagy

Before we can add infinite scrolling, we need to add simple pagination to our application. We could build this ourselves, but in this article we will rely on Pagy, a battle-tested pagination gem that handles all the tricky parts for us.

From your terminal:

bundle add pagy

Add Pagy to the backend of the application by updating app/controllers/application_controller.rb:

class ApplicationController < ActionController::Base
  include Pagy::Backend
end

And add Pagy’s frontend helpers to the application by updating app/helpers/application_helper.rb:

module ApplicationHelper
  include Pagy::Frontend
end

With Pagy installed and ready to use across the application, update app/controllers/cards_controller.rb to paginate records on the index page:

def index
  @pagy, @cards = pagy(Card, items: 10)
end

Finish up our traditional pagination implementation by adding a pager UI to app/views/cards/index.html.erb:

<div class="w-1/2 mx-auto">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Cards</h1>
    <%= link_to 'New card', new_card_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="cards" class="min-w-full">
    <%= render @cards %>
  </div>
  <div id="pager" class="min-w-full my-8 flex justify-between">
    <div>
      <% if @pagy.prev %>
        <%= link_to "< Previous page", cards_path(page: @pagy.prev), class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800" %>
      <% end %>
    </div>
    <div class="text-right">
      <% if @pagy.next %>
        <%= link_to "Next page >", cards_path(page: @pagy.next), class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800" %>
      <% end %>
    </div>
  </div>
</div>

Perfect. With that update in place, we can now paginate records on the cards index page. Test this out by starting up your Rails application with bin/dev from the terminal and heading to localhost:3000/cards.

While Pagy does all of the heavy lifting for us with pagination, the approach to pagination with Turbo Streams and Stimulus that we will build in the rest of this article will work with any paging strategy. Pagy is optional.

Next, let’s switch from traditional paging in our index view to a manual version of infinite scrolling with a “Load More” button powered by a Turbo Stream.

Manual “Load more” button

Our goal in this section is to replace the Next and Previous buttons in our pager with a single “Load more” button. When the user clicks to load more cards, we will send a Turbo Stream request to the server, retrieve the next page of results, and append the new cards to the existing list. When the user hits the last page of records, the button will be hidden.

Let’s start by moving the pager HTML to a partial. Create the new partial from your terminal:

touch app/views/cards/_pager.html.erb

And fill that partial in:

<div id="pager" class="min-w-full my-8 flex justify-center">
  <div>
    <% if pagy.next %>
      <%= link_to(
        "Load more cards",
        cards_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_stream: ""
        }
      ) %>
    <% end %>
  </div>
</div>

Our pager now has a single “Load more cards” link, pointing to the cards index path and passing in the next page as the page param. Easy.

The more interesting thing to note is that the link_to has a new data attribute: data-turbo-stream. This attribute is required to enable the new Turbo Stream GET behavior that allows us to respond with Turbo Stream actions in a GET request.

See the discussion on the PR that introduced this new functionality if you are curious why we need to opt-in to this behavior with a data attribute.

Now update app/views/cards/index.html.erb to use the pager partial:

<div class="w-1/2 mx-auto">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Cards</h1>
    <%= link_to 'New card', new_card_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="cards" class="min-w-full">
    <%= render @cards %>
  </div>
  <%= render "pager", pagy: @pagy %>
</div>

With the Turbo Stream-enabled paging link in place, our last step is to add a Turbo Stream template for the index page. From your terminal:

touch app/views/cards/index.turbo_stream.erb

When turbo-rails receives an inbound Turbo Stream request, it automatically checks for a matching view with a turbo_stream extension. If a matching view is found, it is used. If no Turbo Stream view is found, we fallback to the normal HTML view.

Fill in the new turbo_stream.erb view to finish the manual infinite scroll feature:

<%= turbo_stream_action_tag(
  "append",
  target: "cards",
  template: %(#{render @cards}) 
) %>
<%= turbo_stream_action_tag(
  "replace",
  target: "pager",
  template: %(#{render "pager", pagy: @pagy})
) %>

Here we are sending two Turbo Stream actions to the frontend to be processed by Turbo’s JavaScript. The first appends the new set of cards to the existing card list. The second replaces the pager partial with a new version of the partial. Both actions are built with the turbo_stream_action_tag helper from turbo-rails

We replace the page to update the next page value in the “Load more” button. When there are no pages left, the replace action will remove the pager component from the page. This need to update the pager with a Turbo Stream action is why we moved the pager to a partial at the beginning of the section.

Refresh localhost:3000/cards and try out the new Load more button. If all has gone well, it should work like this:

To follow along with what’s happening here, check your Rails server logs. When you click the button to load more records, you will see Rails receives the request as a TURBO_STREAM request, which is what the data-turbo-stream attribute enables for GET links (and forms).

Try removing the data-turbo-stream attribute from the button. If you do, you will see the request revert to a normal HTML request. Without the data-turbo-stream attribute, clicking the link results in a full page turn, replacing the list of records with the next page of records.

Great work following along so far! The last section of this tutorial will demonstrate automatic infinite scroll with the IntersectionObserver API and a small Stimulus controller. To make using the IntersectionObserver API easier, we will add the handy stimulus-use package to our application.

Infinite scroll with Stimulus controller

While the Load more button we have now is nice, even better (if your goal is trapping people in a social media net, or following along with this tutorial for a few more minutes) is the list of cards updating automatically as the user scrolls to the end of the list.

From your terminal:

yarn add stimulus-use

Staying in your terminal, generate the Stimulus controller:

rails g stimulus autoclick

Head to app/javascript/controllers/autoclick_controller.js and fill it in:

import { Controller } from "@hotwired/stimulus"
import { useIntersection } from 'stimulus-use'

export default class extends Controller {
  options = {
    threshold: 1
  }

  connect() {
    useIntersection(this, this.options)
  }

  appear(entry) {
    this.element.click()
  }
}

This controller pulls in useIntersection from stimulus-use. The appear function is triggered when the element the controller is attached scrolls into view in a user’s browser. appear simply calls click() on the element the controller is attached to, saving the user the effort.

To use this controller, update the pager partial:

<div id="pager" class="min-w-full my-8 flex justify-center">
  <div>
    <% if pagy.next %>
      <%= link_to(
        "Load more cards",
        cards_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_stream: "",
          controller: "autoclick"
        }
      ) %>
    <% end %>
  </div>
</div>

To push our load more button down below the fold so autoscroll doesn’t fire 5 times as soon as the page loads, update app/views/cards/_card.html.erb to add a little extra to each card:

<div id="<%= dom_id card %>">
  <p class="my-5">
    <strong class="block font-medium mb-1">Name:</strong>
    <%= card.name %>
  </p>

  <% if action_name != "show" %>
    <%= link_to "Show this card", card, class: "rounded-lg py-3 px-5 bg-gray-100 inline-block font-medium" %>
    <%= link_to 'Edit this card', edit_card_path(card), class: "rounded-lg py-3 ml-2 px-5 bg-gray-100 inline-block font-medium" %>
    <hr class="mt-6">
  <% end %>
</div>

The markup here is the default markup you get when you use Tailwind through the tailwindcss-rails gem.

Now head back to localhost:3000/cards one more time and see that new records are automatically retrieved as you scroll down the page.

Thanks for following along with me today, you’ve made it to the finish line!

Wrapping up

Today we used the brand new GET Turbo Stream functionality added in Turbo 7.2 to build manual and automatic “infinite” scroll in a Rails application. The big breakthrough here, and the reason I wrote up this article is the removal of the restriction on using Turbo Streams with GET links and forms.

In previous versions of Turbo, to accomplish this we had to sneak in Turbo Stream actions inside of “empty” Turbo Frames. This technique worked fine, but it added a layer of indirection that could make following what was happening more difficult, especially for developers that were new to Turbo. Rendering an empty Turbo Frame so it could be used to insert Turbo Stream actions into the page was odd — it worked fine, but it always felt uncomfortable.

This article demonstrates the new data-turbo-stream functionality with pagination, but a similar approach can be used to simplify the implementation of other common web application use cases like modals and drawers and search forms.

For further reading on the changes in Turbo 7.2, keep an eye on the release notes for Turbo and turbo-rails.

As always, thanks for reading!

Interactive charts with Ruby on Rails, StimulusReflex, and ApexCharts

2022-09-19T00:00:00+00:00

Every B2B web app eventually gets charts. Users love them, buyers love them, sales teams love them. Look at all this data! Look how pretty it is! Charts!

I love charts too. So, let’s build a filterable pie chart with Rails and StimulusReflex. Our chart will rely on data from the database, and, because it is 2022, we will be able to update the chart as the user applies a filter without requiring a full page turn.

To render our charts, we will use ApexCharts, but the technique shown in this article will work fine with any other frontend charting library, I just like ApexCharts.

When we are finished, we will have a Rails app with a chart that looks like this:

(It looks nicer when it isn’t being captured as a low quality gif!)

Before beginning, this article assumes that you are comfortable with Ruby on Rails and that you have a passing familiarity with Stimulus and that you have written a Stimulus controller or two before. You won’t need any prior experience with StimulusReflex to follow along with this article.

As always, you can find the completed code accompanying this article on Github.

Let’s dive in!

Setup

To skip some uninteresting configuration steps, start today by cloning this repo to your local machine, working from the main branch. Make sure you have node installed and redis running locally.

The repo comes with Rails 7 using esbuild via jsbundling-rails to bundle JavaScript, StimulusReflex installed, and two models, Job and Applicant, which will be used to provide data to the fancy chart we are going to build. Jobs and Applicants can’t be interacted with in the UI. A seed file has been provided to build enough test data for our chart to use.

From your terminal, set everything up with:

bundle install
yarn install
bin/rails dev:cache
bin/rails db:prepare db:seed

And then boot the app and build assets with:

bin/dev

Head to localhost:3000 and see the beautiful “dashboard”, where the new chart will live by the end of this article.

Adding a static chart

To display our chart, we are going to use the handy ApexCharts JavaScript library. ApexCharts will handle rendering the pie chart and animating the transitions when we apply filters to the chart — we give it the data, it does the rest.

Install ApexCharts from your terminal:

yarn add apexcharts

To interact with ApexCharts we will use a Stimulus controller. From your terminal again:

rails g stimulus pie_chart

Head to the generated Stimulus controller at app/javascript/controllers/pie_chart_controller.js and fill it in:

import ApplicationController from './application_controller'
import ApexCharts from "apexcharts"

export default class extends ApplicationController {
  static targets = ["chart"]

  static values = {
    labels: Array,
    series: Array
  }

  initialize() {
    this.chart = new ApexCharts(this.chartTarget, this.chartOptions);
    this.chart.render();
  }

  get chartOptions() {
    return {
      chart: {
        type: 'pie',
        height: '400px',
        width: '400px'
      },
      series: this.seriesValue,
      labels: this.labelsValue,
    }
  }
}

Later in this article, this PieChartController will call a server-side reflex, so we extend this controller from the base StimulusReflex ApplicationController.

In the initialize method, we render a new ApexCharts chart, passing in the configuration options with chartOptions.

More details on the options available here can be found in the ApexCharts docs.

series and labels are set using Stimulus’ values. Stimulus values allow us to read and write HTML data attributes, providing a simple way to pass data from the server to our ApexCharts instance. Let’s connect this controller to the DOM to see how that looks in practice.

Head to app/views/dashboards/index.html.erb and update it:

<h1>Applicant Tracking System Dashboard</h1>
<div
  data-controller="pie-chart"
  data-pie-chart-labels-value="<%= @labels %>"
  data-pie-chart-series-value="<%= @series %>"
>
  <div>
    <h2>Applicants over time</h2>
  </div>
  <div data-pie-chart-target="chart"></div>
</div>

Here we use the data-pie-chart-labels-value and data-pie-chart-series-value to pass in the server generated data, and then render the chart in the empty data-pie-chart-target div.

Before this will do anything but render an empty chart we need to set the values of @labels and @series in the controller action.

Update app/controllers/dashboard_controller.rb like this:

class DashboardController < ApplicationController
  def index
    data = Applicant.includes(:job).group('status').count
    @labels ||= data.keys.map(&:humanize)
    @series ||= data.values
  end
end

Nothing too fancy here, just Ruby. We query the database for all of the applicants grouped by the applicant’s status, which in our case will be either “active” or “inactive”.

The query returns a hash, with the status values as a the keys and the number of applicants in each status as the values, something like this:

{
  "active": 50,
  "inactive": 50
}

The keys become the labels in our pie chart and the values become the series data in the pie chart.

With this change in place, refresh the dashboard and you should see the pie chart render, like this:

Your percentage of applicants in each status will probably be different than mine, but you should have a chart!

Right now the pie chart just shows us the status of all applicants in the database which isn’t very useful. Let’s add a filter to drill down to applicants for a specific job, applying the new results without a page turn.

Add job name filter to chart

We want users to be able to filter our chart by the name of a job posting. To do this, we will add a select input to the dashboard. When the user changes the value of the select, we will query the database for data specific to the selected job and re-render the chart with updated data.

To start, let’s add the select input to the dashboard. In app/views/dashboards/index.html.erb:

<h1>Applicant Tracking System Dashboard</h1>
<div
  data-controller="pie-chart"
  data-pie-chart-labels-value="<%= @labels %>"
  data-pie-chart-series-value="<%= @series %>"
>
  <div>
    <h2>Applicants over time</h2>
  </div>
  <form>
    <%= select_tag "job_id",
      options_for_select(
        Job.order(:name).pluck(:name, :id)
      ),
      include_blank: "All jobs",
      data: {
        action: "change->pie-chart#update"
      } %>
  </form>
  <div data-pie-chart-target="chart"></div>
</div>

Note the addition of the <form> input and the select_tag that rolls up all of the jobs in the database and plucks the name and id. In a real application, we might want to be a little more selective in building the option list, but our database only has three jobs so it will be okay.

Also on the select is an associated Stimulus action. When the select changes, pie-chart#update will be called. update doesn’t exist in our Stimulus controller yet, so let’s add that now.

In app/javascript/controllers/pie_chart_controller.js add these two new methods to the controller:

update(event) {
  this.stimulate("PieChart#update", event.target, { serializeForm: true })
}

afterUpdate() {
  this.chart.updateOptions(this.chartOptions);
}

Hey, StimulusReflex code! update calls this.stimulate, a method provided by StimulusReflex that we have access to in PieChartController because it extends ApplicationController.

As described in the docs, stimulate allows us to call server-side reflexes (in this case, the yet to be created PieChartReflex#update method) from our JavaScript code. Because the server needs access to the form data from the frontend, we add the serializeForm option to our stimulate call.

After the reflex runs, afterUpdate tells ApexCharts to re-render our chart with the handy updateOptions method provided by ApexCharts.

Almost there.

Before changing the job name select will do anything but generate an error, the PieChart reflex that our Stimulus controller calls must exist.

From your terminal, create the reflex:

touch app/reflexes/pie_chart_reflex.rb

(In normal times, StimulusReflex provides a generator to create both a new _reflex file and the associated _controller JavaScript file. While they prepare for the official release of the next version the generator rake task is in flux, so we are not using it in this article.)

Fill that new file in:

class PieChartReflex < ApplicationReflex
  def update
    data = retrieve_data(params[:job_id])
    @labels, @series = [data.keys.map(&:humanize).to_json, data.values.to_json]
  end

  def retrieve_data(job_id)
    Applicant.includes(:job).for_job(job_id).group('status').count
  end
end

Accessing the form data from the client side is as easy as params[:job_id] because of the serializeForm option in our stimulate call on the client side. Using the job id, we retrieve updated applicant details from the database and split that data out into @labels and @series instance variables.

Here the really important thing to note is that we are setting the same instance variables that the index method in the DashboardController sets. Recall that in the index method we only set the value of @labels and @series if they are not already set:

data = Applicant.includes(:job).group('status').count
@labels ||= data.keys.map(&:humanize)
@series ||= data.values

When we call a reflex method, StimulusReflex reruns the current page’s controller action and renders that page’s ERB template after processing the reflex, while retaining any instance variables we have set, as described in this handy diagram from the StimulusReflex documentation.

Because we are only updating a piece of the page, we could use a selector morph instead of the default page morph. If we wanted to be really efficient, we could use a CableReady operation combined with a nothing morph to only update the two data attributes that we care about without touching anything else on the page.

For our use case, a page morph works fine. Selector and nothing morphs are more precise tools that come in handy as your pages get larger and use cases get more complex. We could also fiddle a bit with the index method in the DashboardController. When StimulusReflex reruns the index method, we make an unnecessary database call, retrieving data that we will never use. If we wanted to, we could ensure that the index method skips the data query when @labels and @series are not set, but that’s an unnecessary optimization for our tutorial.

After the reflex finishes, the afterUpdate life-cycle callback in the Stimulus controller runs which triggers ApexCharts to animate in the changes to chart.

One last step before this will work. In retrieve_data, we use a for_job scope that doesn’t exist yet.

Head over to app/models/applicant.rb to add that scope:

scope :for_job, ->(job_id) { job_id.present? ? where(job_id: job_id) : all }

Now we are all set. Refresh the dashboard, change the job name input and see the chart smoothly animate in the changes to the data.

Incredible stuff.

Great work following along today, you’ve reached the end of this tutorial!

Wrapping up

Today we worked through building a filterable chart that retrieves new data from the server without requiring a page turn. To do this, we used Ruby on Rails along with Stimulus and StimulusReflex to power the chart interactivity, and ApexCharts to save us from trying to build our own chart renderer in a brief tutorial.

This is a small, simple example of what you can do with StimulusReflex and CableReady, and hopefully it provides a good idea of the simplicity and flexibility of one of my favorite libraries in Rails-land.

To learn more about StimulusReflex, the docs are the best place to start. Note that the team is nearing final release of 3.5, a long-in-the-works release that will bring with it support for installing StimulusReflex with a single command.

Until 3.5 officially releases, installing StimulusReflex into a non-webpacker Rails app requires a bit of manual work. Marco Roth’s Stimulus Reflex esbuild repo outlines the steps to get up and running manually and will be a great resource until the 3.5 release. Thanks to Marco for providing his example repo, which served as my starting point for this tutorial.

If you enjoyed this article, you might enjoy my (free) book, Hotwiring Rails, from which is article was adapted. In the book, we build a fully functional Rails 7 application from scratch, focusing on adding interactive features with Turbo, Stimulus, CableReady, StimulusReflex, and friends.

As always, thanks for reading!

Hotwiring Rails Newsletter - August 2022

2022-08-01T00:00:00+00:00

This newsletter went out via email to the Hotwiring Rails subscriber list, but I know that not everyone wants another email cluttering up their inbox. For those folks, I’ll always publish the full content of the newletter here too, so you can get all the content with none of the emails.

Thoughts or feedback on how I can make this newsletter more valuable? Have something you’d like to include in next month’s edition? Send me an email, or find me on Twitter.

Articles and guides
New and interesting PRs and releases
My recent writing
The best things I read recently

Articles and guides

Fly Reactive Rails by Julian Rubisch

Julian dives deep into how he deploys reactive Rails applications on Fly.io. Fly takes (a lot) more work to get an application deployed compared to Heroku and Render, but for the right type of application, there are major benefits.

Read the article on minthesize.com

Vite-lizing Rails by Vladimir Dementyev

Keeping up with the front end bundling world is exhausting, but Vite is worth taking a look at, especially if you’re considering making a switch from Webpacker to the new cssbundling/jsbundling options added to Rails 7. Vite is a nice alternative to esbuild if you want a more full-featured option with glob imports, HMR, and live reload ready to go (almost) out of the box. You can get there with esbuild, but it requires a bit more work and the developer experience is not quite as nice.

Vladimir’s write up on migrating a Rails app to Vite is very straightforward and, as a nice bonus, it includes a look at his Docker setup for Vite, if you’re into that sort of thing.

Read it on evilmartians.com

Using Turbo Streams with Kredis by Mansa Keïta

This is a quick, interesting look at using Turbo Stream broadcasts without a backing ActiveRecord model. Kredis + a PORO gets the job done as Mansa builds a simple counter app that stays in sync across browsers, even when updates are triggered from the Rails console.

My favorite tip here is using Turbo::StreamsChannel to broadcast directly to a channel, which is a nice power move for Streams users.

Read the tutorial on dev.to

Booleans don’t exist in Ruby by Mike Burns

Mike takes us on a fun trip to answer the question of whether or not Ruby has booleans. Does Ruby have booleans? As Mike says, “That’s a question for you and Ruby to answer together.”

Explore the depths of Ruby’s boolean implementation on thoughtbot.com

Video | Subdomain Routing in Rails by CJ Avila

This video from CJ shows you how to use subdomain constraints to handle subdomain routing in Rails. This is an area that can get tricky, especially if you’ve never needed to think much about dealing with subdomains while developing locally. CJ’s thorough look at how it works (and live trouble shooting) is a good introduction to the problem.

Watch it on Youtube

Galaxy Brain CSS Tricks with Hotwire and Rails by Matt Swanson

This article has a few simple, effective tips for leveraging CSS to clean up your Rails code. Each tip is bite sized, general purpose, and easy to apply to your own code. Note that a few are Tailwind specific, but not all. The highlight for me isn’t a CSS tip specifically, but instead a plea to stop interpolating CSS classes in your views by using Rails’ tag and class_name helpers instead.

5 minutes in this article will probably send you away with something new to add to your bag of tricks.

Read it on boringrails.com

Video | Naming things: rails new by Joel Drapper and Kasper Timm Hansen

Naming Things is a neat new addition to Rails-land from Joel. In the inaugural episode, he pairs with Kasper on designing and building out a bit of an event scheduling application.

As a viewer, you’re listening in on their pairing session as they think through the design of the application and start piecing together some of the code — I really enjoyed hearing two very talented Rails devs talk through problems together. Getting that kind of view of how other folks work is neat, IMO.

Watch it on namingthings.org

New and interesting PRs and releases

Turbo has a new 7.2 release in beta, and there are some really big changes that I was not expecting to see get merged. Some highlights:

Allow Turbo Streams with GET (with an important follow up PR): This is a huge change that should eliminate the need for clunky, Turbo Frame workarounds for responding to GET requests with a Turbo Stream action. There are immediate, obvious applications here improving the DX of building search interfaces and pagination with Turbo.

Support custom rendering in turbo events: This PR came in last year and wasn’t picked up but has gotten a refresh for the 7.2 release. The changes here make it easier to extend Turbo to work with third party libraries like morphdom.

On the other side of the Hotwire stack, Stimulus also has a new release in the works with support for new action options plus support for custom action options merged into the main branch.

I’m excited to see such big changes coming for both Turbo and Stimulus — both projects are growing in popularity and the ongoing development is a great sign of the long term health of the Hotwire stack.

My recent writing

Along with taking a break from this newsletter, I took a break from blogging for the last few months, but I’m off the break and back to writing. In July, I focused on non-tutorial writing so these two pieces might be a little different than what you’re used to. I’ll be back to tutorial writing this month.

Rails is back isn’t enough: Some thoughts on the scarcity of junior level positions in certain parts of the Rails world, and what that scarcity means for the health of Rails on very long time scales. Twitter thought this was a Good One.

Writing effective coding tutorials: I read a lot more articles about Rails than what I include in this newsletter. A lot more. Many of them are great technically, but very hard to get value from.

In this (extremely niche) article, I compiled a few of the common mistakes I see engineers make when writing coding tutorials, along with simple ways to fix those mistakes and deliver a better end product to readers.

The best things I read recently

I’m job searching right now, and so I’ve spent a lot of time thinking about (and retelling) the time I spent building the engineering and product org at my last company. I told a very small piece of that story in Rails is back isn’t enough, linked above.

Expanding a bit on the story of my time at that company: My extremely small team (1 engineer, 0 designers, me as PM + eng in one) was tasked with building an entirely new product adjacent to our existing product. The existing, established product had a long queue of planned work needed to keep our edge in the market. This wasn’t a great plan, in retrospect.

Taking on an entirely new product without a dedicated team to build it was a bad idea. We needed to choose one product, or we needed to add enough resources to execute well on both products at the same time. Instead, the company chose to try to do both without adding enough resources. As a result, the new product suffered from half-baked ideas and a lack of vision because the business continued to put most of its energy into the existing, established product. That product paid the bills, so the math was easy.

This seems like a simple lesson in retrospect, but I see these sorts of poor strategic decisions made all the time — chasing new opportunities without consolidating existing wins, spreading focus across multiple initiatives because narrowing focus is scary, or individuals with 100 ideas, each of which gets 1% of their energy. Companies try to implement things like EOS to combat this focus problem, but ultimately it takes courage and conviction to hold the line when facing constant distractions. Organizational management models only get you so far.

That was a very long introduction to this article from Meta’s CTO, Andrew Bosworth, on ruthless prioritization. Aptly titled “Half staffed is unstaffed” it distills many of the thoughts I’ve had on this recently, and is worth a read if you are thinking about how to allocate your resources (personally, or for a team) more effectively.

At the organizational level the best solution I have found is to always staff from the inside out. You must fully fund the core and if you can’t do that then shut it all down. If you have space beyond that you can continue to fund things fully, one at a time, in accordance with your portfolio of goals. When you can’t fully staff another team then you stop.

Read the article on boz.com

Until next time

It is nice to be back with all ~500 of you after a couple of months away! In case you missed it, Rails conf talks are up on YouTube and there is a new Ruby podcast that you might enjoy.

If you enjoyed this newsletter, send me a note and let me know or share it with a friend who might enjoy it too.

PS — I’m actively looking for a new full-time role (remote, in the US).

I’m most interested in product or engineering leadership roles at early stage companies, but I’ll happily consider IC roles in product or engineering on kind teams doing interesting work. You can reply to this email or find me on Twitter if you know of an opportunity that might be a match.

You can find my odd work history on LinkedIn.

As always, thanks for reading!

Rails is back isn't enough

2022-07-30T00:00:00+00:00

The article below is on a topic that has bothered me for a long time — the lack of opportunity for junior Rails developers — spurred by some recent Twitter energy around Ruby on Rails being “back”. I got mentioned in one of the viral threads about Rails being back which jump started my brain into writing this article, but I don’t write it to admonish folks who are excited about Rails and want to bring in more Rails developers with educational content. I love that work, and appreciate the positive energy, and the commitment to building great content. The Rails is back energy just happened to kick off this train of thought about what the long term outlook of the Rails world is when there are few paths for folks to make their living as junior Rails developers.

2022 is an exciting time for Rails developers.

The release of Rails 7 brought Hotwire into the default Rails package along with new asset bundling solutions to get applications off of the dreaded Webpacker. For fans of Redis, the new Kredis gem is a suggested default for Rails 7 applications, offering tools to make it easier for Rails devs to do fun stuff with Redis.

Outside of officially endorsed Rails projects, the StimulusReflex team is steadily making improvements to StimulusReflex, CableReady, and the related set of projects that make building modern, reactive applications in Rails easier than ever.

Despite all of that excitement, folks in the Rails community have gotten used to dealing with “Is Rails dead? Rails is dead.” questions and comments that discourage developers from choosing Rails for their next project.

“Is Rails dead?” questions are silly if you spend any time keeping up with the Rails ecosystem. Rails isn’t dead, it is alive and kicking, just like it has been for nearly 2 decades. Many billions of dollars have been earned riding the Rails and new businesses continue to be built on the foundation that Rails provides.

As a reaction to “Is Rails dead?”, Rails enthusiasts are leaning into a “Rails is back!” narrative — we aren’t dead, we’ve got all this cool new stuff to share!

I don’t think the noise of “Rails is dead!” and “Rails is back” is particularly interesting. Neither narrative adds any value to the conversation. As a Rails enthusiast (and sometime Rails content creator), I love the idea of “Rails is back”, but I don’t think it gets us where we are trying to go. To begin, “Rails is back!” isn’t very helpful because Rails isn’t back. It never left. Its place in the world of web frameworks just changed. No new JavaScript library or gem will fundamentally change Rails’ value proposition or turn back the clock to a time when Rails was the hottest framework on the block. That time has long gone.

Instead of worrying about Rails coming back, I’m more interested in how to make Rails more accessible to new developers. The long term health of the Rails community relies on attracting and nurturing new Rails enthusiasts, not how loudly we yell about the cool stuff you can build with Hotwire (and I’ve yelled very loud about it myself). I don’t see any evidence that Rails is back for junior developers, and I don’t see much indication that it will be without major philosophical shifts from folks hiring Rails developers.

There are very few options for junior developers to get their start in Rails. New web developers are likely to have far more opportunities if they learn React and Node to land their first gig. The Rails world is welcoming and kind (MINASWAN is real, in my experience), but we don’t make it easy for new developers to earn a living. Unless we fix this problem, Rails will eventually lose real momentum — the people leading Rails today won’t be pushing the framework forward in the decades to come.

Speculation time: I suspect that many companies hiring Rails developers are hiring exclusively for senior level roles because companies using Rails tend to run quite lean.

At my last company we built a multimillion dollar SaaS business with a literal handful of engineers. When I left after ten years, we had two large, complex B2B products supported by eight full-time engineers and one engineer manager. We had less than five engineers for the first five years of the company’s life. Our competitors, mostly using the latest JavaScript frameworks (one even built their own, for unknowable reasons), had dozens or hundreds of engineers. To keep up, we had to be dramatically more efficient than them.

Rails let us run that lean, but it came with an obvious tradeoff. When you are running lean, there is very little space to provide a supportive environment for juniors. In my time there, I hired three junior developers — one miraculously figured it out with very little guidance outside of what she could learn by osmosis and became a very talented lead. The other two left because my team didn’t provide the environment they needed to learn and grow at a reasonable pace. After those failures, I did the same thing a lot of engineering leaders have done and stopped hiring juniors. From what I know of other companies in the Rails world, this is a common experience. Hire juniors, fail to support them because there is no time to do so, see the juniors fail, stop hiring juniors.

Over time this cycle leads to a dearth of junior and entry level positions for Rails developers. And without junior developers, the pipeline of senior talent eventually runs dry. If there are no juniors slots open, where do you expect the seniors to come from?

Eventually, this has severe consequences for the health of the Rails world, and makes me skeptical of any unqualified claims that Rails is back. If you can’t recruit senior Rails talent to keep up with demand as your product grows, that becomes a business risk. I’ve seen this personally — hiring experienced Rails developers is extremely hard, to the point of being nearly impossible for small companies without large budgets and a well-known brand.

In the future, Rails is at risk of becoming non-viable for new businesses with big aspirations. Not for technical reasons but because recruiting Rails developers is just too difficult. We aren’t anywhere near that yet, but the talent deficit will eventually lead us there. Responsible founders and CTOs will choose other options because they know they’ll be able to build a high quality team faster.

In my current job search (I’m available for engineering and product leadership roles, dear reader) I have heard a version of this story multiple times from CTOs who profess a love for Rails but chose another option for their company because they didn’t think they could hire enough Rails developers to keep up with their growth plans. As much as I love Rails and the power that Turbo, Stimulus, and friends provide, cool new toys won’t fix the talent pipeline problem.

Could the new toys attract new talent to Rails (or bring some developers back to Rails)? Yes, sure. I’ve talked to a number of companies that are reevaluating their options because of the efficiency gains they hope to see from Hotwire; however, those are usually companies that are already using Rails extensively and just want to get away from React on the frontend. That’s not going to move the needle much in the long run.

What do we do?

I’m not sure. I don’t have a grand idea of how to fix the talent pipeline problem. I’m just a guy that ran a Rails shop for a while and now spends a lot of time thinking about Ruby on Rails. Maybe all of my ideas here are wrong!

Maybe there isn’t even a talent pipeline problem and I’m just a huge goof that was bad at recruiting and worse at long-term planning. Maybe I’ve had the bad luck to talk exclusively to other companies that are also bad at recruiting and long-term planning.

Assuming there is a talent pipeline problem, and that the pipeline problem is getting worse, here’s some more speculation and a bit about what might help:

I think the talent pipeline problem is tightly coupled to what makes Rails so attractive in the first place. The Basecamp team has a huge influence on the Rails world, and they showed us that shockingly small, self-funded companies can build large, complex, innovative products that compete and win against established players. That’s really cool! They have had extraordinary success with a very small team. Emulating that success is a huge driver for many folks building companies on Rails, for good reason.

But it is easy to move from “we don’t need a big team to get great results” to a related but less defensible idea of staying extremely lean and only hiring when it really hurts. Speaking from experience, if you can do a lot with a little, it starts to feel like your competitive advantage is your tiny, very experienced team. Go far enough down that road and you will find yourself building a company that can’t support and nurture junior talent (and burns out your senior talent).

With the dangers of the extremely lean ethos in mind, one path to fixing the talent pipeline problem might be reevaluating what building a successful company looks like, and rethinking expectations placed on engineers in Rails shops.

Can you build a great product and get it to market with a few really talented Rails engineers? Yes, absolutely. At my last company, we were doing millions in revenue (and profit each year) with two (2) full-time engineers and one of those engineers was me, fresh out of a bootcamp. Eventually, that masochistic commitment to being lean put tremendous strain on our engineering team, and I suspect our experience matches any other company that takes the extremely lean, self-funded route. It just starts to hurt too much and you cut everything that isn’t shipping code.

As you grow, redundancy, sustainability, and reliability (of systems and people) all become more important than continuing to wear the extreme lean badge of honor. Investing in a pipeline of junior talent, giving your senior engineers time to coach and mentor those juniors, and slowing down your cadence to give everyone room to breathe in those new processes likely leads to a more successful company. It certainly leads to a more sustainable pipeline of talent moving up the ranks from junior to senior and beyond.

A single company rethinking the stay lean or die trying approach won’t fix the problem, but if you are in a position to influence growth plans at your Rails shop, think about what the future looks like on the two paths you can take. On one path, you stay lean, you hire seniors, and you keep struggling to find the 5+ years of Rails experience you require for each new hire. On the other path, you slow down a bit. You hire a mix of juniors and seniors, you invest in those juniors, and you build a pipeline of talented folks at all levels of the engineering org.

Which path feels better for the health of your business, your team, and the Rails world?

Writing effective coding tutorials

2022-07-18T00:00:00+00:00

Writing about what you learn, when done well, is a great way to solidify your own knowledge of a topic, stretch yourself professionally, and establish credibility in your field.

In 2020, I started writing tutorial-style articles for Ruby on Rails developers. Writing these articles began as a fun distraction from a stressful job, and as a way to reconnect with web development as a discipline as my day job pulled me further away from regularly writing code. From that beginning as a distracting hobby, I grew to have a passion for writing tutorials as a way to share my knowledge with internet strangers.

Over the last 18 months, my writing has helped me grow my professional network, find new consulting clients, earn a few thousand dollars in (self-published) book sales, and to feel much more confident in my technical skills.

As I’ve worked to improve my own writing, I have read many, many articles from other developers. Many of those articles have been valuable but I also see very common, correctable mistakes from folks who have knowledge to share but struggle with sharing that knowledge effectively.

Today, I’m sharing some of the most common mistakes I see tutorial writers make and how to correct those mistakes to write more effective tutorials. I will also share the simple formula I use when writing new tutorials. Before we begin, there are a few things to keep in mind.

I am writing this from the perspective of someone who writes web development tutorial articles. Most of my articles are “How to build X with Y”, almost always with a concrete, usable component of a web application built by the end of the tutorial. If you are already writing successful tutorials and feel confident in your own formula, you may not learn anything groundbreaking by reading this piece.

The ideas I present in this article should be generally applicable to technical tutorials, but some pieces of advice may not apply directly, depending on what you are writing.

This article does not discuss the distribution of articles or how to build an audience for your hot new tech blog. Today we are focused on improving the quality of your tutorial content. If you write higher quality content, you may be more likely to get some SEO wins or pick up traction in your Twitter niche, but quality is only a piece of the puzzle and, in my experience, only a small piece most of the time. You will not learn how to strike it rich by writing coding tutorials by reading this article, but hopefully you will learn a few things that help you improve your writing.

Also note that I am writing about mistakes I see when reading coding tutorials, especially those published on large platforms like dev.to and Medium where many aspiring writers get started. I am drawing from experience to write this article but I will not link to specific articles as examples of these common mistakes. I am not trying to put anyone contributing free writing to the programming community on blast, so you will have to take my word for it when I call something a “common mistake”.

Let’s dive in.

Write for a specific audience

When you publish a tutorial on the internet, you have no control over who will read your article. Anyone interested in the topic can click on the article and start reading. This means that you are likely to find readers of all experience levels and abilities, and, try as you might, you cannot write an article that effectively caters to all levels of experience.

Despite this, I often see coding tutorials that attempt to write for a huge range of experience levels. This happens most often when the article is demonstrating a very complex interaction requiring strong knowledge in the library or language being used while also spelling out for the reader how to do basic tasks like assigning a variable or opening their terminal.

Instead of trying to write for all possible levels, decide who you are writing for, tailor the content of the article to that level, and then explicitly state who will get the most out of the article in the introduction. If you are writing about an advanced technique that requires a solid grasp of the tech stack you are writing about, tell the reader that and then skip over detailed instructions and explanations for pieces of the project that your advanced readers should understand. If you are writing for more junior folks, slow down and explain things in more detail, narrow the scope if needed, and keep the code simpler to avoid overwhelming the reader with concepts they may be unfamiliar with.

Make a decision about who you are writing for, state that decision clearly, and then write each section of the tutorial with that audience in mind.

Make a promise and keep it

Early in your article, tell the reader what the tutorial will teach them, and then follow through on that commitment.

After reading the first few paragraphs of the article, your reader should know what you are going to help them build and what tools you will use to build it.

Your promise should be clear and specific, like this: “In this article, we are going to use Ruby on Rails and Stimulus to build a drag-and-drop interface for reordering items in a list and saving the users changes to the database. We will use Tailwind to make the list look fancy and the drag-and-drop UX will be powered by Dragula.”

In two sentences, the reader knows exactly what to expect from the rest of the article and they can decide if it is something they are interested in reading.

Once you have made that commitment to the reader, the rest of the article should follow through on that commitment. If you committed to building a drag-and-drop interface with Rails and Stimulus, that’s what every piece of the article should be about.

Most commonly, writers miss the mark on this by never making a promise to the reader. They just dive right into the code without making it clear what is being built. In the worst cases, the tutorial does not have an introduction at all, we just dive right into the code with no context.

Perhaps controversially, I do not think you should rely on the article’s title as the only place you make the article’s goal clear to the reader, especially for more complex tutorials. For the drag-and-drop example, the article title might be “Building a drag-and-drop UI with Rails and Stimulus”. This is a good start, but expanding on that promise early in the body allows us to be more clear about the precise set of tools and techniques used to accomplish the goal. You may find that you can make the promise in the title in some cases, but I find restating the promise in more detail to be more effective at setting the stage for the reader.

Focus on what you’re teaching

After you make a promise to the reader about what you are teaching them, avoid distracting the reader with unexpected or unnecessary code. They signed up to learn X, not Z and W.

This means stripping the tutorial down to its bare essentials so that you can start meeting your promise to readers as efficiently as possible. Distractions often creep in to coding tutorials in these forms:

Long and complex setups before you can begin working on whatever you have promised to teach the reader.
Adding complexity to the project that does not help the reader learn about what you promised to teach them. This often happens when the writer wants to show off, not teach. Unless the purpose of the article is to teach the reader that fancy new technique you just learned, keep it simple.

A useful technique to eliminate complicated or long setup steps is to provide a starter repo for readers to clone so they can bypass setup steps that are not part of what you are teaching and dive right into learning. You can see this approach in action in one of my recent articles which required a non-trivial amount of setup to make good use of the library the article was focused on.

A more general technique to keep your readers focused is to read through your first draft and look out for code that could be simplified or that is unrelated to what you are building. Simplify what you can, remove code that is not necessary, and trim explanatory content that is not essential to understanding how the code works.

Create visual checkpoints

Web development tutorials are almost always focused on building a thing that appears on the screen, but writers commonly forget to show the reader what they’ve built.

Visuals (images, gifs, and videos) serve two purposes in tutorial articles.

First, they show the reader what they will build. “A drag-and-drop UI” probably means the same thing to most readers, but adding a gif that shows the drag-and-drop UI in action removes any ambiguity. I like to add an image or gif of the finished product to the introduction of every tutorial I write to make sure the reader knows exactly what we are building.

Second, visuals are a great way to create checkpoints for the reader as they progress through the article. Working all the way through an article only to reach the end and realize that your code isn’t working is a bummer — help your readers avoid this by pausing after each section to check in on what the code they have written should be doing, and, whenever possible, give them a screenshot or a gif to show it in action.

In our drag-and-drop example, our checkpoints might break down to:

Display a static, unstyled list of items on a page
- Visual support: A screenshot of the list of items in a browser
Add drag-and-drop functionality to the unstyled front end
- Visual support: A gif of the items being reordered via drag-and-drop
Persist new order of elements to the database
- Visual support: A gif of items being reordered via drag-and-drop and then refreshing the page to demonstrate the new order persisting
Styling the list
- Visual support: A gif of drag-and-drop list with updated style and animations in use

Taking the time to create these checkpoints in the tutorial and add a visual for each one is extra work. The payoff is that readers are less likely to get lost as they read your work.

If you are building something without a visual component, it is still helpful to include checkpoints in your tutorial and to try to get creative with ways to include visuals — screenshots of logs, terminal output, or an API response can all be useful ways to give the user a clear way to confirm they are on the right track as they work.

Build incrementally

Closely related to the concept of checkpoints in the last section is building incrementally. Too often, coding tutorials jump right to the finished product. For very simple articles, there may only be one step, but for almost everything worth writing an article about, there are enough steps to break the article down into pieces that can be built sequentially.

If you are reading this article, you have almost certainly encountered a “tutorial” that fails to build incrementally. The author promises to show you how to build a drag-and-drop UI, and then the content of the article is just blocks of code with no context or explanation of how the author arrived there. In the worst cases, you do not even get comments embedded in the code blocks, just a bunch of code that does… something. As a reader, you’re left to figure out the steps on your own and to piece together what all the context-free code does. Reading those tutorials is the owl meme, but for code.

Breaking your articles and explaining each piece as you go has two major benefits.

First, it forces you to think more deeply about the necessary components of what you are building (helping you focus your writing) and makes it much easier for readers to follow your thought process.

Second, breaking your work into coherent, incremental steps helps readers actually learn from your article, rather than just copy and pasting your finished code into their project without understanding what the code does. If you take the time to break your work up you can present the reader with the process you took to arrive at the finished product and you can spend time looking at the interesting decisions you made to build the solution.

Explain the code at the right level

The folks who jump right to the end without showing any of the incremental steps tend to miss the mark on this too. A coding tutorial should teach the reader how to reproduce what you have built. To do this, your article should explain what the code you have written does at an appropriate level of detail for your target audience.

If you are writing for beginners, go into greater detail about concepts and go a bit slower when code gets complex. It is better to explain more than you think you need to than it is to gloss over things that you think might be obvious. Avoid using jargon or terms beginners might not understand, and be generous about linking out to further reading on key concepts they will need to understand.

If a concept is likely to be too confusing for a beginner to grasp, consider whether you need to discuss that topic at all. Perhaps a better approach is to rewrite the code to use a less advanced technique that still solves the problem.

If you are writing for a more advanced audience, do not waste time and energy explaining in detail things they should already know. If you aren’t sure if they know it, mention it briefly and link out to documentation that goes deeper into a concept for those that need a refresher.

If the concept is not core to what you are teaching, feel free to gloss over it and trust that your more advanced audience will be comfortable digging into unknowns.

Check your work

This should be obvious, but all too often I work through an article and realize that the author has skipped a step, made a typo, or otherwise presented code that does not actually work.

Before you publish, read through your article, copy and paste each line, and confirm that it all works as you expect it to. You can’t control readers skipping details, failing to have the right tools installed, or otherwise getting stuck on things because they were in a rush. You can be sure that, when you publish an article, it works like it is supposed to.

You made the reader a promise, keeping it requires checking your work!

A simple formula to get started

Every coding tutorial I write follows the same basic formula. The choice to follow this formula allows me to spend more time thinking about the content of each article and less time worrying about how to present the article to readers.

This formula is intended as an example of what works for me, your mileage may vary. You can see this formula in use in articles like this, this, and this.

Introduction

Tell the reader what you are building in the article, what tools you will use to accomplish the task, and who you are writing for. Include a gif or screenshot of the finished product, when relevant. It usually helps to add a little bit about why the reader might want to read about this topic or why you have chosen to write the article.

In this section you may also find it useful to note the version of tools you are using (“I’m writing this article using Ruby on Rails 7 and Stimulus 3”) so that readers from the distant future know what to expect. I also like to include a link to a Github repo with the final code available so readers can compare their work to yours if they get stuck.

Setup the application

Any setup steps that need to be done to get the reader ready to start building go here. Include a link to a Github repo if setup is complex so readers can skip this step.

Build the application

Break each milestone into a separate section, explain each code block as you go, work in small units, and include visuals frequently to keep the reader on track.

Conclusion

Recap what the reader learned by following along with your tutorial. Add links to further reading and more resources when appropriate. Thank the reader for reading.

Closing thoughts

Writing tutorial articles has been a huge boon to my career. I have grown my skills as a developer and as a writer, built a website that > 0 people visit every day, found new consulting clients, and been able to give back to a community that has given a huge amount to me over the last decade.

My hope in writing this article is that at least one reader will learn something that makes them more successful as a writer. If that’s you, send me a note and say hello.

As always, thanks for reading!

Hotwiring Rails Newsletter - April 2022

2022-04-04T00:00:00+00:00

Thoughts or feedback on how I can make this newsletter more valuable? Have something you’d like to include in next month’s edition? Send me an email, or find me on Twitter.

Articles and guides
New and interesting PRs and releases
My recent writing

Articles and guides

Distributed Systems In Rails series by Nate Hopkins

This three part series is a super valuable and practical guide to thinking about and structuring distributed systems in Rails. Part one of the series is theoretical, parts two and three offer practical approaches to configuring Sidekiq and database connections properly in production.

Read the series on dev.to

The Problem That ViewComponent Solves For Me by Jason Swett

I’ve spent time exploring ViewComponent before, but I’ve always struggled to articulate exactly when I should reach for a component — my brain usually just defaults to helpers and then one day I wake up to a mess of helpers.

In this piece, Jason shares his thought process on why he reaches for ViewComponent in a way that helped me reason more clearly about my own code. No code examples in this one, just thoughts.

Read the article on codewithjason.com

Hotwire Handbook Part 1 by Phil Reynolds

This article is a series of small, well-explained features implemented with Turbo Streams.

While the official documentation lags behind, these types of practical, code-heavy demos are the best way for new folks to get started.

Read the handbook on philreynolds.dev

Import Maps Under The Hood In Rails 7 by Paweł Dąbrowski

Like many of the new front-end tools released with Rails 7, importmap-rails is not very well documented. This article from Paweł is a thorough look at what import maps are, how they work, and how to use them in your Rails application.

Import maps are not right for every use case, but if you want to explore, this is a great starting starting point.

Read it on appsignal.com

Dynamically Re-use and Lazy-load Pages Using Hotwire

This article describes an interesting technique from the Avo team to re-use views across the application, with a healthy dose of Turbo Frames in the mix. The use case here is a bit niche, but the technique is neat.

Read the article on avohq.com

New and interesting PRs and releases

The first Ruby 3.2 preview is out and the headline feature is WASM support. Smart and cool folks will undoubtedly find fun ways to use it, once they’ve had time to explore.

CableReady and StimulusReflex both got new pre-release versions, inching closer to major version bumps that have been in the works for many months. The highlight in both new pre-releases is shipping the package’s JavaScript along with the Ruby gem, making it much easier to use CableReady and StimulusReflex with importmap-rails.

In Turbo-land, a proposal to add a new Stream actions for adding and removing classes from elements has been open since late February — the PR addresses a big limitation of Turbo Streams. Given the philosophy of the project, I will be surprised if this proposal is accepted, but if it is it will be a nice upgrade.

More likely to be accepted is a PR adding support for preloading content with Turbo, improving on limitations of the native rel=prefetch behavior which can be tricky to use for dynamic content.

Finally, an important Turbo bug fix PR was opened, and hopefully will be merged soon. The bug fixed by this PR can be a blocker for migrating cleanly to Turbo, and the double turbo:load event firing can cause all sorts of hard to diagnose problems.

My recent writing

Toggling view layouts with Kredis, Turbo Frames, and Rails is a quick look at using the very fun Kredis gem to implement a common UX element.

User notifications with Rails, Noticed, and Hotwire offers a quick introduction to Noticed, and demonstrates using Noticed and Turbo Streams to build real-time in-app notifications with minimal effort. Stay tuned for another article demonstrating the same approach with CableReady soon.

Finally, if reading about non-Rails topics is your thing, I wrote a piece describing the process of writing and publishing my book on Gumroad and the results of the launch.

Lessons learned from publishing on Gumroad

2022-03-24T00:00:00+00:00

(This post was originally published on Indie Hackers, but I thought some of my regular blog readers might find it interesting, so I’m publishing it here, with light edits.)

One month ago I published my first ebook on Gumroad. I went into the process with absolutely no idea what I was doing, and no real idea of what to expect. I wanted to see if I could write and sell a book, so I struck out with that goal in mind and winged it all the way to the finish line.

Now that I am through the writing and publishing process, I thought others who are considering writing an ebook (especially a technical book like mine), might find value in hearing about my process along with what went well and what could have gone better.

Background

I started consistently publishing Ruby on Rails-focused programming tutorials on this blog in June of 2021. I was fortunate that my articles picked up some traction organically, primarily through high rankings for a couple of articles with very low competition, popular terms for folks looking for information on new pieces of the Rails stack.

From June to December, traffic grew from < 1,000 visitors in June 2021 to > 5,000 visitors in December of 2021. Most of this growth came from Google search traffic, with Twitter and Reddit making up the bulk of the rest.

In addition to the organic search traffic my tutorials attracted, my work was shared regularly on social media (especially Twitter and Rails-focused subreddits) when the niche technology I write about was mentioned.

I had some nice traction and some clear indicators that people were finding my work useful, and I knew that I wanted to write something more substantial than the quick, surface-level content that can fit into a single tutorial.

So, in November I tweeted a rough chapter list to see if there was any interest in a longer form version of my writing and it went about as viral as any tweet can with my tiny Twitter presence (maybe 200 people at the time of the tweet). I took that as another good sign and got to work.

Writing process and pre-launch

Since the book is a step-by-step, line-by-line walkthrough of building a Ruby on Rails application, my time “writing the book” was split in half. I first wrote all of the code for the application, taking detailed notes of each change and splitting the code into what felt like coherent “chapters”. I did not write any prose during the first phase — my chapter outlines were code blocks and file names.

Once I had the application built and each code change captured in my rough outline, I went back through my notes and stepped through each change, writing rough explanations for each block and testing the code for completeness.

After the code and rough outline was assembled, I stepped through each rough chapter and started writing detailed prose and telling the story of each chapter. This process of build > code notes > prose was basically the same as my process for the tutorial blogs I’d been writing for months, so while it was daunting to write 11 coherent, sequential chapters, it helped that the production process felt familiar.

After the detailed prose writing was finished, I went through several rounds of copy editing and polishing, along with 3 full run-throughs of each line of code to ensure accuracy and consistency. During this pass, I also captured all of the gifs and images I needed for each chapter to give readers landmarks to compare their progress to the book.

Once the polishing rounds were past, I sent out the book to beta readers (see below for details) and then spent a month waiting for feedback/incorporating feedback from beta readers.

All told, I spent approximately 150 hours coding, writing, and editing the content of the book before launch. Of that time, roughly 60 hours was coding, and the rest was writing and editing.

Beta readers

In early January, I wanted to give myself a clear deadline to finish the beta draft, so I sent out another tweet asking for early readers.

My idea here was to both get feedback on whether the content was useful/confusing/interesting/awful from strangers and to have an opportunity to test out the web application that I built to host my book (see below for details on that).

Approximately 30 people reached out expressing interest in being beta readers (this felt like a great number given my limited social reach) and I selected 10 to send the book to. I had never run a beta reader program before and had no idea what to expect, but I decided keeping the list of beta readers relatively short would be easier to manage and would keep me from being overwhelmed with feedback.

I eventually received feedback from 5 of the 10 beta readers, 3 sent very detailed feedback and 2 sent more general feedback, all of which was useful. The detailed, chapter-by-chapter feedback was most valuable, and I think I was quite fortunate to get as much detailed feedback as I did.

In retrospect, I should have done more to vet my list of beta readers — without getting lucky with a few very kind folks I may have been left with no early feedback on the book because some of the people I selected were not ready for the commitment required to read and offer feedback on the book.

Publishing the book as an app

I have been coding for a long time, and I have bought quite a few programming ebooks (and plenty of physical programming books) over the years. When I started writing, I assumed that I would publish my book as a PDF, with basic syntax highlighting and otherwise kind of a meh experience. This is how every programming book I’ve read over the years was packaged, so that’s what I went in thinking about.

As I converted my content to PDF, I was very unhappy with the reading experience. I write long, step-by-step code tutorials, which naturally means my readers are looking at a lot of code, and copying/pasting much of it into their editors as they follow along with the tutorial. On my blog, readers can click a “Copy” button to copy and paste each code snippet, which makes a difference when working with dozens of code snippets in a tutorial.

I also rely heavily on gifs to provide visual check-ins on what the application looks like throughout the book and how the elements we build work — click this, this happens, click that, that happens. These visual elements make long coding tutorials much more consumable.

Both copy/pasting with a click and embedding gifs proved to be extremely difficult to accommodate when publishing the book as a PDF.

I started having nightmares about finding typos the day after launch that day one purchasers would all see forever, and worrying about how to distribute major book revisions in a coherent way or otherwise improve the content of the book after launch in a way that ensured readers would be able to find it.

All of this weighed on me for weeks, and I ultimately decided the solution was to build a web app to serve my book as basically a fancy blog with user authentication in front of it. This is the silly engineer thing to do – build a whole dang web app to serve some static text content to readers, but that’s the path I chose.

Ultimately, building the simplest version of the book-as-web-app took ~15 hours to code and deploy to production. Much less than writing the book, but kind of a painful distraction while I was in the polishing phase and finishing the app was a blocker to delivering the book to beta readers for a few days.

I’m now on the hook for ongoing maintenance of the book’s web app, which has been minimal so far, but is something I have to be mindful of as long as the book is relevant.

Fortunately, readers have been happy with the delivery method of the book — so much so that I’m working now on building a SaaS to help other technical writers publish their books to the web without having to code a reading app from scratch like I did.

Launching and marketing the book

My very sophisticated marketing plan for the book consisted of a tweet thread drafted up the night before launch, along with a 20% discount off the list price during the first ~week after launch.

I sent out the thread on February 23rd with no idea what would happen.

My realistic goal for launch was to sell one copy of the book. My “I think I can do this goal” was to sell 20 copies of the book, and my “there’s no way I’ll hit this” goal was to sell 50 copies of the book in the launch period.

My sales chart for the launch week looked like this:

I hit all of my goals, which was a huge surprise.

A few things that worked during the launch week:

The initial tweet thread garnered a lot of attention and retweets from folks with much larger followings. The trust I’ve built up with my free writing gave me enough credibility for others to boost my paid work
I sent one more tweet on March 2, one day before the discount ended letting folks know that the discount was ending and that I was one sale away from my stretch goal (which was true)
I curate a monthly newsletter for Rails developers and sent out March’s edition on March 1 with a link to buy the book and a note about the discounted pricing
I added a share link inside the book to make it easy for folks enjoying the book to share it with others
I got one sale from a free listing on dev.to, which took ~30 seconds to create. Every little bit helps.
I offered purchasing power parity pricing discounts and about five people reached out to get a discount based on where they live. I used a spreadsheet I found online to calculate discounts, which worked fine.

All of my sales but one came from Twitter and direct links:

As you can see from the charts, after the initial burst of sales from the book launch tweet, sales dropped to basically zero until I started talking about the book again in my newsletter and on Twitter at the end of the discount period.

Unless I’m actively promoting the book, sales will be very slow!

Post-launch period

In the three weeks since the discounted launch period ended, sales look like this:

Five total sales, with no more than two in any given day. The two sales in one day came on a day when I published a blog that included a brief blurb about my book at the end — probably not a coincidence.

In this time period I have also had one buyer ask for a refund, which I immediately granted and then helped the person work through the setup issues they had in their environment that made them unable to get value from the book. I wish I were at zero refunds, but 1 out of 63 feels okay.

Lessons learned

Ultimately, I classify the launch of my book as a success based on the original goals I set for myself.

Writing a book was not very profitable on a per hour rate (nearly 200 hours for $3,000 is not a great return compared to my hourly contracting rate!) but I greatly enjoyed the work and I knew from the outset that this work was an investment in myself, rather than for immediate financial gain.

I learned a lot, improved as a writer, built a fun web app, created a product that a few people wanted to pay for, and earned my first real dollars on the internet, which feels like a nice milestone.

If I were going back to do the process over, a few things I would spend more time on:

Working more on marketing. While I never planned to earn a bunch of money from the book, I know that I could have gotten more from the book’s launch with more effort put into promoting the book.
Building a system to ask for reviews. I have zero reviews or testimonials on the book right now, which hurts my ability to sell it to people that do not know me. The price tag is pretty high at $70, which is a big investment to make without more social proof. Having a process in place to follow up with buyers and ask them for feedback and reviews would have helped keep sales up after the initial launch excitement died down.
Up front planning. Writing a book is hard, even when the book is really just a very, very long tutorial. I struggled to keep my ideas clear and ended up scrapping code and spinning my wheels on unimportant parts of the book because I did not go in with a clear enough plan for every stage of the application. A smoother writing process would have meant an earlier launch and a less stressful writing and editing experience.

User notifications with Rails, Noticed, and Hotwire

2022-03-21T00:00:00+00:00

A nearly-universal need in web applications is user notifications. An event happens in the application that the user cares about, and you inform the user of the event. A common example is in applications that have a commenting system — when a user mentions another user in a comment, the application notifies the mentioned user of that comment through email.

The channel(s) used to notify users of new important events depends on the application but the path very often includes an in-app notification widget that shows the user their latest notifications. Other common options include emails, text messages, Slack, and Discord.

Rails developers that need to add a notification system to their application often turn to Noticed. Noticed is a gem that makes it easy to add new, multi-channel notifications to Rails applications.

Today, we are going to take a look at how Noticed works by using Noticed to implement in-app user notifications. We will send those notifications to logged in users in real-time with Turbo Streams and, for extra fun, we will load user notifications inside of a Turbo Frame.

When we are finished, our application will work like this:

Before we begin, this tutorial assumes that you are comfortable building simple Ruby on Rails applications independently. No prior knowledge of Turbo or Noticed is required.

Let’s dive in!

Application setup

To follow along with this tutorial, start by cloning this repository from Github and then set it up:

cd user-notices
bin/setup

The starter repo contains a Rails 7 application with Turbo, Tailwind, and Devise ready to go. The starter repo uses Ruby 3.0.2, but everything in this tutorial will work fine with Ruby 2.7 and 3.1, if you prefer.

If you want to work from your own application instead of cloning the starter, you will need a Rails 7 application with Turbo installed and an authentication system built around a User model.

When you’re ready to start building, start the server and build Tailwind’s css with bin/dev.

Noticed setup

Our starter application comes with a Devise-powered user model and the root path set to the Dashboard#show action which just contains links to sign in or sign out for now. Before diving in to the code, create at least one user through the form at http://localhost:3000/users/sign_up so that you can login and test notifications later in this tutorial.

Eventually, our users will be able to create Messages for other users in the application. Each time a message is a created, a new Notification will be created, and the user the message is for will see that notification on their dashboard.

Before any of that can happen, we need to add Noticed to our application and scaffold a Message resource. Start by adding Noticed, from your terminal:

bundle add noticed
rails generate noticed:model
rails db:migrate

These commands are straight from the Noticed installation docs. If your Rails app is running, be sure to restart it after adding the Noticed gem to your Gemfile with bundle add.

Next, update app/models/user.rb to associate notifications with users:

class User < ApplicationRecord
  devise :database_authenticatable, :registerable,
         :recoverable, :rememberable, :validatable

  has_many :notifications, as: :recipient
end

Here, we added the notifications has_many association, as directed by the Noticed setup script.

Now our users can receive notifications, but we don’t have anything useful to notify them about. We will fix that by scaffolding a Message resource. From your terminal:

rails g scaffold message content:text user:references
rails db:migrate

Thanks to the magic of Rails, the scaffold generator gives us almost everything we need to start creating messages and associating them with users. Because we are using Tailwind via the tailwindcss-rails gem, the scaffold generator also includes some nice looking base styles too.

After the generator runs, update the User model again at app/models/user.rb:

class User < ApplicationRecord
  devise :database_authenticatable, :registerable,
         :recoverable, :rememberable, :validatable

  has_many :messages
  has_many :notifications, as: :recipient
end

Here, we added has_many :messages to set up the other side of the messages relationship.

For convenience, we can also add a link to the messages index page in app/views/dashboard/show.html.erb:

<div>
  <% if user_signed_in? %>
    Signed in as <%= current_user.email %>. <%= button_to "Sign out", destroy_user_session_path, method: :delete, class: "text-blue-500" %>
    <%= link_to "All messages", messages_path, class: "text-blue-500" %>
  <% else %>
    <%= link_to "Sign in", new_user_session_path, class: "text-blue-500" %>
  <% end %>
</div>

And then make a small adjustment to the messages form so we don’t have to memorize user ids:

<%= form_with(model: message, class: "contents") do |form| %>
  <% if message.errors.any? %>
    <div id="error_explanation" class="bg-red-50 text-red-500 px-3 py-2 font-medium rounded-lg mt-3">
      <h2><%= pluralize(message.errors.count, "error") %> prohibited this message from being saved:</h2>

      <ul>
        <% message.errors.each do |error| %>
          <li><%= error.full_message %></li>
        <% end %>
      </ul>
    </div>
  <% end %>

  <div class="my-5">
    <%= form.label :content %>
    <%= form.text_area :content, rows: 4, class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full" %>
  </div>

  <div class="my-5">
    <%= form.label :user_id %>
    <%= form.select :user_id, options_for_select(User.all.pluck(:email, :id)), class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full" %>
  </div>

  <div class="inline">
    <%= form.submit class: "rounded-lg py-3 px-5 bg-blue-600 text-white inline-block font-medium cursor-pointer" %>
  </div>
<% end %>

With these changes in place, head to http://localhost:3000/messages and create a couple of messages to ensure that creating messages works as expected.

Now that we have Noticed installed and messages ready to go, next up we will send users notifications when a new message is created.

Message notifications

Our goal in this section is to create and display notifications to logged in users on the Dashboard#show page.

Step one is to add a new notification, using the the generator built-in to Noticed. From your terminal:

rails generate noticed:notification MessageNotification

This generator creates a new MessageNotification class at app/notifications/message_notification.rb. Head there next and make a few small updates:

class MessageNotification < Noticed::Base
  deliver_by :database

  param :message

  def message
    params[:message].content
  end
  
  def url
    message_path(params[:message])
  end
end

Here, deliver_by :database stores the newly created notification in the database, which will be important for Turbo Stream broadcasts later. If we wanted to send the notification by email too, we could add deliver_by :email, mailer: SomeMailer, as described in the Noticed docs.

param :message serializes the message object and stores it with the notification record in the database. We use that serialized message in the message and url methods. Serializing objects into params in this manner makes it easy to access records related to the notification without managing complex references — just dump the record(s) you need into params and profit.

We can now use our MessageNotification to send notifications to users when a new message is created. Next we need to put the MessageNotification class into use. Head to app/models/message.rb and update it:

class Message < ApplicationRecord
  has_noticed_notifications

  belongs_to :user

  after_create_commit :notify_user

  def notify_user
    MessageNotification.with(message: self).deliver_later(user)
  end
end

Each time a message is created (after_create_commit), notify_user runs and creates a new MessageNotification, serializing the message object and delivering the message to the message’s user. We also add has_noticed_notifications to ensure that when a message is destroyed, any related notifications are destroyed too.

With those small changes, we now have a database-backed notification system up and running. Neat!

We have notifications in the database now but there is no way for users to see those notifications anywhere, which is not very useful. Next up, we will create a Notifications controller to display notifications to users.

From your terminal, generate the controller and a partial to render each notification:

rails g controller Notifications index
touch app/views/notifications/_notification.html.erb

Head to config/routes.rb and add a notifications path helper:

Rails.application.routes.draw do
  resources :notifications, only: [:index]
  resources :messages
  devise_for :users
  get 'dashboard/show'
  root "dashboard#show"
end

Update the NotificationsController at app/controllers/notifications_controller.rb:

class NotificationsController < ApplicationController
  def index
    @notifications = Notification.where(recipient: current_user)
  end
end

Here, we scope notifications to the current_user, so users only see their own notifications when logged in to the application.

Update the new Notifications index view at app/views/notifications/index.html.erb:

<%= turbo_frame_tag "notifications" do %>
  <h1 class="font-bold text-4xl">Notifications</h1>
  <ul>
    <%= render @notifications %>
  </ul>
<% end %>

Note the turbo_frame_tag wrapping the list of notifications. Our plan is to render the list of notifications on the dashboard show page — we will use Turbo Frame’s eager-loading functionality to load the content of the notifications index page on the dashboard show page.

render @notifications relies on Rails’ collection rendering to render each notification. Before this will work, we need to fill in app/views/notifications/_notification.html.erb:

<li>
  <div>
    <p class="text-gray-700">
      <%= notification.to_notification.message %>
    </p>
    <div class="flex justify-between mt-1 text-gray-500 text-sm space-x-4">
      <p>
        Received on <%= notification.created_at.to_date %>
      </p>
      <p>
        Status: <%= notification.read? ? "Read" : "Unread" %>
      </p>
    </div>
  </div>
</li>

Here, we use to_notification from Noticed to access the message method we added in MessageNotification earlier, and use the built-in read? method from Noticed to check if the user has read the notification or not.

One last step here before users can see notifications on the dashboard. Head to app/views/dashboard/show.html.erb and update it to add an eager-loaded Turbo Frame for logged in users:

<div class="flex justify-between">
  <% if user_signed_in? %>
    <div>
      Signed in as <%= current_user.email %>. <%= button_to "Sign out", destroy_user_session_path, method: :delete, class: "text-blue-500" %>
      <%= link_to "All messages", messages_path, class: "text-blue-500" %>
    </div>
    <%= turbo_frame_tag "notifications", src: notifications_path %>
  <% else %>
    <%= link_to "Sign in", new_user_session_path, class: "text-blue-500" %>
  <% end %>
</div>

The turbo_frame_tag has an id of notifications, which matches the Turbo Frame rendered by Notifications#index. When Turbo eager-loads content for a Turbo Frame, it expects the url passed to src to return a response that includes a Turbo Frame with a matching id.

The sequence of events for our eager-loaded notifications index page is this:

A logged in user visits Dashboard#show
Dashboard#show is loaded
Turbo sees the turbo_frame_tag with an src attribute and initiates a new request to /notifications
Notifications#index returns HTML that includes a turbo_frame_tag that matches the tag that initiated the request
Turbo extracts the contents of the turbo_frame_tag and uses that content to replace the content of the existing Turbo Frame

At this point, logged in users can see their notifications on the dashboard. Test it out by logging in as a user and then creating a new messages for that user. Refresh the dashboard as that user and see that your notifications are listed on the dashboard:

While our users can see notifications, they don’t see those notifications in real-time — they have to manually refresh the dashboard before they see new notifications. Let’s wrap up this tutorial by making notifications real-time with Turbo Stream broadcasts.

Real-time notifications with Turbo Streams

Turbo model broadcasts, powered by turbo-rails, make it easy to send real-time updates to users with ActionCable. When we finish this section, each time a new notification is created, a Turbo Stream broadcast will be sent over an ActionCable channel that will automatically insert new notifications for a user into the user’s dashboard notifications list.

To start, update app/models/notification.rb to trigger a Turbo Stream broadcast when a new notification is created:

class Notification < ApplicationRecord
  include Noticed::Model
  belongs_to :recipient, polymorphic: true

  after_create_commit :broadcast_to_recipient

  def broadcast_to_recipient
    broadcast_append_later_to(
      recipient,
      :notifications,
      target: 'notifications-list',
      partial: 'notifications/notification',
      locals: {
        notification: self
      }
    )
  end
end

Here, the broadcast_to_recipient method appends a new notification to the list of notifications. To ensure that only the user the notification is for receives the broadcast, we set the broadcast channel to recipient, :notifications, as described in the turbo-rails source.

The model update handles sending Turbo Stream broadcasts, but before the broadcast will be picked up by the front end we need to subscribe users to a Turbo Stream channel. We also must ensure the markup includes a notifications-list id (matching the target passed to broadcast_append_later_to) for the Turbo Stream to update.

Starting in app/views/notifications/index.html.erb, add the notifications-list id to the ul containing the list of notifications:

<%= turbo_frame_tag "notifications" do %>
  <h1 class="font-bold text-4xl">Notifications</h1>
  <ul id="notifications-list">
    <%= render @notifications %>
  </ul>
<% end %>

And then in app/views/dashboard/show.html.erb:

<div class="flex justify-between">
  <% if user_signed_in? %>
    <div>
      Signed in as <%= current_user.email %>. <%= button_to "Sign out", destroy_user_session_path, method: :delete, class: "text-blue-500" %>
      <%= link_to "All messages", messages_path, class: "text-blue-500" %>
    </div>
    <%= turbo_stream_from current_user, :notifications %>
    <%= turbo_frame_tag "notifications", src: notifications_path %>
  <% else %>
    <%= link_to "Sign in", new_user_session_path, class: "text-blue-500" %>
  <% end %>
</div>

Here, the turbo_stream_from helper from turbo-rails subscribes the user to a channel that matches the channel our model is broadcasting from. current_user, :notifications in the view == recipient, :notifications in the model).

When a user visits the dashboard, the turbo_stream_from helper opens an ActionCable subscription to the matching channel. Broadcasts to that channel are picked up by Turbo and used to update the page when new broadcasts are received. The scoping of the channel to the current_user ensures that users do not receive broadcasts intended for another user so that our new message notifications are only sent to the user the message is for.

With these changes in place, login as a user in one browser and head to the dashboard. Confirm the ActionCable channel subscription is created by checking the server logs for a line that looks like this:

Turbo::StreamsChannel is streaming from Z2lkOi8vdXNlci1ub3RpY2VzL1VzZXIvMQ:notifications

In another browser, head to the new messages form and create a message, setting the user on the form to the user that you are logged in as in the first browser. If all has gone well, the new notification will be added to the list instantly, with no page updates required.

Great work following along with this tutorial, that is all of the code for the day!

Wrapping up & further reading

Today we built a simple notification system with Rails and Noticed, and we used Turbo to display new notifications for users in real-time. Noticed is an extremely powerful gem that makes the work of building and expanding a multi-channel notification system in a Rails app much simpler, and the easy integration with Turbo Streams makes it a great match for any modern Rails application.

In our tutorial application, we displayed notifications as a static list of every notification the user has ever received, with no way to interact with them or remove them from the list. We also required users to be on their dashboard to see the new notification.

In a production application, we might expand our Notifications controller to include an update method that allows users to mark notifications as read and clear those notifications from the list.

We would would also likely build a notification indicator in the main navigation that is rendered on every page of our application with an icon indicating unread notifications (think the ubiquitous bell icon seen across thousands of web applications).

The neat thing about this tutorial’s approach is that the basic approach remains the same even for a more sophisticated implementation. Use Turbo Streams to broadcast new notifications to users and update the UI in real-time. Use built-in Noticed methods to act on notifications (like mark_as_read! to read a notification). Use a Turbo Frame to fetch notifications for a user and load those notifications into a section of a larger page.

For further learning on Noticed, Turbo Streams, and Turbo Frames:

Check out the GoRails tutorial on Noticed (Chris from GoRails wrote Noticed, thanks Chris!)
Read the simple but effective Noticed docs, starting in the repo’s readme and digging in to specific delivery methods as needed
Solidify Turbo concepts with my Turbo 101 article, and my Turbo Frames and Turbo Streams on Rails articles, for more on how to use Turbo with Rails

Finally, if you want to dig deeper into implementing notifications in a Rails application, a chapter of my book is dedicated to building a very light notification system from scratch, with inspiration from Noticed. In the book, we add real-time updates in a more realistic way, with the standard bell icon, pop-out notification list, and the ability to mark notifications as read with a click. The book is written in this same, step-by-step tutorial style, and covers building a modern Rails application from scratch with StimulusReflex, CableReady, Hotwire, and friends.

Building your own notification system is a great exercise if you want to use Noticed in a real production application later, since you will have a much greater understanding of, and appreciation for, what Noticed offers you.

That’s all for today. As always, thanks for reading!

Toggling view layouts with Kredis, Turbo Frames, and Rails

2022-03-07T00:00:00+00:00

Kredis is a new gem that makes it easier to work with Redis keys in Ruby on Rails. Kredis was added a suggested gem for new Rails applications starting with the release of Rails 7.0 in December of 2021 and is likely to become a larger force in the Rails world in the coming years.

From the documentation, Kredis “encapsulates higher-level types and data structures around a single key, so you can interact with them as coherent objects rather than isolated procedural commands”.

What this means for us is that we can use Kredis to make it easier to use Redis as a data store in our application. With Kredis, it is simple to use Redis to read and write complex data structures. Kredis’ integration with ActiveRecord allows us to work with Redis data alongside existing models.

Today we will use Kredis to power a card/list view toggle for a resource’s index page, persisting the user’s view preference across requests. This tutorial will offer a gentle introduction into how Kredis works while exploring how Kredis can help us implement a common real-world UX pattern.

We will also add a bit of Turbo functionality in the form of a Turbo Frame to wrap the list of items to make applying the user’s view preference a bit more efficient.

When we are finished, our application will work like this:

Before we begin, this tutorial is best suited for folks with experience building simple applications with Ruby on Rails. If you have never used Rails before, this tutorial will be difficult to follow. You do not need any prior experience with Turbo or Kredis to follow along with this tutorial.

As usual, you can find the complete code for this application on Github.

Let’s start building.

Application setup

To begin, create a new Rails 7 application from your terminal and scaffold up a Players resource, which we will use as the base for our Kredis-powered view toggle.

rails new kredis-players --css=tailwind
cd kredis-players
./bin/rails g scaffold Player name:string team:string
./bin/rails db:migrate

Note the inclusion of the css=tailwind option in the rails new command. We will use Tailwind to style our application so we can stay focused on building with Kredis instead of copy/pasting CSS.

Once the application is created and the Players resource is scaffolded, you can start up the server and build Tailwind’s CSS with bin/dev.

Because we are building a view toggle for the Players index page, you may also want to seed the database with a few players to save you some manual typing. Head to db/seeds.rb and update it:

10.times do |n|
  Player.create(name: "Player #{n}",  team: "Dallas Mavericks")
end

Then seed the database from your terminal:

./bin/rails db:seed

Build the list and card views

Before introducing Kredis to the application, we will begin by building a list/card view toggle that allows users to swap between the two views by reading URL parameters directly in the view.

Since we used the Rails scaffold generator, we already have a simple list view for Players ready on the index page. Let’s start by updating the generated views to be a little easier to process.

Update app/views/players/_player.html.erb like this:

<li class="border border-gray-200 rounded">
  <div href="#" class="block hover:bg-gray-50">
    <div class="flex items-center px-4 py-4 sm:px-6">
      <div class="min-w-0 flex-1 flex items-center">
        <div class="min-w-0 flex-1 px-4 md:grid md:grid-cols-2 md:gap-4">
          <div>
            <p class="text-sm font-medium"><%= player.name %></p>
          </div>
          <div class="hidden md:block">
            <div>
              <p class="text-sm text-gray-900">
                <%= player.team %>
              </p>
            </div>
          </div>
        </div>
      </div>
      <div>
        <p>
          <%= link_to "View", player, class: "text-blue-700 hover:text-blue-500" %>
        </p>
      </div>
    </div>
  </div>
</li>

Just regular ERB and some Tailwind classes for styling. Now update app/views/players/index.html.erb:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Players</h1>
    <%= link_to 'New player', new_player_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="players" class="min-w-full">
    <div>
      <ul role="list" class="space-y-4">
        <%= render @players %>
      </ul>
    </div>
  </div>
</div>

Again, just ERB and Tailwind, nothing fancy here.

Next we will add the card view, starting with a _card partial. Create the new partial from your terminal:

touch app/views/players/_card.html.erb

And then fill in the new _card partial:

<li class="col-span-1 flex shadow-sm rounded-md">
  <div class="flex-1 flex items-center justify-between border-t border-r border-b border-gray-200 bg-white rounded-r-md">
    <div class="flex-1 px-4 py-2 text-sm">
      <p class="text-gray-900 font-medium hover:text-gray-600"><%= player.name %></p>
      <p class="text-gray-500"><%= player.team %></p>
    </div>
    <div class="px-4 py-2">
      <p><%= link_to "View", player, class: "text-blue-700 hover:text-blue-500" %></p>
    </div>
  </div>
</li>

With the card partial created, we can now add a simple toggle to the Players index page to switch between the list view and the card view.

Head to app/views/players/index.html.erb and update it:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Players</h1>
    <%= link_to 'New player', new_player_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="players" class="min-w-full mt-8">
    <% if params[:view] == "card" %>
      <div class="my-4">
        <%= link_to "List view", players_path(view: "list"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
      </div>
      <ul role="list" class="grid grid-cols-1 gap-5 sm:gap-6 sm:grid-cols-2 lg:grid-cols-4">
        <%= render @players %>
      </ul>
    <% else %>
      <div class="my-4">
        <%= link_to "Card view", players_path(view: "card"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
      </div>
        <ul role="list" class="space-y-4">
          <%= render collection: @players, partial: "players/card", as: :player %>
        </ul>
    <% end %>
  </div>
</div>

Here we add a messy but functional if statement that checks the value of params[:view]. When the view param equals card, we render the players as a grid of cards, otherwise the players are rendered as a list. Both of the view also have a link to toggle the index page to the opposite view, with link_to players_path(view: "list"/"card").

This “works” as a starting point. Head to http://localhost:3000/players and click the button to toggle between the views and the page layout changes. However, we can quickly see the limits of this params based approach:

Toggle the index page to from a list to cards
Click on the view link for any player
Click the “Back to players” link on the show page

Instead of the card layout, the Players index page switches back to the list view. What is happening here? The Players index view relies on the presence of a view URL parameter to determine which layout to display. Because params do not persist between requests, the layout of the Players index page will always fall back to the default list view when visiting /players without any URL parameters.

This is where Kredis comes in. Instead of using the url params to set the view of the Players index page, we can use Kredis to persist the user’s view preference between requests so that the index page retains the expected layout.

Let’s see how Kredis works.

Using Kredis to store preferences

Kredis is a suggested gem in Rails 7, which means it is included in the default Gemfile but it is commented out. To install Kredis, first uncomment it in the Gemfile:

- # gem "kredis"
+ gem "kredis"

Then from your terminal:

bundle install
./bin/rails kredis:install

Restart your Rails application after installing the Kredis gem and running the install task to avoid errors about Kredis being undefined.

At this point, you will also need a Redis server running in your development environment. If you do not already have Redis installed and running, Mac users will find this gist helpful. Linux users may find this guide helpful.

With Kredis installed and Redis running in our local environment, we can now use Kredis to store the user’s view preference instead of relying on the presence of URL parameters.

Head to app/controllers/players_controller.rb and update the index action:

def index
  @players = Player.all
  @user_preferences = Kredis.hash('preferences')
  @user_preferences.update(view: params[:view]) if params[:view].present?
end

Here we are using Kredis to fetch a preferences key from Redis, initializing it as a hash and setting the user_preferences instance variable to be a new instance of a Kredis Hash.

Then, when params[:view] is present in the request, we update the preferences hash to store the value of view. update here updates the preferences key in Redis along with updating the user_preferences instance variable.

Because @user_preferences is an instance of a Kredis Hash, we can treat it (mostly) like a regular hash, as demonstrated by a little bit of experimenting in the console:

irb(main):012:0> preferences = Kredis.hash('preferences')
=>
#<Kredis::Types::Hash:0x00007faed43f0f90
...
irb(main):013:0> preferences.keys
  Kredis Proxy (0.3ms)  HKEYS preferences
=> []
irb(main):014:0> preferences.update(view: 'card')
  Kredis Proxy (0.2ms)  HSET preferences [{:view=>"card"}]
=> 1
irb(main):015:0> preferences.keys
  Kredis Proxy (0.3ms)  HKEYS preferences
=> ["view"]
irb(main):016:0> preferences['view']
  Kredis Proxy (0.3ms)  HGET preferences ["view"]
=> "card"
irb(main):017:0> preferences.delete('view')
  Kredis Proxy (0.3ms)  HDEL preferences ["view"]
=> 1
irb(main):018:0> preferences.keys
  Kredis Proxy (0.2ms)  HKEYS preferences
=> []

To use our new persistent Kredis value instead of URL parameters when rendering the index page, head back to app/views/players/index.html.erb and replace the existing if block with the below:

<% if @user_preferences[:view] == "card" %>
  <div class="my-4">
    <%= link_to "List view", players_path(view: "list"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
  </div>
  <ul role="list" class="grid grid-cols-1 gap-5 sm:gap-6 sm:grid-cols-2 lg:grid-cols-4">
    <%= render @players, partial: "players/card", as: :player %>
  </ul>
<% else %>
  <div class="my-4">
    <%= link_to "Card view", players_path(view: "card"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
  </div>
  <ul role="list" class="space-y-4">
    <%= render @players %>
  </ul>
<% end %>

Here we updated the if condition to check @user_preferences instead of params to determine which layout to render.

With that change in place, head back to http://localhost:3000/players and toggle between the list and card views. If all has gone well, toggling should still work. Even better, if you click on a player and then click to return back to the players index page, your list view preference will be retained.

Neat!

Adding a Turbo Frame

Now that we do not need the URL parameter on every request, it also makes sense to think about what actually needs to be updated when the user toggles between the list and card views. The rest of the page will not change — only the list of players will.

Turbo Frames give us the tools to adjust the application to only update the players section of the page when the user toggles the view, making the application feel faster and more responsive to user input.

For our application, Turbo Frames also resolve a subtle problem that you might have encountered while testing Kredis.

Turbo Drive takes a snapshot of every page to speed up applications. In almost all circumstances, this caching is something we just get for free without needing to think about; however, in this case, caching our players index page creates some issues that we need to resolve.

The issue works like this:

Head to http://localhost:3000/players
Toggle the view from list to card
Click on the view link for a player
Click on the link to go back to the players index page
Notice that for a brief moment, the index page renders the list view before replacing it with the card view

This flash of content happens because Turbo Drive caches the index page before navigating from /players to /players?view=card. The next time you visit /players, Drive uses the original, list-view version of /players from the cache to “preview” the index page before updating it with the new, card-view version of the index page rendered from the server.

Turbo Frames resolve this issue because Turbo does not cache a page when navigation is scoped within a Turbo Frame. Turbo Drive’s snapshot caching only occurs when a full-page navigation occurs.

This means that with Turbo Frames users are free to toggle between list and card views on the players index page as often as they like — the content of the index page will not be cached until they navigate away from the index page entirely.

Because the content is not cached until the final navigation, the correct layout of the page gets cached and the “preview” displayed by Turbo on the next visit to the index page matches the final version, eliminating the flash of bad cached content.

Let’s see how this works in practice.

First, update app/views/players/index.html.erb to wrap the list of players in a Turbo Frame:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Players</h1>
    <%= link_to 'New player', new_player_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <%= turbo_frame_tag "players", class: "min-w-full mt-8" do %>
    <% if @user_preferences[:view] == "card" %>
      <div class="my-4">
        <%= link_to "List view", players_path(view: "list"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
      </div>
      <ul role="list" class="grid grid-cols-1 gap-5 sm:gap-6 sm:grid-cols-2 lg:grid-cols-4">
        <%= render @players, partial: "players/card", as: :player %>
      </ul>
    <% else %>
      <div class="my-4">
        <%= link_to "Card view", players_path(view: "card"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
      </div>
        <ul role="list" class="space-y-4">
          <%= render @players %>
        </ul>
    <% end %>
  <% end %>
</div>

Here we removed the “players” div and replaced that div with a <turbo-frame id="players", using the turbo-rails provided turbo_frame_tag helper method.

Now that the list of players is wrapped in a Turbo Frame, links inside of that frame will update the content of that frame instead of updating the entire page.

This means that when the user clicks on the view toggle links, Turbo will extract the content of the players Turbo Frame from the response and use that content to update the players frame while discarding the rest of the content. When a Turbo Frame request is initiated, Rails helpfully renders the response without a layout, saving a (very small amount of) work on the server.

Finish up adding Turbo Frame support by updating the links to view players in both the card and player partials like this:

<%= link_to "View", player, class: "text-blue-700 hover:text-blue-500", data: { turbo_frame: "_top" } %>

The addition of data-turbo-frame="_top" to links wrapped inside of a <turbo-frame> tells Turbo to perform a normal full-page navigation, instead of scoping the navigation within the frame. This ensures that going to players#show still works as expected.

After making these changes, head back to http://localhost:3000/players, toggle the view from list to card (or card to list) and then click to view a player, and finally click back out to the players index page. If all has gone well you will not see any flashing cached content.

Improving our Kredis usage

Our current implementation of Kredis will only work if our application has exactly one user. This is because we are using a static key, preferences, to set the players list layout.

In the real world, we will (hopefully!) have more than one user in our application. Each of our users should be able to store their own view preferences or our view toggle will not be a very useful feature! Let’s wrap up this tutorial by looking at how we can use Kredis in ActiveRecord models to store user-specific preferences, making view toggling much more useful.

From your terminal, create a User model:

./bin/rails g model User session:string
./bin/rails db:migrate

And then update app/controllers/application_controller.rb:

class ApplicationController < ActionController::Base
  helper_method :current_user

  def current_user    
    @current_user ||= User.find_or_create_by(session: session.id.to_s)
  end
end

We do not want to build out a whole user authentication system to learn more about Kredis, so here we are faking it by using session.id to find or create a new user in the database and then defining current_user as a helper_method available throughout our application.

In a real application, current_user would be a real, actual, logged in user, but our fake users will be good enough to demonstrate the key concept here.

Our goal is to be able to associate user preferences with a User using Kredis. Our first implementation of Kredis hardcoded a preferences key to store all preferences. Our new implementation will give each User their own unique key through Kredis’ integration with ActiveRecord.

Head to app/models/user.rb and update it like this:

class User < ApplicationRecord
  kredis_hash :preferences
end

This update means that all users will have a unique key/value pair that stores their preferences in Redis. We can access this attribute with user.preferences, which will return an instance of a Kredis::Hash, just like Kredis.hash('preferences') did in our controller.

To use this new preferences attribute, update the PlayersController#index action in app/controllers/players_controller.rb:

def index
  @players = Player.all
  current_user.preferences.update(view: params[:view]) if params[:view].present?
end

Here we use the same update method we used before, this time using current_user.preferences to access a unique hash for the current user.

Then update the players index view to reference the right hash:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Players</h1>
    <%= link_to 'New player', new_player_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <%= turbo_frame_tag "players", class: "min-w-full mt-8" do %>
    <% if current_user.preferences[:view] == "card" %>
      <div class="my-4">
        <%= link_to "List view", players_path(view: "list"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
      </div>
      <ul role="list" class="grid grid-cols-1 gap-5 sm:gap-6 sm:grid-cols-2 lg:grid-cols-4">
        <%= render @players, partial: "players/card", as: :player %>
      </ul>
    <% else %>
      <div class="my-4">
        <%= link_to "Card view", players_path(view: "card"), class: "bg-blue-800 px-4 py-2 text-white hover:bg-blue-600 rounded" %>
      </div>
        <ul role="list" class="space-y-4">
          <%= render @players %>
        </ul>
    <% end %>
  <% end %>
</div>

The if statement now checks current_user.preferences instead of @user_preferences.

To test out these changes, open two separate browsers, toggle the list/card view to different options in each window, and then refresh each window to see that each “user” has their own unique preferences.

With that last change, you have reached the end of this tutorial! Incredible work today.

Wrapping up

Today we learned a little about how to use Kredis to unlock more power from our Redis-enabled Ruby on Rails applications.

Kredis’ tight integration with ActiveRecord gives it a leg up on other tools that are often used to do this type of work — if you have used Rails for a while, you have probably stuffed view preferences, filter and search terms, and other information into the session to persist the information across page turns. Redis is a more resilient and more appropriate tool for storing this type of information, and Kredis makes it simple to use Redis for this type of work in Ruby on Rails applications.

While we used a hash for storing data, Kredis supports a variety of datatypes to suit your needs — explore the repo to see the full list of available types.

One of the more exciting aspects of Kredis is the tools that can be built on top of it — we can build our own simple functionality like the preference storage we created today, but other smart folks are building gems on top of Kredis to enable functionality like presence tracking for application resources or complex, multi-stage forms.

That’s all for today. As always, thanks for reading!

Hotwiring Rails Newsletter - March 2022

2022-03-02T00:00:00+00:00

Thoughts or feedback on how I can make this newsletter more valuable? Have something you’d like to include in next month’s edition? Send me an email, or find me on Twitter.

Articles and guides
New and interesting PRs and releases
My recent writing
The best things I read recently

Articles and guides

Stripe checkout and customer portal Ruby on Rails Tutorial by Andy Leverenz

Building payment infrastructure is always challenging. This in-depth tutorial and accompanying screencast, if video is your thing, is an excellent resource for quickly integrating Stripe into your application.

I love long, detailed tutorials, and this one checks all the boxes. If you need to get Stripe checkout setup and aren’t sure where to start, this is the place.

Read the tutorial on web-crunch

The In-depth Guide to ActiveRecord load_async in Rails 7 by Paweł Urbanek

Paweł’s article on the load_async ActiveRecord feature added in Rails 7 is an exceptionally detailed look at how load_async works, its potential dangers, and the right way to approach using load_async in your Rails applications.

Highly recommended for anyone working on production Rails apps to keep one around as a resource.

Read the guide on pawelurbanek.com

Exploring Trix by Konnor Rogers

This in-progress series explores techniques for extending and enhancing the functionality of Trix. Trix’s integration with Rails is really convenient, but the limitations of the editor have made it hard to adopt for many use cases. Konnor’s work in this series makes Trix more valuable.

Read the series on dev.to

Hotrails.dev by Alexandre Rubin

This multi-part tutorial on building an application with Turbo and one line of custom JavaScript is a wonderful introduction to Turbo. I especially appreciate the annotated illustrations of common Turbo concepts.

This is very much worth the time for folks new to turbo-rails.

Start the course

Toggling page content with Turbo Frames and Kredis by Peter Phillips

I love Kredis, and I am excited for more folks to start creating cool stuff with it. In this quick tutorial, you’ll learn how to combine Kredis and Turbo Frames to toggle the visibility of page content, and to persist that preference across page loads.

This is a great introduction to Kredis, and the types of problems that Kredis can help you solve.

Read it on dev.to

Global presence by overstimulated

This is a really neat demonstration of a multi-region application built on fly.io, exploring this extremely complex question: “what does it look like to hook up a system of live-DOM-updates-over-Websockets to a caching layer that’s distributed across multiple regions?”

The end result is a really neat example of some of the problem spaces that are now accessible to determined Rails developers.

Read the overview, try out the demo, or view the source on Github.

Julian’s take on adding a simple lightbox to a Rails application is a great resource, approachable for all levels of Rails developers and the step-by-step approach makes it accessible even if you are new to Turbo and Stimulus.

This is a great one to bookmark for the next time you need to add a lightbox to a Rails application you maintain :)

Read the article on minthesize.com

Modern Front End Magic With Rails 7: A Visual Editor for Markdown by KUY.io

This series released in October 2021 but I first came across it this month and found it to be an exceptionally thoughtful and detailed look at solving a hard problem with Rails and Hotwire.

The two part series walks through building a WYSIWYG markdown editor from scratch in a Rails application with plenty of Turbo and Stimulus in the mix.

Read part one and part two of the series on KUY.io

Load More Pagination in Rails with Hotwire Turbo Streams by Dale Zak

Everyone needs to paginate resources — Dale Zak has a great, quick write up on building pagination from scratch with Turbo and Stimulus.

This article includes a really neat way to send Turbo Stream GET requests with a small Stimulus controller to modify headers — a handy trick to have in your Turbo bag.

Read the write up on dev.to

New and interesting PRs and releases

Devise alternatives

Last month, I linked to Steve Polito’s write up on building a Rails authentication system from scratch. A few weeks ago, Steve released an authentication generator for Rails 7 applications that allows anyone to add a simple authentication system to their Rails application.

The generator-based approach makes it easier to wrap your head around what your authentication system is doing, and to extend and modify it as needed.

Lázaro Nixon released a similar generator-based approach this month too.

I would still choose Devise for most commercial applications today, but the movement towards a more Rails-native approach to scalable, secure authentication is exciting and continues to receive support from the Rails core team. The challenges of integrating Devise into Turbo-enabled applications are likely to drive even more interest here in the coming months.

An extremely comprehensive Hotwire Rails template

Georg Ledermann’s opinionated Hotwire Rails template is a great starting place for a new Rails 7 application. The template goes well beyond the standard configuration options you’ll find in most templates — you’ll find linting, testing, and deployment needs are fully covered here too.

My recent writing

I wrote a book!

I am very pleased to share that the book on modern Rails development that I have been working on since the fall is now “published”, which means a little bit of self-promotion in this edition of the newsletter, and a much more regular cadence of blogs in the coming months now that I’m not writing a book and building a web app to publish the book as something more useful than a PDF.

The book, Hotwired ATS, is available on Gumroad at a 20% discount until March 3rd — if you prefer reading context-less source code, you can also find the source on Github for free.

Real-time previews with Rails and StimulusReflex

My exploration of markdown editors that led to the wonderful series from KUY.io linked above started with putting together this introductory-level tutorial on using StimulusReflex to preview markdown content as the user types.

Pagination and infinite scrolling with Rails and the Hotwire stack

Dale Zak’s article on pagination linked above, and a conversation on the StimulusReflex discord, inspired this article that introduces another approach to both traditional pagination and infinite scrolling via a small Stimulus controller.

Turbo Rails 101: Build a todo app with Turbo

This one does what it says on the tin: It is a brand new, “what the hell is a Turbo Frame and why do I care”-level introduction to Turbo Frames and Turbo Streams aimed at alleviating a couple of common sticking points for new Turbo users.

The best things I read recently

Spam accounts in 2022 by Josh Beckman

This is a quick, interesting read on the tactics that spammers use to game search engines and drive affiliate sales (in the best cases) or steal identities and cash (in the worst cases).

If you have ever built an SaaS that gives customers access to post jobs, build a website, or otherwise create public-facing content that can be used to run a scam, you’ve probably dealt with tactics like this.

Server-Sent Events, WebSockets, and HTTP by Mark Notttingham

This piece was near the top of the dreaded orange site recently and I found it to be a really interesting read on the challenges of making pub/sub a standards-defined part of the web.

The work that goes into defining and expanding the standards that enables web developers to build incredibly complex experiences that work seamlessly across browsers and devices always leaves me feeling awed and slightly intimated. Writing like this that helps illuminate some of thinking that goes into how to solve these types of standards-defining problems is always a treat.

Until next time

If you’re a Rails developer who also happens to love filling out surveys, the folks at Planet Argon have a treat for you.

The link to apply for a RailsConf opportunity scholarship is open until March 11.

Turbo Rails 101: Building a todo app with Turbo

2022-02-11T00:00:00+00:00

When Rails 7 released in December, Turbo became a default component of all new Ruby on Rails applications. This was an exciting moment for Rails developers that want to use Rails full-stack — especially folks on small teams with limited resources to pour into building and maintaining a React or Vue frontend.

Along with that excitement has come a constant stream of developers trying to learn how Turbo works, and how to integrate it into their Rails applications successfully.

For many, that learning journey has included a lot of roadblocks — Turbo requires some changes to how you structure applications and the official documentation is not yet as detailed as it could be. The good news is, most folks hit the same roadblocks early on their journey, which means we can help folks faster by addressing the common points of confusion.

In particular, there is confusion about how to use Turbo Frames and Turbo Streams together, and confusion about how Turbo Streams work.

Today, we are going to build a simple todo list application, powered entirely by Turbo. While building, we will take a few detours to look more deeply at a few common Turbo behaviors, and we will directly address two of the most common misconceptions that I see from folks who are brand new to using Turbo in their Rails applications.

When we are finished, our application will work like this:

This tutorial is written for Rails developers who are brand new to Turbo. The content is very much Turbo 101 and may not be useful if you are already comfortable working with Turbo Frames and Turbo Streams. This article also assumes comfort with writing standard CRUD applications in Rails — if you are have never used Rails before, this is not the right place to start!

As usual, you can find the complete code for our demo application on Github.

Let’s start building!

Application setup

We will start with a brand new Rails 7 application which comes with Turbo out of the box.

Generate a new Rails application with Tailwind CSS for styling. From your terminal:

rails new turbo-todo --css=tailwind
cd turbo-todo

And then scaffold up a Todo resource. From your terminal again:

rails g scaffold Todo name:string status:integer

Update migration to set default value for status:

class CreateTodos < ActiveRecord::Migration[7.0]
  def change
    create_table :todos do |t|
      t.string :name
      t.integer :status, default: 0

      t.timestamps
    end
  end
end

Finally, create and migrate the database:

rails db:create db:migrate

Create new todos with Turbo Streams

The Rails scaffold generator provides a fully functional implementation of todos out of the box. If you start up the Rails app and head to /todos you can create, edit, and delete todos to your heart’s content, but every request will initiate a full page turn. Not very exciting.

Our goal in this section is to update the existing todo scaffold to use Turbo Streams to insert newly created todos into the DOM without a full page turn or any custom JavaScript.

Start by replacing the content of the index view, app/views/todos/index.html.erb, with the following:

<div class="mx-auto w-1/2">
  <h2 class="text-2xl text-gray-900">
    Your todos
  </h2>
  <div class="w-full max-w-2xl bg-gray-100 py-8 px-4 border border-gray-200 rounded shadow-sm">
    <div class="py-2 px-4">
      <%= render "form", todo: Todo.new %>
    </div>
    <ul id="todos">
      <%= render @todos %>
    </ul>
  </div>
</div>

Here, we updated the page layout so it looks a little nicer and inserted the new todo form directly on to the page. Users will use this form to add new todos, and existing todos will be rendered in the <ul> below the form.

Note that the <ul> has an id of todos. Turbo Streams target elements in the DOM by id, and the todos id will be used to insert newly created todos into the DOM.

Update app/views/todos/_todo.html.erb to render each todo properly inside of the <ul>:

<li class="py-2 px-4 border-b border-gray-300">
  <%= todo.name %>
</li>

The form partial we render in the index view needs a few adjustments too. In app/views/todos/_form.html.erb:

<%= form_with(model: todo, id: "#{dom_id(todo)}_form") do |form| %>
  <% if todo.errors.any? %>
    <div id="error_explanation" class="bg-red-50 text-red-500 px-3 py-2 font-medium rounded-lg mt-3">
      <h2><%= pluralize(todo.errors.count, "error") %> prohibited this todo from being saved:</h2>

      <ul>
        <% todo.errors.each do |error| %>
          <li><%= error.full_message %></li>
        <% end %>
      </ul>
    </div>
  <% end %>
  <div class="flex items-stretch flex-grow">
    <%= form.label :name, class: "sr-only" %>
    <%= form.text_field :name, class: "block w-full rounded-none rounded-l-md sm:text-sm border-gray-300", placeholder: "Add a new todo..." %>
    <%= form.submit class: "-ml-px relative px-4 py-2 border border-blue-600 text-sm font-medium rounded-r-md text-white bg-blue-600 hover:bg-blue-700" %>
  </div>
<% end %>

Note the addition of an id to the <form>, using the dom_id of the todo passed to the partial, which will be used to target Turbo Stream updates.

To use these new ids to update the DOM, we need to tell our controller to render a Turbo Stream when the form is submitted.

To do this, head to the TodosController and update the create action:

def create
  @todo = Todo.new(todo_params)

  respond_to do |format|
    if @todo.save
      format.turbo_stream
      format.html { redirect_to todo_url(@todo), notice: "Todo was successfully created." }
    else
      format.html { render :new, status: :unprocessable_entity }
    end
  end
end

The change here is the addition of format.turbo_stream to the happy path in the action. format.turbo_stream tells Rails that when a Turbo Stream request is sent to the create action, respond with a matching create.turbo_stream.erb file.

If you are a long-time Rails developer, this will feel very similar to responding with js.erb files in response to ajax requests.

In order for this to work, we need to create the create.turbo_stream.erb file, otherwise you will get an error about a missing template.

From your terminal:

touch app/views/todos/create.turbo_stream.erb

And then fill that new file in:

<%= turbo_stream.prepend "todos" do %>
  <%= render "todo", todo: @todo %>
<% end %>
<%= turbo_stream.replace "#{dom_id(Todo.new)}_form" do %>
  <%= render "form", todo: Todo.new %>
<% end %>

Our first look at a Turbo Stream! In create.turbo_stream.erb we render two <turbo-stream> elements.

The first prepends the newly created todo to the list of todos, targeting the <ul> with the id of todos. The second replaces the todo form with a fresh copy of the form, allowing us to clear the todo form after each successful submission.

At this point, you can start up your Rails application with bin/dev. Head to localhost:3000/todos, create a couple of todos and see that they automatically append to the list of todos. Magic.

Let’s pause here and review what is happening in a little more detail. Each time the user submits the new todo form, the request sent to the server includes an Accept header that identifies the request as a Turbo Stream request:

text/vnd.turbo-stream.html, text/html, application/xhtml+xml

Turbo sets this header automatically on all POST, PUT, PATCH, DELETE form submissions, with no intervention required from the developer. As of Turbo 7.2, you can optionally respond Turbo Streams in response to GET requests. Just include data-turbo-stream on the link or GET form you want to process as a Turbo Stream.

turbo-rails registers a turbo_stream Mime type to enable responding to inbound Turbo Stream form submissions with turbo_stream content. We see this in action in the TodosController:

def create
  respond_to do |format|
    format.turbo_stream
  end
end

When calling format.turbo_stream without passing a block, Rails conventions expect that a file that matches the action and Mime type exists — in our case, create.turbo_stream.erb.

In create.turbo_stream.erb, we render <turbo-stream> elements using the turbo_stream helper. Rails renders the create.turbo_stream view to HTML and sends that HTML back to the frontend:

<turbo-stream action="prepend" target="todos">
  <template>
    <li class="py-2 px-4 border-b border-gray-300">
      A new todo
    </li>
  </template>
</turbo-stream>

<turbo-stream action="replace" target="new_todo_form">
  <template>
    <form id="new_todo_form" action="/todos" accept-charset="UTF-8" method="post"><input type="hidden" name="authenticity_token" value="TmpPYfxQOln1t3jmigbzZ49ciBWjivGEjp_nJUWJMeUJZSoeA8dvbkLeD6kLkmHZx-zc8kOzcqu69tWGYASlpQ" autocomplete="off" />
      <div class="flex items-stretch flex-grow">
        <label class="sr-only" for="todo_name">Name</label>
        <input class="block w-full rounded-none rounded-l-md sm:text-sm border-gray-300" placeholder="Add a new todo..." type="text" name="todo[name]" id="todo_name" />
        <input type="submit" name="commit" value="Create Todo" class="-ml-px relative px-4 py-2 border border-blue-600 text-sm font-medium rounded-r-md text-white bg-blue-600 hover:bg-blue-700" data-disable-with="Create Todo" />
      </div>
    </form>
  </template>
</turbo-stream>

Turbo extracts the Turbo Stream elements from the HTML and uses each element’s action and content to update the DOM.

Now we understand a bit about what is happening when our Turbo-powered from is submitted, which also gives us the knowledge to knock out a few common misconceptions about Turbo.

Turbo Streams can target any element, not just Turbo Frames

First, many new Turbo developers think that Turbo Streams can only target Turbo Frames. This misconception causes them to run into issues nesting their forms within an unnecessary Turbo Frame or to end up with invalid HTML by wrapping <tr> or <li> elements in <turbo-frame> elements.

Issues caused by this misconception come up almost daily on the Rails Internet and Turbo Streams can be very difficult to work with while laboring under this misconception. Although the documentation never links Turbo Frames and Turbo Streams in this way, the issue persists.

So, let’s get very clear here: Turbo Streams target elements in the DOM by id (or class, less commonly). Any element with an id can be targeted by a Turbo Stream, not just Turbo Frame elements.

Turbo Streams do not require WebSockets

Turbo Streams have gotten a lot of attention because they can be used with WebSockets to proactively send updates to many users at once outside of the standard request/response cycle.

With turbo-rails, developers can easily broadcast updates from models and background jobs to send <turbo-stream> snippets over WebSockets with ActionCable.

These types of WebSockets-powered Turbo Stream broadcasts are great — but as you saw in the TodosController, you can also just render Turbo Stream tags in response to a request from a browser. You can make great use of Turbo Streams without WebSockets.

Now that we have gotten way down into the weeds of Turbo Streams, let’s zoom back out a bit and take our first look at Turbo Frames.

Editing existing todos

Users will edit their todos by clicking on the name of the todo. When they click on the name, the edit form for that todo will render in place of the todo in the list, like this:

To build this functionality, we will use a Turbo Frame to scope navigation to the piece of the page we want to update. Start by updating the existing todo partial like this:

<li id="<%= "#{dom_id(todo)}_container" %>" class="py-2 px-4 border-b border-gray-300">
  <%= turbo_frame_tag dom_id(todo) do %>
    <%= link_to todo.name, edit_todo_path(todo) %>
  <% end %>
</li>

Here, we added a unique id to each <li>. Nested within the <li>, we added a <turbo-frame> using the turbo-rails provided turbo_frame_tag helper method.

Within the Turbo Frame is a link_to pointing to the edit action in the TodosController. Because the link is within the Turbo Frame, Turbo will expect the server to return a Turbo Frame with a matching id. Turbo will extract the matching Turbo Frame from the response HTML and use it to replace the original content of the frame.

In our case, that means when the user clicks on the todo’s name, edit.html.erb will render an edit form and that form will replace the link to the edit page.

Let’s see this in action. Update app/views/edit.html.erb:

<%= turbo_frame_tag dom_id(@todo) do %>
  <%= render "form", todo: @todo %>
<% end %>

The turbo_frame_tag has an id that matches the turbo_frame_tag in the todo partial.

With that change in place, refresh the todos index page and click on the name of a todo. If all has gone well, you will see that the edit form replaces that todo in the list. If you submit the form, you will see that you get redirected to the show page of the todo you edited — not quite there yet!

We will fix this issue with another Turbo Stream rendered from the server, this time for TodosController#update.

From your terminal:

touch app/views/todos/update.turbo_stream.erb

Update the new update.turbo_stream.erb view like this:

<%= turbo_stream.replace "#{dom_id(@todo)}_container" do %>
  <%= render "todo", todo: @todo %>
<% end %>

We are using a Turbo Stream replace action again. This time the Turbo Stream action replaces the content of the <li> wrapping the todo with the content of the todo partial. Because the edit form is replaced by the updated Todo, we do not need to reset the edit form like we did the new form in create.turbo_stream.erb.

Update the TodosController to respond to Turbo Stream requests:

def update
  respond_to do |format|
    if @todo.update(todo_params)
      format.turbo_stream
      format.html { redirect_to todo_url(@todo), notice: "Todo was successfully updated." }
      format.json { render :show, status: :ok, location: @todo }
    else
      format.html { render :edit, status: :unprocessable_entity }
      format.json { render json: @todo.errors, status: :unprocessable_entity }
    end
  end
end

Now when the edit form is submitted successfully the edit form is replaced with an updated version of the edited todo’s partial.

At this point, we can create and edit todos without a full page turn but you may have noticed that we are not using Turbo Streams to handle invalid form submissions. The else path in the create and update actions is missing a turbo_stream response. We will fix that in the next section.

Handling form errors

To demonstrate handling form errors, we need to first add a validation to the Todo model so that we can send invalid form submissions to the server. In app/models/todo.rb:

validates_presence_of :name

Form submissions with a blank name will fail to save, letting us test out error handling with Turbo Streams.

Head back to the TodosController and update the create and update actions:

def create
  @todo = Todo.new(todo_params)

  respond_to do |format|
    if @todo.save
      format.turbo_stream
      format.html { redirect_to todo_url(@todo), notice: "Todo was successfully created." }
    else
      format.turbo_stream { render turbo_stream: turbo_stream.replace("#{helpers.dom_id(@todo)}_form", partial: "form", locals: { todo: @todo }) }
      format.html { render :new, status: :unprocessable_entity }
    end
  end
end

def update
  respond_to do |format|
    if @todo.update(todo_params)
      format.turbo_stream
      format.html { redirect_to todo_url(@todo), notice: "Todo was successfully updated." }
      format.json { render :show, status: :ok, location: @todo }
    else
      format.turbo_stream { render turbo_stream: turbo_stream.replace("#{helpers.dom_id(@todo)}_form", partial: "form", locals: { todo: @todo }) }
      format.html { render :edit, status: :unprocessable_entity }
      format.json { render json: @todo.errors, status: :unprocessable_entity }
    end
  end
end

When the todo fails to save, we render a turbo_stream directly from the controller, replacing the content of the form with an updated version of the form so that errors are displayed to the user.

This method of rendering Turbo Streams inline in the controller is an alternative to creating views like create.turbo_stream.erb — either approach will work. In practice, it tends to be easier to manage complex Turbo Stream responses with dedicated views while rendering simple responses inline works fine for single stream responses.

Deleting todos

Next up, we will add the ability to delete todos without a page turn by using another Turbo Stream rendered from the controller.

Start by updating the todo partial to add a delete button:

<li id="<%= "#{dom_id(todo)}_container" %>" class="py-2 px-4 border-b border-gray-300">
  <%= turbo_frame_tag dom_id(todo) do %>
    <div class="flex justify-between items-center space-x-2">
      <%= link_to todo.name, edit_todo_path(todo) %>

      <%= button_to todo_path(todo), class: "bg-red-600 px-4 py-2 rounded hover:bg-red-700", method: :delete do %>
        <span class="sr-only">Delete</span>
        <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-white" viewBox="0 0 20 20" fill="currentColor" title="Delete todo">
          <path fill-rule="evenodd" d="M9 2a1 1 0 00-.894.553L7.382 4H4a1 1 0 000 2v10a2 2 0 002 2h8a2 2 0 002-2V6a1 1 0 100-2h-3.382l-.724-1.447A1 1 0 0011 2H9zM7 8a1 1 0 012 0v6a1 1 0 11-2 0V8zm5-1a1 1 0 00-1 1v6a1 1 0 102 0V8a1 1 0 00-1-1z" clip-rule="evenodd" />
        </svg>
      <% end %>
    </div>
  <% end %>
</li>

Note the method: :delete on the button, which ensures the button hits the destroy action on the controller. The svg icon here is from Heroicons — feel free to just make the button say “Delete” if you like, that will work fine too.

In the TodosController, update the destroy action:

def destroy
  @todo.destroy

  respond_to do |format|
    format.turbo_stream { render turbo_stream: turbo_stream.remove("#{helpers.dom_id(@todo)}_container") }
    format.html { redirect_to todos_url, notice: "Todo was successfully destroyed." }
    format.json { head :no_content }
  end
end

This inline turbo_stream uses the Turbo Stream remove action to remove the target element from the DOM entirely.

Refresh the index page, click the delete button on a todo and see that the todo is removed from the DOM without a full page turn.

Mark todos as complete

A todo list is not very helpful if todos cannot be marked as complete. In this section, we will add a button to toggle todos complete and incomplete, relying as usual on Turbo Streams to update the DOM for us.

To begin, let’s define a simple status enum in the Todo model:

enum status: {
  incomplete: 0,
  complete: 1
}

Back to app/views/todos/_todo.html.erb:

<li id="<%= "#{dom_id(todo)}_container" %>" class="py-2 px-4 border-b border-gray-300">
  <%= turbo_frame_tag dom_id(todo) do %>
    <div class="flex justify-between items-center space-x-2">
      <%= link_to todo.name, edit_todo_path(todo), class: todo.complete? ? 'line-through' : '' %>

      <div class="flex justify-end space-x-3">
        <% if todo.complete? %>
          <%= button_to todo_path(todo, todo: { status: 'incomplete' }), class: "bg-green-600 px-4 py-2 rounded hover:bg-green-700", method: :patch do %>
            <span class="sr-only">Mark as incomplete</span>
            <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-white" viewBox="0 0 20 20" fill="currentColor">
              <path fill-rule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clip-rule="evenodd" />
            </svg>
          <% end %>
        <% else %>
          <%= button_to todo_path(todo, todo: { status: 'complete' }), class: "bg-gray-400 px-4 py-2 rounded hover:bg-green-700", method: :patch do %>
            <span class="sr-only">Mark as complete</span>
            <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-white" viewBox="0 0 20 20" fill="currentColor">
              <path fill-rule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clip-rule="evenodd" />
            </svg>
          <% end %>
        <% end %>

        <%= button_to todo_path(todo), class: "bg-red-600 px-4 py-2 rounded hover:bg-red-700", method: :delete do %>
          <span class="sr-only">Delete</span>
          <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-white" viewBox="0 0 20 20" fill="currentColor" title="Delete todo">
            <path fill-rule="evenodd" d="M9 2a1 1 0 00-.894.553L7.382 4H4a1 1 0 000 2v10a2 2 0 002 2h8a2 2 0 002-2V6a1 1 0 100-2h-3.382l-.724-1.447A1 1 0 0011 2H9zM7 8a1 1 0 012 0v6a1 1 0 11-2 0V8zm5-1a1 1 0 00-1 1v6a1 1 0 102 0V8a1 1 0 00-1-1z" clip-rule="evenodd" />
          </svg>
        <% end %>
      </div>
    </div>
  <% end %>
</li>

There’s a lot here, let’s cut through the noise to highlight the important functional pieces.

The todo edit link gets struck through when the todo is complete:

<%= link_to todo.name, edit_todo_path(todo), class: todo.complete? ? 'line-through' : '' %>

If the todo is complete, we render button_to to mark the todo as incomplete. Incomplete todos get a button_to to mark the todo as complete.

<% if todo.complete? %>
  <%= button_to todo_path(todo, todo: { status: 'incomplete' }), class: "bg-green-600 px-4 py-2 rounded hover:bg-green-700", method: :patch do %>
<% else %>
  <%= button_to todo_path(todo, todo: { status: 'complete' }), class: "bg-gray-400 px-4 py-2 rounded hover:bg-green-700", method: :patch do %>
<% end %>

In either case the patch request goes to TodosController#update as a Turbo Stream request, and the existing update.turbo_stream.erb view is rendered.

If this were a real application, we would pull these buttons out into helper methods or into a view component, but for our purposes, we can live with a messy partial.

Complete/incomplete todos in separate tabs

Now that users can mark todos as complete, it would be nice to not have to see completed todos all of the time. We will finish up our Turbo-powered todo application by adding a tabbed interface to the todos index page, allowing users to toggle between incomplete and complete todos.

Get started by adding simple filtering logic to the index action in the TodosController:

def index
  @todos = Todo.where(status: params[:status].presence || 'incomplete')
end

Then update app/views/todos/index.html.erb:

<div class="mx-auto w-1/2">
  <h2 class="text-2xl text-gray-900">
    Your todos
  </h2>
  <%= turbo_frame_tag "todos-container", class: "block max-w-2xl w-full bg-gray-100 py-8 px-4 border border-gray-200 rounded shadow-sm" do %>
    <div class="border-b border-gray-200 w-full">
      <ul class="flex space-x-2 justify-center">
        <li>
          <%= link_to "Incomplete",
            todos_path(status: "incomplete"),
            class: "inline-block py-4 px-4 text-sm font-medium text-center text-gray-500 border-b-2 border-transparent hover:text-gray-600 hover:border-gray-300" 
          %>
        <li>
          <%= link_to "Complete",
            todos_path(status: "complete"),
            class: "inline-block py-4 px-4 text-sm font-medium text-center text-gray-500 border-b-2 border-transparent hover:text-gray-600 hover:border-gray-300"
          %>
        </li>
    </ul>
    </div>
    <% unless params[:status] == "complete" %>
      <div class="py-2 px-4">
        <%= render "form", todo: Todo.new %>
      </div>
    <% end %>
    <ul id="todos">
      <%= render @todos %>
    </ul>
  <% end %>
</div>

The index view now wraps the todo content in a todos-container <turbo-frame>. As with the edit links for each individual todo, this Turbo Frame will scope navigation within the frame, allowing the list of Todos to be updated without changing the content on the rest of the page.

Inside of the new todos-container frame, we added links to view incomplete and complete todos. Logic to hide the new todo form when viewing complete todos was also added, since newly created todos are always incomplete.

Because the links to view incomplete and complete todos are within the todos-container Turbo Frame, each time those links are clicked, Turbo will replace the content of the todos-container with updated content from the server.

Conveniently, we do not need to change anything about the index action to render Turbo Frame content. Even though the entire page re-renders when the index action is called, Turbo will extract the todos-container frame from the response and discard the rest. If that small bit of inefficiency bothers you, it is possible to be more efficient.

With this change in place, we have a slight problem with the status toggle behavior. Right now, when the user marks a todo as complete, the todo is updated but it stays on the list. Instead, when a todo’s status is updated, we would like to remove it from the list.

Implementing this functionality will require adding a new, non-RESTful action to the TodosController. We will call this new action change_status. Start in the TodosController:

before_action :set_todo, only: %i[ show edit update destroy change_status ]

def change_status
  @todo.update(status: todo_params[:status])
  respond_to do |format|
    format.turbo_stream { render turbo_stream: turbo_stream.remove("#{helpers.dom_id(@todo)}_container") }
    format.html { redirect_to todos_path, notice: "Updated todo status." }
  end
end

Here, we updated the set_todo before_action to set the @todo instance variable when change_status is called and we defined change_status.

change_status updates the status of the given todo and then removes that that todo from the DOM. This will work for marking todos as complete and for marking them as incomplete — either way, we just target the id of the <li> and use a Turbo Stream remove action.

We added this new action because the update action we used in the first implementation of this feature use a replace Turbo Stream action, instead of removing the todo from the DOM. We could have hacked the update action to handle status changes differently or created a whole new TodoStatusChangesController for this, but there’s no reason to do that in our learning application.

Update config/routes.rb to add the new route to the application:

resources :todos do
  patch :change_status, on: :member
end

And finally, update the todo partial one last time to use the change_status_todo_path on the status toggle buttons:

<% if todo.complete? %>
  <%= button_to change_status_todo_path(todo, todo: { status: 'incomplete' }), class: "bg-green-600 px-4 py-2 rounded hover:bg-green-700", method: :patch do %>
    <span class="sr-only">Mark as incomplete</span>
    <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-white" viewBox="0 0 20 20" fill="currentColor">
      <path fill-rule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clip-rule="evenodd" />
    </svg>
  <% end %>
<% else %>
  <%= button_to change_status_todo_path(todo, todo: { status: 'complete' }), class: "bg-gray-400 px-4 py-2 rounded hover:bg-green-700", method: :patch do %>
    <span class="sr-only">Mark as complete</span>
    <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 text-white" viewBox="0 0 20 20" fill="currentColor">
      <path fill-rule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clip-rule="evenodd" />
    </svg>
  <% end %>
<% end %>

With that last change in place, you can refresh the page and see that todos are now grouped into tabs. Toggle the status on a few todos and see that they are removed from the list of todos. Change tabs and see that the todo list updates:

Our application is so small that there’s no real way to tell that changing tabs is only updating the content of the todos-container Turbo Frame, right? It could just be updating the whole page and we would never be able to tell. A quick way to test the Turbo Frame out (and to see why partial page updates can be so useful) is to add a dummy input to the page, outside of the todos-container frame:

Neat!

Degrading gracefully

Throughout this application, we use respond_to blocks to render responses to Turbo Stream requests. The nice thing about this approach is that we always have a format.html back up ready to go if a user does not have JavaScript enabled in their browser.

Because we constructed our application in this way, the application remains fully functional even when JavaScript is disabled. The partial page updates powered by Turbo give way to regular full page turns. Turbo applications, constructed thoughtfully, tend to rely less on JavaScript to function, making it easier to build applications that can gracefully fall back to normal, server rendered HTML and full page turns.

Depending on your application’s audience, this may not be a top priority, but Turbo-powered applications give you a solid base to build from if your application needs to serve JavaScript-free users.

Wrapping up

Today we built a Turbo-powered todo application, using Turbo Streams and Turbo Frames to make fast, efficient page updates in response to user actions.

This simple application served as a base to explore the basics of Turbo Streams and Turbo Frames and gave us a chance to debunk a few common misconceptions about Turbo Streams in the process.

As you move forward in your Turbo journey, remember that Turbo Streams are for responding to form submissions. Streams give you the tools to update one or many elements after a form submission. You can render Turbo Streams inline in a controller, or from views.

Turbo Frames are for scoping GET requests to a single piece of the page. Use Turbo Frames to add tabbed content to a page, to power search and filter interfaces, or for inline editing like we saw today. Turbo Frames always replace the entire content of the target frame, and only one frame can be updated per GET request.

If you need more sophisticated update behavior (like appending items) or you need to update multiple elements at once, you cannot (easily) use a Turbo Frame.

In this tutorial, we looked at basic use cases for Streams and Frames; however, we only just scratched the surface of what you can do with Turbo.

To continue learning, a thorough review of the Turbo reference documentation is a good starting point. In particular, familiarizing yourself with the events Turbo emits is important for more advanced use cases. Understanding Turbo Frame options is also important — functionality like eager and lazy loaded frames, breaking out of frames, and targeting frames from the outside all help unlock powerful Turbo Frame-powered experiences.

In addition to the Turbo documentation, you might find my more in-depth explorations of using Turbo Streams and Turbo Frames in Rails useful.

For a much deeper dive into building a real application with Turbo, Alexandre Ruban’s (in-progress) Hotrails course is a great resource.

That’s all for today. As always, thanks for reading!

Pagination and infinite scrolling with Rails and the Hotwire stack

2022-02-04T00:00:00+00:00

September 2022 Update: The techniques described in this article still work. However, with the release of Turbo 7.2, there is a simpler approach than the technique used in this article. You can find an updated version of this article here.

Nearly every web application will eventually need to add pagination to improve page load times and allow users to process information in a more consumable way — you don’t want to load 1,000 records in one request!

Today, we are going to use the Hotwire stack (Turbo and Stimulus) to implement pagination in a Ruby on Rails application. We will implement pagination in three different ways, to give ourselves a chance to explore Turbo Frames, Turbo Streams, and Stimulus.

This article was inspired by a conversation on the StimulusReflex discord and the great article by Dale Zak published as a result of that conversation.

In Dale’s article, a purpose-built Stimulus controller is used to respond to a GET request with a Turbo Stream template. After reading that article, I decided to explore another method for achieving the same result, which is what we will tackle today.

In the article, we will start with a simple Rails 7 application, build standard pagination with Pagy, and then layer on three different implementations of Turbo-powered pagination:

Pagination with Previous and Next page buttons
Manual “infinite scroll” with a load more button
Automatic infinite scroll

When we are finished, the infinite scroll version will look like this:

Before we begin, this article assumes that you are comfortable with Ruby on Rails and you have had a bit of exposure to Turbo and Stimulus. The techniques described in this article will work without Ruby on Rails, but the code will be easiest to follow if you are comfortable developing simple Ruby on Rails applications.

You can find the complete code for this tutorial on Github, and you can try out a “production” version of the application on Heroku.

Let’s get started!

Application setup

We will work from a new Rails 7 application, using importmap-rails to manage JavaScript and Tailwind for styling.

Create a new Rails 7 application from your terminal:

rails new turbo-pagination --css=tailwind
cd turbo-pagination

To demonstrate pagination, we will create a simple Widget resource. From your terminal again, use the built-in scaffold generator:

rails g scaffold Widget name:string
rails db:migrate

Because we are using Tailwind via the tailwindcss-rails gem, the scaffold generator applies some basic Tailwind styling to generated views, so we have nice looking Widget pages right out of the box.

In order to test pagination as we work, we will need some Widgets in the database. Open your rails console with rails c and add test data to the Widgets table:

50.times do |n|
  Widget.create(name: "Widget ##{n}")
end

Pagination the old fashioned way

We are going to start by implementing pagination with standard Rails techniques. Each time a user requests a new page, we will load the new page with a full page turn, no Turbo required. Once pagination is working with full-page turns, we will add in Turbo to enhance the experience.

In our application, we will use Pagy to implement pagination. Let’s install Pagy now, following along with the Pagy quick start guide.

From your terminal, add pagy to your Gemfile:

bundle add pagy

Add Pagy’s backend to app/controllers/application_controller.rb:

class ApplicationController < ActionController::Base
  include Pagy::Backend
end

Add Pagy’s frontend helpers to app/helpers/application_helper.rb:

module ApplicationHelper
  include Pagy::Frontend
end

With Pagy installed and ready to use across the application, update app/controllers/widgets_controller.rb to paginate records on the index page:

def index
  @pagy, @widgets = pagy(Widget.all, items: 10)
end

And then finish up our traditional pagination implementation by adding a simple pager UI to app/views/widgets/index.html.erb:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Widgets</h1>
    <%= link_to 'New widget', new_widget_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="widgets" class="min-w-full">
    <%= render @widgets %>
  </div>
  <div id="pager" class="min-w-full my-8 flex justify-between">
    <div>
      <% if @pagy.prev %>
        <%= link_to "< Previous page", widgets_path(page: @pagy.prev), class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800" %>
      <% end %>
    </div>
    <div class="text-right">
      <% if @pagy.next %>
        <%= link_to "Next page >", widgets_path(page: @pagy.next), class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800" %>
      <% end %>
    </div>
  </div>
</div>

Here, we added the pager div and its contents, with a next and previous page buttons that render when a page exists to navigate to. The prev and next methods used on the links are supplied by pagy.

With this change in place, boot up the rails application with bin/dev and head to http://localhost:3000/widgets and see that widgets are paginated at 10 items per page. Click the next and previous links to move between pages as desired. Notice that each time a paging button is clicked, a full page turn is initiated and the entire content of the page is replaced.

In the next section, we will adjust our paging functionality to update only the widgets list and the pagination buttons, instead of performing a full-page turn on each request.

Navigate pages with Turbo

In this section, we will use a Turbo Frame to update the content of the widgets area with the new page data. Turbo Frames allow us to scope navigation to specific part of the page instead of replacing the entire page with each request.

Scoped navigation with Turbo Frames speeds up requests and allows us to build UIs that feel modern and fast, while continuing to use server-rendered HTML for each request.

To begin, we will wrap the widgets list and the pagination controls in a Turbo Frame. In app/views/widgets/index.html.erb:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Widgets</h1>
    <%= link_to 'New widget', new_widget_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <%= turbo_frame_tag "widgets", class: "min-w-full" do %>
    <%= render @widgets %>
    <%= render "pager", pagy: @pagy %>
  <% end %>
</div>

Here, we replaced the widgets div with a turbo_frame_tag, and moved the pager into that Turbo Frame.

This change means that all link clicks within the widgets Turbo Frame will now expect to receive a matching turbo_frame response from the server. Turbo will then replace the content of that frame with the content supplied by the server, leaving the rest of the page content untouched.

Before this will work, we need to add the pager partial and move the pagination controls into that partial. We don’t technically need to use a partial to render the pagination controls, but it helps keep the index page readable.

From your terminal:

touch app/views/widgets/_pager.html.erb

And then move the paging controls into the new pager partial:

<div id="pager" class="min-w-full my-8 flex justify-between">
  <div>
    <% if pagy.prev %>
      <%= link_to(
        "< Previous page",
        widgets_path(page: pagy.prev),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_action: "advance"
        }
      ) %>
    <% end %>
  </div>
  <div class="text-right">
    <% if pagy.next %>
      <%= link_to(
        "Next page >",
        widgets_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_action: "advance"
        }
      ) %>
    <% end %>
  </div>
</div>

The content here is nearly identical to what was previously in the index page. The only change is the addition of a new data-turbo-action data attribute on each link.

By default, when navigating within a Turbo Frame, the page URL does not change. Normally, this is correct behavior navigation within a Turbo Frame, but in our case it is not.

When a user moves from page one to page two, they expect to be able to refresh the page and stay on page two and to be able to use the back button in their browser to get back to page one.

The advance value for the data-turbo-action attribute tells Turbo to update the current URL and insert the previous page URL into the browser’s history, retaining the intuitive forward, back, and refresh behavior users expect.

At this point, refresh /widgets and see that clicking the Previous and Next page buttons correctly updates the content of the widgets frame. When you do this, you will notice one issue — navigating between pages does not update the user’s scroll position. They have to manually scroll back up to the top of the list to see the results.

We can fix this issue by updating the widgets Turbo Frame in the widgets index view:

<%= turbo_frame_tag "widgets", class: "min-w-full", autoscroll: "true" do %>

The autoscroll attribute tells Turbo to scroll the frame into view when the frame is loaded, automatically scrolling us back up to the top of the frame when a new page is loaded.

Nice work so far! We now have standard pagination implemented, powered by Turbo Frames. In the next section, we’ll transition to a manual version of an infinite scroll experience.

Manual “infinite scroll”

The first version of “infinite scroll” in our application will replace the Next and Previous pagination controls with a single load more button. When the user clicks this button, we will fetch the next page of widgets from the server, append them to the existing list of widgets, and update the load more button to prepare to fetch the next set of records.

The major functional change is that instead of replacing the content of the widgets list with entirely new content, we need to keep the current widgets in the list and add the new widgets to the end of the list.

This change will introduce us to a limitation of Turbo Frames. Today, navigation within a Turbo Frame always replaces the entire content of the Frame with new content. There is no concept of appending content using Turbo Frames — its replace or nothing.

This means that to implement an infinite scroll experience, we need to reach for Turbo Streams. In contrast to Turbo Frames, which always replace the target content, Turbo Streams can replace, remove, append, and prepend content as desired.

Our goal is to use the pagination controls to retrieve new widgets from the server and then append those widgets to the existing list with Turbo Streams. When we are finished, our server will render turbo-stream elements as HTML, which Turbo will use to update the widgets list and the pagination controls without touching the rest of the page.

To complicate matters a bit, Turbo expects Turbo Streams to be used with non-GET requests (like form submissions). There is no built-in way to render a Turbo Stream in response to a GET request, like the requests generated by clicks on our pagination controls.

One way to work around this is described in Dale’s article. In it, a Stimulus controller and request.js are used to insert a Turbo Stream header into GET requests, getting Turbo to see the request as a Turbo Stream request despite not originating from a form submission.

The approach is Dale’s article is a completely valid way to solve the problem and it works quite well. However, we are going to use a different method to reach the same destination. Our approach will use a not-obvious but built-in Turbo behavior to get a Turbo Stream response without modifying headers.

Whew. Let’s look at some code.

To start, we need an empty Turbo Frame. Update app/views/widgets/index.html.erb like this:

<div class="w-full">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Widgets</h1>
    <%= link_to 'New widget', new_widget_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <%= turbo_frame_tag "page_handler" %>
  <div id="widgets" class="min-w-full">
    <%= render @widgets %>
  </div>
  <%= render "pager", pagy: @pagy %>
</div>

Here, we added a page_handler Turbo Frame with no content inside and we removed the widgets Turbo Frame, which we no longer need.

This empty page_handler frame will be the messenger that sneaks our Turbo Stream content in from the server, no header modification required.

To see this in action, update the pager partial to remove the old pagination controls and replace them with a single load more link:

<div id="pager" class="min-w-full my-8 flex justify-center">
  <div>
    <% if pagy.next %>
      <%= link_to(
        "Load more widgets",
        widgets_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_frame: "page_handler"
        }
      ) %>
    <% end %>
  </div>
</div>

Notice the load more link is targeting the page_handler Turbo Frame, informing Turbo that clicks on that link should replace the content of the page_handler frame, instead of navigating the entire page. Because the load more link is not nested within the page_handler frame, we need this attribute to target that frame.

Now we have pagination controls targeting an empty Turbo Frame, but clicking on the link will just re-render app/views/widgets/index.html.erb with an empty page_handler frame. That’s not very useful.

To make this work, we need to update our controller to enable turbo_frame variants, so that we can render different content from the index action in response to a Turbo Frame request.

In app/controllers/application_controller.rb:

class ApplicationController < ActionController::Base
  include Pagy::Backend

  before_action :turbo_frame_request_variant
  
  private

  def turbo_frame_request_variant
    request.variant = :turbo_frame if turbo_frame_request?
  end
end

Here we are using a turbo-rails method, turbo_frame_request?, to identify inbound Turbo Frame requests. When the inbound request is a Turbo Frame, we tell our controller to respond with a turbo_frame variant instead of the normal html.erb content.

To see this in action, create the new Turbo Frame variant for the index action. From your terminal:

touch app/views/widgets/index.html+turbo_frame.erb

And then fill the new view in:

<%= turbo_frame_tag "page_handler" do %>
  <%= turbo_stream_action_tag(
    "append",
    target: "widgets",
    template: %(#{render @widgets}) 
  ) %>
  <%= turbo_stream_action_tag(
    "replace",
    target: "pager",
    template: %(#{render "pager", pagy: @pagy})
    ) %>
<% end %>

Here, we respond with a page_handler Turbo Frame because Turbo expects us to render content for that frame when the load more link is clicked.

Inside of that Turbo Frame is where the magic happens. We first render a Turbo Stream that appends the @widgets to the existing list of widgets (using the widgets id). Then we render another Turbo Stream to replace the content of the pager div with an updated version of the pager.

Now, when the user clicks the load more link, a Turbo Frame request is sent to the /widgets, Rails sees the index.html+turbo_frame.erb view and responds with the content of that view, rendered as plain HTML.

Turbo then sees the response on client-side, “replaces” the content of the page_handler Turbo Frame tag with the two turbo-stream elements, and then processes the actions defined in those turbo-streams. The end result is a new set of widgets appended to the list, and a load more button updated to fetch the next page of results.

See this in action by heading to the widgets index page and clicking the load more button. If all has gone well, each click of the load more button will append more widgets to the list and increment the page number each time.

Note that the Turbo Frame + Turbo Stream technique we used here was originally found on the Turbo discussion forums — the folks there figured it out, I’m just building on their great work.

Now we have a manual “infinite scroll” experience in place. Let’s finish this article by using Stimulus to fetch new widgets automatically as the user scrolls down the page.

Automatic infinite scroll

Our infinite scroll experience will be powered by a Stimulus controller and will rely on the IntersectionObserver API to fetch new widgets automatically as the user scrolls the page.

To make using the IntersectionObserver API easier, we will add the wonderful stimulus-use package to our application. This is not a requirement, but it does simplify the code a bit.

From your terminal:

bin/importmap pin stimulus-use

We also need a Stimulus controller to add the automatic fetch behavior to the DOM as the user scrolls. Again from your terminal, generate a new Stimulus controller:

rails g stimulus autoclick

Fill in the new Stimulus controller at app/javascript/controllers/autoclick_controller.js:

import { Controller } from "@hotwired/stimulus"
import { useIntersection } from 'stimulus-use'

export default class extends Controller {
  options = {
    threshold: 1
  }

  connect() {
    useIntersection(this, this.options)
  }

  appear(entry) {
    this.element.click()
  }
}

This controller pulls in the useIntersection from stimulus-use. The appear function is triggered when the element the controller is attached scrolls into view in a user’s browser. appear simply calls click() on the element the controller is attached to.

To use this controller, update the pager partial:

<div id="pager" class="min-w-full my-8 flex justify-center">
  <div>
    <% if pagy.next %>
      <%= link_to(
        "Load more widgets",
        widgets_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_frame: "page_handler",
          controller: "autoclick"
        }
      ) %>
    <% end %>
  </div>
</div>

Here, we added data-controller="autoclick" to the load more link. With this change in place, each time the load more link is scrolled into view, the Stimulus controller will programmatically click the load more link. Each time this occurs, a Turbo Frame request to the index action is fired to fetch and append the next set of widgets.

The autoclick controller we are using here was lightly adapted from Sean Doyle’s autoclick controller in his own implementation of infinite scrolling with Turbo.

Sean’s implementation of infinite scrolling presents yet another approach to working around the limits of Turbo Frames and is worth reviewing in full, if you are interested in more advanced Turbo use cases. In Sean’s work, the key thing to note is his use of the code from this Turbo draft PR which adds additional “actions” to Turbo Frames.

If you plan to use an approach like this one in a production application: A reader surfaced that the infinite scrolling technique we use in this book does not work reliably on Firefox for Android. Those users have to click the Load More button manually to load new records.

That’s all for this tutorial, great work following along!

Wrapping up

Today we implemented multiple pagination styles in a Rails 7 application with Turbo Frames, Turbo Streams, and Stimulus. While building pagination, we got to see a couple of useful, more advanced uses of Turbo Frames in Rails:

Rendering Turbo Frame variants to respond with different content in response to Turbo Frame requests
Rendering Turbo Streams inside of empty Turbo Frame tags to use Turbo Streams in response to GET requests

These types of techniques dramatically expand the usefulness of Turbo without adding significant complexity to your code, and are helpful tools to add to your Turbo kit.

The very smart folks on the StimulusReflex discord inspired this article, and the excellent work done by Dale Zak and Sean Doyle served as a great foundation to build on.

This article is intended to serve as a supplement to their work, presenting alternative approaches to help expand the set of tools we have to work with in Turbo.

That’s all for today. As always, thanks for reading!

Real-time previews with Rails and StimulusReflex

2022-02-03T00:00:00+00:00

Today we are going to build a live, server-rendered, liquid-enabled markdown previewer with Ruby on Rails and StimulusReflex. It’ll be pretty neat.

Allowing users to preview their content before saving it is a common need in web applications — posts, products, emails. Any user-created content that gets turned into HTML can benefit from a preview function to help users check their work before they save it.

Our StimulusReflex-powered previewer will parse user-generated markdown on the server, insert dynamic content with liquid, and update the DOM in ~100ms, fast enough that the preview feels instant.

When our work is done, the end product will look like this:

If you’d like to try out the previewer in a production environment, you can try it out on Heroku. The demo application is hosted on Heroku’s free tier so expect a delay on the first page load!

Before diving in, this tutorial assumes a basic level of Rails knowledge. If you have never written Rails before, this tutorial may not be for you. You do not need any prior experience with Stimulus or StimulusReflex to follow along — in fact, this tutorial will be most useful to you if you are new to StimulusReflex and curious about how it can help you build great experiences faster.

Let’s get started!

Application setup

We are working from a base Rails 7 application with TailwindCSS and StimulusReflex installed. To follow along with this tutorial, start by cloning the starter repo so we can skip past the install steps and get to build the application. All of the code changes we will make in this tutorial can be found in this pull request.

Our application will allow users to create and edit Posts. As they modify a post, the preview display beside the post will update. Before we can preview posts, we’ll need a Post resource to work with, so let’s begin by scaffolding that resource up.

From your terminal:

rails g scaffold Post title:string body:text
rails db:migrate

After you scaffold the resource, start up the rails application and build CSS and JavaScript with bin/dev. Head to http://localhost:3000/posts and make sure that creating and editing posts works before moving on.

As a reminder, our end goal is a live, updated-as-you-type preview displayed beside the post form. We’ll construct the UI first. Create a new preview partial from your terminal:

touch app/views/posts/_preview.html.erb

Fill the new partial in with:

<div class="bg-gray-200 p-4 shadow-sm rounded-sm">
  <article class="prose">
    <h2 class="text-2xl font-bold"><%= post.title %></h2>
    <div>
      <%= post.body %>
    </div>
  </article>
</div>

Update app/views/posts/new.html.erb to render the preview partial beside the post form, like this:

<div class="mx-auto md:w-2/3 w-full">
  <div class="flex space-x-12">
    <div class="w-1/2">
      <%= render "form", post: @post %>
    </div>
    <div class="w-1/2">
      <%= render "preview", post: @post %>
    </div>
  </div>
</div>

Now update app/views/posts/edit.html.erb with the same content:

<div class="mx-auto md:w-2/3 w-full">
  <div class="flex space-x-12">
    <div class="w-1/2">
      <%= render "form", post: @post %>
    </div>
    <div class="w-1/2">
      <%= render "preview", post: @post %>
    </div>
  </div>
</div>

Here, we could have created a new partial with the content we added to new and edit, but it is not strictly necessary. Feel free to move this content to a partial if you prefer.

Now we have preview content displayed beside the post form, but the preview content is static — visit posts/new and see that the “preview” is just an empty gray box. Not very useful yet.

In the next section, we will create a StimulusReflex-enabled Stimulus controller to update the preview content as the user types, making the gray preview box a little more useful.

Adding a Stimulus controller

Our application will eventually rely on a server-side reflex to update the content of the preview partial that we created in the last section. However, before we do that, let’s build a purely client-side implementation of the preview function to ease into things.

From your terminal, use the stimulus_reflex generator:

rails g stimulus_reflex post
rails stimulus:manifest:update

After this generator runs, you will see that the generator created both a client-side Stimulus controller and a server-side reflex: app/javascript/controllers/post_controller.js and app/reflexes/post_reflex.rb.

On the client-side, StimulusReflex enhances vanilla Stimulus controllers. StimulusReflex-powered Stimulus controllers have all of the same functionality of a regular Stimulus controller, with extra functionality layered on top.

To demonstrate this, we are going to start by using the new Stimulus controller, post_controller.js to update the content of the post preview as the user types. We will not have any markdown formatting or liquid substitution, but the page will react to user input and we will get to see a few Stimulus concepts in action.

Head to app/javascript/controllers/post_controller.js and update it:

import ApplicationController from './application_controller'

export default class extends ApplicationController {
  static targets = [ "title", "body", "titlePreview", "bodyPreview" ]

  preview() {
    this.titlePreviewTarget.innerText = this.titleTarget.value
    this.bodyPreviewTarget.innerText = this.bodyTarget.value
  }
}

Here, we inherit from ApplicationController, the base StimulusReflex controller.

In the controller, we define targets that we will use to obtain an easy reference to the elements that we care about. We use these targets in the preview function. Each time preview is called, the titlePreview and bodyPreview elements are updated with the current value of titleTarget and bodyTarget, respectively.

For the most part, this is regular JavaScript. If we wanted to, we could remove the targets and instead select each element by an id, with something like document.getElementById('body-target'). Stimulus targets give us a more convenient method to access any number of elements by a name, but nothing magical is happening.

To use this new preview function, we need to connect the new PostController to the DOM and add target elements and actions to the preview and form partials.

In new.html.erb:

<div class="mx-auto md:w-2/3 w-full">
  <div class="flex space-x-12" data-controller="post">
    <div class="w-1/2">
      <%= render "form", post: @post %>
    </div>
    <div id="post-preview" class="w-1/2">
      <%= render "preview", post: @post %>
    </div>
  </div>
</div>

Here we added data-controller="post" to the div wrapping the form and preview partials.

To use a Stimulus controller, you must connect the controller to a DOM element with the data-controller attribute. Stimulus scopes controllers based on the DOM hierarchy — the element with the data-controller attribute and all of the children of that element will be within that controller’s scope. targets and actions for a controller must be within the scope to function.

This scoping mechanism allows developers to have multiple independent instances of the same controller present on the page at once, when needed.

Update the form partial next:

<%= form_with(model: post, class: "contents") do |form| %>
  <% if post.errors.any? %>
    <div id="error_explanation" class="bg-red-50 text-red-500 px-3 py-2 font-medium rounded-lg mt-3">
      <h2><%= pluralize(post.errors.count, "error") %> prohibited this post from being saved:</h2>

      <ul>
        <% post.errors.each do |error| %>
          <li><%= error.full_message %></li>
        <% end %>
      </ul>
    </div>
  <% end %>

  <div class="my-5">
    <%= form.label :title %>
    <%= form.text_field(
      :title,
      class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full",
      data: {
        post_target: "title",
        action: "input->post#preview"
      } 
    ) %>
  </div>

  <div class="my-5">
    <%= form.label :body %>
    <%= form.text_area(
      :body,
      rows: 4,
      class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full",
      data: {
        post_target: "body",
        action: "input->post#preview"
      } 
    ) %>
  </div>

  <div class="inline">
    <%= form.submit class: "rounded-lg py-3 px-5 bg-blue-600 text-white inline-block font-medium cursor-pointer" %>
  </div>
<% end %>

The important updates here are on the two form inputs. On each, we added a data-post-target and a data-action attribute. The post-target attribute defines a target for the PostController, and the action attribute tells Stimulus to fire the PostController's preview function when an input event occurs on that input.

With this change in place, each time a user types a character in the title or body inputs, the PostController will run the preview function to update the titlePreview and bodyPreview elements.

This won’t work just yet though. Before it will, we need to set the titlePreview and bodyPreview targets. Head to app/views/posts/_preview.html.erb and update it:

<div class="bg-gray-200 p-4 shadow-sm rounded-sm">
  <article class="prose">
    <h2 class="text-2xl font-bold" data-post-target="titlePreview"><%= post.title %></h2>
    <div data-post-target="bodyPreview">
      <%= post.body %>
    </div>
  </article>
</div>

Notice the data-post-target attributes on the header and the body tags.

With this change in place, refresh the new post page, start typing and see that the content of the preview updates as you type.

This is a great start, but we aren’t really “previewing” anything because the Stimulus controller is not parsing markdown content or substituting liquid tags.

To add useful previews, we will use the server-side PostReflex that we generated at the beginning of this section.

Server-side previews with StimulusReflex

Stimulus controllers do their work on the client-side, adding and removing elements, updating classes or attributes. Even when we are using StimulusReflex, this is still true — Stimulus controllers stay on the client-side. To add server-side functionality, we need to move to the PostReflex, at app/reflexes/post_reflex.rb.

This reflex class will be responsible for transforming the content of the preview body from markdown to HTML, and for substituting any liquid tags present in the content. Once the content is transformed, StimulusReflex will send the transformed content back to the client, where it will be inserted seamlessly into the DOM, all in ~100ms.

Before adding markdown and liquid parsing, let’s move the client-side preview updates to the server-side reflex. Update PostReflex like this:

class PostReflex < ApplicationReflex  
  def preview
    morph '#preview-title', post_params[:title]
    morph '#preview-body', post_params[:body]
  end

  private

  def post_params
    params.require(:post).permit(:title, :body)
  end
end

In this reflex, we are using two selector morphs to update the preview content, identifying the element to update with an id. Selector morphs allow us to run reflex actions without a full pass through a controller action with ActionDispatch, and are perfect for features that only update a small piece of the page, like our preview functionality.

post_params is identical to attribute whitelisting that you’ll find in a standard Rails controller — each time this reflex is called, the data from the post form will be serialized and sent to the server, giving us a handy way to reference the current content of the form.

In StimulusReflex, there are two ways to call a server-side reflex.

The first option is through a reflex data attribute on a DOM element. This approach completely bypasses the reflex’s related Stimulus controller and directly invokes the server-side reflex. This option works well when you have a simple reflex that does not need custom options to function.

Our use case requires us to use the second method for calling reflexes: using this.stimulate in a Stimulus controller. stimulate is extremely flexible and allows us to override the element passed to the server-side reflex, define options, and run any JavaScript we like before sending the reflex to the server.

To call a reflex with stimulate, head back to app/javascript/controllers/post_controller.js and update it like this:

import ApplicationController from './application_controller'

export default class extends ApplicationController {
  connect() {
    super.connect()
  }

  preview() { 
    this.stimulate("Post#preview", { serializeForm: true })
  }
}

Here, we removed the targets from the controller and preview now just calls this.stimulate.

The serializeForm option ensures the content of the closest form element in the DOM is passed to the reflex on the server. The data in the form is accessible on the server via params, as we saw in the PostReflex earlier in this section.

Now we have a reflex defined on the server, and a Stimulus controller ready to call that reflex. To make this work, we need to update the DOM to match the structure expected by the reflex.

Recall in the preview method in PostReflex, we have two morphs that rely on ids to target the right element in the DOM:

def preview
  morph '#preview-title', post_params[:title]
  morph '#preview-body', post_params[:body]
end

For the reflex to work, we need an element with a preview-title id and another with a preview-body id. Update the preview partial like this:

<div class="bg-gray-200 p-4 shadow-sm rounded-sm">
  <article class="prose">
    <h2 id="preview-title" class="text-2xl font-bold"><%= post.title %></h2>
    <div id="preview-body">
      <%= post.body %>
    </div>
  </article>
</div>

Now both the title element and the body element have ids that match the expectations of the reflex, so when the reflex runs, those elements will be updated.

Now move over to the form partial to connect the post Stimulus controller to the form:

<%= form_with(model: post, class: "contents", data: { controller: "post" }) do |form| %>
  <% if post.errors.any? %>
    <div id="error_explanation" class="bg-red-50 text-red-500 px-3 py-2 font-medium rounded-lg mt-3">
      <h2><%= pluralize(post.errors.count, "error") %> prohibited this post from being saved:</h2>

      <ul>
        <% post.errors.each do |error| %>
          <li><%= error.full_message %></li>
        <% end %>
      </ul>
    </div>
  <% end %>

  <div class="my-5">
    <%= form.label :title %>
    <%= form.text_field(
      :title,
      class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full",
      data: {
        action: "input->post#preview"
      } 
    ) %>
  </div>

  <div class="my-5">
    <%= form.label :body %>
    <%= form.text_area(
      :body,
      rows: 4,
      class: "block shadow rounded-md border border-gray-200 outline-none px-3 py-2 mt-2 w-full",
      data: {
        action: "input->post#preview"
      } 
    ) %>
  </div>

  <div class="inline">
    <%= form.submit class: "rounded-lg py-3 px-5 bg-blue-600 text-white inline-block font-medium cursor-pointer" %>
  </div>
<% end %>

Here, notice the form element now has a data-controller="post" attribute, scoping the Stimulus controller to the form. We also updated the form inputs to remove the unnecessary data-post-target attributes.

Because we are serializing the entire content of the form, we no longer need a direct reference to the individual input elements in the Stimulus controller, so we do not need the target attributes.

One last step to move the preview functionality to the server. Because we connected the PostController to the form, we need to remove the controller from the wrapper div in app/view/posts/new.html.erb:

<div class="mx-auto md:w-2/3 w-full">
  <div class="flex space-x-12">
    <div class="w-1/2">
      <%= render "form", post: @post %>
    </div>
    <div class="w-1/2">
      <%= render "preview", post: @post %>
    </div>
  </div>
</div>

Previously, we needed a direct reference in the Stimulus controller to the preview elements, so we needed both the form and preview partials to be within the scope of the PostController.

Because the preview elements are now updated in the server-side reflex and referenced by id, the preview partial no longer needs to be in the PostController’s scope.

With this last change in place, head back to http://localhost:3000/posts/new, start typing and see that the preview content updates each time the form changes. To confirm that StimulusReflex is doing the work on the server, watch the Rails server logs as you type. With each key press, the server logs will output information about the reflex:

Now we are “previewing” content as the user types, but that preview still is not doing anything useful. We just added a round trip to the server, but the server is not doing anything useful with the content yet.

In the next section, we will make the server-side reflex more valuable by adding markdown and liquid tag parsing when the preview content is updated.

Add markdown and liquid parsing

We will use redcarpet to parse markdown. To do so, We need to add redcarpet to our application.

From your terminal:

bundle add redcarpet

Restart your Rails application and head to app/helpers/posts_helper.rb and add a method to parse markdown content:

module PostsHelper
  def to_markdown(content)
    return '' if content.blank?

    markdown = Redcarpet::Markdown.new(
      Redcarpet::Render::HTML,
      fenced_code_blocks: true,
      autolink: true
    )
    markdown.render(content).html_safe
  end
end

This helper takes a string, initializes a new instance of Redcarpet::Markdown as described in the documentation, and then renders the content to HTML.

The to_markdown helper will be used in two places in our application: When the preview partial is loaded during a normal request (like when visiting the post edit page) and when parsing content on the fly in the PostReflex.

Update the preview partial first, at app/views/posts/_preview.html.erb:

<div class="bg-gray-200 p-4 shadow-sm rounded-sm">
  <article class="prose">
    <h2 class="text-2xl font-bold" id="preview-title">
      <%= post.title %>
    </h2>
    <div id="preview-body">
      <%= to_markdown(post.body) %>
    </div>
  </article>
</div>

When a user visits the new or edit pages, markdown content in the body will be parsed and rendered as HTML.

Update the PostReflex to use the new to_markdown helper:

def preview
  morph '#preview-title', post_params[:title]
  morph '#preview-body', ApplicationController.helpers.to_markdown(post_params[:body])
end

Head to new or edit now, type some markdown content in the body input and see that as you type, the markdown is transformed into HTML instantly.

At this point, we have server-powered markdown parsing — nice work!

Our next step is adding liquid support, allowing users to transform liquid content into real data and see that content processed in real time.

From your terminal:

bundle add liquid

Back to the PostsHelper:

module PostsHelper
  def to_markdown(content)
    return '' if content.blank?

    markdown = Redcarpet::Markdown.new(
      Redcarpet::Render::HTML,
      fenced_code_blocks: true,
      autolink: true
    )
    markdown.render(liquified(content)).html_safe
  end

  def liquified(content)
    Liquid::Template
      .parse(content)
      .render('company_name' => 'Hotwiring Rails')
  end
end

liquified takes a string of content, scans it for matching liquid tags and objects, and translates them.

Our example implementation mocks up a simple company_name tag — in a real application we could use liquid tags to insert data about the object we are working on.

Restart your Rails application again and then go back to the new or edit posts page, enter in a body with some markdown content and the `` tag and see that the tags are replaced as you type:

Debouncing requests

Right now, every keystroke triggers a round trip to the server. Many of these requests are unnecessary because the user will have typed another character before the request completes. A simple way to reduce the load on the server is to debounce preview function, waiting for the user to pause before triggering the server-side reflex.

To debounce the preview function, we will use lodash’s debounce. From your terminal:

yarn add lodash

And then update the Post Stimulus controller:

import ApplicationController from './application_controller'
import debounce from "lodash/debounce"

export default class extends ApplicationController {
  connect() {
    super.connect()
    this.preview = debounce(this.preview.bind(this), 50)
  }

  preview() { 
    this.stimulate("Post#preview", { serializeForm: true })
  }
}

Now, preview will wait 50ms before firing. Feel free to play around with the wait period to find the time that feels right to you.

And with that change, you have reached the end of this tutorial, great work today!

Wrapping up

Today we built a fully functional, StimulusReflex-powered markdown and liquid tag parser. Our application processes content on the server and returns it to the client without page turns, saving records in the database, or dealing with the overhead of a Rails controller action.

While building this application, we learned a bit about how to use Stimulus and StimulusReflex in Rails application and applied some basic techniques of both to create a real-time experience while staying close to core Rails principles.

Before using something like what we built today in production, a couple of things to think about:

Why not just parse things on the client?

Throughout this tutorial, you may have wondered “Why don’t we just parse the markdown on the client?”

Liquid processing was introduced to give us a reason to parse content like this on the server. We are here to learn about Stimulus and StimulusReflex, so I fit the requirements to that goal.

If your application just needs markdown parsing without the extra complications of liquid, a client-side parser is a completely reasonable (and more performant) choice!

Do you need real-time previews?

The live preview we built is useful for learning and showing off what StimulusReflex can do, but it may not be the most desirable user experience in a real application. The live preview experience works fine for some use cases, particularly when the content is only a few paragraphs in length.

As the content gets longer, a better approach is moving the “preview” content into a separate tab, hidden by default. As the user types, update the content in the hidden tab as usual, but keep the content hidden until the user requests it. That experience is likely to scale a bit better, while not being any more technically complex than the experience we built.

Hotwiring Rails Newsletter - January 2022

2022-02-01T00:00:00+00:00

Thoughts or feedback on how I can make this newsletter more valuable? Have something you’d like to include in next month’s edition? Send me an email, or find me on Twitter.

Articles and guides
New and interesting PRs and releases
The best things I read recently

Articles and guides

The Hotwire example treasure trove by Sean Doyle

This Github repo has been making the Twitter and Discord rounds lately, and for good reason.

Each branch in the repo contains a writeup and full implementation of a common UX pattern built with Stimulus and Turbo. Keep this handy and reference the examples when you need them.

View the treasure trove on Github

Live reloading for Hotwire Rails apps by Kirill Platonov

With the demise of Webpacker (more on that below), replacing webpack-dev-server is top of mind for many folks. Kirill explored this problem and built a gem to tackle it.

Read the how-to on kirillplatonov.com

Remote modals with Rails, Hotwire, and Bootstrap by Benito Serna

This quick article offers a slightly different take on implementing server-rendered modal content with Turbo, hooking into the before-render callback to neatly transition the modal out when needed.

Read the full article on bhserna.com

Pagination in Rails from scratch by Ana Nunes da Silva

Implementing common application needs from scratch can be a really interesting learning exercise. It can be worth doing even if just to get more comfortable reasoning about the problem before reaching for a battle-tested gem.

In this article, you’ll build a simple pagination system from scratch, complete with a front end implementation and the standard set of convenience methods.

Build your own pagination on ananunesdasilva.com

Speaking of building things from scratch, ICYMI, Steve Polito’s recent look at building an authentication system from scratch is a good read.

Dueling tutorials: Two ways to lazy load tooltips with modern Rails

Steve Polito published “Hotwire: Asynchronously loaded tooltips” last week based on code from Sean Doyle. This is a well-constructed article guiding you through remotely loaded tooltips powered by Turbo Frames. As a bonus, it includes a nice use of TailwindCSS’s peer class.

Julian Rubisch read Steve’s article and was inspired to write up an alternative implementation of the same functionality with Futurism. As Julian notes in the article, Futurism can offer a simpler migration path for legacy applications that were not designed from zero with Turbo Frames in mind.

Sometimes we can get stuck in one single path forward — Julian’s response article is a great reminder that there are usually alternatives out there when we need them.

On Stimulus Reflex: Why and Why by Stephen Margheim

This is a thoughtful, code-free read from Stephen. Inspired by conversations on the StimulusReflex Discord, Stephen presents a take on how StimulusReflex fits into a Turbo-powered Rails world.

Read it on fractaledmind.com

Rails 7 features, before and after by Zakaria Fatahi

This gist is a nice, compact introduction to some of the most interesting new features in Rails 7. A nice bookmark to read when you’re making the Rails 7 upgrade.

View it on Github

New and interesting PRs and releases

Farewell, Webpacker

Webpacker was officially retired in January. Sadly, it was retired before an official release of Webpacker 6 was cut.

Rails 7’s import maps and js/cssbundling gems made Webpacker’s retirement a foregone conclusion, but it was surprising to see it retired without a 6.0 release.

For folks on Webpacker 5.4, despite the retirement I do not think there is an urgent need to switch off of Webpacker — you can still upgrade existing apps to Rails 7 without changing bundling strategies.

For folks who love Webpacker and want to stick with it long term, Webpacker’s development has moved to Shakapacker, which is likely to be maintained well into the future.

Custom Turbo Stream actions

This is technically a comment on a PR, but its my newsletter and I make the rules. Sean Doyle stars again, this time demonstrating a method for “adding” new actions to Turbo Streams through clever use template tags and Stimulus.

If you need more power than Turbo Streams seven actions offer (and you can’t use CableReady), this technique might help.

View the technique on Github

The best things I read recently

This is the third Hotwiring Rails newsletter I’ve written, and Baldur’s writing has been in the best thing(s) I’ve read recently twice.

Apologies for the lack of variety but this piece was a great read on why keeping up with the latest and greatest in web development can feel so overwhelming compared to the same problem in other fields. Baldur’s article is well-worth reading for anyone feeling overwhelmed by trying to keep up with all the newest tools, tricks, and techniques in web development.

In particular, shifting from “keeping up” to doing research is a great mental model for cutting through the noise and keeping yourself focused on resources that are useful for you and interesting to you. This specific article is a funny thing to share in a newsletter about the latest Rails developments — the whole deal is that I share a bunch of Rails-related content because who doesn’t love learning as much as possible all of the time, right?

Putting the article into context with this newsletter: I resonate with Baldur’s “do research” approach and I model my own content consumption in a similar way.

In Baldur’s terms, my “Work Question” is what’s happening at the edge of Ruby on Rails. Most of the content I consume is only useful to me in that context, and that’s where I keep my brain while I’m reading. Before organizing myself around this Work Question, I spent a lot of time trying to keep up with React and Vue, and I spent hours following tutorials on building serverless apps (yikes) — things I had no real interest in or need for, but that seemed like I should be aware of.

One day I just stopped reading all of that stuff. I have no idea what’s going on with React, or Vue, or even if those are still the cool JavaScript frameworks to use. Instead, I just learn about what’s going on in Rails, and I’m happier and better informed because of it.

If you are deep in information overload, read Baldur’s piece for help and then apply his approach to this newsletter — if an article I linked doesn’t fit your Work Question, do you need to read it or can you ignore it?

Read it on baldurbjarnason.com

Until next time

RailsConf 2022’s Call For Proposals is open — you have until February 28th to submit!

Thanks for reading! Tune in next month for news on Hotwired ATS.

Hotwiring Rails Newsletter - December 2021

2021-12-07T00:00:00+00:00

Thoughts or feedback on how I can make this newsletter more valuable? Have something you’d like to include in next month’s edition? Send me an email, or find me on Twitter.

Articles and guides
New and interesting PRs and releases
Recent pieces from my blog
The best things I read recently

Articles and guides

Rails, Hotwire, CableReady, and StimulusReflex are BFFs by Nate Hopkins

This article from the creator of CableReady and StimulusReflex offers perspective on how the different pieces of the modern Rails landscape fit together.

This one is full of useful links and thoughtfully lays out a key philosophical distinction between Turbo’s aggressive adherence to RESTful principles and SR’s embrace of flexibility.

Read it on dev.to

Kubing Rails: stressless Kubernetes deployments with Kuby by Vladimir Dementyev and Travis Turner

If you (like me) have been intimidated by the complexity of Kubernetes, you might be interested in taking Kuby for a spin.

In this detailed tutorial on deploying a real Rails application with Kuby, the Evil Martians team gets down in the weeds and describes challenges they encountered and how they overcame them while deploying a Rails + AnyCable application to DigitalOcean with Kuby.

Read it on evilmartians.com

Why Hotwire in 2021 by Cédric Fabianski and Elizabeth Braae

The Bearer team talks about their transition from a complex React-powered frontend to a simplified, Hotwire-powered approach with a small team and a rapidly growing product.

If you’re considering whether your team can make the switch, I’ll leave this quote from the article here, click through to read the rest:

“We were essentially maintaining two applications where one should have been enough. The big learning was that this architecture might have made sense for large, stable applications, but it didn’t make sense for a company iterating fast to find a good product market fit.”

Read it on bearer.com

Building a Real-Time Chat App in Rails Using ActionCable and Turbo by Abiodun Ajibade

This implementation of the classic chat application demo relies on Turbo Streams to build a multi-room chat application, with a little bit of Stimulus in the mix. Turbo Stream model broadcasts keep things simple and clean — a nice introduction to the power of Turbo Streams and, more generally, the power that ActionCable unlocks for Rails libraries.

Read it on honeybadger.io

Rails Remote Elements Tutorial by Steve Polito

Steve takes us through an interesting approach to implementing a SPA with rails-ujs and Stimulus, showing us that the tools to build highly responsive user experiences in Rails have been available for years.

Leveraging Stimulus data attributes and built in rails-ujs events, we get a classic SPA todo application without React. An interesting experiment and learning experience, and a nice reminder of the power of rails-ujs, forgotten, but not gone.

Read it on stevepolito.design

A Rails Multi-Tenant Strategy That’s ~30 Lines and “Just Works” by Jason Meller

This quick read demonstrates a simple multi-tenant implementation without reliance on gems. Thoughtfully designed and battle tested in a large production application, this is a nice implementation of a common requirement for Rails-powered SaaS applications.

Included is a neat trick for setting Current in the Rails console, discussions of testing considerations, and a thoughtful acknowledgement of criticisms of Rails’ Current attributes implementation.

Read it on dev.to

Rails 7, Railties, and Sorbet at Shopify with Rafael França from Rails Core by Remote Ruby

I’ll confess that I’m not much for listening to technical podcasts, but this conversation between the Remote Ruby team and Rafael França was very enjoyable.

Come for the (now confirmed) hint at the Rails 7 release timeline, stay for the thoughtful words from Rafael on Shopify’s development practices, types, and Hotwire.

Listen on transistor.fm

The CableReady Language Implementation Project

The StimulusReflex team has a vision to expand CableReady outside of the Rails world and into a “universal standard tool for developers to dynamically control browsers from the server”. This is a big project, and they’re looking for help from developers capable of building server-side libraries in languages like Go, Python, and Java. If this sounds like you, give their call for support a review and get in touch with the team.

Read it on dev.to

New and interesting PRs and releases

Turbo

Turbo 7.1 is out!

This is a huge release that includes changes that adds rails-ujs style functionality to Turbo and fixes for a number of outstanding bugs, a few of the highlights in this release for me:

Push state history on frame navigations — This is the most exciting addition in 7.1 in my opinion, making it trivial to update URLs on frame navigation. This change dramatically improves the usability of tabbed interfaces and search forms powered by Turbo Frames
Add requestSubmit polyfill — This small change eliminates the need to manually add the requestSubmit polyfill to projects using Turbo. If you’ve moved away from rails-ujs, you may have gotten bit by the lack of access to Rails.fire(‘submit’) like me. requestSubmit with this polyfill is a nice alternative
Add data-confirm behavior to Turbo — Another rails-ujs behavior cleanly migrated to Turbo. The implementation is slightly more verbose so that it doesn’t clash with existing rails-ujs behavior, just in case your project has Turbo alongside old rails-ujs code.

Rails

Rails 7 RC1 is out!

With this release, we’re on track to see Rails 7.0 out before the end of the year.

Since Alpha 2 in September, much of the work has been on bug fixes, deprecation, and documentation. One interesting PR that made the cut was the addition of a standardized error reporting interface, via Rails.error that should simplify handling and reporting errors in Rails apps.

Recent pieces from my blog

I won’t be publishing many brand new pieces for the next few months because I’m focused on finishing my modern Rails development book/course (~2/3 done with the code).

Before going heads down, I published this quick piece on my Rails + esbuild config for live reloads.

I’m also committed to keeping my turbo-rails documentation pieces up to date with new Turbo releases — Turbo Frames on Rails was updated after the release of Turbo 7.1.

Expect to see many more shorter form tutorials from me after I publish my Hotwiring Rails course in (fingers crossed) late January. ___

The best things I read recently

The unreasonable effectiveness of simple HTML by Terence Eden — This newsletter, and all of my writing, is about modern web development. I’m interested in building highly interactive websites, which some times means using techniques absolutely will not work on Internet Explorer, much less on a PSP or a TV.

This piece from January of 2021 is a thoughtful reminder that, for many use cases, your users are best served with simple, old school websites that work, all the time, in every situation, with no fluff.

Exploring new things is fun, cool, and sometimes profitable, but if you’re building web sites for critical services, very broad audiences, or vulnerable populations, you probably don’t need that slick animation or cool new JavaScript tech.

The pragmatic engineer test: 12 questions on engineering culture by Gergely Orosz — Hiring Rails developers seems to get harder by the day and retaining your current team can be a challenge.

Building the right culture — and sharing that culture widely — can help you get a leg up. Gergely’s thoughtful overview of the questions you should be able to answer as a leader about your engineering team’s culture is a nice place to get started.

Bonus — Jamie Lawrence walks through how his team at Podia scores on these 12 questions.

Until next time

RailsConf 2022 has been announced. I’ll be there — I hope to see some of you there too!

That’s all for this month’s edition of Hotwiring Rails. For those of you who celebrate, happy holidays! I’ll be back in the new year.

Live reloading with Ruby on Rails and esbuild

2021-11-06T00:00:00+00:00

As you may have heard by now, Rails 7 comes out of the box with importmap-rails and the mighty Webpacker is no longer the default for new Rails applications.

For those who aren’t ready to switch to import maps and don’t want to use Webpacker now that it is no longer a Rails default, jsbundling-rails was created. This gem adds the option to use webpack, rollup, or esbuild to bundle JavaScript while using the asset pipeline to deliver the bundled files.

Of the three JavaScript bundling options, the Rails community seems to be most interested in using esbuild, which aims to bring about a “new era of build tool performance” and offers extremely fast build times and enough features for most users’ needs.

Using esbuild with Rails, via jsbundling-rails is very simple, especially in a new Rails 7 application; however, the default esbuild configuration is missing a few quality of life features. Most important among these missing features is live reloading. Out of the box, each time you change a file, you need to refresh the page to see your changes.

Once you’ve gotten used to live reloading (or its fancier cousin, Hot Module Replacement), losing it is tough.

Today, esbuild doesn’t support HMR, but with some effort it is possible to configure esbuild to support live reloading via automatic page refreshing, and that’s what we’re going to do today.

We’ll start from a fresh Rails 7 install and then modify esbuild to support live reloading when JavaScript, CSS, and HTML files change.

Before we get started, please note that this very much an experiment that hasn’t been battle-tested. I’m hoping that this is a nice jumping off point for discussion and improvements. YMMV.

With that disclaimer out of the way, let’s get started!

Application setup

We’ll start by creating a new Rails 7 application.

If you aren’t already using Rails 7 for new Rails applications locally, this article can help you get your local environment ready.

Once your rails new command is ready for Rails 7, from your terminal:

rails new live_esbuild -j esbuild
cd live_esbuild
rails db:create
rails g controller Home index

Here we created a new Rails application set to use jsbundling-rails with esbuild and then generated a controller we’ll use to verify that the esbuild configuration works.

Booting up

In addition to installing esbuild for us, jsbundling-rails creates a few files that simplify starting the server and building assets for development. It also changes how you’ll boot up your Rails app locally.

Rather than using rails s, you’ll use bin/dev. bin/dev uses foreman to run multiple start up scripts, via Procfile.dev. We’ll make a change to the Procfile.dev later, but for now just know that when you’re ready to boot up your app, use bin/dev to make sure your assets are built properly.

Configure esbuild for live reloading

To enable live reloading, we’ll start by creating an esbuild config file. From your terminal:

touch esbuild-dev.config.js

To make things a bit more consumable, we’ll first enable live reloading for JavaScript files only, leaving CSS and HTML changes to wait for manual page refreshes.

We’ll add reloading for views and CSS next, but we’ll start simpler.

To enable live reloading on JavaScript changes, update esbuild-dev.config.js like this:

#!/usr/bin/env node

const path = require('path')
const http = require('http')

const watch = process.argv.includes('--watch')
const clients = []

const watchOptions = {
  onRebuild: (error, result) => {
    if (error) {
      console.error('Build failed:', error)
    } else {
      console.log('Build succeeded')
      clients.forEach((res) => res.write('data: update\n\n'))
      clients.length = 0
    }
  }
}

require("esbuild").build({
  entryPoints: ["application.js"],
  bundle: true,
  outdir: path.join(process.cwd(), "app/assets/builds"),
  absWorkingDir: path.join(process.cwd(), "app/javascript"),
  watch: watch && watchOptions,
  banner: {
    js: ' (() => new EventSource("http://localhost:8082").onmessage = () => location.reload())();',
  },
}).catch(() => process.exit(1));

http.createServer((req, res) => {
  return clients.push(
    res.writeHead(200, {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      "Access-Control-Allow-Origin": "*",
      Connection: "keep-alive",
    }),
  );
}).listen(8082);

There’s a lot going on here, let’s walk through it a section at a time:

const path = require('path')
const http = require('http')

const watch = process.argv.includes('--watch')
let clients = []

First we require packages and define a few variables, easy so far, right?

Next, watchOptions:

const watchOptions = {
  onRebuild: (error, result) => {
    if (error) {
      console.error('Build failed:', error)
    } else {
      console.log('Build succeeded')
      clients.forEach((res) => res.write('data: update\n\n'))
      clients.length = 0
    }
  }
}

watchOptions will be passed to esbuild to define what happens each time an esbuild rebuild is triggered.

When there’s an error, we output the error, otherwise, we output a success message and then use res.write to send data out to each client.

Finally, clients.length = 0 empties the clients array to prepare it for the next rebuild.

require("esbuild").build({
  entryPoints: ["application.js"],
  bundle: true,
  outdir: path.join(process.cwd(), "app/assets/builds"),
  absWorkingDir: path.join(process.cwd(), "app/javascript"),
  watch: watch && watchOptions,
  banner: {
    js: ' (() => new EventSource("http://localhost:8082").onmessage = () => location.reload())();',
  },
}).catch(() => process.exit(1));

This section defines the esbuild build command, passing in the options we need to make our (JavaScript only) live reload work.

The important options are the watch option, which takes the watch and watchOptions variables we defined earlier and banner.

esbuild’s banner option allows us to prepend arbitrary code to the JavaScript file built by esbuild. In this case, we insert an EventSource that fires location.reload() each time a message is received from localhost:8082.

Inserting the EventSource banner and sending a new request from 8082 each time rebuild runs is what enables live reloading for JavaScript files to work. Without the EventSource and the local request sent on each rebuild, we would need to refresh the page manually to see changes in our JavaScript files.

http.createServer((req, res) => {
  return clients.push(
    res.writeHead(200, {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      "Access-Control-Allow-Origin": "*",
      Connection: "keep-alive",
    }),
  );
}).listen(8082);

This section at the end of the file simply starts up a local web server using node’s http module.

With the esbuild file updated, we need to update package.json to use the new config file:

"scripts": {
  "build": "esbuild app/javascript/*.* --bundle --outdir=app/assets/builds",
  "start": "node esbuild-dev.config.js"
}

Here we updated the scripts section of package.json to add a new start script that uses our new config file. We’ve left build as-is since build will be used on production deployments where our live reloading isn’t needed.

Next, update Procfile.dev to use the start script:

web: bin/rails server -p 3000
js: yarn start --watch

Finally, let’s make sure our JavaScript reloading works. Update app/views/home/index.html.erb to connect the default hello Stimulus controller:

<h1 data-controller="hello">Home#index</h1>
<p>Find me in app/views/home/index.html.erb</p>

Now boot up the app with bin/dev and head to http://localhost:3000/home/index.

Then open up app/javascript/hello_controller.js and make a change to the connect method, maybe something like this:

import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  connect() {
    this.element.textContent = "Hello Peter. What's happening?"
  }
}

If all has gone well, you should see the new Hello Peter header on the page, replacing the Hello World header.

If all you want is JavaScript live reloading, feel free to stop here. If you want live reloading for your HTML and CSS files, that’s where we’re heading next.

HTML and CSS live reloading

esbuild helpfully watches our JavaScript files and rebuilds every time they change. It doesn’t know anything about non-JS files, and so we’ll need to branch out a bit to get full live reloading in place.

Our basic approach will be to scrap esbuild’s watch mechanism and replace it with our own file system monitoring that triggers rebuilds and pushes updates over the local server when needed.

To start, we’re going to use chokidar to watch our file system for changes, so that we can reload when we update a view or a CSS file, not just JavaScript files.

Install chokidar from your terminal with:

yarn add chokidar -D

With chokidar installed, we’ll update esbuild-dev.config.js again, like this:

#!/usr/bin/env node

const path = require('path')
const chokidar = require('chokidar')
const http = require('http')

const clients = []

http.createServer((req, res) => {
  return clients.push(
    res.writeHead(200, {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      "Access-Control-Allow-Origin": "*",
      Connection: "keep-alive",
    }),
  );
}).listen(8082);

async function builder() {
  let result = await require("esbuild").build({
    entryPoints: ["application.js"],
    bundle: true,
    outdir: path.join(process.cwd(), "app/assets/builds"),
    absWorkingDir: path.join(process.cwd(), "app/javascript"),
    incremental: true,
    banner: {
      js: ' (() => new EventSource("http://localhost:8082").onmessage = () => location.reload())();',
    },
  })
  chokidar.watch(["./app/javascript/**/*.js", "./app/views/**/*.html.erb", "./app/assets/stylesheets/*.css"]).on('all', (event, path) => {
    if (path.includes("javascript")) {
      result.rebuild()
    }
    clients.forEach((res) => res.write('data: update\n\n'))
    clients.length = 0
  });
}
builder()

Again, lots going on here. Let’s step through the important bits.

const chokidar = require('chokidar')

First, we require chokidar, which we need to setup file system watching. Starting easy again.

Next, we setup the build task:

async function builder() {
  let result = await require("esbuild").build({
    // snip unchanged options
    incremental: true,
  })
  chokidar.watch(["./app/javascript/**/*.js", "./app/views/**/*.html.erb", "./app/assets/stylesheets/*.css"]).on('all', (event, path) => {
    if (path.includes("javascript")) {
      result.rebuild()
    }
    clients.forEach((res) => res.write('data: update\n\n'))
    clients.length = 0
  });
}

Here we’ve moved the build setup into an async function that assigns result to build.

We also added the incremental flag to the builder, which makes repeated builds (which we’ll be doing) more efficient.

The watch option was removed since we no longer want esbuild to watch for changes on rebuild on its own.

Next, we setup chokidar to watch files in the javascript, views, and stylesheets directories. When a change is detected, we check the path to see if the file was a javascript file. If it was, we manually trigger a rebuild of our JavaScript.

Finally, we send a request out from our local server, notifying the browser that it should reload the current page.

With these changes in place, stop the server if it is running and then bin/dev again. Open up or refresh http://localhost:3000/home/index, make changes to index.html.erb and application.css and see that those changes trigger page reloads and that updating hello_controller.js still triggers a reload.

Wrapping up

Today we created an esbuild config file that enables live reloading (but not HMR) for our jsbundling-rails powered Rails application. As I mentioned at the beginning of this article, this is very much an experiment and this configuration has not been tested on an application of any meaningful size. You can find the finished code for this example application on Github.

I’m certain that there are better routes out there to the same end result, and I’d love to hear from others on pitfalls to watch out for and ways to improve my approach.

While researching this problem, I leaned heavily on previous examples of esbuild configs. In particular, the examples found at these two links were very helpful in getting live reload to a functional state:

This example esbuild config, from an issue on the jsbundling-rails Github repo
This discussion on the esbuild Github repo

If you, like me, are a Rails developer that needs to learn more about bundling and bundlers, a great starting point is this deep dive into the world of bundlers. If you’re intested in full HMR without any speed loss, and you’re willing to break out of the standard Rails offerings, you might enjoy vite-ruby.

Finally, if you’re using esbuild with Rails and Stimulus, you’ll probably find the esbuild-rails plugin from Chris Oliver useful.

That’s all for today. As always - thanks for reading!

Rendering view components with Turbo Stream broadcasts

2021-11-03T00:00:00+00:00

View components, via Github’s view_component gem, are growing in popularity in the Rails community but until recently, view components and Turbo Stream broadcasts didn’t play well together. This made using both view components and Turbo Streams in the same application clunky and a little frustrating.

While there were ways to get components working with streams, thanks to a recent addition to turbo-rails, rendering view components from Turbo Streams now works seamlessly out of the box.

To demonstrate how to connect these two powerful tools together, we’ll be building a very simple Rails application that allows users to manage a list of Spies.

Each spy in the list of spies will be rendered using a view component, and when new spies are added to the database, the newly created spy record will be rendered and broadcast via a callback in the spy model.

When we’re finished, it’ll work like this:

Beautiful, I know.

This article assumes that you’re comfortable building Rails applications. You won’t need any previous experience with Turbo Streams or view components to follow along.

If you want to skip right to the end the complete code for this article can be found on Github.

Let’s start building!

Application setup

We’ll be working from a fresh Rails application. I’m working off of the latest Rails 7 release (alpha2 at the time of this writing) but everything in this tutorial will work fine on Rails 6.1 too.

If you want to follow along, you can run the below commands from your terminal, or you can clone this Github repo and skip ahead to the Building the spy component section.

rails new turbo-view-components -T
cd turbo-view-components
rails g scaffold Spy name:string mission:string
bundle add view_component
rails db:create db:migrate

If you’re using Rails 6.1 instead of 7, you’ll also need to install turbo-rails manually.

bundle add turbo-rails
rails turbo:install

Whichever route you choose, you’re ready to move on to the next section once you have view_component and turbo-rails installed in your application and a Spy resource created. Once you’re setup, start up your server with rails s and head to http://localhost:3000/spies.

Building the spy component

To use view components, we need to create a component to render a spy object. We can create new components with the built-in generator view_component provides:

rails g component Spy spy

This generator will create both a spy_component.rb file and a spy_component.html.erb file in the components directory.

Since we’re building a very simple component, spy_component.rb is good to go out of the box. For reference, it should look like this:

class SpyComponent < ViewComponent::Base
  def initialize(spy:)
    @spy = spy
  end
end

spy_component.html.erb contains placeholder content provided by the generator right now, so let’s update it to render information about each spy:

<div id="<%= dom_id @spy %>">
  <p>
    <strong>Name:</strong>
    <%= @spy.name %>
  </p>

  <p>
    <strong>Mission:</strong>
    <%= @spy.mission %>
  </p>

  <p>
    <%= link_to "Show this spy", @spy %>
  </p>
</div>

This is just regular old erb, essentially a copy of the default content of _spy.html.erb generated by the Rails scaffold generator we ran during application setup.

With the spy component in place, we now need to actually use it. To do that, we’ll update the spies index view to use the our new component. Update spies/index.html.erb like this:

<p id="notice"><%= notice %></p>

<h1>Spy</h1>

<div id="spies">
  <%= render(SpyComponent.with_collection(@spies)) %>
</div>

<%= link_to "New spy", new_spy_path %>

Here we’re using collection rendering to loop through each spy in @spies and render each with spy_component.html.erb.

We’ve now got our list of spies rendering using view components — next up we’ll add Turbo Stream broadcasts so that newly created spies are appended to the list automatically.

Add Turbo Stream broadcasts

First, we need to ensure that visitors to the spies index page are subscribed to the appropriate turbo stream channel.

To do this, we can use the turbo_stream_from helper from turbo-rails. Update spies/index.html.erb like this:

<!-- Snip -->
<%= turbo_stream_from "spies" %>
<div id="spies">
  <%= render(SpyComponent.with_collection(@spies)) %>
</div>
<!-- Snip -->

There are two important pieces here. First, we added the turbo_stream_from helper, with spies as the name.

This helper creates a <turbo-cable-stream-source> in the rendered HTML with a signed-stream-name that looks something like this:

<turbo-cable-stream-source channel="Turbo::StreamsChannel" signed-stream-name="aLongSecureName"></turbo-cable-stream-source>

Next, the div wrapping the list of spy components has an id of spies. This id must match the target of the Turbo Stream broadcast, which defaults to the plural name of the model we are broadcasting from. If our wrapper div doesn’t have an id, the broadcast we add next will fail.

With the stream subscription added to the view, the last step is to add a model callback in models/spy.rb to broadcast newly created spies on the spies channel.

Update spy.rb like this:

class Spy < ApplicationRecord
  after_create_commit :append_new_record

  private

  def append_new_record
    broadcast_append_to(
      'spies',
      html: ApplicationController.render(
        SpyComponent.new(spy: self)
      )
    )
  end
end

Here we’re using the new html option added to Turbo Stream broadcast methods to render the SpyComponent instead of rendering a partial.

Note that using ApplicationController.render to render a view_component isn’t officially sanctioned by the view_component team.

The view_component folks are actively discussing an official way to add support for stream broadcasts, which you can track on this issue.

With the model broadcast in place, we’re ready to test our Turbo Stream-enabled view components.

A note for Rails 7 users: If you’re on Rails 7 alpha2 (the latest release at the time of this writing) an issue exists that will prevent the broadcast from working because of a disabled session error. This issue is entirely unrelated to view components but it will break our broadcast all the same.

This issue will be fixed in the next Rails release, but until then you can prevent the issue by updating development.rb with this line:

config.action_controller.silence_disabled_session_errors = true

To see it action, refresh the index page, and then open a new tab to http://localhost:3000/spies/new, create a new spy, and see that the newly created spy is automatically appended to the list of spies automatically.

Wrapping up

Today we looked at a technique to render View Components in Turbo Stream broadcasts, leveraging the recently added ability to render html in Turbo Stream broadcasts.

This article’s focus is on Turbo Stream broadcasts + view_component, so we didn’t dive deep into how Turbo Streams work or how to take full advantage of the real power of view_component.

To dig deeper, you might find these resources helpful starting points:

The view_component documentation is excellent, and worth reviewing in detail to understand more about how you can use view components
Encapsulating Ruby on Rails views from Github’s blog is a nice introduction to the work behind the ViewComponent library and includes links to see how Github uses ViewComponents in their application
The Turbo Rails source code. If you’re serious about using Turbo in your Rails application, thoroughly reviewing the source and accompanying comments is highly recommend.
The Turbo handbook is the best place to start if you’re new to Turbo and want to understand the basics of Streams, Frames, and Drive

That’s all for today. As always, thanks for reading!

Hotwiring Rails Newsletter - November 2021

2021-11-01T00:00:00+00:00

Thoughts or feedback on how I can make this newsletter more valuable? Have something you’d like to include in next month’s edition? Send me an email, or find me on Twitter.

Articles and guides
New and interesting Rails-land PRs
Recent pieces from my blog
The best thing I read recently

Articles and guides

Frontend bundler braindump by Konnor Rogers

This deep dive into the world of frontend bundling is a dense, delightful trip down into the weeds. With Rails 7 dropping Webpacker as the default for new apps and the introduction of importmap-rails, cssbundling-rails, and jsbundling-rails, understanding more about bundling will help Rails devs make sense of all of the new options.

Read it on dev.to

Using Hotwire with Rails for a SPA like experience by Mike Wilson

A neat implementation of list item navigation without a full page turn. Built with Turbo and Stimulus, making great use of a few Turbo events to animate loading states and highlight the active list item. A great read for folks new to the full Hotwire stack

Read it on mikewilson.dev

Hotwire: Server-rendered live previews by Sean Doyle

Sean’s work is always thoughtful and thorough, and this article is no different. In it, you’ll build a live preview component with Rails, Stimulus, and Turbo that previews the user’s content as they type. As a nice bonus, the implementation gracefully degrades for users without JavaScript enabled.

Read it on thoughtbot.com

Hotwire with StimulusReflex by Julian Rubisch

Hotwire isn’t the only option out there for building modern Rails applications — CableReady and StimulusReflex are excellent tools to have in your bag. In this slide deck, Julian (a StimulusReflex core team member) talks through how he thinks about combining the Hotwire stack with StimulusReflex. Essential reading if you’re trying to figure out how all of the pieces fit together in the real world.

View the slides on speakerdeck.com

Hotwire: best practices for Stimulus by Pete Hawkins

This article shows off a few simple, reusable Stimulus controllers that can be dropped in to any Stimulus-enabled application. Autosubmitting, empty states, toggling and more. Simple, generic components that make a great introduction to some useful Stimulus patterns and libraries like stimulus-use for folks getting started with Stimulus.

Read it on dev.to

Hotwire Turbo: Increment likes count without any page refreshes by Yaroslav Shmarov

A short and sweet piece from Yaroslav demonstrating the implementation of a Turbo Stream powered like button. This is a simple introduction for Turbo Stream beginners who are already comfortable with Rails and want to explore what Turbo can add to toolkit.

Read it on blog.corsego.com

New and interesting PRs

Rails 7

Stop failing CSRF token generation when session is disabled — This PR, merged into Rails main last month, fixes a bug introduced in Rails 7 alpha 1 that breaks rendering forms and buttons outside of the normal controller context. Until this bug fix, rendering something like a delete button from a Turbo Stream or CableReady broadcast was broken in Rails 7, as first described here.
ActionCable client ensures subscription command is confirmed — This PR, merged into Rails main last week, should eliminate nagging ActionCable connection failures that were all to common, especially when using Turbo Stream broadcasts.
Add support for generated columns in PostgreSQL — This PR, merged into Rails main in September, adds support for generated columns in Postgres to ActiveRecord, eliminating the need to drop down to writing raw SQL when you need a generated column. Especially useful for things like pg_search migrations.

CableReady

Receive morph updates from model/PORO callbacks — This PR, merged into CableReady’s 5.0 prerelease, adds the ability to broadcast updates from a model to many subscribers, customizing the content for each user. Perfect for SaaS apps with complex permission levels that need to reference current_user in broadcasted content, inspired by Bullet Train’s Sprinkles library.

cssbundling-rails

Add Tailwind styled scaffold templates — This PR, merged into cssbundling-rails in September, does what it says on the tin. Adding some basic, Tailwind-powered scaffolds (with designs sanctioned by the Tailwind team) makes it even easier to get nice looking prototypes together with Rails and Tailwind

Recent pieces from my blog

Turbo Frames on Rails — Turbo’s documentation, especially for the turbo-rails gem, is… lacking. While the official docs may eventually catch up, I spent time compiling what I’ve learned over the last year of building with Turbo to write up some simple how-tos, tricks, and gotchas as a companion to my Turbo Streams on Rails article from August.
Sorting, searching, and filtering with Hotwire — This is a two part series for folks newer to Turbo that walks through building an (almost) instant search, filter, and sort experience with Turbo Frames and a little bit of Stimulus.
- Part 1: Sort tables (almost) instantly with Ruby on Rails and Turbo Frames
- Part 2: Filter, search, and sort tables with Rails and Turbo Frames

The best thing I read recently

The Single-Page-App Morality Play by Baldur Bjarnason

Baldur uses the never ending debate about SPAs as a jumping off point for a broader discussion of the pitfalls that almost every team of web developers deals with as they try (and usually fail) to deliver great software. There’s no code here, and no easy answers, but if you’re reading this newsletter you’ll probably enjoy reading and thinking about Baldur’s piece.

Until we accept that most of the industry is poorly managed and try to figure out how to solve that problem, sites and web apps won’t get better, just shinier and with more bounce.

Read the entire article at baldurbjarnason.com

That’s all for this month, thanks for reading and for your support! If you enjoyed this newsletter, I would appreciate you sharing it with your friends.

Filter, search, and sort tables with Rails and Turbo Frames

2021-10-15T00:00:00+00:00

Today we are going to build a table that can be sorted, searched, and filtered all at once, using Ruby on Rails, Turbo Frames, a tiny Stimulus controller, and a little bit of Tailwind for styling.

We will start with a sortable, Turbo Frame-powered table that displays a list of Players from a database. We built this sortable table in a previous article — you might find it helpful to start with that article, especially if you are new to Turbo.

When we are finished, users will be able to search for players by name, filter them by their team, and sort the table. Sorting, searching, and filtering all work together, in any combination, and they each occur without a full page turn. The end result will work like this:

This article is intended for folks who are comfortable with Ruby and Rails code. You won’t need any prior experience with Turbo Frames or Stimulus.

Let’s dive in!

Project setup

If you want to follow along with this article and you haven’t already completed the sortable table article locally, you’ll want to begin by cloning this Github repo. If you have completed the sortable table article, this one picks up exactly where that one ends, so go ahead and work from where that article finished.

To set up the application after cloning from Github, from your terminal:

bundle install
yarn install
rails db:setup

Once the application is ready, checkout the sortable branch, where this article picks up, and then run bin/dev to compile assets and start your development server.

After you start the server, head to http://localhost:3000 and see that you have a seeded database of players and that you can sort the table by clicking on each column header.

If you’re curious how the the sorting works, the sortable tables article goes through the frame-powered sorting mechanism in detail.

With setup complete, let’s start building!

We’ll start with a simple search form, added inside the players turbo frame in app/views/players/_players.html.erb.

<%= turbo_frame_tag "players", class: "shadow overflow-hidden rounded border-b border-gray-200" do %>
  <div class="flex justify-end mb-1">
    <%= form_with url: list_players_path, method: :get do |form| %>
      <%= form.text_field :name, placeholder: "Search by name", value: params[:name], class: "border border-blue-500 rounded p-2" %>
      <%= form.button "Search", class: "bg-blue-500 text-white p-2 rounded-sm" %>
    <% end %>
  </div>
  <!-- Snip the table -->
<% end %>

This is a standard-issue Rails search form. When the form is submitted, a GET request is dispatched and PlayersController#list responds to the request. For now, the form requires the user to click the Search button to submit the search request.

Next, we’ll update the list method in app/controllers/players_controller.rb to filter the list of players when the search form is submitted:

def list
  players = Player.includes(:team)
  players = players.where('name ilike ?', "%#{params[:name]}%") if params[:name].present?
  players = players.order("#{params[:column]} #{params[:direction]}")
  render(partial: 'players', locals: { players: players })
end

Here we’ve got a clunky, functional implementation of searching by name — when the name parameter from the form’s text_field is present, we run a case insensitive query for players in the database with a name that matches the search query.

With these changes in place, refresh the page, type a query into the search form and submit it. You should see the players list update with the results of your query.

You’ll notice right away that searching clears any previously applied sorting on the table, and sorting the table clears out any search. So we’ve got a search form, but we have to click to submit the form and doing so clears out the user’s sorting preference.

We’ll fix both of those issues, starting with remove the submit button and searching as the user types instead.

Real-time searching

We’ll use a small Stimulus controller to submit the search form as the user types. First, generate the Stimulus controller with the generator built in to stimulus-rails. From your terminal:

rails g stimulus search_form

Then, fill that controller in with:

// app/javascript/controllers/search_form_controller.js
import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  static targets = [ "form" ]

  search() {
    clearTimeout(this.timeout)
    this.timeout = setTimeout(() => {
      this.formTarget.requestSubmit()
    }, 200)
  }
}

This controller’s search function calls requestSubmit on a form target. Since we’ll call this search function as the user is typing in a text input, we’ve add clearTimeout and setTimeout to ensure that the form isn’t submitted every time the user types a new character.

Note that requestSubmit needs a polyfill for support on Safari and IE11. As an alternative to requestSubmit, if you are using Rails/ujs in your application, Rails.fire(this.formTarget, 'submit') works without a polyfill.

With the Stimulus controller ready to go, next we’ll connect that controller the search form:

<div class="flex justify-end mb-1">
  <%= form_with url: list_players_path, method: :get, data: { controller: "search-form", search_form_target: "form", turbo_frame: "players" } do |form| %>
    <%= form.text_field :name,
      placeholder: "Search by name",
      class: "border border-blue-500 rounded p-2",
      autocomplete: "off",
      data: { action: "input->search-form#search" }
    %>
  <% end %>
</div>
<%= turbo_frame_tag "players", class: "shadow overflow-hidden rounded border-b border-gray-200" do %>
<!-- Snip table -->
<% end %>

Here we first moved the search form outside of the players turbo-frame. This is necessary because if the search form is inside of the turbo frame, the search input will be reset every time the players frame is rerendered (meaning every time our search form is submitted), like this:

Because the form is now outside of the frame, we add a data-turbo-frame to target the players frame. This tells Turbo to use the response from the form submission to replace the content of the players frame.

We also added data-controller="search-form" and data-search_form_target="form" to the form element. These Stimulus attributes connect the search-form controller to the DOM and set the form target.

Finally, we add data-action=“input->search-form#search” to the name field, which tells Stimulus to call the search function each time the input event is fired on the name field.

We moved fairly quickly through some core Stimulus concepts here. The Stimulus handbook is a great reference point if you need to spend more time with any of these concepts.

With these changes in place, we can refresh the page and see that search results update as the user types. We still cannot sort and search at the same time though, so let’s tackle that next.

Sorting and searching at the same time

The reason we can’t search and sort at the same time is because the list method relies on URL parameters to apply search and filter options. Because the search form doesn’t include the sort parameters and sorting doesn’t include the search parameter, list has no way of retaining sort and search options across requests — every new request to list starts from scratch.

There are a variety of ways to address this issue. The most direct is to move from using params to storing filter options in the session.

Our basic approach will be to use params to update a filters hash in the session object. Since session is maintained across requests, as long as we update the session filters hash with the search and sort params on each request, we can persist search and sort options across requests.

A very ugly implementation of this concept, done directly in the list method looks like this:

def list
  session['filters'] = {} if session['filters'].blank?

  session['filters'].merge!(filter_params)
  players = Player.includes(:team)
  players = players.where('players.name ilike ?', "%#{session['filters']['name']}%") if session['filters']['name'].present?
  players = players.order("#{session['filters']['column']} #{session['filters']['direction']}")
  
  render(partial: 'players', locals: { players: players })
end

private

def filter_params
  params.permit(:name, :column, :direction)
end

Here we ensure that session['filters'] is a hash, update its value by merging in whitelisted filter_params and then use the the filters hash to search for players by name and order the list of players, as appropriate.

This ugly code is fully functional — if you update the list method and add the filter_params method to your controller you will be able to search and sort at the same time; however, this code is clunky, difficult to follow, and quickly becomes unmaintainable as your application grows.

So, if this code is ugly and unmaintainable, why are we looking at it?

Because we are going to refactor it into something neater and more scalable. Before we do that it is helpful to see what the most direct implementation can be so we can understand what is happening at a basic level.

When we’re done, we’ll still store params in a hash in the session object, and we’ll still use the hash values to query the database based on the user’s preferences. Our code will be nicer, but it’ll still be the same basic concept.

Let’s refactor this code next.

Building a Filterable concern

To make our filtering code more scalable and less error prone, we are going to start with a generalized Filterable concern. We’ll include Filterable in the PlayersController and use it to filter players.

Filterable won’t know anything about the specifics of querying Players, instead it will just implement logic to store query parameters in the session. Once the values are stored, they’ll be used to apply filters.

First, create the filterable concern. From your terminal:

touch app/controllers/concerns/filterable.rb

Then, fill in filterable.rb with the following:

module Filterable
  def filter!(resource)
    store_filters(resource)
    apply_filters(resource)
  end

  private

  def store_filters(resource)
    session["#{resource.to_s.underscore}_filters"] = {} unless session.key?("#{resource.to_s.underscore}_filters")

    session["#{resource.to_s.underscore}_filters"].merge!(filter_params_for(resource))
  end

  def filter_params_for(resource)
    params.permit(resource::FILTER_PARAMS)
  end

  def apply_filters(resource)
    resource.filter(session["#{resource.to_s.underscore}_filters"])
  end
end

There’s a lot of code here, let’s break it down.

The filter! method is what we’ll call from the controller to apply filters in response to a request from a user. It takes a resource argument. resource will be an ActiveRecord class, like Player. filter! simply calls out to two internal methods, store_filters and apply_filters, which do the heavy lifting.

store_filters ensures that session['class_name_filters'] exists, and then writes whitelisted parameters into the session key, replicating the first two lines of our ugly implementation directly in the list method in the last section.

Once the filters are stored, apply_filters calls a filter class method from the class we’re interested in which should return a list of ActiveRecord objects.

You’ll notice here that Filterable isn’t doing much on its own. Instead, it is relying on methods to exist on the class passed in to filter!. This is by design — in a real application, we would likely need to build filtering mechanisms for many different ActiveRecord classes, and each will need to be able to filter by a unique set of columns. Trying to build all of that logic into the Filterable module would quickly become even harder to maintain than tossing everything in the controller.

Rather than building all of that complexity in to Filterable, we instead just rely on the target class to define FILTER_PARAMS and a filter method in whatever way works for that particular class.

Let’s see this in action by updating app/models/player.rb to work with our new Filterable concern.

class Player < ApplicationRecord
  belongs_to :team

  FILTER_PARAMS = %i[name column direction].freeze

  scope :by_name, ->(query) { where('players.name ilike ?', "%#{query}%") }

  def self.filter(filters)
    Player.includes(:team)
          .by_name(filters['name'])
          .order("#{filters['column']} #{filters['direction']}")
  end
end

Here we’ve defined the FILTER_PARAMS constant with the three filtering we support on the players table.

Next, we added the by_name scope to handle searching the players table by name.

Finally, we implement filter, which replaces the queries that we previously built in PlayersController#list and makes use of the new by_name scope to be a bit more readable.

With Player set up for filtering, we can update PlayersController to include Filterable and replace the filtering logic in list with the new filter! method.

class PlayersController < ApplicationController
  include Filterable
  # snip
  def list
    players = filter!(Player)
    render(partial: 'players', locals: { players: players })
  end
end

Here we simply include Filterable in the controller and then set the value of players using the filter! method provided by Filterable.

Much nicer, right?

Before moving on, you’ll also notice that Filterable is in the controller/concerns directory, but it isn’t truly a Concern. In our case, controller concerns is the simplest place for this module to live, but we don’t need the full functionality of a true concern. You could just as easily place this module in another place if you prefer.

Our last step is to update the views to read values from the session instead of params so that we always display the correct set of applied filters to users.

First, update the table header in _players like this:

<tr>
  <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
    <%= sort_indicator if session.dig('player_filters', 'column') == "name" %>
    <%= build_order_link(column: "name", label: "Name") %>
  </th>
  <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
    <%= sort_indicator if session.dig('player_filters', 'column') == "teams.name" %>
    <%= build_order_link(column: "teams.name", label: "Team") %>
  </th>
  <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
    <%= sort_indicator if session.dig('player_filters', 'column') == "seasons" %>
    <%= build_order_link(column: "seasons", label: "Seasons") %>
  </th>
</tr>

We’ve replaced references to params with session.dig calls with this change. When no filters have been applied, session['player_filters'] won’t exist, so we use dig to avoid nil errors in those cases.

Next, update app/heleprs/players_helper.rb like this:

module PlayersHelper
  def build_order_link(column:, label:)
    if column == session.dig('player_filters', 'column')
      link_to(label, list_players_path(column: column, direction: next_direction))
    else
      link_to(label, list_players_path(column: column, direction: 'asc'))
    end
  end

  def next_direction
    session['player_filters']['direction'] == 'asc' ? 'desc' : 'asc'
  end

  def sort_indicator
    tag.span(class: "sort sort-#{session['player_filters']['direction']}")
  end
end

Again we’re just replacing params with the equivalent session values so that sorting works correctly all the time and visual indicators are shown consistently.

With those changes in place, refresh the page and see that you can search and sort the table at the same time, and the UI always shows the proper sort indicator.

Nice work making it this far! We spent a good amount of time building a more scalable filtering solution, so let’s wrap up this article by putting that scalable solution to use by adding a filtering option to the table.

Add filtering by team

Right now users can search by player name and sort the table, but they can’t filter by team or season. Let’s add filtering by team to the filter options.

We’re going to use a select input for the Team filter, since team is a belongs_to relationship on the Player class — we’ll have a dropdown menu that displays all available teams by name, each dropdown option will have team_id as the value sent back to the server.

Since we’re using a select input, let’s make it look nice by taking a slight detour to install Tailwind’s forms plugin:

From your terminal:

yarn add @tailwindcss/forms

And then update tailwind.config.js to include the plugin:

plugins: [
  require('@tailwindcss/forms'),
]

Incredible stuff.

Back to adding the team filter. We’ll start by adding the team filter to the UI. In the players partial:

<%= form_with url: list_players_path, method: :get, data: { controller: "search-form", search_form_target: "form", turbo_frame: "players" } do |form| %>
  <%= form.select :team_id,
    options_for_select(
      Team.all.pluck(:name, :id),
      session.dig('player_filters', 'team_id')
    ),
    { include_blank: 'All Teams' },
    class: "border-blue-500 rounded",
    data: { 
      action: "change->search-form#search" 
    } 
  %>
<!-- Snip the search input -->
<% end %>

This is a regular Rails select helper. In it, we build a list of all teams in the database, set the selected value when one is present (session.dig, again) and fire the search function of the search-form controller each time the select input changes.

Refresh the page, change the team input and see that it doesn’t work yet. We haven’t updated Player to support filtering by team yet.

Head to app/models/player.rb and update it like this:

FILTER_PARAMS = %i[name team_id column direction].freeze

scope :by_team, ->(team_id) { where(team_id: team_id) if team_id.present? }

def self.filter(filters)
  Player.includes(:team)
        .by_name(filters['name'])
        .by_team(filters['team_id'])
        .order("#{filters['column']} #{filters['direction']}")
end

Now we can start to see the benefit of the work we did in the last section. We can add filtering by a new option with just a few simple changes to the model.

We updated the model to add team_id to the list of valid FILTER_PARAMS, added the by_team scope, and then added by_team to the filters method.

No need to change our controller or touch the Filterable module — updating the model is all we need.

Refresh the page, apply a team filter and see that filtering by team works along with searching by name and sorting. Great work!

Filtering the index action

We’ll wrap up this exercise by make a few more small adjustments to avoid weirdness when a user visits the /players with filtering values already saved in their session.

First, now that we have access to the filter! method, we can update the index action to use that method instead of always setting the value of players to all players in the database.

To do that, update index in app/controllers/players_controller.rb like this:

def index
  @players = filter!(Player)
end

With that change in place, we’ll now filter the list of players properly when a user reloads players page after applying filters in a previous request, making the experience on the index page a bit more consistent.

Finally, since we’re using the session values to restore previously applied filters, we need to update the name search field to set its value from the session. Without this change, when the user applies a name search and then refreshes the page, the list of players will be filtered by name, but the search term won’t be visible in the name field.

To fix this issue, update the name field in the players partial like this:

<%= form.text_field :name,
  placeholder: "Search by name",
  value: session.dig('player_filters', 'name'),
  class: "border border-blue-500 rounded p-2",
  autocomplete: "off",
  data: { action: "input->search-form#search" }
%>

With value added to the name field, we’ll always show the correct value to users, even on the initial page load.

With these small changes in places, we’ve now got consistent filtering experience that persists cleanly across requests and can be extended with new options as our user experience requirements change.

Nice work making it this far!

You’ve reached the end of the tutorial portion of the article, we’ll finish up by discussing a few ways we could improve this implementation in a production application.

Production-grade considerations

While what we built today works fine, there are some things to consider if we were building a real, consumer-facing application.

Our code works and is pretty easy to maintain, but before we go, let’s touch on a few points to think about for production-grade applications. These are intended simply as things to think about as you build, and we won’t be going through any code here:

Session storage limitations

We are relying on the session object to store filter options. This is fine for small applications, but as you grow, you may run into the limits of storing options like this in the session.

By default, Rails stores the session in a cookie which can only be ~4kb before it will start raising errors. You can use a different session store but you may want to consider a more flexible solution for storing filter options.

One option here is to use Kredis, a Redis-based solution that provides a nice interface for solving problems like our session persistance problem today.

Replace helper methods

The helper methods we are using to render sorting links and indicators could be implemented as ViewComponents.

This pattern allows us to write more testable and reusable view code and excels in cases like our example application. As this application grows, we should expect to have multiple tables across our application that all need sort links.

Rather than implementing them as helpers that are very specific to the players table, we could create them as generic components that can be thoroughly tested and then reused throughout the code base.

Expand Filterable implementation

Right now, Filterable relies on each model to define FILTER_PARAMS and a filter method from scratch.

In the real world, there would likely be enough overlap between the different implementations of filter in each model that we would benefit from moving filter out of each model and into a Filter class that defines some shared logic, like applying order, which is likely to be the same for every class.

For a really detailed implementation of a Filterable pattern, take a look at the filterable_reflex from StimulusReflex Patterns.

Wrapping up

Today we built on a Turbo Frame foundation to expand a sortable table view to a table that can be searched, filtered, and sorted without full page turns or lots of custom JavaScript.

Because frames are so powerful, we were able to easily hook into our existing frame code and add in searching and filtering without thinking too much about the front end implementation. It mostly just worked, once we built the filtering logic on the server.

This is the power of Turbo Frames — we can build fast, efficient user interfaces without stepping much outside of standard Rails code. The client side code stays light and maintainable, while our server looks and feels familiar to any level of Rails developer.

To dig deeper into Turbo and building modern Ruby on Rails applications with the Hotwire stack:

Read my articles on Turbo Frames and Turbo Streams on Rails
Dive into the Turbo and turbo-rails source and follow along with the Github activity for both
Join the hotwire discussion forums

That’s all for today. As always, thanks for reading!

Turbo Frames on Rails

2021-10-08T00:00:00+00:00

Turbo, part of the Hotwire tripartite, gives you the tools to write dramatically less custom JavaScript than you would otherwise need to build modern, performant web applications.

Turbo is composed of Turbo Drive, Turbo Frames, Turbo Streams, and Turbo Native. Each is a valuable piece of the puzzle but today we’re going to focus on Turbo Frames.

Turbo Frames “allow predefined parts of a page to be updated on request.” Used wisely, frames allow developers to decompose their UI into independently updated pieces, quickly.

Why should I use Turbo Frames?

Turbo Frames unlock a huge amount of potential with minimal changes to existing code and they can be gradually introduced to existing projects without necessitating major architectural changes.

In greenfield projects designed with Turbo Frames in mind, a small team of developers can use frames to deliver fast, efficient user interfaces in dramatically less time than would be required when build a SPA-powered front end.

While Turbo Frames can be used with many different tech stacks, Rails developers will find the tight integration of frames into Rails (via the turbo-rails gem) makes using frames in Rails a breeze.

The Turbo documentation gap

As with many new technologies, one of the barriers to getting started with Turbo Frames is inconsistent/incomplete documentation. Along with problems in the official documentation, much of the tutorial content written about Turbo Frames is already out of date.

Turbo has evolved quickly since its release in December of 2020, and the documentation and supporting tutorials from content creators have struggled to keep pace.

As an example of the documentation gap, turbo-rails doesn’t have any dedicated usage documentation, instead just linking to the base Turbo docs and letting users figure out the rest.

This means that learning to use Turbo Frames in your Rails application today often requires reading the docs, then digging through source code, and then Googling your way through Github issues and forum posts. Turbo Frames can be difficult to approach.

In time the documentation will improve and the community will coalesce around best practices and standards that can be more easily communicated to new users.

In the meantime, here we are.

What is this article? Who is it for?

Today, I’m going to share what I’ve learned over nearly a year building applications with Turbo Frames, from small hobby projects and experiments to multiple large, legacy production applications.

By doing so, I’m hoping to address the gaps in documentation, to save you from spending too much time reading source code, trawling forum posts and Github issues.

I will cover content you can find in the documentation (if you know where to look), gotchas and tricks learned from real world experience and picked up from reading way too much about Turbo and Turbo Frames since Turbo’s initial release. I’ll also share patterns that illuminate some of the common web application needs that frames are perfect for.

This content is geared towards Rails developers interested in using Turbo. The code samples and concepts will be illustrated using Ruby and Rails code and conventions. If you aren’t working in Rails, you should still find plenty of value here, but know that you may not have access to convenience methods like <%= turbo_frame_tag %> in your chosen language and framework.

Let’s dive in!

What are Turbo Frames?

At a high level, Turbo Frames are pieces of a webpage that can be updated independently, without impacting the rest of the content on the page.

Links and forms within a frame will, by default, attempt to update only the content of the containing frame, whether the server sends a completely new HTML document or only a page fragment.

Turbo Frames allow developers to decompose a page into pieces of content that can be updated individually as new information is received from a server.

In practice, common use cases for Turbo Frames include:

Tabbed content
In-line editing
Searching, sorting, and filtering data

Let’s start looking at code.

Constructing a frame

A basic Turbo Frame, rendered with the built-in helper from Turbo Rails (and using erb), looks like this:

<%= turbo_frame_tag "some_id" do %>
  <div>
    Some framed content
  </div>
<% end %>

The only required argument for turbo_frame_tag is an id.

When the helper is processed, the final HTML is:

<turbo-frame id="some_id">
  <div>
    Some framed content
  </div>
</turbo-frame>

Instead of providing a static id, we can also pass in an active record object, which the helper will use to generate a unique id via dom_id(object.id).

<%= turbo_frame_tag Comment.first %>

Frames can also receive an src attribute. When src is supplied, the frame will be populated after the initial page load via a separate HTTP request to the frame’s src:

<%= turbo_frame_tag "comments", src: comments_path do %>
  <div>
    Placeholder content
  </div>
<% end %>

Here, the page will initially load the turbo frame with the placeholder div and then immediately a request to comments_path endpoint is made and the content of the Turbo Frame is replaced with the response from the server, provided our server returns HTML with a matching Turbo Frame tag.

What’s a matching Turbo Frame tag? Let’s look at that next.

Responding with frame content

The power of Turbo Frames is in replacing pieces of the page with content sent from the server — without touching the rest of the DOM.

Updating an existing frame just requires responding to a request with HTML that contains a Turbo Frame element with an id that matches the id of the frame target sent in the request.

Taking the comments example from above, when a request is made to /comments, the server should respond with HTML that contains a frame like this:

<turbo-frame id="comments">
  <!-- A list of comments, perhaps -->
</turbo-frame>

The content inside of the matched frame is replaced with the updated content from the server, without touching the rest of the page.

Navigating within frames

By default, links and forms within a Turbo Frame will perform the navigation within the frame, rather than performing a full page turn.

For a simple example, we can imagine that we have a list of comments on a page and that each comment is wrapped in a Turbo Frame tag, like this:

<% @comments do |comment| %>
  <%= turbo_frame_tag comment do %>
    <div>
      <%= link_to comment.body, edit_comment_path(comment) %>
    </div>
  <% end %>
<% end %>

This ERB results in rendered HTML:

<turbo-frame id="comment_1">
  <div>
    <a href="/comments/1/edit">Comment 1</a>
  </div>
</turbo-frame>
<turbo-frame id="comment_2">
  <div>
    <a href="/comments/2/edit">Comment 2</a>
  </div>
</turbo-frame>
<turbo-frame id="comment_3">
  <div>
    <a href="/comments/3/edit">Comment 3</a>
  </div>
</turbo-frame>

With this Turbo-powered HTML in place, clicks on the edit links will make a Turbo Frame request to /comments/:id/edit, retrieve some HTML, and return that HTML to the browser.

The Turbo Frame request is a normal HTML request with an additional Turbo-Frame header included in the request, with a value that matches the id of the target Turbo Frame.

In turbo-rails, the presence of this header can be used to identify a Turbo Frame request and respond with appropriate content, like this:

# This approach gets messy in controller actions. Read further on for a variant based approach that scales much more cleanly
if turbo_frame_request?
  render partial: "some_turbo_frame_partial"
else
  render partial: "some_other_partial"
end

When turbo-rails responds to a Turbo Frame request, it automatically removes the layout from the HTML response. Since Turbo will discard all of the HTML but the requested frame, Rails skips rendering content that it knows won’t be used.

When a response to a Turbo Frame request is received, Turbo Drive replaces the content of the target frame, leaving the rest of the page untouched.

Continuing with the above comments example, the edit view may look something like this:

<%= turbo_frame_tag @comment do %>
  <%= form_with(model: @comment) do |form| %>
    <div class="inline-field">
      <%= form.label :body %>
      <%= form.text_field :body %>
    </div>
    <div class="actions">
      <%= form.submit %>
    </div>
  <% end %>
<% end %>

After clicking to edit a comment then, the updated HTML after Turbo processes the response would look something like this:

<!-- Form content rendered from /edit -->
<turbo-frame id="comments_1" src="http://localhost:3000/comments/1/edit">
  <form action="/comments/1" accept-charset="UTF-8" method="post">
    <input type="hidden" name="_method" value="patch">
    <input type="hidden" name="authenticity_token" value="some_token">
    <div class="inline-field">
      <label for="comment_body">Body</label>
      <input type="text" value="Comment 1" name="comment[body]" id="comment_body">
    </div>
    <div class="actions">
      <input type="submit" name="commit" value="Update Comment">
    </div>
  </form>
</turbo-frame>
<!-- The other frames, untouched by the /edit request -->
<turbo-frame id="comment_2">
  <div>
    <a href="/comments/2/edit">Comment 2</a>
  </div>
</turbo-frame>
<turbo-frame id="comment_3">
  <div>
    <a href="/comments/3/edit">Comment 3</a>
  </div>
</turbo-frame>

Finishing up this example, what happens when the user submits the comment form?

After submit in a normal Rails action, we might redirect the user to the comment show page. When the request occurs within a Turbo Frame, a “redirect” still occurs, but rather than redirecting the page in the browser, the redirect is followed, the new HTML content is sent from the server, and that HTML is used to update the frame.

In our example, the comment#update controller action might look like this:

def update
  respond_to do |format|
    if @comment.update(comment_params)
      format.html { redirect_to @comment }
    else
      format.html { render :edit, status: :unprocessable_entity }
    end
  end
end

And the show view:

<%= turbo_frame_tag @comment do %>
  <div>
    <%= link_to @comment.body, edit_comment_path(@comment) %>
  </div>
<% end %>

This behavior allows us to build Turbo Frame-powered interfaces while making very few or no changes to a standard Rails controller, making it possible to prototype in Rails very quickly.

With just what we’ve learned so far, we’re already in position to start building simple interfaces with Turbo Frames that allow us to add significant amounts of interactivity to our application with very few changes to the Ruby and HTML we’d have in any Rails app.

Let’s keep exploring, there’s plenty more to see.

Breaking out of a frame

While navigating within a frame is very handy, sometimes, a form or link within a frame needs to break out of the frame and perform a normal page turn.

To add that behavior to a navigation element within a frame, simply add data-turbo-frame="_top" to the element, like this:

<%= link_to comment.body, edit_comment_path(comment), data: { turbo_frame: "_top" } %>

Targeting a frame from the outside

Eventually, you’ll find yourself needing to update the content of a frame using a link that isn’t wrapped in your target frame. This often comes up when building tabbed content; the navigation menu will be placed outside of the Turbo Frame that contains the actual content.

To handle situations like this, we can use the same turbo-frame data attribute to tell Turbo that the navigation should occur in a specific turbo frame.

Using our tabbed content example, we may have a page layout like this:

<ul>
  <li>
    <a href="user/1/profile" data-turbo-frame="main">
      Profile
    </a>
  </li>
  <li>
    <a href="user/1/favorites" data-turbo-frame="main">
      Favorites
    </a>
  </li>
</ul>
<turbo-frame id="main"></turbo-frame>

Clicks on either of the user links will generate a request with a Turbo-Frame: main header and the response will update the content of the main turbo frame, even though the links are not wrapped within the frame.

Lazy loading Frames

The src attribute on a frame can be combined with loading=lazy to mark a frame as lazy-loaded. Lazy-loaded frames will not fetch their content from the server until they’re visible on the page.

Lazy loading is especially helpful for modals, popovers, and other non-critical, below-the-fold content.

Lazy loaded frames can be combined with a spinner or other loading indicator, like this:

<%= turbo_frame_tag "lazy_frame", src: comments_path, loading: "lazy" do %>
  <div>I'm a loading spinner</div>
<% end %>

Note that loading="lazy" must be combined with a src attribute, otherwise lazy loading does nothing

Frame events

Turbo comes packed with lifecycle events, including two frame specific events:

turbo:frame-render
turbo:frame-load

These events (added in Turbo 7.0.0-rc2) fire each time a frame is loaded or reloaded. Developers can use these events to attach behavior to an element within a frame or animate the entry of a frame element, and they pair nicely with Stimulus.

In addition to these built in events, a useful complement to the emitted events is the busy attribute. The busy attribute is automatically applied to a frame when a request for the frame’s content begins and is removed when the request finishes.

// From Turbo's source code: turbo/src/util.ts
export function markAsBusy(...elements: Element[]) {
  for (const element of elements) {
    if (element.localName == "turbo-frame") {
      element.setAttribute("busy", "")
    }
    element.setAttribute("aria-busy", "true")
  }
}

This attribute can be useful for adding loading indicators to long running requests and otherwise tracking the state of frames between the beginning of a frame request and the turbo:frame-render and turbo:frame-load events firing.

Refreshing Frames

Sometimes, users need the ability to refresh the contents of a frame without refreshing the entire page.

In ideal circumstances, a Turbo Stream broadcast automatically updates the contents of the frame without needing a manual refresh, we don’t always get to work in ideal circumstances.

In earlier versions of Turbo, refreshing a frame required workarounds like appending a timestamp to the src attribute. Today, those workarounds are no longer necessary and Turbo will happily update a frame any number of times with an indentical src on each request.

We can build a refreshable frame like this:

<%# app/views/games/show.html.erb %>
<%= turbo_frame_tag @game do %>
  <div><%= @game.home_score %></div>
  <div><%= @game.away_score %></div>
  <%= link_to "Update score", game_path(@game) %>
<% end %>

Here, the first time a user clicks the update link, the turbo-frame will be updated with an src like this:

<turbo-frame id="game_10" reloadable="" src="http://localhost:3000/games/10">

Additional clicks on the update link will fetch updated scores from the server and reload the content of the frame.

Note that a currently open PR (as of the time this article was last updated, December 2021) proposes an alternative implementation of reloadable frames that will prevent the reloadable attribute from being exposed while keeping the behavior intact. In future versions of Turbo the reloadable attribute may not longer exist or be neccessary.

Conditional template rendering using variants

As you use Turbo Frames in Rails, you’ll eventually run into the desire to have a single controller action respond to both Turbo Frame requests and regular, full-page requests.

A common example of this is rendering a /new page as both a standalone page and as frame content inside of a modal. You could implement a conditional in your controller action to check for the Turbo Frame header, but that quickly starts to clutter up your controllers.

An alternative approach is to use variants to render different content based on the inbound request headers, to keep your controllers cleaner.

class ApplicationController < ActionController::Base
  before_action :turbo_frame_request_variant
  private

  def turbo_frame_request_variant
    request.variant = :turbo_frame if turbo_frame_request?
  end
end

This approach adds a before_action to the ApplicationController that checks if the inbound request is a Turbo Frame request using the turbo_frame_request? method provided by turbo-rails. When the request is a turbo frame, Rails will look for a Turbo Frame variant (new.html+turbo_frame.erb, for example) and render that variant when it exists. Otherwise, the default html.erb view will be used

This approach was originally described here. While there was a proposal to add this variant-based approach to turbo-rails, those proposals were rejected. The guidance from the maintainers is that the turbo_frame_request? to detect Turbo Frame requests is the only support that will be added because “If you’re branching all your responses for frames vs not, something isn’t right, and we should investigate those pressures in a different way.”.

In my experience, judicious use of variants to serve different content for Frame requests is sometimes the simplest and cleanest path to achieve a desired outcome but YMMV.

Limitations & Gotchas

Turbo Frames and page layouts

Turbo Frames in the DOM are implemented as custom HTML elements (the <turbo-frame> tag we’ve seen in this article). We can add classes to them (<%= turbo_frame_tag post, class: "classes" %>) and style them; however, because they exist in the DOM and need to wrap their content to be useful, inserting them into existing layouts can cause issues without some planning.

In particular, you can’t place a frame inside of a table row. While there are workarounds for this, it is worth knowing that tables and Turbo Frames don’t play well together out of the box.

In addition to table issues, you may sometimes run into layout issues with flex box and grid layouts when frames are the direct descendant of the container element. You’ll sometimes find that your nicely spaced three element flex layout now breaks because all three elements need to be wrapped in a single <turbo-frame> tag.

A route worth exploring in these situations is adding display: contents to the <turbo-frame> element, as mentioned here.

Responding to a Turbo Frame request without a matching Turbo Frame

As you build applications with Turbo Frames, you will inevitably run into an issue in development where you click a link within a frame and all of the content within the frame disappears. You’ll check the Rails logs and see the view you expected was rendered without errors.

Why did everything disappear?

The first thing to do when this happens is to check the JavaScript console for errors. Turbo’s JavaScript expects that the HTML response to a Turbo Frame request will contain a matching Turbo Frame element. If no match is found, Turbo will empty the frame of all content and raise an error like this in the JavaScript console:

Response has no matching <turbo-frame id="comments"> element

This can be a little confusing at first, since the content renders fine from the server but once you get used to Turbo processing the rendered HTML and updating the DOM, you’ll be able to quickly debug missing frame element errors like this one.

When in doubt, check for JavaScript errors.

Updating page URL when navigating within frame

When a user clicks on a link within a frame, the URL of the page won’t change. This makes sense most of the time — the user hasn’t navigated to a new page, they’ve simply updated some of the content on the existing page.

One use case where the URL not changing can be problematic is when the user is applying sorting and filtering options by clicking links within a frame. Imagine a table layout with links to sort the table by each column, for example.

In those cases, a developer may wish to update the URL of the page to reflect the currently applied filters so that the user can copy/paste the URL to share a specific view of the table.

As of Turbo’s 7.1 release, developers can now update the page URL from a frame navigation by adding a turbo-action data attribute to the form or link that triggers the frame navigation. Options for turbo-action are replace, to replace the current URL in the browser’s history, or advance, to push a new entry in the history. When building search forms, you’ll likely want to use advance to update the page URL with each new search. In Rails, using a standard form_with search form, your code to push frame navigation into the browser’s history might look like this:

<%= form_with url: customers_path, method: :get, data: { turbo_frame: "customers", turbo_action: "advance" } do |form| %>
  <%# Some form content goes here %>
<% end %>

Wrapping up

Today we looked in detail at the current state of Turbo Frames — what they are, how to use them in Rails, and the some of the ways frames can help you build modern, efficient applications.

Turbo Frames in Rails applications naturally pair very well with Stimulus and work exceptionally well when paired with Turbo Streams to manage broadcasted updates and more targeted content updates.

With Hotwire coming by default to new Rails 7 applications, I’m happy to report that after a year of use, building Ruby on Rails applications with the Hotwire stack is simple, straightforward, and adds very little overhead to your development experience. For many Rails applications, all of the reactivity you need can be achieved with the Hotwire stack, and Turbo Frames will play a key role in many applications in the years to come.

While you begin to experiment, keep in mind that Turbo is still in very active development. Throughout this article, I linked to open PRs that will improve Turbo Frames and make some of the techniques described in this article simpler to implement — as time goes on, expect Turbo and Turbo Frames to continue to improve and for turbo-rails to continue to add functionality to help Rails developers use frames effectively in their applications.

Ready to go deeper? (Some shameless self-promotion ahead, sorry!)

I wrote about using the Hotwire stack to build server rendered modal forms in Rails
Build a typeahead search interface with Turbo Frames
Use a Turbo Stream + Turbo Frame combo to render different content to different users from a single stream broadcast
Read my exploration of Turbo Streams

Finally, if you want to learn about implementing Turbo Frames in a complex Rails application, I wrote a book on building a modern Ruby on Rails application with my preferred stack, including Hotwire, StimulusReflex, CableReady, and friends. The book is written in this same, step-by-step tutorial style that you find throughout this blog and aims to offer solutions for a variety of common, real-world web application problems without reaching for the weight of a full frontend framework.

That’s all for today. As always, thanks for reading!

Sort tables (almost) instantly with Ruby on Rails and Turbo Frames

2021-09-19T00:00:00+00:00

One of the wonderful things about working in Rails in 2021 is that we are spoiled for options when it comes to building modern, reactive applications. When we need to build highly interactive user experiences, we don’t need to reach for JavaScript frameworks like Vue or React — the Rails ecosystem has all the tools we need to deliver exceptionally fast, efficient, and easy-to-maintain front ends.

Yesterday, I published an article demonstrating a simple implementation of a sortable table with StimulusReflex. Today, we’re going to build the same experience with Turbo Frames instead.

Why build the same thing with two different tools? Because we have great options to choose from in Rails-land, and understanding each option is a great place to start when considering which tool is right for you and your team.

Like yesterday, our application is going to allow users to view a table of players. They’ll be able to click on each header cell to sort the table in ascending and descending order.

Sorting will happen very quickly, without a full-page turn, and we won’t be writing any custom JavaScript or doing anything outside of writing ordinary Ruby code and ERB templates.

When we’re finished, the application will work like this:

You can demo the application for yourself on Heroku (the free dyno might need a moment to wake up when you visit it) or view the complete source on Github.

This article assumes that you are comfortable building applications with Ruby on Rails and may be difficult to follow if you have never worked with Rails before. Previous experience with Turbo Frames is not required.

Let’s get started!

Setup

To begin, we’re going to create a new Rails application, skipping webpacker in favor of the newly released css and jsbundling gems.

If you prefer skipping the setup steps, you can clone down the example repo from Github. The main branch is pinned to the end of the setup process and ready for you to start building.

This article is being written using Rails 6.1. When Rails 7 releases, the install command listed below will change. Until then we need to manually add the bundling gems to our Gemfile.

First, from your terminal:

rails new player_sort_frames_1 --skip-javascript --skip-webpack-install --skip-turbolinks
cd player_sort_frames_1
bundle add cssbundling-rails jsbundling-rails hotwire-rails faker
rails g model Team name:string                                                  
rails g scaffold Player name:string team:references seasons:integer
rails db:migrate

Here, in addition to the bundling gems, we added Faker to our project to allow us to quickly add seed data to the database and added a Team model and Players resource to our project.

The core of our application will be a list of players, we won’t interact with Teams directly so we skip adding a controller and views for Teams.

Next up, we’ll use the new bundling gems to install webpack to handle our JavaScript and Tailwind for CSS. From your terminal:

rails javascript:install:webpack
rails css:install:tailwind

Finally, we’ll wrap up setup by installing Hotwire in our application. Technically, we only need Turbo for this project, but having Stimulus setup won’t hurt anything. One more time, from your terminal:

rails hotwire:install

With everything installed, you can start up the server and build your assets with bin/dev from your terminal.

Build table layout

We’ll begin by updating the players index page to display a nicely styled table of all of the players in the database. Rails’ scaffolding gets us most of the way there, but instead of rendering the players table in index.html.erb we’ll move the table to a partial.

This partial will come in handy later when we use Turbo Frames to sort the list of players.

First, update app/views/players/index.html.erb like this:

<div class="max-w-7xl mx-auto mt-12">
  <%= render "players", players: @players %>
</div>

The index is now rendering a partial (players) that hasn’t been created yet, so we’ll create that partial next.

From your terminal:

touch app/views/players/_players.html.erb

And fill the partial in like this:

<div class="shadow overflow-hidden rounded border-b border-gray-200">
  <table class="min-w-full bg-white">
    <thead class="bg-gray-800 text-white">
      <tr>
        <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
          <span>Name</span>
        </th>
        <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
          <span>Team</span>
        </th>
        <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
          <span>Seasons</span>
        </th>
      </tr>
    </thead>

    <tbody class="text-gray-700">
      <% players.each do |player| %>
        <tr>
          <td class="text-left py-3 px-6"><%= player.name %></td>
          <td class="text-left py-3 px-6"><%= player.team.name %></td>
          <td class="text-left py-3 px-6"><%= player.seasons %></td>
        </tr>
      <% end %>
    </tbody>
  </table>
</div>

This partial simply renders a table that contains a header row and then rows for each player in the players variable. The classes are all built-in Tailwind classes that are not integral to the function of the application.

Now we’ve got a nice looking table ready to display our players, but no players in the database! Let’s fix that by creating enough seed data that we can see sorting in action.

First, update db/seeds.rb:

['Dallas Mavericks', 'LA Clippers', 'LA Lakers', 'San Antonio Spurs', 'Boston Celtics', 'Miami Heat', 'New Orleans Pelicans'].each do |name|
  Team.create(name: name)
end

100.times do
  Player.create(name: Faker::Name.name, team: Team.find(Team.pluck(:id).sample), seasons: rand(25))
end

And then from your terminal, seed the database:

rails db:seed

If you haven’t already, boot up the app and build assets with bin/dev from your terminal, and then head to localhost:3000/players and see the list of randomly generated players, ready to sort.

Now that we’ve got our table populated and displaying, let’s start on the fun stuff by making the table sortable with Turbo Frames.

Add turbo frame sorting

As a reminder, our goal is for users to be able to click on a table header to sort the table by that column. We want sorting to happen without a page turn, and we want to use a Turbo Frame to update the list of players in the UI.

To start, we’ll convert the wrapper div in the players partial to a turbo_frame_tag with an id of players and the same classes that the wrapper div had. In app/views/players/_players.html.erb

<%= turbo_frame_tag "players", class: "shadow overflow-hidden rounded border-b border-gray-200" do %>
  <table class="min-w-full bg-white">
  <!-- snip -->
  </table>
<% end %>

Don’t forget to replace the closing </div> with the <% end %> tag!

Now that the players turbo frame wraps the content of the table, links inside of this frame will automatically attempt to replace the content of the turbo frame with the content they receive from the server.

We’ll come back to this concept in a minute, but first, let’s build out the remainder of the code we need for the first pass at sorting the table.

Next, inside of the players partial still, update the table header like this:

<thead class="bg-gray-800 text-white">
  <tr>
    <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
      <%= sort_link(column: "name", label: "Name") %>
    </th>
    <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
      <%= sort_link(column: "teams.name", label: "Team") %>
    </th>
    <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
      <%= sort_link(column: "seasons", label: "Seasons") %>
    </th>
  </tr>
</thead>

Here we’ve replaced the static text labeling each column with a call to the sort_link helper method, passing in a column and a label for each header cell.

This helper method doesn’t exist yet, so let’s add that next. In app/helpers/players_helper.rb:

module PlayersHelper
  def sort_link(column:, label:)
    link_to(label, list_players_path(column: column))
  end
end

sort_link renders a standard Rails link_to, using label to display the link to the user, and column to append a parameter to the generated URL.

The link_to points to list_players_path, which we haven’t defined yet. Let’s do that next, in routes.rb:

resources :players do
  collection do
    get 'list'
  end 
end

And finally, we need to update our controller to define an action for the new list route we’ve added.

In app/controllers/players_controller.rb, add a new list method, like this:

def list
  players = Player.includes(:team).order("#{params[:column]} asc")
  render(partial: 'players', locals: { players: players })
end

Here we’ve defined a new controller action that queries the database for all of the players, includes the teams table in the query, and orders the players using the value of params[:column].

The includes(:team) is necessary both to allow sorting players by their team name and for performance, since the players partial makes a call to player.team.name to display the players name on each row.

Once the players are retrieved, the players partial is rendered and sent back to the browser.

With the controller update in place, refresh the page, click on a column header, and, if all has gone well, you should see that the list of players is updated (almost) instantly with the correct column sorting applied.

When you sort by a column, you should see output in your server logs like this:

Started GET "/players/list?column=seasons"
Processing by PlayersController#list as HTML
  Parameters: {"column"=>"seasons"}
  Player Load (0.5ms)  SELECT "players".* FROM "players" ORDER BY seasons asc
  ↳ app/views/players/_players.html.erb:18
  Team Load (0.2ms)  SELECT "teams".* FROM "teams" WHERE "teams"."id" IN (?, ?, ?, ?, ?, ?, ?)  [["id", 3], ["id", 1], ["id", 2], ["id", 6], ["id", 4], ["id", 5], ["id", 7]]
  ↳ app/views/players/_players.html.erb:18
  Rendered players/_players.html.erb (Duration: 5.9ms | Allocations: 3994)
Completed 200 OK in 7ms (Views: 5.6ms | ActiveRecord: 0.6ms | Allocations: 4224)

Nice work so far, let’s pause here to step back and talk about how this all ties together.

We started by wrapping the players table in a Turbo Frame with an id of players. Then we added a link to each of the header cells, pointing to players/list.

Because the link is wrapped in a turbo frame, when the response is sent back from the server, Turbo scans the response for a turbo frame with the same id (players) and replaces the existing frame content with the new frame content.

In our case, this means that the updated list of players, rendered by players/list, replaces the content of the players table without touching the rest of the page.

This use of Turbo Frames, to replace the content of a matching frame, is the simplest approach to using Turbo Frames.

In more advanced cases, we can have a link inside of a frame break out of a frame, with target=_top or use data-turbo-frame to tell Turbo that a link from outside of a frame should target that frame.

Turbo Frames enable us to make fast, efficient page updates without adding significant complexity to our code.

Now that we’ve got a little more context for how Turbo Frames are enabling sorting in our application, let’s finish up by adding the ability to sort in both ascending and descending order and inserting a visual indicator when sorting is applied.

Add descending sorting

We want users to be able to sort in descending order by clicking on the same column header twice in a row. The first click will sort in ascending order, the next click will sort in descending order.

Here’s a demonstration of the desired user experience for this section of the article:

First, we’ll modify the sort_link helper that we added in the last section to send a direction as well as a column in the list request.

In app/helpers/players_helper.rb:

def sort_link(column:, label:)
  if column == params[:column]
    link_to(label, list_players_path(column: column, direction: next_direction))
  else
    link_to(label, list_players_path(column: column, direction: 'asc'))
  end
end

def next_direction
  params[:direction] == 'asc' ? 'desc' : 'asc'
end

sort_link now adds a second parameter into the link_to. The new direction parameter is set to the value of the next_direction helper method if we are generating the sort link for a column that is being used for sorting; otherwise, direction is set to ascending.

next_direction simply inverts the current sort direction so the sort direction changes with each click.

Next we’ll update the list method in players_controller.rb to use the new direction parameter in the order clause:

def list
  players = Player.includes(:team).order("#{params[:column]} #{params[:direction]}")
  # snip
end

With this change in place, we can now sort our table in either direction. Incredible stuff, nice work making it this far!

One more set of changes left: Giving users feedback in the UI that sorting is active.

Add visual indicator of sort direction

Our final task is to add an indicator next to the column name when that column is being used to sort the table. The indicator will be a small triangle, pointing up when sorting in ascending order and down when sorting in descending order.

We’ll start by updating the table header in the players partial like this:

<thead class="bg-gray-800 text-white">
  <tr>
    <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
      <%= sort_indicator if params[:column] == "name" %>
      <%= sort_link(column: "name", label: "Name") %>
    </th>
    <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
      <%= sort_indicator if params[:column] == "teams.name" %>
      <%= sort_link(column: "teams.name", label: "Team") %>
    </th>
    <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
      <%= sort_indicator if params[:column] == "seasons" %>
      <%= sort_link(column: "seasons", label: "Seasons") %>
    </th>
  </tr>
</thead>

Here we’ve added a conditional call to sort_indicator into each header cell.

Since we only want to add the indicator when the column is being used to sort the table, we use if params[:column] to skip the sort_indicator call on inactive columns.

We also added relative to the <th> elements class lists because the sort indicator will be absolutely positioned inside of the header cell.

Next up, we’ll define sort_indicator in app/helpers/players_helper.rb:

def sort_indicator
  tag.span(class: "sort sort-#{params[:direction]}")
end

sort_indicator simply returns a span with a class that matches the current sort direction, read from params[:direction]

Finally, we’ll add css so our sort indicator isn’t just an invisible span on the page.

For convenience, we’ll insert the css right into app/assets/stylesheets/application.css

.sort {
  position: absolute;
  top: 1rem;
  left: 0.5rem;
  width: 0;
  height: 0;
  border-left: 6px solid transparent;
  border-right: 6px solid transparent;
}

.sort-desc {
  border-top: 8px solid #fff;
}

.sort-asc {
  border-bottom: 8px solid #fff;
}

With the CSS added and our new sort_indicator helper defined, refresh the page, click on the column headers, and see that the table is sorted and a visual indicator is shown for the current column and sort direction:

Great job following along, we’ve reached the end of the code for today!

Wrapping up

Today we learned how to build a sortable table with Rails and Turbo Frames. Our table can be sorted without a page turn, and we built the interface using basic set of tools familiar to any Rails developer.

While our table only supports sorting today, our frame-based approach can be enhanced to support filtering and searching without deviating from the core concept of using Turbo Frames to display and update the table.

Turbo Frames, along with the rest of the Hotwire stack, give Rails developers the ability to quickly build fast, modern user experiences without adding the weight and complexity that can come with JavaScript frameworks.

You’ll note that I did not compare the approach in this article directly to the approach in yesterday’s article where we built the same UI with StimulusReflex. Rather than attempting a direct, side-by-side comparison, I prefer presenting both options as standalone projects that show how to build simple experiences using each tool so that readers can learn how each tool works, begin experimenting, and see which feels right.

Both StimulusReflex and Turbo (Frames and Streams) can deliver world-class user experiences in production applications while providing an unbeatable developer experience. The right choice for you and your team is almost certain to be the option that your team feels most comfortable with and most productive in.

Whichever routet you choose, with the Hotwire stack, StimulusReflex, and CableReady available, Rails is well-positioned for the future.

If you’re ready to go further with on your Turbo journey, here are a few places to start:

Review the Turbo documentation
Dive in to the Turbo Rails source code on Github
Spend time with Stimulus, the other side of the Hotwire stack
Learn from the community in the Hotwire discussion forum
Join the StimulusReflex discord, where folks will happily help you learn more about building reactive web applications with Rails

That’s all for today. As always, thanks for reading!

Sort tables (almost) instantly with Ruby on Rails and StimulusReflex

2021-09-18T00:00:00+00:00

Today we’re going to use Ruby on Rails and StimulusReflex to build a table that sorts itself each time a user clicks on a header column.

Sorting will occur without a page turn, in less than 100ms, won’t require any custom JavaScript, and we’ll build the whole thing with regular old ERB templates and a little bit of Ruby.

The end result will be very fast, efficient, simple to reason about, and easy to extend as new functionality is required.

It’ll be pretty fancy.

When we’re finished, it will work like this:

You can view the finished product on Heroku, or find the full source on Github.

This article will be most useful to folks who are familiar with Ruby on Rails, but you will not need any previous experience with Stimulus or StimulusReflex to follow along. If you’ve never worked with Rails before, some concepts here may be a little tough to follow.

Let’s get started!

Setup

As usual, we’ll start with a brand new Rails 6.1 application, with Tailwind and StimulusReflex installed. Tailwind is not a requirement, but it helps us make our table look a little nicer and the extra setup time is worth the cost.

If you’d like to skip the copy/pasting setup steps, you can clone down the example repo and skip ahead to the Building the Table section. The main branch of the repo is pinned to the end of the setup process and ready for you to start writing code.

If you’re going to follow along with the setup, first, from your terminal:

rails new player_sorting -T
cd player_sorting
bundle add stimulus_reflex
bundle add faker
rake stimulus_reflex:install
rails g model Team name:string
rails g scaffold Player name:string team:references seasons:integer
rails db:migrate

Assuming you’ve got Rails and Yarn installed, this will produce a brand new Rails 6.1 application (at the time of this writing), install StimulusReflex, and scaffold up the Team and Player models that we’ll use to build our sortable table.

If you don’t care to use Tailwind for this article, feel free to skip past this next section, Tailwind is a convenient way to make things look presentable, but if you just want to focus on sorting the table without any styling, Tailwind is not necessary!

If you want to install Tailwind, start in your terminal:

yarn add tailwindcss@latest postcss@latest autoprefixer@latest
npx tailwindcss init
mkdir app/javascript/stylesheets
touch app/javascript/stylesheets/application.scss

And then update tailwind.config.js:

module.exports = {
  purge: [
    './app/**/*/*.html.erb',
    './app/helpers/**/*/*.rb',
    './app/javascript/**/*/*.js',
    './app/javascript/**/*/*.vue',
    './app/javascript/**/*/*.react'
  ],
  darkMode: false,
  theme: {
    extend: {},
  },
  variants: {
    extend: {},
  },
  plugins: [],
}

And postcss.config.js:

// postcss.config.js
module.exports = {
  plugins: [
    require("tailwindcss")("./tailwind.config.js"),
    require("postcss-import"),
    require("postcss-flexbugs-fixes"),
    require("postcss-preset-env")({
      autoprefixer: {
        flexbox: "no-2009",
      },
      stage: 3,
    }),
  ],
}

Next we’ll update app/javascripts/stylesheets/application.scss to import Tailwind:

@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";

And then include that stylesheet in app/javascripts/packs/application.js:

import "../stylesheets/application.scss"

Finally, update the application layout to include the stylesheet generated by webpack:

<%= stylesheet_pack_tag 'application', 'data-turbolinks-track': 'reload', media: 'all' %>

Whew. We’re through the setup and ready to start building.

Building the table

First up, let’s get our bearings.

The core of our application is the Players resource. We are going to construct a table that displays all of the players in our database, with the players name, their team’s name, and their seasons played as columns.

We’ll only use the Team model created during setup in the context of Players, so we don’t need a controller or views for teams.

We’ll start by moving the table from the players index view to a partial. From your terminal:

touch app/views/players/_players.html.erb

And fill that partial in with:

<div id="players" class="shadow overflow-hidden rounded border-b border-gray-200">
  <table class="min-w-full bg-white">
    <thead class="bg-gray-800 text-white">
      <tr>
        <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
          <span>Name</span>
        </th>
        <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
          <span>Team</span>
        </th>
        <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer">
          <span>Seasons</span>
        </th>
      </tr>
    </thead>

    <tbody class="text-gray-700">
      <% players.each do |player| %>
        <tr>
          <td class="text-left py-3 px-6"><%= player.name %></td>
          <td class="text-left py-3 px-6"><%= player.team.name %></td>
          <td class="text-left py-3 px-6"><%= player.seasons %></td>
        </tr>
      <% end %>
    </tbody>
  </table>
</div>

Most of the markup here is Tailwind classes for styling the table.

The functionally important pieces are the ids set on the wrapper div (#players) and the ids set on the table header cells. These ids will be used later to update the DOM when the user clicks to sort the table.

Next update the index view to use the new partial:

<div class="max-w-7xl mx-auto mt-12">
  <%= render "players", players: @players %>
</div>

With these changes in place, we have a nice looking table ready to display the players in the database, but we don’t have any players. Since we’re going to be sorting, let’s make sure the database has plenty of data in it.

Copy this into db/seeds.rb:

['Dallas Mavericks', 'LA Clippers', 'LA Lakers', 'San Antonio Spurs', 'Boston Celtics', 'Miami Heat', 'New Orleans Pelicans'].each do |name|
  Team.create(name: name)
end

100.times do
  Player.create(name: Faker::Name.name, team: Team.find(Team.pluck(:id).sample), seasons: rand(25))
end

And then, from your terminal:

rails db:seed

Now start up your Rails server and head to localhost:3000/players.

If all has gone well, you should see a table populated with 100 randomly generated players.

Next up, we’ll build our sorting mechanism with StimulusReflex.

Sorting with StimulusReflex

To sort the table, we’re going to create a reflex class with a single action, sort.

When the user clicks on a table header, we’ll call the reflex action to sort the players and update the DOM.

We’ll start by generating a new Reflex. From your terminal:

rails g stimulus_reflex Table

This generator creates both a reflex in app/reflexes and a related Stimulus controller in app/javascripts/controllers.

For this article, we won’t make any modifications to the Stimulus controller, so we’ll focus on the Ruby reflex found at app/reflexes/table_reflex.rb

Fill that reflex in with:

class TableReflex < ApplicationReflex
  def sort
    players = Player.order("#{element.dataset.column} #{element.dataset.direction}")
    morph '#players', render(partial: 'players', locals: { players: players })
  end
end

The first line of sort is a standard Rails ActiveRecord query. In it, we retrieve all of the players from the database, ordered by attributes sent from the DOM when the reflex action is triggered.

Reflex actions have access to a variety of properties. In our case, the property we’re interested in is element.

element is a representation of the DOM element that triggered the reflex and it includes all of the data attributes set on that element, accessible via element.dataset.

This means that in reflex actions, we can always access data attributes from the element that triggered the reflex as if we were working with that element in JavaScript. Handy.

For our purposes, we care about two data elements that don’t yet exist in the DOM — column and direction. The ActiveRecord query to retrieve and order players uses those values to know which column to order the results by, and in which direction (ascending or descending) the results should be ordered.

After we’ve retrieved the ordered list of players from the database, we use a selector morph to update the DOM, replacing the content of the players partial we created earlier with the updated list of players.

Our reflex is built, but there’s no way for a user to trigger the reflex. Let’s add that next.

In the players partial, update the header row like this:

<tr>
  <th 
    id="players-name" 
    class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer"
    data-reflex="click->Table#sort"
    data-column="name"
    data-direction="asc"
  >
    <span>Name</span>
  </th>
  <th 
    id="players-team" 
    class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer"
    data-reflex="click->Table#sort"
    data-column="teams.name"
    data-direction="asc"
  >
    <span>Team</span>
  </th>
  <th 
    id="players-seasons" 
    class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer"
    data-reflex="click->Table#sort"
    data-column="seasons"
    data-direction="asc"
  >
    <span>Seasons</span>
  </th>
</tr>

Here we’ve updated each header cell with the three data attributes we need for the reflex to be triggered and to run successfully.

First, data-reflex is used to tell StimulusReflex that this element should trigger a reflex when some action occurs. In our case, it will be called on click.

Each cell also gets a unique data-column value, which we use to sort the players result by the matching database column. Finally, each header cell also starts with a direction of asc which is used to set the direction of the order query.

Let’s look at the Reflex code again and review what’s happening now that we’ve updated the DOM to call this reflex.

def sort
  players = Player.order("#{element.dataset.column} #{element.dataset.direction}")
  morph '#players', render(partial: 'players', locals: { players: players })
end

The sort method we’ve defined matches the name of the data-reflex on each header cell. When the header cell is clicked, this reflex will run.

The header cell that that the user clicks will be passed to the reflex as element, giving us access to the element’s data attributes, which we access through element.dataset.

Once the value of players is set by the database query, we use a selector morph to tell the browser to update the element with the id of players with the content of the players partial, using the updated, reordered list of players.

This is the magic of StimulusReflex in action. With just a couple of data attributes and a few lines of simple Ruby code, our users can now click on a table header and, in < 100ms, they’ll see a table sorted to match their request.

Refresh the page and try it out for yourself. If all has gone well, clicking on a header cell should sort the table by that column in ascending order. While this is nice, we have a few more items to address before our work is complete.

Next up, we’ll address an error with the order query, and then finish this article by modifying the sort reflex to allow users to sort in both ascending and descending order and display visual feedback to indicate what column is being sorted.

Fixing an ordering error

First, sharp eyed readers might have noticed that sorting by the team name column doesn’t work yet.

Each header cell’s column data attribute matches a column in the database, so we can generate the order query dynamically. This works fine for the name and season because those columns live on the Players table. ActiveRecord knows how to order by name and seasons without any extra effort.

For the team name column, we’re passing teams.name to the order call in our query, which ActiveRecord trips over with an error like this:

# The text will vary depending on the database adapter you're using!
Reflex Table#sort failed: SQLite3::SQLException: no such column: teams.name

We can fix this by updating the query slightly:

players = Player.includes(:team).order("#{element.dataset.column} #{element.dataset.direction}")

Here we added includes(:team) to the existing order query, making the Teams table accessible in the order clause and fixing the “no such column” error that was thrown when attempting to sort by team name.

Note that joins instead of includes would also fix the error, but since we need to use team name when we render the players partial (to display each player’s team), includes is the better choice.

Before moving on, you’ll notice that we are using user-accessible values to generate a SQL query — anyone can modify data-attributes in their browser’s dev tools.

Prior to Rails 6, this could have opened up our application to SQL injection; however, since Rails 6, Rails will raise an error automatically if anything but a table/column name + a sort direction are passed in to order.

Adding descending ordering

With the work we’ve done so far, sorting the table works great as long as the user only wants to sort in ascending (A - Z) order. Since the sort direction is read from a data attribute that is always “asc”, there is no way to sort in descending (Z - A) order. Let’s add that functionality next.

Before jumping in to the code, let’s outline the desired user experience.

When a user clicks on a column header, the table should be sorted by that column, in ascending order. When the user clicks on the same column header again, the table should be sorted by that column in descending order. And then we alternate, forever, between ascending and descending on subsequent clicks.

Sorting by a different column should always sort in ascending order on the first click.

Here’s a gif of what we’re aiming for:

To achieve the desired user experience, we need to track the next sort direction for each header cell, so that when the sort reflex is called, the right direction can be sent to the order query.

To do this, we’re going to take advantage of CableReady’s tight integration with StimulusReflex. We’ll update the sort reflex to include a cable_ready operation that changes the direction data attribute of the element that triggered the reflex.

To do this, update TableReflex like this:

def sort
  # snip
  set_sort_direction
end

private

def next_direction(direction)
  direction == 'asc' ? 'desc' : 'asc'
end

def set_sort_direction
  cable_ready
    .set_dataset_property(
      selector: "##{element.id}",
      name: 'direction',
      value: next_direction(element.dataset.direction)
    )
end

Here we’ve added two private methods to the TableReflex class. next_direction is a simple helper method that takes the current value of direction and returns the next value.

set_sort_direction is more interesting. In it, we use CableReady’s set_dataset_property operation to set the value of the element’s direction data attribute to the value of next_direction .

Finally, we call set_sort_direction in the sort reflex, which adds the set_dataset_property operation to the queue each time the sort reflex runs.

With this in place, refresh and click on the same column multiple times to see that each click reorders the table, toggling between ascending and descending order.

Order of operations: Not just for math class

Before moving on, it is important to pause and think about how this code works. When a reflex includes CableReady operations, a specific order of operations is always followed.

First, broadcasted CableReady operations execute. Next, StimulusReflex morphs execute. Finally, CableReady operations that are not broadcasted execute (that’s our set_dataset_property operation).

Because the StimulusReflex morph runs before the CableReady operation, each table header cell has its direction data attribute reset to asc when sort is triggered. This behavior lets us “reset” sort directions when moving between columns without having to add logic in the partial.

Immediately after the StimulusReflex morph, set_dataset_property runs and updates the value of direction on the currently active sort column.

If we appended .broadcast to the set_dataset_property operation, the direction property would be updated before the StimulusReflex morph, causing the CableReady update to be overwritten by the morph, breaking the ability to sort in descending order.

This order of operations is important to understand, and helps unlock a new level of functionality within reflexes.

Before moving on, now that we understand the order of operations in reflex actions, we can use that knowledge to make a small optimization to the sort reflex.

We know that every time the reflex runs, each header cell will have its data-direction value set to asc before the active sort column is updated by the CableReady set_dataset_property operation.

Since the value of data-direction is already asc, if the next sort direction is asc, set_dataset_property won’t do anything useful.

Let’s update sort to skip the CableReady operation in that case:

def sort
# snip
  set_sort_direction if next_direction(element.dataset.direction) == 'desc'
end

Now set_sort_direction will only be run when necessary, simplifying our DOM updates at the cost of slightly more complexity in our ruby code.

Let’s finish up our sortable table implementation by adding a visual indicator to the table when sorting is active.

Sort direction visuals

To indicate which column is being sorted, and in which direction, we’ll draw a triangle with CSS, with an upward pointing triangle indicating ascending order, and a downward triangle indicating descending order.

Only the column that is being used for sorting will display the icon.

When we’re finished, the indicator will look like this:

Let’s start with the CSS.

We’ll insert the CSS directly into our main application.scss file to keep things simple:

.sort {
  position: absolute;
  top: 5px;
  left: -1rem;
  width: 0;
  height: 0;
  border-left: 6px solid transparent;
  border-right: 6px solid transparent;
}

.sort-desc {
  border-top: 8px solid #fff;
}

.sort-asc {
  border-bottom: 8px solid #fff;
}

We’ll use the base .sort class along with a dynamic .sort-asc or .sort-desc to display the sort indicator. If you’re interested in how this CSS works, this is a nice introduction to drawing shapes with CSS.

With the CSS ready, next we’ll create a partial to render the indicator, from your terminal:

touch app/views/players/_sort_indicator.html.erb

And fill that in with:

<div class="relative">
  <span class="sort sort-<%= direction %>"></span>
</div>

Nothing fancy here, just appending the local direction variable to the class name to ensure our triangle points in the right direction.

We’ll finish up by updating the sort reflex to insert the sort indicator into the DOM, again relying on a CableReady operation that runs immediately after the StimulusReflex morph.

class TableReflex < ApplicationReflex
  def sort
    # snip
    insert_indicator
  end

  private

  # snip
  def insert_indicator
    cable_ready
      .prepend(
        selector: "##{element.id}",
        html: render(
          partial: 'players/sort_indicator',
          locals: { direction: element.dataset.direction }
        )
      )
  end
end

Here we added another private method to the reflex to handle inserting the sort indicator when the sort reflex is called.

insert_indicator uses CableReady’s prepend operation to insert the content of the sort_indicator partial into the DOM, before the target element’s first child.

With this in place, we can refresh the page and see the sort indicator added each time the sort reflex runs, pointing up for ascending sorts and down for descending sorts:

An implementation note

During this article I made a choice to use cable_ready operations to add the sort indicator and update the data attribute.

Instead of cable_ready operations, another approach would be to assign instance or local variables for things like “active column” and “next direction” during the sort reflex. We could then read those variables when rendering the players partial, using them to render the sort indicator and set the next sort direction.

This approach would allow us to eliminate the cable_ready operations; however, doing so would complicate the view. Either approach is fine, my personal preference is to rely on the very fast cable_ready operations to simplify the view.

Using cable_ready also has the added benefit of letting us talk more about how StimulusReflex works, which is a bonus in a tutorial article like this one. As you spend more time with StimulusReflex, experiment with different approaches and find what works best for you.

Wrapping up

Today we built a sortable table interface with Ruby on Rails, StimulusReflex, and CableReady. Our table is fast, updates efficiently, is easy to extend, and is no where near production ready yet. What we built today was part one of a two part series. Next up, we’ll extend the sortable table by adding filtering and pagination, getting closer to the full-featured implementation seen in Beast Mode.

While there are numerous ways to implement a sortable table interface, for Rails developers, StimulusReflex is worthy of strong consideration. SR’s fast, efficient mechanisms for updating and re-rendering the DOM, including bypassing ActionDispatch’s overhead with selector morphs, allow us to sort and render the updated table extremely quickly, with minimal code complexity or additional mental overhead. Its tight integrations with CableReady and Stimulus combine into an extremely powerful tool in any Rails developers kit.

To go further into StimulusReflex and CableReady:

Review StimulusReflex patterns for thoughtfully designed solutions in StimulusReflex, including filterable for working with complex sorting and filtering requirements
Join the StimulusReflex discord and connect with other folks building cool stuff with StimulusReflex, CableReady, and Rails

That’s all for today, as always, thanks for reading!

Server-rendered modal forms on Rails with CableReady, Mrujs, Stimulus, and Tailwind

2021-09-09T00:00:00+00:00

The Rails ecosystem continues to thrive, and Rails developers have all the tools they need to build modern, reactive, scalable web applications quickly and efficiently. If you care about delivering exceptional user experiences, your options in Rails-land have never been better.

Today we’re going to dive into this ecosystem to use two cutting edge Rails projects to allow users to submit forms that are rendered inside of a modal.

The form will open in a modal with content populated dynamically by the server, the server will process the form submission, and the DOM will updated without a full-page turn.

To accomplish this, we’ll use Stimulus for the front-end interactivity, CableReady’s brand new CableCar feature to send content back from the server, and Mrujs to enable AJAX requests and to automatically process CableCar’s operations.

It’ll be pretty fancy.

When we’re finished, our application will look like this:

This article includes a fair amount of JavaScript and assumes a solid understanding of the basics of Ruby on Rails.

If you’ve never used Rails before, this article might move a little too quickly for you. While comfort with Rails and JavaScript are needed, you don’t need to have any prior experience with CableReady or Stimulus.

As usual, you can find the complete source code for this article on Github.

Let’s dive in!

Setup

If you prefer to skip the setup steps and jump right in to coding, you can clone the main branch of this repo and then scroll down to the Customers Layout section.

To get everything installed, we’re going to walk on the wild side by using the newly released, very-much-still-alpha jsbundling-rails and cssbundling-rails gems.

First, we’ll create a Rails application and use the alpha js/cssbundling gems to install Webpack and Tailwind, from your terminal:

rails new tiny_crm --skip-webpack-install --skip-javascript
cd tiny_crm
bundle add jsbundling-rails cssbundling-rails
rails javascript:install:webpack
rails css:install:tailwind
bin/dev

Note that these gems are VERY new, if you bump into errors, check the documentation to see if commands have changed or reach out to me and let me know what error you’re encountering.

With Webpack and Tailwind installed, next we’ll install the core dependencies for this guide, Stimulus, CableReady (plus Action Cable), and Mrujs, from your terminal:

bundle add hotwire-rails
be rails hotwire:install

And then update your Gemfile to pull in the latest cable_ready.

gem "cable_ready", github: "stimulusreflex/cable_ready"

Note that if you’re reading this in the future, we’re using 5.0 for this guide.

And then from your terminal:

bundle
yarn add mrujs cable_ready @rails/actioncable

Next, update app/javascript/packs/application.js like this, to pull in the new dependencies and configure Mrujs to use its CableCar plugin.

import "./controllers"
import mrujs from "mrujs";
import CableReady from "cable_ready"
import { CableCar } from "mrujs/plugins"
import * as Turbo from "@hotwired/turbo"

window.Turbo = Turbo;

mrujs.start({
  plugins: [
    new CableCar(CableReady)
  ]
})

That’s a lot of dependencies to setup. Do we really need all of this to display a modal? No, no not really.

The techniques we’ll use in this article only require Action Cable (a core Rails library), CableReady, and Mrujs.

Tailwind and Stimulus are requirements to follow along with the guide step-by-step, but we’re just using them to do things that can be done with your own CSS and vanilla JavaScript, if that’s your preference.

Ultimately, the only UI component you need is a modal that can open and close. Stimulus and Tailwind are a simple way to get there, but they’re not the only way!

Moving on, to wrap up the copy/pasting setup work, we’ll be creating and editing Customers in this application, so let’s scaffold up that resource:

rails g scaffold Customer name:string
rails db:migrate

Setup is complete! Great work so far. Now we can start writing code.

Customers layout

First, we’ll apply some basic styling to the customers index page:

<div class="max-w-3xl mx-auto mt-8">
  <div data-controller="modal" class="flex justify-between items-baseline mb-6">
    <h1 class="text-3xl text-gray-900">Customers</h1>
    <%= link_to 'New Customer', new_customer_path, class: "text-blue-600", data: { action: "click->modal#open" } %>
    <%= render 'modal' %>
  </div>
  <div id="customers" class="flex flex-col items-baseline space-y-6 p-4 shadow-lg rounded">
    <% @customers.each do |customer| %>
      <%= render "customer", customer: customer %>
    <% end %>
  </div>
</div>

The index view renders a list of customers, plus a header that includes a link to create a new customer.

The header container div includes a controller="modal" data attribute which is a reference to a Stimulus controller that doesn’t exist yet. Likewise, the new customer link references the same modal controller in its data-action attribute.

We’ll create that controller soon, for now though, clicking on the new customer link will navigate the browser to customers/new

The index view also renders two partials that don’t exist yet, modal and customer. Let’s create and fill those in next so that we can render the index page again.

First, the customer partial:

touch app/views/customers/_customer.html.erb

The customer partial just renders the customer’s name for now:

<div class="text-gray-700 border-b border-gray-200 w-full pb-2">
  <%= customer.name %>
</div>

Next create the modal partial:

touch app/views/customers/_modal.html.erb

And fill that in:

<div data-modal-target="container"
     class="hidden fixed inset-0 overflow-y-auto flex items-center justify-center"
     style="z-index: 9999;">
  <div class="max-w-lg max-h-screen w-full relative">
    <div class="m-1 bg-white rounded shadow">
      <div class="px-4 py-5 border-b border-gray-200 sm:px-6">
        <h3 class="text-lg leading-6 font-medium text-gray-900">
          Customer
        </h3>
      </div>
      <form id="customer_form"></form>
    </div>
  </div>
</div>

The important items here are the modal-target="container" data attribute, which the Stimulus controller will use to open/close the modal and the empty <form> element.

This form element will eventually be filled in with content from the server when the user opens the modal.

With the index markup in place and your server running via bin/dev, head to http://localhost:3000/customers and make sure everything is displaying as expected.

Next we will create the modal Stimulus controller and fill it in with content rendered from the server. I’m excited too.

First we need to create a new Stimulus controller:

rails g stimulus modal

And fill the controller in with:

// Credit: This controller is an edited-to-the-essentials version
// of the modal component created by @excid3 as part of the essential 
// tailwind-stimulus-components package found here:
// https://github.com/excid3/tailwindcss-stimulus-components

// In production, use the full component from the 
// library or expand this controller to allow for 
// keyboard closing and dealing with scroll positions

import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
static targets = ['container'];

  connect() {
    this.toggleClass = 'hidden';
    this.backgroundId = 'modal-background';
    this.backgroundHtml = this._backgroundHTML();
    this.allowBackgroundClose = true;
  }

  disconnect() {
    this.close();
  }

  open() {
    document.body.classList.add('fixed', 'inset-x-0', 'overflow-hidden');
    this.containerTarget.classList.remove(this.toggleClass);
    document.body.insertAdjacentHTML('beforeend', this.backgroundHtml);
    this.background = document.querySelector(`#${this.backgroundId}`);
  }

  close() {
    if (typeof event !== 'undefined') {
      event.preventDefault()
    }
    this.containerTarget.classList.add(this.toggleClass);
    if (this.background) { this.background.remove() }
  }

  _backgroundHTML() {
    return `<div id="${this.backgroundId}" class="fixed top-0 left-0 w-full h-full" style="background-color: rgba(0, 0, 0, 0.7); z-index: 9998;"></div>`;
  }
}

There’s a lot of JavaScript here, but it isn’t doing anything too fancy.

In connect, we set default values the controller needs to function.

open simply applies classes to the body and the modal container to make the modal visible on the screen and apply the standard grayed-out background to the rest of the page.

When close is called, the background is removed and the modal is hidden.

If you decide to use this approach in a real project, consider using the Stimulus component this code is derived from. The above code was edited for brevity and the edits will introduce issues with scrolling and accessibility that the full component handles cleanly.

With the Stimulus controller created, we’re almost ready to render the modal. Before we proceed, let’s step back and make sure we’re clear on what we want to achieve.

Our goal is to create a server-rendered modal that allows a user to create a new customer. After the form in the modal is submitted, the newly created customer should be inserted into the list of customers, and the modal should close.

The first task is to open the modal and display the content from the server, which means that when a user clicks on the New Customer link on the index page:

A request should be made to the server to retrieve the content for the customer form
The content should replace the empty customer form that the modal partial renders on the initial page load
The modal should open

This will be easier than it sounds.

We’ll use Mrujs to make an AJAX request to customers#new, we’ll queue up operations with CableCar, and Mrujs will automatically process those operations for us.

First we need to tell Mrujs to convert the New Customer link to a CableCar-enabled link.

As described in the documentation, we’ll do that by updating the link on the index page like this:

<%= link_to 'New Customer', new_customer_path, class: "text-blue-600", data: { action: "click->modal#open", remote: true } %>

Here we’ve added data-remote="true" to the link. Once we do this Mrujs takes care of the rest for us by adding a special cable-ready Accept header when the link is clicked.

With this change in place, when the user clicks on the New Customer link, a request will be sent to customers#new that our controller will (eventually) respond to with Cable Ready operations.

Since we’re going to be rendering the form partial shortly, let’s go ahead and update that partial now:

<%= form_with(model: customer, id: "customer_form") do |form| %>
  <div class="px-4 pt-5 pb-4 sm:p-6 sm:pb-4">
    <% if customer.errors.any? %>
      <div class="p-4 border border-red-600">
        <h2>
          Could not save customer
        </h2>
        <ul>
          <% customer.errors.full_messages.each do |message| %>
            <li><%= message %></li>
          <% end %>
        </ul>
      </div>
    <% end %>

    <div class="form-group">
      <%= form.label :name %>
      <%= form.text_field :name %>
    </div>

  </div>
  <div class="rounded-b mt-6 px-4 sm:grid sm:grid-cols-2 sm:gap-3 sm:grid-flow-row-dense">
    <%= form.button class: "w-full sm:col-start-2 bg-blue-600 px-4 py-2 mb-4 text-white rounded-sm hover:bg-blue-700" %>
    <button data-action="click->modal#close" class="mt-3 w-full sm:mt-0 sm:col-start-1 mb-4 bg-gray-100 hover:bg-gray-200 rounded-sm px-4 py-2">
      Cancel
    </button>
  </div>
<% end %>

Most of this is standard Tailwind classes to apply some light styling to the form.

The important pieces are the id of the form, assigned on line 1, and the data-action assigned to the close button, which fires the close function we defined in the modal Stimulus controller earlier.

We can also make the form look nicer with Tailwind’s form plugin. This is optional, but if you’d like to use it, first install it with yarn, from your terminal:

yarn add @tailwindcss/forms

Then update tailwind.config.js:

module.exports = {
  mode: 'jit',
  purge: [
    './app/views/**/*.html.erb',
    './app/helpers/**/*.rb',
    './app/javascript/**/*.js'
  ],
  plugins: [
    require('@tailwindcss/forms')
  ]
}

Next we need to update the new method in the CustomersController to render CableReady operations, using the newly introduced CableCar. To do that, we’ll make two changes to the CustomersController.

First update the controller to include CableReady::Broadcaster to give the controller access to cable_car:

class CustomersController < ApplicationController
  include CableReady::Broadcaster
  # snip
end

Feel free to place the include in ApplicationController if you prefer.

Then update the CustomersController new method as follows:

def new
    html = render_to_string(partial: 'form', locals: { customer: Customer.new })
    render operations: cable_car
      .outer_html('#customer_form', html: html)
end

Here we’re rendering the form partial to a string which we then pass to cable_car and use in an outer_html operation, targeting the (currently empty) customer form.

With all this in place, head back to http://localhost:3000/customers and click on the New Customer link. If all has gone well, you should see the modal open and the customer form render.

Incredible work so far.

Next up we’ll use this same CableCar approach to handle form submissions.

Submitting the form

This section is going to look pretty familiar. We’ll start by updating the customer form with the remote data attribute, just like we added to the new customer link in the last section:

<%= form_with(model: customer, id: "customer_form", data: { remote: true }) do |form| %>

Again, this tells Mrujs to submit the form with an AJAX request and to expect CableReady operations to perform in response.

Next, head back to customers_controller.rb and update the create method:

def create
  @customer = Customer.new(customer_params)

  if @customer.save
    html = render_to_string(partial: 'customer', locals: { customer: @customer })
    render operations: cable_car
      .append('#customers', html: html)
  else
    # TODO: Handle errors
  end
end

Here we’re again rendering a partial to a string and passing that string to an operation (this time, append). The target is the list of the customers rendered in the customers index view, where the newly created customer will be added to the bottom of the list. If you prefer, use prepend to add the customer to the top of the list instead.

With this in place, open up the modal, type in a name, and submit the form. You should see the newly created customer get appended to the list like expected but the modal doesn’t close.

That’s not ideal.

How do we close the modal when the form submission is successful? By tapping into some of what makes CableReady so powerful — chaining operations and emitting custom DOM events.

To make this work, we’ll first add another operation to the operations chain sent back to Mrujs:

def create
    # snip
    render operations: cable_car
      .append('#customers', html: html)
      .dispatch_event(name: 'submit:success')
end

The dispatch_event operation allows us to emit whatever event we like. With this new event dispatched on successful submission, closing the modal is as simple as adding an event listener to the modal’s Stimulus controller, like this:

open() {
  document.addEventListener("submit:success", () => {
    this.close()
  }, { once: true });
  // snip
}

When the modal opens, an event listener is created, tuned to the event name that is dispatched from the cable_car payload.

Now when you submit the modal form, both the append and the dispatch_event operations are sent back in response to a successful form submission, Mrujs magic automatically performs the operations, and the submit:success event listener closes the modal.

Handling errors

Wonderful work stuff so far. Next we’ll deal with form errors using render operations again.

First, make it possible for a customer submission to have a validation error by adding validates_presence_of :name to models/customer.rb

With that in place, when the form is submitted with a blank name, the form submission will fail. When that happens, we want to render the customer form inside of the modal, with the validation errors attached.

To render errors in response to a failed submission, update the create method like this:

def create
  @customer = Customer.new(customer_params)

  if @customer.save
    html = render_to_string(partial: 'customer', locals: { customer: @customer })
    render operations: cable_car
      .append('#customers', html: html)
      .dispatch_event(name: 'submit:success')
  else
    html = render_to_string(partial: 'form', locals: { customer: @customer })
    render operations: cable_car
      .inner_html('#customer_form', html: html), status: :unprocessable_entity
  end
end

In the else branch, we again render the partial to a string and render operations. This time, since we don’t want the modal to close and we don’t need to replace the <form> element itself, we can just use one inner_html operation.

Open up the modal, submit a blank form, and see that the form is re-rendered with the errors as expected.

You’re a star for making it this far. Let’s finish up by seeing how easy it is to reuse this modal for editing customers, and adding some small optimizations to the modal opening.

Cable Car customer edits

A cool thing about the empty customer form modal is that we can reuse it with no modifications for editing existing customers, leaving us with just one tiny modal container that we can reuse for any number of modals on the page.

First, add a cable_car enabled modal link to the customer partial:

<div class="text-gray-700 border-b border-gray-200 w-full pb-2" id='<%= "customer-#{customer.id}" %>'>
  <%= link_to customer.name, edit_customer_path(customer), data: { remote: true, action: "click->modal#open" } %>
</div>

Here we setup the relevant data attributes on the link and, on the wrapper div, we added a unique id. We’ll use that id to replace the content of the customer when the edit form is submitted.

Next up, back to the CustomersController to adjust the edit and update methods:

def edit
  html = render_to_string(partial: 'form', locals: { customer: @customer })
  render operations: cable_car
    .replace('#customer_form', html: html)
end

def update
  if @customer.update(customer_params)
    html = render_to_string(partial: 'customer', locals: { customer: @customer })
    render operations: cable_car
      .replace("#customer-#{@customer.id}", html: html)
      .dispatch_event(name: 'submit:success')
  else
    html = render_to_string(partial: 'form', locals: { customer: @customer })
    render operations: cable_car
      .inner_html('#customer_form', html: html), status: :unprocessable_entity
  end
end

This should look pretty familiar. The edit method is a mirror of the new method, and the update method is a mirror of the create method. Again, we dispatch the submit:success event when the customer is updated, otherwise the form re-renders with errors.

Finally, to use the same modal controller for every modal link on the page, we’ll move the data-controller="modal" declaration one level up the DOM tree. In customers/index.html.erb:

<div class="max-w-3xl mx-auto mt-8" data-controller="modal">
  <div class="flex justify-between items-baseline mb-6">
    <!-- Snip -->
  </div>
</div>

With these changes in place, refresh the customers index page, click on a customer’s name, and see that updating the customer happens in a modal, and the customer is updated in place in the list on a successful form submission.

Something you may have noticed as you’ve worked through this guide is that the modal opens before the content from the server has rendered, causing a very brief flash as the modal opens and then quickly replaces the empty form or the form’s previous contents:

This happens because the modal opens instantly when a modal link is clicked but the round trip to the server to retrieve the form partial is not quite instant.

We have options for how to prevent this, including adding a loading state to the modal to make the re-render less jarring, but the method I’ll demonstrate is keeping the modal hidden until the content has been retrieved from the server. This gives us another chance to use CableReady and Stimulus, and that’s what we’re all here for, right?

First, add another event listener to the modal controller:

open() {
  document.addEventListener("modal:loaded", () => {
    this.containerTarget.classList.remove(this.toggleClass);
  }, { once: true });
  document.addEventListener("submit:success", () => {
    this.close()
  }, { once: true });
    
  document.body.classList.add('fixed', 'inset-x-0', 'overflow-hidden');
  document.body.insertAdjacentHTML('beforeend', this.backgroundHtml);
  this.background = document.querySelector(`#${this.backgroundId}`);
}

Here we updated open to move the containerTarget.classList.remove call from happening instantly to happening in response to modal:loaded DOM event.

This change means that all of the modal links are now broken because the modal:loaded event never occurs and so containerTarget.classList.remove never runs and the modal container stays hidden.

We can fix the modal links by updating CustomersController like this:

def new
  html = render_to_string(partial: 'form', locals: { customer: Customer.new })
  render operations: cable_car
    .outer_html('#customer_form', html: html)
    .dispatch_event(name: 'modal:loaded')
end

def edit
  html = render_to_string(partial: 'form', locals: { customer: @customer })
  render operations: cable_car
    .outer_html('#customer_form', html: html)
    .dispatch_event(name: 'modal:loaded')
end

In both the new and edit methods, we again take advantage of CableReady’s chainable operations to dispatch modal:loaded after the outer_html is replaced.

With this change, the sequence of events when the user clicks on a modal link is:

Request to server begins
Open action is triggered
Modal backdrop is applied to the page, no visible modal yet
Form content is replaced
Modal loaded event is dispatched
Hidden class is removed from the modal, making it visible

This sequence happens rapidly enough in our circumstances for the user to barely notice the delay between the backdrop being applied and the modal displaying. In a production environment, you may find that a loading state for an immediately-opened modal is a more scalable option, but we’re here to learn about CableReady and Mrujs, not build a production application.

With these changes in place, the modal will open with the updated content already populated, eliminating the flash of old content.

The single modal connected div can be used to display any number of modals, serving as a way to reduce the initial page load in a more traditional application which might pre-render each edit modal.

Wrapping up

Today we learned how to build a server rendered modal form, powered by Stimulus, CableReady, and Mrujs.

Stimulus and CableReady are two powerful, battle-tested tools with a mature feature set that should be considered for any modern Rails application. CableReady can stand alone as a way to deliver real-time updates to end users through a variety of methods or it can be powered-up with StimulusReflex to deliver a SPA-link experience, minus the SPA.

Mrujs is a newer tool, under active development, and is intended to serve as a modern, stable replacement for rails/ujs, which is no longer under active development and which will be deprecated when Rails 7 releases.

In addition to the tight integration with CableReady’s Cable Car that we saw today, Mrujs gives you access to simple confirmation dialogs, disabled links, and the other niceties from Rails UJS, in a modern package.

An important note before we go: we could build a very similar user experience with a variety of tools in the Rails ecosystem, including Turbo Streams (here’s a guide for that).

While the full Hotwire stack can deliver this experience with about the same amount of effort, the power and flexibility of CableReady’s chainable operations makes CableReady + Mrujs a better fit for this particular use case than the full Hotwire stack, in my very, very humble opinion.

What’s really exciting about this is that as Rails developers, our cups are overflowing with powerful tools to build real-time, reactive applications. That means we all win, no matter which tool we reach for most often.

Continue your journey with CableReady, Stimulus, and Mrujs with these resources:

The CableReady documentation
The Stimulus docs
The Mrujs docs
Explore the StimulusReflex documentation when you’re ready
Join the StimulusReflex discord if you get stuck with CableReady or StimulusReflex
(Shameless plug) Subscribe to my monthly newsletter, Hotwiring Rails, to stay up to date on the latest on building modern, performant applications with Rails and tools like CableReady and Stimulus

That’s all for today.

As always, thanks for reading!

Doing the Impossible — Building a Persistent Audio Player in Ruby on Rails

2021-09-05T00:00:00+00:00

Today we’re going to learn how to build a Ruby on Rails app that accomplishes what some folks think is impossible in a multi-page application — persisting an audio player while navigating around an application.

We’ll use Ruby on Rails and Turbo to accomplish this, but we could use Turbo’s predecessor, Turbolinks to achieve the same result, and Rails is only incidental to the finished project. We could just as easily use Turbo with any other “multi-page” framework and deliver the same experience.

When we’re finished, our application will allow users to create and manage Ideas along with a persistent audio player tuned to a white noise internet radio station, to help them focus while they generate ideas.

Users will be able to start and stop the audio using a standard <audio> input. When audio is playing, it will continue playing as the user navigates around the application.

The application will look like this:

After we’ve built our application, we’ll spend a time talking about myths in web development, and how to avoid falling into the expert-led myth-trap.

This article assumes basic familiarity with Ruby on Rails, but no prior experience with Turbo is required, and you should be able to follow along even if you’ve never seen Rails code before.

Let’s get started.

Application Setup

To begin, we’ll create a standard Ruby on Rails app, install Turbo, and scaffold an Ideas resource that we’ll use to demonstrate the “multi-page” part of the application.

rails new rails_focused_ideas
cd rails_focused_ideas
bundle add turbo-rails
rails turbo:install
rails g scaffold Idea title:string description:text
rails db:migrate

To finish installing Turbo, open app/packs/application.js and update it like this:

import Rails from "@rails/ujs"
import "@hotwired/turbo-rails"
import * as ActiveStorage from "@rails/activestorage"
import "channels"

Rails.start()
ActiveStorage.start()

Here we’ve removed any references to turbolinks and added the import "@hotwired/turbo-rails" line to use Turbo throughout our application.

When you’ve finished setting up the application, start your rails server (rails s from your terminal) and navigate to http://localhost:3000/ideas.

Add audio player to layout

Next up, we’ll add an audio element to our application layout (app/views/layouts/application.html.erb), just after the opening <body> tag:

<audio 
  controls
  src="http://uk1.internet-radio.com:8267/listen.pls&t=.m3u">
      Your browser does not support the
      <code>audio</code> element.
</audio>

The src attribute in this code is a random white noise internet station and it could go down tomorrow or ten years from now. You’re free to use whatever you like for the source, including a local file accessible to your Rails server or any other internet radio station you like.

Now if we refresh the page, we’ll see our audio element is there and we can start and stop it, but every time we navigate to a new page, the audio stops playing.

How do we make it persist?

Persisting the audio player across page turns

To make the audio persist across page turns, update the <audio> element like this:

<audio 
  data-turbo-permanent
  id="white-noise-player"
  controls
  src="http://uk1.internet-radio.com:8267/listen.pls&t=.m3u">
    Your browser does not support the
    <code>audio</code> element.
</audio>

We’ve done the impossible and made our audio element persistent across page turns by making two changes.

First, we added data-turbo-permanent to the element to tell Turbo that this element should persisted across page loads. Next, we added an id to the audio element, which Turbo uses to match the element when it renders pages.

With these small changes in place, we can refresh the page, start up the audio player, and then navigate around our application, clicking links and navigating through the browser history to our hearts content without ever stopping the audio player.

Magical.

Why write this article?

The problem we solved today — building a persistent audio player with Turbo(links) — has been solved for a long time.

Why take the time to write up something that’s so simple?

Because, as we saw in the tweet linked in the introduction, dangerous misconceptions exist in web development. These misconceptions are often spread by folks who want you to believe that their way is the One True Path.

Downstream of this bad information are developers researching problems who absorb the bad information and trust it because it comes from someone who has such a big platform that they must know what they’re talking about.

Some of those developers will then make decisions based on the bad information and justify their decisions in conversations with other developers by repeating the bad information until it is eventually picked up and presented as truth by a new flock of experts.

The bad information continues flowing, poisoning the conversation for years to come, turning into myths based on half-truths, misunderstandings, and overconfidence.

This is how myths like “Rails doesn’t scale” propagate and persist, when even a cursory amount of research disproves the idea.

Defending against myths

You can avoid falling for these myths by remembering that everyone writing about web development on the internet only knows a tiny fraction of all that there is to know about web development.

No one is an expert on every language and every method for building web applications. Instead, all of us, at best, know a lot about a few things, and a little about a lot more things.

We make decisions and recommendations based on that very limited knowledge, which means we can never confidently speak in absolutes. We can only share what we think is best, based on our own knowledge, for better or worse.

There is no One True Path in web development. Many different valid solutions exist, each with their own set of benefits and tradeoffs.

Fortunately, we don’t need One True Path. Many paths can lead to the same destination.

With this in mind, try to follow these rules to be a good consumer when you’re exposed to web development content:

Discard any absolute opinion about one technology or framework being “the best” way to solve any particular problem
Don’t listen to people who frequently speak in absolutes about languages or frameworks without contextualizing their opinion
When comparing languages or frameworks, anyone who says “[x] can’t be used for [y] you have to use [z] instead” is probably just trying to sell you on [z]
If someone refuses to acknowledge the weaknesses of their chosen approach, assume they’re not being honest
When someone is an expert on a tool, you can probably trust what they’re saying about that specific tool
If you don’t like a particular tool or framework there are probably a dozen other tools to solve the same problem that you can use instead

Wait… aren’t you presenting yourself as an expert? Why should I listen to you?

Fair. Since I am 1) on the internet and 2) talking about web development, I should share what my particular motivations and interests are so you can have that information as you consider this article.

I don’t have a big platform (~150 followers on dev.to and the same on Twitter, and ~2,000 people per month reading my blog). I write weekly about Ruby on Rails and the broader Rails ecosystem to a very small audience.

My writing is mostly aimed at helping newer Rails developers learn how to solve common UX problems that they’ll encounter as they work, and to share some of the modern tools available in the broader Rails ecosystem that enable performant, modern user experiences without needing to turn to a SPA.

My opinions on Rails are built from ~a decade of working on large production Rails applications and I consider myself to be Pretty Good at building Rails-powered web applications.

I encourage folks to consider Rails as an alternative to SPAs and JavaScript-everything not because I think JavaScript frameworks are bad, but because I think that Ruby and Rails are wonderful tools, and I’d like more developers to experience them.

That’s me — now you can decide whether to throw my thoughts on everything in the trash or not.

Wrapping up

Today we built a persistent audio player inside of a simple, multi-page application — from one perspective, an impossible task, from another, very simple.

We used this impossible/simple problem to talk a little about the myths of web development, and how experts with large platforms can spread those myths (often unintentionally!) to their audience. These myths then propagate through the community at large, and cause unnecessary, impossible to resolve arguments for years to come.

While we can’t stop experts with large platforms from sharing bad information, as developers, we can be informed about how we consume content. We can be more thoughtful about how we approach learning and growing when we know that absolutes rarely exist and objectively correct answers about which language or framework to use are few and far between in the world of web development.

As always, thanks for reading!

Building a Live Search Experience with StimulusReflex and Ruby on Rails

2021-08-28T00:00:00+00:00

As we approach the release of Rails 7, the Rails ecosystem is full of options to build modern web applications, fast. Over the last 9 months, I’ve written articles on building type-as-you-search interfaces with Stimulus and with the full Hotwire stack, exploring a few of the options available to Rails developers.

Today, we’re going to build a live search experience once more. This time with StimulusReflex, a “new way to craft modern, reactive web interface with Ruby on Rails”. StimulusReflex relies on WebSockets to pass events from the browser to Rails, and back again, and uses morphdom to make efficient updates on the client-side.

When we’re finished, our application will look like this:

It won’t win any beauty contests, but it will give us a chance to explore a few of the core concepts of StimulusReflex.

As we work, you will notice some conceptual similarities between StimulusReflex and Turbo Streams, but there are major differences between the two projects, and StimulusReflex brings functionality and options that don’t exist in Turbo.

Before we get started, this article will be most useful for folks who are comfortable with Ruby on Rails and who are new to StimulusReflex. If you prefer to skip ahead to the source for the finished project, you can find the full code that accompanies this article on Github.

Let’s dive in.

Setup

To get started, we’ll create a new Rails application, install StimulusReflex, and scaffold up a Player resource that users will be able to search.

From your terminal:

rails new stimulus-reflex-search -T
cd stimulus-reflex-search
bundle add stimulus_reflex
bundle exec rails stimulus_reflex:install
rails g scaffold Players name:string
rails db:migrate

In addition to the above, you’ll also need to have Redis installed and running in your development environment.

The stimulus_reflex:install task will be enough to get things working in development but you should review the installation documentation in detail ahead of any production deployment of a StimulusReflex application.

With the core of the application ready to go, start up your rails server and head to http://localhost:3000/players.

Create a few players in the UI or from the Rails console before moving on.

Search, with just Rails

We’ll start by adding the ability to search players without any StimulusReflex at all, just a normal search form that hits the existing players#index action.

To start, update players/index.html.erb as shown below:

<p id="notice"><%= notice %></p>

<h1>Players</h1>
<%= form_with(url: players_path, method: :get) do |form| %>
  <%= form.label :query, "Search by name:" %>
  <%= form.text_field :query, value: params[:query] %>
  <%= form.submit 'Search', name: nil %>
<% end %>
<table>
  <thead>
    <tr>
      <th>Name</th>
      <th colspan="3"></th>
    </tr>
  </thead>

  <%= render "players", players: @players %>
</table>

<br>

<%= link_to 'New Player', new_player_path %>

Here we’re rendering a search form at the top of the page and we’ve moved rendering players to a partial that doesn’t exist yet.

Create that partial next:

touch app/views/players/_players.html.erb

And fill it in with:

<tbody>
  <% players.each do |player| %>
    <tr>
      <td><%= player.name %></td>
      <td><%= link_to 'Show', player %></td>
      <td><%= link_to 'Edit', edit_player_path(player) %></td>
      <td><%= link_to 'Destroy', player, method: :delete, data: { confirm: 'Are you sure?' } %></td>
    </tr>
  <% end %>
</tbody>

Finally, we’ll add a very rudimentary search implementation to the index method in the PlayersController, like this:

def index
  @players = Player.where("name LIKE ?", "%#{params[:query]}%")
end

If you refresh /players now, you should be able to type in a search term, submit the search request to the server, and see the search applied when the index page reloads.

Now let’s start adding StimulusReflex, one layer at a time.

Creating a reflex

StimulusReflex is built around the concept of reflexes. A reflex, to oversimplify it a bit, is a Ruby class that responds to user interactions from the front end.

When you’re working with StimulusReflex, you’ll write a lot of reflexes that do work that might otherwise require controller actions and a lot of client-side JavaScript logic.

Reflexes can do a lot, from re-rendering an entire page on demand to kicking off background jobs, but for our purposes we’re going to create one reflex that handles user interactions with the search form we added in the last section.

Instead of a GET request to the players#index action, form submissions will call a method in the reflex class that processes the search request and updates the DOM with the results of the search.

We’ll start with generating the reflex using the built-in generator:

rails g stimulus_reflex PlayerSearch

This generator creates two files, a PlayerSearch reflex class in the reflexes directory and a player-search Stimulus controller in javascripts/controllers.

We’ll define the reflex in the PlayerSearch class, and then, optionally, we can use the player-search controller to trigger that reflex from a front end action and hook into reflex lifecycle methods that we might care about on the front end.

The simplest implementation of a working PlayerSearch reflex is to update an instance variable from the reflex, and then rely on a bit of StimulusReflex magic to do everything else. We’ll start with the magical version.

First, add a search method to the PlayerSearch reflex:

def search
  @players = Player.where('name LIKE ?', "%#{params[:query]}%")
end

Then update the search form to trigger the reflex on submit:

<%= form_with(method: :get, data: { reflex: "submit->PlayerSearch#search" }) do |form| %>
  <%= form.label :query, "Search by name:" %>
  <%= form.text_field :query, value: params[:query] %>
  <%= form.submit 'Search', name: nil %>
<% end %>

And update PlayersController#index to only assign a new value to @players when it hasn’t already been set by the reflex action.

def index
  @players ||= Player.all
end

With these changes in place, we can refresh the players page, submit a search, and see that searching works fine. So what’s going on here?

In the form, we’re listening for the submit event and, when it is triggered, the data-reflex attribute fires the search method that we defined in the PlayerSearch reflex class.

PlayerSearch.search automatically gets access the params from the nearest form so we can use params[:query] like we would in a controller action.

We use the query param to assign a value to @players and, because we haven’t told it to do anything else, the reflex finishes by processing PlayersController#index, passing the updated players instance variable along the way and using morphdom to update the content of the page as efficiently as possible.

So we can finish this article by deleting the form’s submit button and moving the reflex from the submit event on the form to the input event on the text field, right?

Not so fast.

While what we have “works”, our implementation is currently inefficient and hard to maintain and expand. Future developers will have to piece together what’s going on in search. We’re also re-rendering the entire HTML body even though we know that only a small part of the page actually needs to change.

We can do a little better.

Using Selector Morphs

The magical re-processing of the index action happens because the default behavior of a reflex is to trigger a full-page morph when a reflex method runs.

While page morphs are easy to work with, we can be more explicit about our intentions and more precise in our updates by using selector morphs.

Selector morphs are more efficient than page morphs because selector morphs skip routing, controller actions, and template rendering. Selector morphs are also more clear in their intention and easier to reason about since we know exactly what will change on the page when the reflex runs.

Full page morphs are powerful and simple to use, but my preference is to use selector morphs when the use case calls for updating small portions of the page.

Let’s replace the magical page morph with a selector morph.

First, as you might have guessed, selector morphs use an identifier to target their DOM changes. We’ll add an id to the <tbody> in the players partial to give the selector morph something to target.

<tbody id="players-list">
<!-- Snip -->
</tbody>

Next we’ll update the search form:

<%= form_with(url: players_path, method: :get) do |form| %>
  <%= form.label :query, "Search by name:" %>
  <%= form.text_field :query, value: params[:query], data: { controller: "player-search", action: "input->player-search#search" }  %>
<% end %>

Here we’ve scrapped the submit button and we’ve replaced the data-reflex on the submit button with a Stimulus controller directly on the query text field.

The player-search controller was created by the generator we ran earlier to create the PlayerSearch reflex, and we’ll fill in the Stimulus controller next:

import ApplicationController from './application_controller'

export default class extends ApplicationController {
  search() {
    this.stimulate('PlayerSearch#search')
  }
}

Here we’re inheriting from a Stimulus ApplicationController, which was automatically created by the stimulus_reflex:install task we ran at the beginning of this article. Since we’re inheriting from ApplicationController, we have access to this.stimulate, which we can use to trigger any reflex we like.

Why would we use a Stimulus controller instead of a data-reflex attribute on a DOM element?

Using a Stimulus controller gives us a little more flexibility and power than if we attach the reflex to the DOM directly, which we’ll explore in the next section.

Before we expand the Stimulus controller, let’s finish up the implementation of the selector morph by updating the PlayerSearch#search like this:

def search
  players = Player.where('name LIKE ?', "%#{params[:query]}%")
  morph '#players-list', render(partial: 'players/players', locals: { players: players })
end

Here we no longer need players to be an instance variable. Instead, we pass it in as a local to the players partial which the selector morph renders to replace the children of #players-list.

With this in place, we can refresh the page and start typing in the search form. If you’ve followed along so far, you should see that as you type, the content of the players table is updated.

If you check the server logs, you’ll see that instead of the controller action processing and the entire application layout re-rendering, the server only runs the database query to filter the players and then renders the players partial. Skipping routing and full page rendering dramatically reduces the amount of time and resources used to handle the request.

Expanding the Stimulus controller

Now we’ve got live search in place using a selector morph. Incredible work so far!

Let’s finish up by expanding the Stimulus controller to make the user experience a bit cleaner and learn a little more about StimulusReflex in the process.

First, searching on each keystroke isn’t ideal. Let’s adjust search to wait for the user to stop typing before calling the PlayerSearch reflex.

search() {
  clearTimeout(this.timeout)
  this.timeout = setTimeout(() => {
    this.stimulate('PlayerSearch#search')
  }, 200)
}

Nothing fancy here, and you should probably consider a more battle tested debounce function in production, but it’ll do for today.

Next, it would be nice to give the user a visual cue that the list of players has updated. One way to do that is to animate the list when it updates and StimulusReflex helpfully gives us an easy way to listen for and react to reflex life-cycle events.

beforeSearch() {
  this.playerList.animate(
    this.fadeIn,
    this.fadeInTiming
  )
}

get fadeIn() {
  return [
    { opacity: 0 },
    { opacity: 1 }
  ]
}

get fadeInTiming() {
  return { duration: 300 }
}

get playerList() {
  return document.getElementById('players-list')
}

Here we’re combining a custom StimulusReflex client-side life-cycle callback (beforeSearch) with the Web Animations API to add a simple fade effect to the players list each time it updates.

In addition to the client-side events, StimulusReflex provides server-side life-cycle callbacks, which we don’t have a use for in this particular article, but they exist if you need them.

Now we have visual feedback for users as they type. Let’s finish this article by allowing users to clear a search without having to backspace the input until its empty.

This last exercise will give us a chance to look at using more than one selector morph in a single reflex and to expand the Stimulus controller a bit more.

Resetting search results

Our goal is to add a link to the page that displays whenever the search text box isn’t empty. When a user clicks the link, the search box should be cleared, the players list should be updated to list all of the players in the database, and the reset link should be hidden.

We’ll start by adding a new partial to render the link:

touch app/views/players/_reset_link.html.erb

And fill that in with:

<% if query.present? %>
  <a href="#" data-action="click->player-search#reset">Clear search</a>
<% end %>

The reset link will only display if the local query variable is present. Clicks on the link are routed to a player-search Stimulus controller, calling the reset function that doesn’t exist yet.

Before we update the Stimulus controller, let’s adjust the index view, like this:

<p id="notice"><%= notice %></p>
<h1>Players</h1>
<div data-controller="player-search">
  <%= form_with(method: :get) do |form| %>
    <%= form.label :query, "Search by name:" %>
    <%= form.text_field :query, data: { action: "input->player-search#search" }, autocomplete: "off", value: params[:query] %>
  <% end %>
  <div id="reset-link">
    <%= render "reset_link", query: params[:query] %>
  </div>
  <table>
    <thead>
      <tr>
        <th>Name</th>
        <th colspan="3"></th>
      </tr>
    </thead>
    <%= render "players", players: @players %>
  </table>
</div>
<br>

<%= link_to 'New Player', new_player_path %>

Here we’ve inserted the a new reset_link partial, wrapped in a #reset-link div.

More importantly, we’ve adjusted how the player-search Stimulus controller is connected to the DOM. Instead of the controller being attached to the search text field, the controller is now on a wrapper div.

While we didn’t have to make this change to the controller connection, doing so makes it clear that the controller is interested in more than just the text input and opens up the possibility of using targets to more specifically reference DOM elements in the future.

This change also gives us an opportunity to look at one more piece of functionality of StimulusReflex-enabled functionality in Stimulus controllers.

Update the Stimulus controller like this:

search(event) {
  clearTimeout(this.timeout)
  this.timeout = setTimeout(() => {
    this.stimulate('PlayerSearch#search', event.target.value)
  }, 200)
}

reset(event) {
  event.preventDefault()
  this.stimulate('PlayerSearch#search')
}

We’ve made two important changes here.

First, since the Stimulus controller is no longer inside of the search form, the search reflex will no longer be able to reference params implicitly. We handle this change by passing the value of the search box to stimulate as an additional argument. Stimulate “is extremely flexible” and we take advantage of that flexibility to ensure the search reflex receives the search query even without access to the search form’s params.

Next, we added reset, which simply triggers the search reflex without an additional argument.

On the server side, we need to update PlayerSearch#search like this:

def search(query = '')
  players = Player.where('name LIKE ?', "%#{query}%")
  morph '#players-list', render(partial: 'players/players', locals: { players: players })
  morph '#reset-link', render(partial: 'players/reset_link', locals: { query: query })
end

Here we updated search to take an optional query argument. The value of query is used to set the value of players and then two selector morphs replace the content of players-list and reset-link.

In action, our final product looks like this:

An alternative approach

If you review the method signature of stimuluate, you’ll notice that we could have solved the problem of passing the value of the search box to the server in other ways.

Instead of passing in event.target.value, we could have passed event.target like this: this.stimulate(‘PlayerSearch#search, event.target).

This approach would override the default value of the server-side Reflex element, allowing us to call event.target.value to access the value of the search box from the server.

While this would work for the search function, it wouldn’t work for reset since we need to ignore the value of the search box when resetting the form. We could make it all work by passing an element to override the default element assignment, but it would take more effort.

Passing in the value explicitly allows us to use PlayerSearch#search to handle both search and reset requests and keeps our code a bit cleaner on the server side.

This is a matter of preference without a definitive answer on which approach is “best”. Implementing a solution overriding element on the server side would work fine. Also viable would be using an entirely different reflex action for the reset link.

StimulusReflex offers plenty of flexibility, and some choices will come down to what feels best to you and your team.

Wrapping up

Today we looked at implementing a simple search-as-you-type interface with Ruby on Rails and StimulusReflex. This simple example should give you some indication of the power StimulusReflex has to deliver modern, fast web applications while keeping code complexity low and developer happiness high.

Even better, StimulusReflex plays nicely with Turbo Drive and Turbo Frames, giving developers the ability to mix-and-match to choose the best tool for the job.

To keep learning about building Rails applications with StimulusReflex:

Dive into the (excellent, very well-maintained) official documentation
Check out demo applications demonstrating some core StimulusReflex concepts
Join the StimulusReflex discord to learn from lots of folks way sharper than me
Learn more advanced usage patterns with StimulusReflexPatterns from Julian Rubisch

As always, thanks for reading!

Building a Real Time Scoreboard with Ruby on Rails and CableReady

2021-08-18T00:00:00+00:00

The release of Hotwire in late 2020 brought attention to a growing interest within the Rails community in building modern, reactive Rails applications without needing the complexity of an API + SPA.

Although Hotwire’s Turbo library garnered a lot of attention, the Rails community has been working for years to improve the toolset we have to build modern, full-stack Rails applications. Turbo isn’t the first attempt at giving Rails developers the tools they need.

One of the most important of these projects is CableReady, which powers StimulusReflex and Optimism, along with standing on its own as a tool to:

Create great real-time user experiences by triggering client-side DOM changes, events and notifications over ActionCable web sockets

Source: CableReady Welcome

Today we’re going to explore CableReady by using Rails, CableReady, and Stimulus to build a scoreboard that updates for viewers in real-time, with just a few lines of Ruby and JavaScript.

When we’re finished, our scoreboard will look like this:

This article assumes that you’re comfortable working with Rails but you won’t need any prior knowledge of CableReady or ActionCable to follow along. If you’ve never used Rails before, this article isn’t the best place to start.

Let’s dive in!

Application Setup

First, let’s create our Rails application, pull in CableReady and Stimulus, and scaffold up a Game model that we’ll use to power our scoreboard.

rails new scoreboard_ready -T
cd scoreboard_ready
bundle add cable_ready
yarn add cable_ready
rails webpacker:install:stimulus
rails g scaffold Game home_team:string away_team:string home_team_score:integer away_team_score:integer
rails db:migrate

Although Redis is not technically required in development for CableReady, we’ll use it and hiredis to match the installation guidance from CableReady.

Update your Gemfile with these gems:

gem 'redis', '~> 4.0' # Uncomment this line, it should already be in your Gemfile
gem 'hiredis'

And then bundle install from your terminal.

Don’t have Redis installed in your development environment? Installing Redis on your machine is outside the scope of this article, but you can find instructions for Linux and OSX online.

Finally, we need to update our ActionCable configuration in config/cable.yml to use Redis in development:

development:
  adapter: redis
  url: <%= ENV.fetch("REDIS_URL") { "redis://localhost:6379/1" } %>
  channel_prefix: scoreboard_ready_development

test:
  adapter: test

production:
  adapter: redis
  url: <%= ENV.fetch("REDIS_URL") { "redis://localhost:6379/1" } %>
  channel_prefix: scoreboard_ready_production

With the application setup complete, we’ll build the basic layout for our scoreboard next.

Setup the scoreboard view

First, update the games show view:

<div style="text-align: center;">
  <div style="margin-top: 2rem;">
    <%= link_to 'Back to all games', games_path %>
  </div>
  <%= render "game_detail", game: @game %>
</div>

Here we’re inlining a couple of styles to make the scoreboard a little more legible and rendering a game_detail partial that doesn’t exist yet. Add that next, from your terminal:

touch app/views/games/_game_detail.html.erb

And fill it in with:

<div>
  <h1><%= "#{game.home_team} vs. #{game.away_team}" %></h1>
  <div style="color: gray; font-size: 1.2rem; margin-bottom: 0.8rem;">
    <%= "#{game.home_team}: #{game.home_team_score}" %>
  </div>
  <div style="color: gray; font-size: 1.2rem;">
    <%= "#{game.away_team}: #{game.away_team_score}" %>
  </div>
</div>

Some more inline styles (we’ll remove these later!) with some standard erb to render each team’s name and score.

At this point, we can go to localhost:3000/games and create a game, and then go to the game show page to view it.

We don’t have real-time updates in place yet, we’ll start building that with CableReady next.

Create channel and controller

Our first step to delivering real-time updates is to add a channel to broadcast updates on. When a user visits a game’s show page, they’ll be subscribed via a WebSocket connection to an ActionCable channel.

Without a channel subscription, the CableReady broadcasts we’ll be sending soon won’t be received.

To create a channel we can use the built-in generator:

rails g channel Game

This will create a few files for us. For the purposes of this article, we’re interested in the game_channel.rb file created in app/channels.

Open that file and update the subscribed method:

def subscribed
  stream_or_reject_for Game.find_by(id: params[:id])
end

The subscribed method is called each time a new Consumer connects to the channel.

In this method, we’re using the ActionCable method stream_or_reject_for to create a Stream that will send subscribed users broadcasts for a specific instance of a game, based on an id parameter.

When no game is found, the subscription request will be rejected.

With the channel built, next we need to allow consumers to subscribe to the channel so they can receive broadcasted updates.

The channel generator we ran automatically creates a file at javascripts/channels/game_channel.js that we could use to handle the subscription on the frontend; however, CableReady really shines when combined with Stimulus.

To do that, we’ll create a new Stimulus controller, from the terminal:

touch app/javascript/controllers/game_controller.js

And fill it in with:

import { Controller } from 'stimulus'
import CableReady from 'cable_ready'

export default class extends Controller {
  static values = { id: Number }

  connect() {
    this.channel = this.application.consumer.subscriptions.create(
      {
        channel: 'GameChannel',
        id: this.idValue
      },
      {
        received (data) { if (data.cableReady) CableReady.perform(data.operations) }
      }
    )
  }
  
  disconnect () {
    this.channel.unsubscribe()
  }
}

This Stimulus controller is very close to game_channel.js created by the channel generator, with a little Stimulus and CableReady power added.

Each time the Stimulus controller connects to the DOM, we create a new consumer subscription to the GameChannel, passing an id parameter. When the Stimulus controller disconnects from the DOM, the subscription is removed.

When a broadcast is received by the consumer, we use CableReady to perform the requested operations.

Before the Stimulus controller will work, we need to update app/javascript/controllers/index.js to import consumer.js (part of the ActionCable package) and attach consumer to the Stimulus Application object.

Update controllers/index.js with these two lines of code to accomplish that:

import consumer from '../channels/consumer'
application.consumer = consumer

Read more about why this is the right way to combine ActionCable and Stimulus here.

With our Stimulus controller built, we can update the game_detail partial to connect the controller to the DOM.

<div 
  id="<%= dom_id(game) %>"
  data-controller="game"
  data-game-id-value="<%= game.id %>"
>
  <h1><%= "#{game.home_team} vs. #{game.away_team}" %></h1>
  <div style="color: gray; font-size: 1.2rem; margin-bottom: 0.8rem;"><%= "#{game.home_team}: #{game.home_team_score}" %></div>
  <div style="color: gray; font-size: 1.2rem;"><%= "#{game.away_team}: #{game.away_team_score}" %></div>
</div>

Here we accomplished a lot with one change to the parent div:

We attached the Stimulus controller to the parent div
Set the id value that the Stimulus controller uses to send the id param in the channel subscription request
Set the id of the div to the dom_id of the rendered game instance. We’ll use this id in the CableReady broadcast we’ll generate in our model, up next.

With all of this in place, visit a game show page and check the Rails server logs. If everything is setup correctly, you should see log entries that look like this after the show page renders:

GameChannel is transmitting the subscription confirmation
GameChannel is streaming from game:Z2lkOi8vc2NvcmVib2FyZC1yZWFkeS9HYW1lLzE

Broadcast game updates from the model

With the channel built and consumers subscribing to updates, our last step to real-time scoreboard updates is sending a broadcast each time a game is updated.

The simplest way to do this is to broadcast a CableReady operation in an after_update callback in the Game model.

To make this possible, we first need to include the CableReady Broadcaster in our models and delegate calls to render to the ApplicationController, as described in the (excellent) CableReady documentation.

Update app/models/application_record.rb as follows:

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true
  include CableReady::Broadcaster
  delegate :render, to: :ApplicationController
end

And then update app/models/game.rb:

class Game < ApplicationRecord
  after_update do
    cable_ready[GameChannel].morph(
      selector: dom_id(self),
      html: render(partial: "games/game_detail", locals: { game: self})
    ).broadcast_to(self)
  end
end

Here we’ve added an after_update callback to trigger a CableReady broadcast. The broadcast is sent on the GameChannel, queuing up a morph operation targeting the current game instance, and rendering the existing game_detail partial.

With this callback in place, our scoreboard should now update in real-time.

You can test this yourself by heading to a game show page and then opening your Rails console and running something like Game.find(some_id).update(home_team_score: 100).

You should see the score update in the browser window immediately after submitting the update command in the Rails console.

While this works pretty well, our scoreboard really only needs to receive updates when the score changes, and it would be helpful to provide a little feedback to the user when the score changes.

Let’s finish up this article by updating our implementation to broadcast only on score changes, and to animate newly updated scores.

Getting fancier

To start, we’ve got some clunky inline styling that makes our erb code pretty hard to follow. Let’s move those styles out of the HTML and into a stylesheet. From your terminal:

mkdir app/javascript/stylesheets
touch app/javascript/stylesheets/application.scss

And add the below to application.scss:

.score-container {
  display: flex;
  color: gray;
  font-size: 1.2rem;
  margin-bottom: 0.8rem;
  justify-content: center;
}
.swing-in-top-fwd {
  -webkit-animation: swing-in-top-fwd 0.5s cubic-bezier(0.175, 0.885, 0.320, 1.275) both;
          animation: swing-in-top-fwd 0.5s cubic-bezier(0.175, 0.885, 0.320, 1.275) both;
}
@-webkit-keyframes swing-in-top-fwd {
  0% {
    -webkit-transform: rotateX(-100deg);
            transform: rotateX(-100deg);
    -webkit-transform-origin: top;
            transform-origin: top;
    opacity: 0;
  }
  100% {
    -webkit-transform: rotateX(0deg);
            transform: rotateX(0deg);
    -webkit-transform-origin: top;
            transform-origin: top;
    opacity: 1;
  }
}
@keyframes swing-in-top-fwd {
  0% {
    -webkit-transform: rotateX(-100deg);
            transform: rotateX(-100deg);
    -webkit-transform-origin: top;
            transform-origin: top;
    opacity: 0;
  }
  100% {
    -webkit-transform: rotateX(0deg);
            transform: rotateX(0deg);
    -webkit-transform-origin: top;
            transform-origin: top;
    opacity: 1;
  }
}

To animate the scores, we’re just using a simple CSS swing animation, copy/pasted directly from the always handy Animista.

Finally import that new stylesheet into the webpack bundle:

// app/javascripts/application.js
import "stylesheets/application"

We want to be able to update scores individually. To enable that, we’ll move the score portion of the scoreboard into a dedicated partial that we can then render in a broadcast.

From your terminal:

touch app/views/games/_score.html.erb

And fill that in with:

<div id="<%= "#{team}_score" %>" class="swing-in-top-fwd">
  <%= score %>
</div>

Then update the game_detail partial to remove the inline styles and to use our new score partial:

<div data-controller="game" 
  data-game-id-value="<%= game.id %>" 
  id="<%= dom_id(game) %>"
>
  <h1><%= "#{game.home_team} vs. #{game.away_team}" %></h1>
  <div class="score-container">
    <div>
      <%= game.home_team %>:
    </div>
    <%= render "score", 
      score: game.home_team_score, 
      team: "home" 
    %>
  </div>
  <div class="score-container">
    <div>
      <%= game.away_team %>:
    </div>
    <%= render "score", 
      score: game.away_team_score, 
      team: "away" 
    %>
  </div>
</div>

Finally, we’ll update the callback in the Game model:

class Game < ApplicationRecord
  after_update_commit { broadcast_changes }

  def broadcast_changes
    update_score(team: 'home') if saved_change_to_home_team_score? 
    update_score(team: 'away') if saved_change_to_away_team_score?
  end

  def update_score(team:)
    cable_ready[GameChannel].outer_html(
      selector: "##{team}_score",
      html: render(partial: 'games/score', locals: { score: send("#{team}_team_score"), team: team })
    ).broadcast_to(self)
  end
end

Here we’ve updated our callback to check for changes to the two attributes we care about (home_team_score and away_team_score). When either attribute is changed, a broadcast is triggered from update_score to replace the target div’s contents with the content of the score partial.

We use the outer_html CableReady operation in this case to completely replace the DOM content and ensure that our animation triggers when the new score content enters the DOM.

And with that in place, we can now see our isolated, animated real-time updates:

Wrapping up

Today we explored how to add CableReady and Stimulus onto the core Rails ActionCable package to enable real-time DOM manipulations without writing tons of JavaScript, worrying about client-side state management, or doing much outside of writing pretty standard Rails code. The complete source code for this demo application is on Github.

CableReady (and StimulusReflex, which we’ll explore in a future article) are mature, powerful tools that allow Rails developers to create modern, reactive applications while keeping development time and code complexity low. CableReady also plays well with most of Turbo, and can fit seamlessly into a Hotwire-powered application.

Start your journey deeper into CableReady and reactive Rails applications with these resources:

Read the CableReady documentation
Read the StimulusReflex documentation
Read the Hotwire documentation
Talk to the helpful folks on the StimulusReflex discord

As always, thanks for reading!

Conditional Rendering With Turbo Stream Broadcasts

2021-08-14T00:00:00+00:00

A very common pattern in Rails development is for a view to contain checks for things like current_user.can?(:take_some_action). These types of checks are common, especially in B2B applications that implement role-based permissions powered by a solution like Pundit.

So, naturally, when a Rails developer begins working with Turbo Stream broadcasts, they wonder how to access the current_user or other session-level variables. The short answer? You can’t.

Partials rendered via a stream broadcast have to be free of global variables or they’ll throw an error because they are not rendered within the context of a specific request. Without that request context variables like current_user will always be undefined.

While we can’t access global variables from a broadcast, we can, with creative use of Turbo Frames, still deliver real-time broadcasts that retain access to key variables like current_user.

To demonstrate the concept today we’re going to build a simple application that displays a list of Spies.

Each spy will have a mission; however, not everyone will be able to see the spy’s mission. Only visitors with secret clearance can see the mission field. Everyone else will see a “Classified” string in place of the mission.

When a new spy is created, we’ll broadcast an update to any viewers of the spy list. Viewers with clearance will see the mission for the newly created spy, viewers without won’t.

The end result will work like this:

This article will be most useful for folks who are already comfortable with Ruby on Rails and who have a little experience with Turbo already. If you’re comfortable with Rails but you’ve never used Turbo before, an article like this might be a better introduction.

As usual, you’ll find the full code for this article on GitHub.

Ready? Let’s get started.

Application Setup

Let’s get our application setup first, from your terminal:

rails new hotwire-spies -T
cd hotwire-spies
bundle add hotwire-rails
rails hotwire:install
rails g scaffold Spy name:string mission:string
rails db:migrate

Setup Frames

We’re going to make some updates to the Spy index view that our scaffold generated to incorporate a few Turbo Frames and prepare to use broadcasts to append new records.

Open spies/index.html.erb and update it:

  
<p id="notice"><%= notice %></p>

<h1>Spies</h1>

<%= turbo_stream_from "spies" %>
<div id="spies">
  <% @spies.each do |spy| %>
    <%= render spy, spy: spy %>
  <% end %>
</div>

<br>
<%= render "spy_link" %>

Here we’re subscribing to the spies turbo stream channel, setting an id on the container div for our rendered spies, and then looping through and rendering the spy partial for each spy.

The default scaffold generator doesn’t create a spy partial for us, so we need to create that:

touch app/views/spies/_spy.html.erb

And fill it in with:

  
<%= turbo_frame_tag spy do %>
  <div>
    <%= spy.name %> | Current mission: <%= spy.mission %>
  </div>
<% end %>

We’re also rendering a new spy_link partial, which doesn’t exist yet. Go ahead and create it now:

touch app/views/spies/_spy_link.html.erb

And fill that it in:

  
<%= turbo_frame_tag "new_spy" do %>
  <%= link_to 'New Spy', new_spy_path %>
<% end %>

Wrapping the new spy link in a dedicated frame means that when the link is clicked, we can replace the link on the page with the content returned from the call to /spies/new.

To make that work, we’ll need to update new.html.erb next to render its content in a matching <turbo-frame> tag:

  
<%= turbo_frame_tag "new_spy" do %>
  <%= render 'form', spy: @spy %>
<% end %>

In your browser, head to http://localhost:3000/spies and you should be able to click on the New Spy link and see the spy form render. Fill in a name and give your spy a mission, submit the form, and see that the form disappears but the spy doesn’t render on the page and the new spy link doesn’t come back.

Our spy creation form POST succeeds, but we aren’t sending back a useful response from the controller or broadcast any updates on the Spies channel.

Let’s finish up spy creation by tackling those issues next.

Handling spy creation

When our new spy form is submitted, a turbo stream request is sent to the server, and right now our server isn’t responding back with anything useful. Let’s fix that first by heading to the SpiesController and updating the create method:

  
def create
  @spy = Spy.new(spy_params)

  respond_to do |format|
    if @spy.save
      # Our addition
      format.turbo_stream { render turbo_stream: turbo_stream.replace("new_spy", partial: "spy_link")}
      format.html { redirect_to @spy, notice: "Spy was successfully created." }
    else
      format.html { render :new, status: :unprocessable_entity }
      format.json { render json: @spy.errors, status: :unprocessable_entity }
    end
  end
end
  

  

Now our create method will respond to turbo_stream requests by replacing the new_spy frame with the content in the spy_link partial.

Try creating a new spy now and, after a successful POST, the form should be removed from the page and the New Spy link should appear in its place.

Finally, we need to append the newly created spy to the list of spies without the user needing to refresh the page.

We’ll do that with a broadcast from the Spy model:

  
after_create_commit { broadcast_append_to('spies', target: 'spies', locals: { spy: self }) }

This callback runs after a new spy is created and sends a broadcast on the Spies channel containing a Turbo Stream element that looks like this <turbo-stream action="append" target="spies">. Nested within the stream element is the with the content of the spy partial

Now when we create a new spy, the new record should instantly be added to the list of spies, no refresh required.

With all of this in place, we can move to the really fun part of this article, conditionally rendering content for different users from a Turbo Stream broadcast.

Let’s see how it works.

Conditional rendering from a broadcast

First, let’s remember that our original goal was that each spy’s mission would only be displayed to users with secret clearance. Everyone else should just see “Classified” displayed in place of the mission text.

This is simple to do on the initial page load, but since we’re broadcasting new agent creation via a WebSocket connection, things are a little more complicated. Without access to the user’s session, we’ll have to get creative.

To meet our secret clearance requirement, we’re going to use a technique that I first saw described on the Hotwire discussion forums that allows us to bypass the need to access session-level variables in partials rendered by a broadcast.

We’ll implement this requirement by adding a check for a session variable into the spy partial that our broadcast renders. When the session variable exists, the user will see mission text, otherwise the user will not.

This session variable is a rough mock of what, in a real application, might be accessed through something like Current.user

To start, we’ll add a helper method to our ApplicationController to make it easy to access this session variable:

  
class ApplicationController < ActionController::Base
  helper_method :secret_clearance

  def secret_clearance
    session[:clearance].presence || false
  end
end

Then we’ll reference that helper method in spies/_spy.html.erb like this:

  
<%= turbo_frame_tag spy do %>
  <div>
    <%= spy.name %> | Current mission: <%= secret_clearance ? spy.mission : "Classified" %>
  </div>
<% end %>

We’ll need a way to set the value of clearance for different users. To do that, we’ll add a new endpoint to the SpiesController.

First, update our routes file:

  
resources :spies do
  collection do
    post 'clearance'
  end
end

Then in the controller, add the new clearance method:

  
def clearance
  secret_clearance ? session.delete(:clearance) : session[:clearance] = true
  redirect_to spies_path
end

Then, we’ll add a way to the set the clearance session variable from the index page in our extremely secure spy application.

  
Your current clearance is <%= secret_clearance ? "Secret" : "None" %>
<%= link_to "Change clearance", clearance_spies_path, method: :post %>

Before we move on, create a new spy from the Spies index page and see that secret clearance is always false when broadcasting through a channel, even when the current session has secret clearance set to true.

This is happening because the value of session when rendering from a broadcast is an empty hash. The session isn’t read from the user’s current session, and so our secret_clearance check will always fail.

To fix this issue and ensure users always see what they’re supposed to see, we’ll start by creating a new partial:

touch app/views/spies/_spy_frame.html.erb

Fill that new partial in with the following content:

  
<%= turbo_frame_tag spy, src: spy_path(spy, frame: true) %>

Here we’re taking advantage of the fact that Turbo Frames can be given an src attribute which causes the frame load its content from a URL when it is inserted onto the page.

In this case, the frame will retrieve the content from spies#show. We’ll update that method in the SpiesController next:

  
def show
  if request.headers["turbo-frame"]
    render partial: 'spy', locals: { spy: @spy }
  else
    render 'show'
  end
end
  

  

Our show method checks the request headers to see if the request includes a turbo-frame header. When the frame header is present, the existing spy partial is rendered.

Finally, update our model broadcast to render the new spy_frame partial:

  
after_create_commit { broadcast_append_to('spies', target: 'spies', partial: 'spies/spy_frame', locals: { agent: self }) }

With these changes in place, everything should work. To test it out, first make sure you’ve got two sessions active (open one in an incognito window). In one browser window, give the user clearance, in the other don’t.

Create a new spy in either session, and see that the broadcast triggers an update in both windows. In the window with clearance, the mission should display, in the window without clearance, Classified displays instead, like this:

So what’s going on here?

The broadcast sends the empty, src powered frame from the spy_frame partial to all subscribed users and that frame is rendered safely since it does not request any session variables
When the frame is inserted onto each user’s page, the src attribute on the frame immediately triggers an HTTP request to spies/:id
Since the request is sent from the browser to an endpoint on the server, we have access to all the session data we expect in the rendered response
The server responds to the request with the content of the spy partial which includes a <turbo-frame> with an id that matches the frame id of the empty frame in step 1.
Since the frame ids match, Turbo replaces the content of the frame with the response from the server, and the new spy appears in the list.

With this trick, we can now make use of Turbo Stream broadcasts in situations where the rendered content changes based on non-local variables. Magical.

Wrapping up

Today we looked at a potential solution to a very common blocker for folks building more complex applications with Hotwire.

Real-world use cases for this approach include conditionally rendering edit/delete controls based on user permissions, or not displaying certain information to users without appropriate permissions inside of a multi-user account.

Other recommended approaches to these types of problems typically rely on using inline styles to hide/show content but that is not an acceptable solution for all use cases, particularly in more sensitive applications where the data being included in the markup but hidden on the page isn’t acceptable.

Although this method works well for some use cases, it certainly isn’t without downsides or limitations, which include:

Each broadcast now requires an additional round trip to the server. We’re gaining a lot of flexibility in return for additional load on our server but we are adding load and potentially slowing down our application. Be sure you need the extra flexibility gained by this approach.
The complexity of your code is increased. Following what’s happening in a broadcast takes a little more effort, even for the simplest possible use case outlined in this article.
If you’re using this method to update content that already exists on the page, you might see flickering issues. There are ways to address this, including one described in the discussion on this approach on the Hotwire forum. Depending on your UX needs, you could also include an “updating content” loading state in the frame broadcast.

When considering taking this approach in a production application, think about the tradeoff you’re making. We learned about a hammer today, but that doesn’t mean every problem related to session variables and broadcasts is a nail.

Sometimes, you might be better off with another approach like:

Broadcasting a generic “refresh to see changes” message
Using inline styles to conditionally show/hide information
Adjusting the content rendered in a broadcast to eliminate the need to access session variables
Scoping your streams appropriately to ensure that every recipient of a particular broadcast should see what is being broadcast
Not broadcasting updates at all. Sometimes, real-time visibility into changes isn’t helpful or necessary and folks will be comfortable seeing the updated information on the next page load.

While not without tradeoffs and potential drawbacks, this technique demonstrates some of the power that can be unlocked with creative applications of Turbo Streams and Frames in Rails applications. With Turbo still in early days, we can expect to see even more powerful techniques developed in the coming years to continue to push Turbo and Rails applications powered by Turbo ahead.

As always, thanks for reading!

Turbo Streams on Rails

2021-08-09T00:00:00+00:00

Updated March 17th, 2024 to account for the release of refresh and morph Stream actions in Turbo 8

Turbo, part of the Hotwire tripartite, helps you build modern, performant web applications with dramatically less custom JavaScript.

Turbo is composed of Turbo Drive, Turbo Frames, Turbo Streams, and Strada. Each is a valuable piece of the puzzle but today we’re going to focus on Turbo Streams. Turbo Streams “deliver page changes over WebSocket, Server-Sent Events, or in response to form submissions” and they might be the most exciting part of the Turbo package.

Streams allow you to deliver fast and efficient user experiences with minimal effort and, for Rails developers, you’ll find that streams fit comfortably into your existing toolkit. With just a few lines of code, you can send tiny snippets of HTML over the wire and and use those snippets to update the DOM with no custom JavaScript.

For fancier use cases, you can broadcast changes over a WebSocket connection to keep every interested user updated in real-time. No custom JavaScript to write, just HTML and a little bit of Ruby.

Let’s dive in.

What’s a Turbo Stream?

At their core, streams allow you to send snippets of HTML to the browser to make small changes to the DOM in response to events that happen on the server.

The basic structure of a Turbo Stream is an HTML snippet that looks like this:

<turbo-stream action="action_to_take" target="element_to_update">
  <template>
    <div id="element_to_update">
      <!-- Some more html -->
    </div>
  </template>
</turbo-stream>

The <turbo-stream> element always includes an action, from the following possibilities:

append
prepend
before
after
replace
update
remove
morph
refresh

In addition to an action, a target (or targets) must be provided for all actions except for refresh. Refreshing and morphing are special cases introduced in Turbo 8, and you can read more about morphing and Turbo Stream refresh broadcasts in this article.

Inside of the stream element, the HTML to render gets wrapped in a <template> tag. Turbo’s JavaScript processes these HTML snippets by reading the action and target from the stream tag and inserting (or removing) the wrapped HTML as appropriate.

Turbo Streams can be sent in response to a direct request from a browser (like submitting a form) or broadcast to subscribers over a WebSocket connection.

With turbo-rails, we have the tools we need to take either path, and we’ll look at both methods in detail next.

Turbo Streams in controllers

The simplest way to get started with Turbo Streams is to have a controller respond to turbo_stream requests and render a turbo_stream response in return.

This approach looks something like this:

# players_controller.rb
def create
  @player = Player.new(player_params)

  respond_to do |format|
    if @player.save
      format.html { redirect_to @player, notice: "Player was successfully created." }
      format.turbo_stream
    else
      format.html { render :new, status: :unprocessable_entity }
    end
  end
end

# app/views/players/create.turbo_stream.erb
<%= turbo_stream.replace "players_form" do %>
  <%= render partial: "new_player" %>
<% end %>

In this example, a user is attempting to save a new player record in the database.

When the save is successful and the request format is turbo_stream, we use Rails’ implicit rendering to respond with create.turbo_stream.erb which renders a Turbo Stream response with a replace action targeting the players_form element.

The turbo_stream method in the create view generates the necessary <turbo-stream> tag and wraps the content within the turbo_stream block in a <template> tag.

The response back to the browser will look like this:

<turbo-stream action="replace" target="players_form">
  <template>
    <!-- Content of new_player.html.erb partial -->
    <turbo-frame id="players_form">
      <a href="/players/new">New Player</a>
    </turbo-frame>
  </template>
</turbo-stream>

Turbo’s JavaScript then steps in, processes the response, and updates the DOM. We do not need to make any adjustments to the failure path in our controller above. Even though the form submission is a turbo_stream request, Rails falls back to responding with HTML when a turbo_stream format to respond with does not exist.

Our example demonstrates handling a POST request with Turbo Streams, but we can use the same approach with a GET request, add data-turbo-stream to the link or GET form you want to send as a Turbo Stream request and Turbo will handle the rest.

Rendering inline

For simple responses we do not always need dedicated .erb files. Instead we can render the turbo_stream response inline in the controller, like this:

def create
  @player = Player.new(player_params)

  respond_to do |format|
    if @player.save
      format.html { redirect_to @player, notice: "Player was successfully created." }
      format.turbo_stream { render turbo_stream: turbo_stream.replace('players_form', partial: 'new_player') }

    else
      format.html { render :new, status: :unprocessable_entity }
    end
  end
end

There is no functional difference between inline rendering and rendering from a template, for simple actions you can use whichever feels better.

Updating multiple elements at once

The create.turbo_stream.erb file we saw earlier renders a single <turbo-stream> tag, but we aren’t limited to updating a single stream per response. We can update multiple elements at once by simply adding them to our turbo_stream.erb file:

<%= turbo_stream.replace "players_form" do %>
  <%= render partial: "new_player" %>
<% end %>
<%= turbo_stream.append "players" do %>
  <%= render partial: "player", locals: { player: @player } %>
<% end %>

Now after a successful form POST, we will send back a response with two <turbo-stream> tags, each of which will be processed by Turbo’s JavaScript on the front end.

If you really dislike creating turbo_stream templates, you can also render multiple Streams inline in the controller like this:

format.turbo_stream do
  # Pass an array of turbo_streams to the render call
  render turbo_stream: 
    [
      turbo_stream.replace('players_form', partial: 'new_player'),
      turbo_stream.append('players', partial: 'player', locals: { player: @player })
    ]
end

Should you do this? Probably not, but it works.

Broadcasting changes

So far, we have seen rendering streams from controller actions. This method works great for dealing with form submissions made by individual users; however, there is another method of broadcasting updates to a wider audience that you can call on when needed.

As we saw in the Hotwire introduction video, stream broadcasts sent from the controller only update the page that made the original request. If other people are viewing the same page, they won’t see the update until they refresh the page.

If your use case requires every interested user to see changes as they happen, you can use broadcasts from the turbo-rails gem to send streams to every subscriber at once, in real time.

The basic syntax for these updates looks like this, in your model:

# app/models/player.rb
after_create_commit { broadcast_append_to('players') }

Here we’re using a callback, fired each time a new player is created, to broadcast the newly created player to the players channel.

Broadcast options

By default, the Stream target will be set to the plural name of the model; however you can override the target as needed by passing in a target to the method, like this:

after_create_commit { broadcast_append_to('players', target: 'players_list') }

Broadcasting from the model will attempt to use to_partial_path to guess the name of the partial that should be rendered.

In our examples so far, if we have a partial in app/views/players named _player.html.erb that partial will be used. As with targeting, you can override the partial like this:

after_create_commit { broadcast_append_to('players', partial: 'a_special_player') }

When streaming in this manner, you must create a WebSocket connection to the channel you’re broadcasting updates on.

To do that, you can add <%= turbo_stream_from "some_channel" %> to the view. The name of the channel passed to turbo_stream_from must match the name of the channel passed to broadcast_action_to , otherwise your updates will get lost in space.

broadcast_action_to vs. broadcast_action_later_to

In the source code for broadcastable, a comment near the top of the file advises us to use broadcast_action_later_to instead of broadcast_action_to .

Using _later methods moves the stream broadcast into a background job, which means that broadcasts can happen asynchronously, moving the potentially expensive rendering work out of the web request.

Since we want everything to be fast, not slow, we’ll use broadcast_action_later_to, like this:

after_create_commit { broadcast_append_later_to('players') }

The only exception to the later rule is delete actions. broadcast_remove_to simply removes an element from the DOM without rendering a template and so does not need to be moved into a background job. To reinforce this exception, broadcast_remove_later_to is not defined and calling it will throw an undefined method error.

Broadcasting from a controller

So far we’ve seen that we can render a Turbo Stream response from a controller action and broadcast from a model.

Another, less common, path is to call broadcast_append|remove|replace on an instance of a model from within a controller (or a service, or anywhere else you like), like this:

@player.broadcast_append_later_to("players")

This does the same thing as calling this method in a callback in the model, meaning the append will broadcast to all users subscribed to the players channel.

More magical methods

So far we’ve been adding _to at the end of our broadcast methods and explicitly passing in the channel name. If you like magic, you can omit the _to from your broadcasts and just use broadcast_append|remove|replace.

Using this form requires a channel streaming from the model instance, like the below.

# player.rb
after_update_commit { broadcast_replace }

# _player.html.erb
<%= turbo_stream_from @player %>

To type even less, you can add broadcasts_to to your model, like this:

# player.rb
broadcasts_to ->(_) { 'players' }

Here we’re using a non-standard stream name of players. If we’re using the model instance as the stream name, we can get down to a single magic word:

# player.rb
broadcasts

Both broadcasts and broadcasts_to automatically add broadcasts on create, update, and delete commits to the model, as seen here

In practice, these methods often do not add much value since they rely so much on magical naming convention.

The magic fails when attempting to, for example, append a newly created record to an index list. In that scenario, our stream won’t be tied to an instance of the class, and so we’ll need to use the long form version of broadcast_action_to, like we saw above.

When in doubt, just use the longer, slightly less magical methods. I’m sharing these magic methods because they’re commonly used in guides and feature prominently in the announcement video so you’ll likely find them in the wild for years to come. If your use case allows you to use them, go for it, but the longer versions work just fine too.

Gotchas

Scope your channels!

In most of the examples I’ve shown so far, we are broadcasting to streams with hardcoded strings for names (“players”). This works fine; however, in most web applications resources aren’t globally available. Account 123 should only see updates on their account, not every account. You can avoid streaming updates to the wrong user by ensuring you’re broadcasting to properly scoped channels and subscribing users to those scoped channels.

In the players example code we’ve used so far, we can imagine that a player belongs to a team, and updates to those players should be broadcast on the team channel.

To implement this in our code, it might look like this:

# teams/show.html.erb
<%= turbo_stream_from @team %>

# player.rb
belongs_to :team

after_create_commit { broadcast_append_later_to(team) }
after_destroy_commit { broadcast_remove_to(team) }

With this in place, the team show page subscribes to updates related to that specific team, keeping data from leaking between different teams. The same concept can be applied to scope channels by user or account, as needed.

Note that this scoping is only necessary for broadcasts sent over WebSocket. If your controller renders a turbo_stream response, only the client that made the initial request will receive the update. There is no risk of data leaking in this manner when responding to a form submission with a turbo stream template.

Error handling

When you render a turbo_stream from a controller action and the DOM doesn’t have a matching target element to update… nothing happens. There’s no error anywhere, the stack trace indicates that the partial was rendered, and the DOM doesn’t update.

This is normal, expected behavior. There was not an error, there just wasn’t anything to change in the DOM - but it can be confusing when you’re getting started and the lack of errors can make troubleshooting more difficult.

If you are not getting errors, the turbo_stream response is showing in the server logs, and nothing is updating the page, the problem is (almost) always that the target passed to <turbo-stream> isn’t matching any elements in your page’s markup. Start by checking what you are rendering on the page and compare that to the target passed to the stream.

Wrapping up

Turbo is an incredibly powerful tool, and the tight integration between Rails and Turbo means that forward-looking Rails developers should be exploring ways to bring Turbo into their applications.

Using streams with turbo-rails opens up a world of possible user experiences for Rails developers that previously meant more work, more effort, and more code to maintain.

Hopefully what I’ve shared here helps you feel a little more comfortable thinking about streams, and how streams can fit into your existing code base. As always, thanks for reading! More resources Ready to dive deeper?

Spend some time with turbo-rails source code
Spend some time with the turbo source code
Dive into the docs
Read some practical guides, like this guide on building a live search interface, or this one on handling modal form submissions
GoRails has published a variety of Hotwire-specific guides

Finally, if you want to learn about implementing Turbo Streams in a complex Rails application, I wrote a book on building a modern Ruby on Rails application with Hotwire. The book is written in this same, step-by-step tutorial style that you find throughout this blog and aims to offer solutions for a variety of common, real-world web application problems without reaching for the weight of a full frontend framework.

Instant search with Rails and Hotwire

2021-08-05T00:00:00+00:00

Last year I wrote an article on building an instant search form with Rails and Stimulus. Since then, Turbo, the other half of Hotwire for the web has been released and the Hotwire stack is now the default for new applications in Rails 7.

Turbo opens the door for an even simpler, cleaner implementation of an instant search form.

So, today we’re taking another look at how to build a search-as-you-type interface that allows us to query our database for matches and update the UI with those matches (almost) instantly.

When we’re done, our code will look almost identical to what you’re used to writing to build Rails applications, and the only JavaScript we’ll need will be to trigger form submissions as the user types.

The finished product will work like this:

I’m writing this assuming that you’re comfortable with Ruby on Rails - you won’t need any knowledge of the Hotwire stack and you’ll find the most value from this article if you’re new to Turbo. While this article uses Stimulus, its core focus is on Turbo, and specifically how to use Turbo Frames in a Rails application.

You can find the complete code for this guide on Github. Follow along with this guide from the main branch, or see the complete implementation in the instant search branch.

Let’s get started.

Application Setup

First, we’ll need an application with both Turbo and Stimulus installed and ready to go. If you’ve already got an application setup, feel free to use that instead.

To follow along line-by-line, start by running these commands from your terminal. Your machine will need to be setup for standard Rails development, with Ruby and Rails. For reference, I’m writing this guide using Rails 7 and Turbo 7.1.

  
rails new hotwire-search -T && cd hotwire-search
rails g scaffold Player name:string
rails db:migrate
rails s

If you’re using Rails 6 or working from an existing application, you’ll also need to install Turbo and Stimulus. The simplest way to do this is to use the turbo-rails and stimulus-rails gems.

With the above commands run, we should have a new rails application up and running, and visiting https://localhost:3000/players should open a page that lists the players in your database and includes a link to create new players.

Go ahead and add a few new players from the UI, or open up your Rails console and add some players there.

Give them different names since we’ll be filtering the list of players by name in this project.

With setup complete, open up the code in your favorite editor.

We’ll start by updating the players index view to add a form that we’ll use to filter the list of players.

Update the players index view to add the form:

<p style="color: green"><%= notice %></p>

<h1>Players</h1>
<%= form_with(url: players_path, method: "GET") do |form| %>
  <%= form.label :query, "Search by name:" %>
  <%= form.text_field :query %>
  <%= form.submit 'Search', name: nil %>
<% end %>
<div id="players">
  <% @players.each do |player| %>
    <%= render player %>
    <p>
      <%= link_to "Show this player", player %>
    </p>
  <% end %>
</div>

<%= link_to "New player", new_player_path %>

Next update the index action in the PlayersController to process search form submissions:

def index
  if params[:query].present?
    @players = Player.where("name LIKE ?", "%#{params[:query]}%")
  else
    @players = Player.all
  end
end

At this point, our search form works — submit it and the page will reload with the list of players filtered to match your search query; however, each form submission has to be manually triggered and the entire page reloads instead of only rendering the new list of players. We won’t win any awards for the elegance of our code, but it works just fine and will serve as a simple base for us to build from.

Let’s start with adding rendering our search results with Turbo Frames first, and then we’ll wrap up by adding a Stimulus controller to submit the form automatically as the user types.

Adding Turbo Frames

Our goal is to use a Turbo Frame to update only the content of the list of players each time the form is submitted. To do this, we’ll first need to wrap the content we want to update in a <turbo-frame> HTML tag.

We can use the turbo_frame_tag helper method from turbo-rails for this.

We’ll first create players partial that renders the list of players wrapped in a turbo frame.

From your terminal:

touch app/views/players/_players.html.erb

And then fill that in with:

<%= turbo_frame_tag "players" do %>
  <%= render players %>
<% end %>

Here we’re rendering a collection of players, wrapped inside of a turbo frame tag.

Next update the players index to render the partial:

<p style="color: green"><%= notice %></p>

<h1>Players</h1>
<%= form_with(url: players_path, method: "GET") do |form| %>
  <%= form.label :query, "Search by name:" %>
  <%= form.text_field :query %>
  <%= form.submit 'Search', name: nil %>
<% end %>
<!-- Rendering the new partial here -->
<%= render "players", players: @players %>

<%= link_to "New player", new_player_path %>

Finally, update the player partial:

<div id="<%= dom_id player %>">
  <p>
    <%= player.name %> <%= link_to "View Player", player_path(player) %>
  </p>
</div>

With the turbo frame in place, refresh the page and see that it looks identical to what we had before but in the markup you’ll see a <turbo-frame> element wrapping the list of players:

<turbo-frame id="players">
	<!-- Snip player divs -->
</turbo-frame>

Now we’re wrapping our players in a Turbo Frame, but we’re not making use of that Turbo Frame on the server.

Rendering a Frame from the server

To do that, update the index action in the players controller to check the inbound request.

When the request targets a Turbo Frame, we’ll respond with just the Turbo Frame we want to update, otherwise we’ll render the full page:

def index
  if params[:query].present?
    @players = Player.where("name like ?", "%#{params[:query]}%")
  else
    @players = Player.all
  end
  if turbo_frame_request?
    render partial: "players", locals: { players: @players }
  else
    render "index"
  end
end

This a bit a clunky (we’ll clean it up very soon!) but it demonstrates an important piece of how Turbo Frames work with Rails.

We’re using the built-in turbo_frame_request? helper to check for a specific turbo-frame header in the request from the browser.

When this header is present, our index action renders a partial that contains a matching Turbo Frame. Otherwise, we render the index page as normal.

Finally, we need to update our search form to tell Turbo that we want to target the players Turbo Frame we just added:

<%= form_with(url: players_path, method: "GET", data: { turbo_frame: "players" }) do |form| %>
  <!-- Snip the form content -->
<% end %>

Here we added the turbo-frame data attribute. When using Turbo Frames, we can add this data attribute to tell Turbo that the response to the form submission (or link click) should target the given Turbo Frame.

This extra effort is necessary because the search form and the list of players are not wrapped inside of a single Turbo Frame.

When a form submission or link click occurs inside of a <turbo-frame> element, Turbo will automatically attempts to retrieve and replace that frame’s content without any extra work from us.

While we could try wrapping the form inside of the players Turbo Frame, doing so is undesirable. If a form is inside of a Turbo Frame, each time that frame is re-rendered, form inputs will lose their current values and any focus will be lost — not an ideal user experience.

Since we don’t want to wrap the form inside of the frame that contains our list of players, we need to explicitly tell Turbo that our form submission should update the players Turbo Frame.

Cleaning up the Frame request

The if turbo_frame_request? logic in the controller action is a bit messy. Thankfully we can clean this up a bit.

First, create a new turbo_frame view:

touch app/views/players/index.html+turbo_frame.erb

And fill that view in with:

<%= turbo_frame_tag "players" do %>
  <%= render @players %>
<% end %>

Then update your ApplicationController to automatically look for turbo_frame views when a Turbo Frame request is received:

class ApplicationController < ActionController::Base
  before_action :turbo_frame_request_variant
  private

  def turbo_frame_request_variant
    request.variant = :turbo_frame if turbo_frame_request?
  end
end

Here we’re using the same turbo_frame_request? helper to set request.variant on all inbound requests that contain a turbo-frame header.

With that update to the ApplicationController, we can simplify the index action in the PlayersController to:

def index
  if params[:query].present?
    @players = Player.where("name like ?", "%#{params[:query]}%")
  else
    @players = Player.all
  end
end

Since the ApplicationController automatically sets the variant for us, individual controller actions no longer need to care about whether the request is for a Turbo Frame or not and Rails will invisibly render the right view for us.

With these changes in place, refresh the page, type in a search query and see that the page content updates as before… so how do we know it is working?

Check the server logs and see that when a form submission hits the server, Rails responds with players partial content without re-rendering the application layout or the index view.

If things are set up correctly, you should see something like this:

Processing by PlayersController#index as HTML
  Parameters: {"query"=>"Dirk"}
   (0.1ms)  SELECT sqlite_version(*)
  ↳ app/controllers/players_controller.rb:7:in `index'
  Rendering players/index.html+turbo_frame.erb
  Player Load (0.1ms)  SELECT "players".* FROM "players" WHERE (name like '%Dirk%')
  ↳ app/views/players/index.html+turbo_frame.erb:2
  Rendered collection of players/_player.html.erb [1 times] (Duration: 0.2ms | Allocations: 65)
  Rendered players/index.html+turbo_frame.erb (Duration: 2.1ms | Allocations: 922)
Completed 200 OK in 5ms (Views: 2.4ms | ActiveRecord: 0.4ms | Allocations: 1903

Notice the rendering players/index.html+turbo_frame.erb in the logs which tells us everything is working as expected.

Updating the URL on form submission

A requirement for many search and filter experiences is updating the URL as the user makes selections so that they can bookmark or copy and paste a particular set of filters. While this isn’t really useful for our search form use case, it is important to know that Turbo makes it trivial to update the URL on a Turbo Frame request.

To do this, we can update our form like this:

<%= form_with(url: players_path, method: "GET", data: { turbo_frame: "players", turbo_action: "advance" }) do |form| %>
  <!-- Snip -->
<% end %>

Here we added the turbo-action data attribute, giving it a value of advance. This attribute tells Turbo to push the frame navigation into the browser’s history, updating the page’s URL in the process. This option is outlined in the Turbo documentation here and the PR that added the functionality provides full details on how it works.

Once this change is made, refresh the page, type in a search query and see that the URL updates in your browser. You can then copy/paste that URL into a new window and see that searching works during a full page visit as well (just remember to set the value of the input to the params[:query] value if you go this route).

Magical stuff.

Automatic form submission

With our Turbo Frame powered form we already have a nice, efficient search experience in our application; however, it would be even better if the list of players could be updated as the user types.

To do this, we’ll add a small Stimulus controller that automatically submits the form as the user types.

To get started, generate the Stimulus controller with the built-in generator:

rails g stimulus form-submission

Then fill that controller (located at app/javascript/controllers/form_submission_controller.js) with the following:

import { Controller } from "@hotwired/stimulus"

export default class extends Controller {

  search() {
    clearTimeout(this.timeout)
    this.timeout = setTimeout(() => {
      this.element.requestSubmit()
    }, 200)
  }
}

Here our search function waits for 200ms before calling requestSubmit() on the element the controller is attached to in the DOM. requestSubmit() requires a polyfill to work on Safari but Turbo helpfully includes that polyfill for us.

Next we’ll update the DOM to attach the Stimulus controller to the search form:

<%= form_with(url: players_path, method: "GET", data: { controller: "form-submission", turbo_frame: "players", turbo_action: "advance" }) do |form| %>
  <%= form.label :query, "Search by name:" %>
  <%= form.text_field :query, data: { action: "input->form-submission#search" } %>
<% end %>

Here we added two data attributes, one data-controller attribute to <form> element, connecting our FormSubmission controller. The other is a data-action attribute, telling Stimulus to call form-submission#search on each input event emitted by the form’s text field.

With this in place, we can refresh the page and see that the form submits as we type, with a short delay to let the user pause typing before submitting the form.

Here’s what the final experience looks like if you’ve been following along from start:

Wrapping up

This small example should help you get a taste for how simple it can be to build modern, highly-responsive applications with Ruby on Rails and the Hotwire stack, combining the feel of a SPA with the developer experience that the Rails world (usually) loves.

To use this in the real world, we’d need to think about things like:

An empty state: The list partial could render different content when players is empty
Cleaner, more performant database queries: Definitely don’t just leave your query sitting in the controller! For production use cases, you’d want to consider an option like pg_search

For a deeper dive into Turbo Frames, you might enjoy my article dedicated to Turbo Frames. To learn about the other key component of Turbo, Turbo Streams, you can dig into my Turbo Streams on Rails article. To explore the full power of Turbo, the official Handbook and Reference docs are a good starting place.

While this example is not yet production-ready, I hope it gives you a good starting point into the world of Turbo & Stimulus-powered Rails applications. The next time you’re building something new, consider whether Rails + the Hotwire Stack might be enough to deliver the experience your users expect while keeping your team happy and productive.

As always, thanks for reading!

Handling modal forms with Rails, Tailwind CSS, and Hotwire

2021-07-15T00:00:00+00:00

September 2022 update: Since this article was written, Rails 7 and Turbo 7.2 have been released. There are new, simpler techniques to building modal forms with Turbo now. I published an updated version of this article after the release of Turbo 7.2, I recommend reading that article instead.

The release of the Hotwire package of Stimulus and Turbo has brought a tremendously powerful set of tools to Rails developers who want to bring the speed of a single page application to their users without sacrificing the development speed and happiness that Rails is known for.

Today we are using Hotwire, Rails, and Tailwind CSS to build a modal form submission flow.

With just a little bit of Stimulus and Turbo, we will create a user experience that opens a modal on click, retrieves modal content remotely, and cleanly handles successful form submission and form errors.

This modal flow will help us learn a few of the core concepts of Hotwire while demonstrating how to implement a very common user experience in modern web development.

The finished product will look something like this:

I am writing this guide assuming that you are comfortable with Ruby on Rails and that you have some level of familiarity with Stimulus. You do not need to be an expert on either.

While we use Tailwind CSS to make our modal implementation simple, you do not need experience with Tailwind. If you prefer, you can implement your own, non-Tailwind modal and still get value from this guide.

You can find the complete code for this project on Github.

Let’s get started.

Project setup

We are going to start with a fresh Rails 6.1 application. If you prefer, you can follow along in an existing Rails 6 application, just be sure you’ve got Hotwire and Tailwind 2 installed.

If you want to skip the setup steps and don’t have a repo handy, clone down this repository, bundle install, and you’re good to go. If you take this route, jump ahead to the setting up comments section below.

To start from scratch, first open your terminal and run these commands:

rails new hotwire_modals -T
cd hotwire_modals
bundle add hotwire-rails
rails hotwire:install

Next, we will setup Tailwind CSS.

In your terminal, run:

yarn add tailwindcss@latest postcss@latest autoprefixer@latest @tailwindcss/forms
npx tailwind init

Then update postcss.config.js:

  
module.exports = {
  plugins: [
    require("tailwindcss")("./tailwind.config.js"),
    require("postcss-import"),
    require("postcss-flexbugs-fixes"),
    require("postcss-preset-env")({
      autoprefixer: {
        flexbox: "no-2009",
      },
      stage: 3,
    }),
  ],
}
  

  

And update tailwind.config.js:

  
module.exports = {
  purge: [
    './app/**/*/*.html.erb',
    './app/helpers/**/*/*.rb',
    './app/javascript/**/*/*.js',
  ],
  darkMode: false, // or 'media' or 'class'
  theme: {
    extend: {},
  },
  variants: {
    extend: {},
  },
  plugins: [
    require('@tailwindcss/forms')
  ],
}
  

  

Next we’ll import Tailwind into the application. In your terminal run:

mkdir app/javascript/stylesheets
touch app/javascript/stylesheets/application.scss

And update the newly created application.scss file to import tailwind:

  
@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";

Next, update your existing application.js file to import application.scss:

  
import Rails from "@rails/ujs"
import "@hotwired/turbo-rails"
import * as ActiveStorage from "@rails/activestorage"
import "channels"

// This is the line we're adding
import "stylesheets/application"

Rails.start()
ActiveStorage.start()

import "controllers"

That’s a lot of copy and pasting, hang in there. One more to go before we get to the fun stuff.

Update app/views/layouts/application.html.erb with the below:

  
<!DOCTYPE html>
<html>
  <head>
    <title>HotwireModals</title>
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>

    <%= stylesheet_pack_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
    <%= javascript_pack_tag 'application', 'data-turbolinks-track': 'reload' %>
  </head>

  <body class="max-w-7xl mx-auto mt-8">
    <%= yield %>
  </body>
</html>
  

  

Perfect. Next we will scaffold up a Comment resource, which we’ll use to build our modal form submission flow.

Setting up comments

First, generate a Comment resource. In your terminal:

rails g scaffold Comment body:string
rails db:migrate
touch app/views/comments/_comment.html.erb

Then update the comments index view with a simple list view layout:

  
<!-- app/view/comments/index.html.erb -->
<div class="max-w-3xl mx-auto mt-8">
  <div class="flex justify-between items-baseline mb-6">
    <h1 class="text-3xl text-gray-900">Comments</h1>
    <%= link_to 'New Comment', new_comment_path, class: "text-blue-600" %>
  </div>
  <div class="flex flex-col items-baseline space-y-6 p-4 shadow-lg rounded">
    <% @comments.each do |comment| %>
      <%= render "comment", comment: comment %>
    <% end %>
  </div>
</div>
  

  

The index view loops through the comments and renders the _comment partial for each one. Let’s fill that in with the below:

  
<!-- app/view/comments/index.html.erb -->
<div class="text-gray-700 border-b border-gray-200 w-full pb-2">
  <%= comment.body %>
</div>

At this point, boot up your Rails server and head to http://localhost:3000/comments. You should see a layout that looks something like this.

Go ahead and create a few comments to test things out.

Once you confirm everything looks right, head on to the next section where we will add a modal to the replace the standalone New view that the scaffold generator provided.

Add Tailwind Stimulus Components

Rather than reinventing the wheel and writing a Stimulus controller to open and close our modal, we’re going to use Tailwind CSS Stimulus Components. This project provides easy-to-use, production-ready components for a variety of common front-end user interactions built with Stimulus and Tailwind, including a ready-to-use modal component.

One of the joys of Ruby on Rails development is the incredibly robust ecosystem of community-built libraries that help reduce the effort that goes in to building Rails applications.

While the Stimulus community is still growing, projects like Tailwind Stimulus Components are already making it easier to build Stimulus-powered frontend experiences.

First add tailwind-stimulus-components to your application. From your terminal:

yarn add tailwindcss-stimulus-components

Then import and register the Modal component in app/javascript/controllers/index.js:

  
import { Modal } from "tailwindcss-stimulus-components"
application.register('modal', Modal)

While we are going to use the basic behavior of the Modal component, for our purposes we’ll need to add a little bit of extra functionality to the modal controller. We can do this by extending the Modal controller.

First, in your terminal:

touch app/javascript/controllers/extended_modal_controller.js

Then update the newly created controller as shown below:

  
import { Modal } from "tailwindcss-stimulus-components"

export default class ExtendedModal extends Modal {
  static targets = ["form"]

  connect() {
    super.connect()
  }
}

With our ExtendedModal controller created, we are ready to start building our Hotwire-powered modal interface.

First, create a partial to hold the modal content. From your terminal:

touch app/views/comments/_modal.html.erb

And fill that partial in with the below:

  
<div data-extended-modal-target="container" class="hidden animated fadeIn fixed inset-0 overflow-y-auto flex items-center justify-center" style="z-index: 9999;">
  <div class="max-w-lg max-h-screen w-full relative">
    <div class="m-1 bg-white rounded shadow">
      <div class="px-4 py-5 border-b border-gray-200 sm:px-6">
        <h3 class="text-lg leading-6 font-medium text-gray-900">
          Comment
        </h3>
      </div>
      <form id="comment_form"></form>
    </div>
  </div>
</div>
  

  

This partial contains the body of the modal.

The style is mostly copied from the tailwind-stimulus-components documentation, the most important parts for our project are setting the data-extended-modal-target, which is required for the modal to open and the empty <form id="comment_form"> tag. This empty tag serves as a placeholder for the comments form content that will be populated when the modal opens.

Next up, we’ll update the comments index view to connect our modal controller to the DOM:

  
<!-- comments/index.html.erb -->

<%= turbo_frame_tag 'comment_modal' %>

<div class="max-w-3xl mx-auto mt-8">
  <div class="flex justify-between items-baseline mb-6" data-controller="extended-modal" data-extended-modal-prevent-default-action-opening="false">
    <%= render 'modal' %>
    <h1 class="text-3xl text-gray-900">Comments</h1>
    <%= link_to 'New Comment', new_comment_path, class: "text-blue-600", data: { action: "click->extended-modal#open", 'turbo-frame': 'comment_modal' } %>
  </div>
  <%= turbo_frame_tag "comments", class: "flex flex-col items-baseline space-y-6 p-4 shadow-lg rounded" do %>
    <% @comments.each do |comment| %>
      <%= render "comment", comment: comment %>
    <% end %>
  <% end %>
</div>
  

  

Now we are into the meat of the project. Let’s walk through what we changed here.

First, we render an empty <turbo_frame>. We’ll populate this frame with the content of the modal when the New Comment link is clicked.

Next, we connect our controller to the dom with data-controller="extended-modal" and override the default behavior of the modal controller with prevent-default-action-opening="false" which tells our modal controller to allow the link_to action to run as normal.

The New Comment link_to which has two data attributes. The first is a standard Stimulus action attribute which fires the open function of our modal controller on click.

The second is a Turbo attribute which tells Turbo to render the response from the server in the comment_modal <turbo_frame> that we added at the top of the view. Without this data attribute, Turbo would not know where to place the response from comments/new and clicking on the New Comment link would just render an empty modal.

The basic flow of our modal is this:

User clicks on New Comment link
The click fires the extended-modal#open function which opens the modal
Because we told the open function to allow the default action to be followed, a request is sent to comments/new
When a response is received from comments/new, Turbo steps in and attempts to insert the response into the comment_modal turbo frame

Finally, we replace our comments wrapper div with a “comments” turbo_frame_tag. We will use this frame later to insert new comments after a successful form submission.

Fancy stuff, but we’re not quite there yet.

Next we need to update the new comments view and the comment form partial in order for the modal to properly render the comment form after it opens.

Let’s start with updating the new view:

  
<!-- comments/new.html.erb -->

<%= turbo_frame_tag 'comment_modal' do %>
  <turbo-stream target="comment_form" action="replace">
    <template>
      <%= render partial: "form", locals: { comment: @comment } %>
    </template>
  </turbo-stream>
<% end %>
  

  

Here we’re rendering a comment_modal turbo_frame tag and inside of that rendering a turbo-stream that looks for an element with an id of comment_form and replaces it.

Next, the form partial:

  
<%= form_with(model: @comment, id: "comment_form", data: { extended_modal_target: 'form' }) do |form| %>
  <div class="px-4 pt-5 pb-4 sm:p-6 sm:pb-4">
    <% if @comment.errors.any? %>
      <div class="p-4 border border-red-600">
        <ul>
          <% @comment.errors.full_messages.each do |message| %>
            <li><%= message %></li>
          <% end %>
        </ul>
      </div>
    <% end %>

    <div class="form-group">
      <%= form.label :body %>
      <%= form.text_field :body %>
    </div>

  </div>
  <div class="rounded-b mt-6 px-4 sm:grid sm:grid-cols-2 sm:gap-3 sm:grid-flow-row-dense">
    <%= form.button class: "w-full sm:col-start-2 bg-blue-600 px-4 py-2 mb-4 text-white rounded-sm hover:bg-blue-700" %>
    <button data-action="click->extended-modal#close" class="mt-3 w-full sm:mt-0 sm:col-start-1 mb-4 bg-gray-100 hover:bg-gray-200 rounded-sm px-4 py-2">
      Cancel
    </button>
  </div>
<% end %>
  

  

The form is almost entirely standard Rails form_with content. The important bits are passed in to the form_with. First we set the id of the form to comment_form. Recall that the turbo-stream in comments/new is looking for an element with that id.

Next, we add a data-extended-modal-target attribute of “form”. We will use this target attribute later to reset the form after a successful submission.

With those views in place, let’s update the comments controller to send turbo_stream responses when the posts#create action is called:

  
# app/controllers/comments_controller.rb
# Snipped content
def create
  @comment = Comment.new(comment_params)

  if @comment.save
    render turbo_stream: turbo_stream.append(
      'comments',
      partial: 'comment',
      locals: {
        comment: @comment
      }
    )
  else
  # TODO: Handle errors
  end
end
# Snipped content
  

  

Here’s where the turbo_stream magic happens.

When our comment form is submitted, the create action is called and, if the comment saves, the server sends a turbo_stream response that appends the comment partial to a comments turbo_frame. Note that we do not need a comments.turbo_stream.erb partial, the existing html comments partial we have gets used automatically.

The final step to properly handling our form submission is to close the modal after the form is submitted successfully. To do this, we will add a small function to the modal Stimulus controller and then attach an event listener to the DOM.

First, add this function to app/javascript/controllers/extended_modal_controller.js

  
handleSuccess({ detail: { success } }) {
  if (success) {
    super.close()
    this.formTarget.reset()
  }
}
  

  

handleSuccess simply checks a boolean success value and, if success is true, we close the modal and reset the form. If success is false (meaning the form submission failed), our Stimulus controller does nothing.

Finally, to ensure handleSuccess is called after the form is submitted, update the modal partial as shown:

  
<div data-extended-modal-target="container" 
     data-action="turbo:submit-end->extended-modal#handleSuccess"
     class="hidden animated fadeIn fixed inset-0 overflow-y-auto flex items-center justify-center"
     style="z-index: 9999;">
<!-- Snip inner content -->
</div>
  

  

Here we’ve added a data-action attribute listening for the turbo:submit-end event. When that event fires, our modal controller’s handleSuccess function is called, allowing our modal to close when the form is submitted successfully.

Whew. At this point, if you’ve been following along with each step, you should be able to load up the comments index page, click the New Comment link, type in a comment and submit it successfully. If everything is working, the new comment link will open a modal, and submitting the modal’s form will close the modal and append the new comment onto the comment list.

Handling form errors

To wrap up, we’ll update our CommentsController to handle errors. This’ll be quick.

First, update the comments model to validate that body is present so that it is possible to have an invalid comment submission.

  
class Comment < ApplicationRecord
  validates_presence_of :body
end

Then head to the CommentsController and update the create method as shown:

  
def create
  @comment = Comment.new(comment_params)

  if @comment.save
    render turbo_stream: turbo_stream.append(
      'comments',
      partial: 'comment',
      locals: {
        comment: @comment
      }
    )
  else
    render turbo_stream: turbo_stream.replace(
      'comment_form',
      partial: 'form',
      locals: {
        comment: @comment
      }
    ), status: :unprocessable_entity
  end
end
  

  

In the else block, we’re again rendering a turbo_stream response. This time, we target the comment_form turbo_frame and replace the content of frame instead of appending. We also send along a 422 status code so that the turbo:submit-end properly sets success to false. Otherwise, the default status code of 200 would cause our modal to close.

With this error handling in place, submitting the modal form without typing anything into the body field will cause the form to rerender with the errors listed above the form, like this:

As our very last step, thanks to Turbo’s caching, an edge-case UI error can happen now that we have error handling in place.

If a user attempts to submit the form with a blank body, and then corrects their error and successfully submits the form, clicking the New Comment link again will open the form with the error section still present on the modal.

We can clear that up with a small addition to our modal Stimulus controller:

  
import { Modal } from "tailwindcss-stimulus-components"

export default class ExtendedModal extends Modal {
  static targets = ["form", "errors"]

  connect() {
    super.connect()
  }

  handleSuccess({ detail: { success } }) {
    if (success) {
      super.close()
      this.clearErrors()
      this.formTarget.reset()
    }
  }

  clearErrors() {
    if (this.hasErrorsTarget) {
      this.errorsTarget.remove()
    }
  }
}
  

  

Here the clearErrors function looks for the presence of errors on the form and removes them from the DOM if they exist. To make this work, update the form partial to add the data-extended-modal-target onto the errors container:

  
<!-- app/views/comments/_form.html.erb -->
<% if @comment.errors.any? %>
  <div class="p-4 border border-red-600" data-extended-modal-target="errors">
    <h2>
      Could not save comment
    </h2>
    <ul>
      <% @comment.errors.full_messages.each do |message| %>
        <li><%= message %></li>
      <% end %>
    </ul>
  </div>
<% end %>
  

  

Wrapping up

That’s all for today. I hope that this guide has helped you see some of the power you can unlock by adding Hotwire to your Rails stack.

With Hotwire, we were able to build a modal-based form submission flow that eliminates full page turns and delivers a modern-feeling, performant solution while still keeping our code easy to follow and build upon for any Rails developer.

Hotwire is still new, and still evolving rapidly, in the future we can expect even more powerful functionality and even tighter integration with Rails as the community continues to add Hotwire to new and existing Rails applications.

Here are a few resources I used as I wrote this guide:

Tailwind Stimulus Components: In addition to modals, this project gives you ready-to-use Alerts, Dropdowns, and Tabs, among other things. Before you roll your own component, check here.
How to use modals with forms in Rails using Turbo: This article served as a great starting point for my work and helped me reach the solution that I presented. Highly recommended if you’re interested in another approach to handling modal forms with Turbo
The Turbo and Stimulus handbooks
The Turbo source code: Turbo is evolving rapidly enough that even the official documentation can sometimes lag behind. Reading through the source can be a great way to learn more about the options you have for solving a problem, and for understanding what’s going on behind the scenes.

Thanks for reading!

About the author:

I’m David - a solo founder and consultant. I write about software engineering, Ruby on Rails and the Rails-y ecosystem, product management, and the lessons I learn as I work to build sustainable SaaS businesses.

My current project is Vestimonials, an async video communication tool to help companies collect and share video testimonials from their employees and customers.

I’m available for Rails, product management, and strategy consulting. Get in touch if you have questions about this article or anything else that’s on your mind.

Building a collapsible sidebar with Stimulus and Tailwind CSS

2021-07-06T00:00:00+00:00

Today we’re building one of the most common elements in web design - a lefthand sidebar for navigation - using Tailwind CSS and Stimulus.js.

Our sidebar will always take up 100% of the height of the page and we’ll be able to expand and collapse the sidebar by clicking a button. The whole thing will be accomplished with just a bit of Stimulus for the expand/collapse action and a small amount of Tailwind in our markup.

Here’s what it will look like when we’re finished.

To accomplish this, we’ll start with a plain HTML file, pull in Tailwind CSS to make things look nice, and use Stimulus for interactivity.

I’m writing this assuming a solid understanding of HTML and CSS, and some level of comfort with JavaScript.

If you’ve never seen Tailwind before, some of the classes we add for styling might feel a little odd. You don’t need any knowledge of how Stimulus works, but if you’re brand new you might want to read the Stimulus Handbook to help solidify some concepts as we go.

You can find the complete code for this project on Github.

Let’s dive in.

Project setup

For simplicity, we’re just going to use a plain old HTML file and pull in Tailwind and Stimulus from a CDN. In a real project, you should use a build system but we don’t need all that to demonstrate the concept!

Let’s start with our plain HTML. Go ahead and copy and paste the below into a file called sidebar.html

  
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Collapsible sidebar with Stimulus and Tailwind</title>
  </head>

  <body>
    <aside>
      Here's our sidebar, eventually
    </aside>
    <main>
      Here's where our main content lives
    </main>
  </body>
</html>
  

  

Now we’ll add in Stimulus and make Stimulus available through window.Stimulus. Add these script tags to the head tag, copied right from the Stimulus docs.

  
<script src="https://unpkg.com/stimulus/dist/stimulus.umd.js"></script>
<script>
  (() => {
    const application = Stimulus.Application.start()

    application.register("sidebar", class extends Stimulus.Controller {
      static get targets() {
        return [ "" ]
      }
    })
  })()
</script>
  

  

And then pull in Tailwind CSS from CDN, which is not recommended for uses outside of demos like this. Tailwind has extensive documentation for how to include Tailwind for just about any build system and framework you can imagine.

  
<link href="https://unpkg.com/tailwindcss@^2/dist/tailwind.min.css" rel="stylesheet">

Perfect, now when we open sidebar.html in our browser we should be able to access Stimulus in the JavaScript console and the defaults applied by Tailwind should be visible on our placeholder text.

Let’s create the sidebar layout with Tailwind now.

Replace the <body> of your sidebar.html file with the below to add the basic structure of the sidebar to your page.

  
<body>
  <div class="container">
    <div class="flex">
      <aside class="sm:w-1/5 bg-blue-500 min-h-screen">
        <div class="sticky top-0 pt-12 px-2 w-full">
          <div class="absolute right-2 top-2 cursor-pointer">
            <svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
              <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11 19l-7-7 7-7m8 14l-7-7 7-7" />
            </svg>
          </div>
          <nav>
            <ul class="flex flex-col overflow-hidden space-y-2">
              <li class="hover:text-gray-200 h-8">
                <a href="#" class="flex items-center h-8">
                  <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1 inline-block" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M3 12l2-2m0 0l7-7 7 7M5 10v10a1 1 0 001 1h3m10-11l2 2m-2-2v10a1 1 0 01-1 1h-3m-6 0a1 1 0 001-1v-4a1 1 0 011-1h2a1 1 0 011 1v4a1 1 0 001 1m-6 0h6" />
                  </svg>
                  <span>
                    Home
                  </span>
                </a>
              </li>
              <li class="hover:text-gray-200 h-8">
                <a href="#" class="flex items-center h-8">
                  <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M21 12a9 9 0 01-9 9m9-9a9 9 0 00-9-9m9 9H3m9 9a9 9 0 01-9-9m9 9c1.657 0 3-4.03 3-9s-1.343-9-3-9m0 18c-1.657 0-3-4.03-3-9s1.343-9 3-9m-9 9a9 9 0 019-9" />
                  </svg>
                  <span>
                    Learn
                  </span>
                </a>
              </li>
              <li class="hover:text-gray-200 h-8">
                <a href="#" class="flex items-center h-8">
                  <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M16 7a4 4 0 11-8 0 4 4 0 018 0zM12 14a7 7 0 00-7 7h14a7 7 0 00-7-7z" />
                  </svg>
                  <span>
                    About
                  </span>
                </a>
              </li>
              <li class="flex items-center hover:text-gray-200 h-8">
                <a href="#" class="flex items-center h-8">
                  <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M20 13V6a2 2 0 00-2-2H6a2 2 0 00-2 2v7m16 0v5a2 2 0 01-2 2H6a2 2 0 01-2-2v-5m16 0h-2.586a1 1 0 00-.707.293l-2.414 2.414a1 1 0 01-.707.293h-3.172a1 1 0 01-.707-.293l-2.414-2.414A1 1 0 006.586 13H4" />
                  </svg>
                  <span>
                    Contact
                  </span>
                </a>
              </li>
            </ul>
          </nav>
        </div>
      </aside>
      <main class="sm:w-4/5 p-4">
        Main content goes here.
      </main>
    </div>
  </div>
</body>
  

  

This method of adding a static sidebar uses flexbox to create two side-by-side containers, with the smaller content on the left and the content on the right. Using Tailwind, we don’t need to touch CSS at all to create the layout, instead relying entirely on existing Tailwind utility classes.

Because we want the sidebar to have a background color that stretches (at minimum) the full height of the page, we add the min-h-screen class to the sidebar’s container. We want to keep the navigation links on the page at all times, even if the main content scrolls, so add the sticky class to the element that wraps the navigation menu.

Our navigation links each have an icon. When we have our collapse/expand functionality in place, the collapsed sidebar will display the link icons without any text.

With the body updated, refresh the page and see the basic layout in place. Clicking on the collapse/expand icon won’t do anything just yet.

The Stimulus controller will handle collapsing and expanding the sidebar content when the user clicks the collapse/expand icon.

First, we’ll add a function that toggles the sidebar from expanded to collapsed and back again. This function will look like this:

  
toggle() {
  if (this.sidebarContainerTarget.dataset.expanded === "1") {
    this.collapse()
  } else {
    this.expand()
  }
}
  

  

The toggle() function uses a simple if statement that checks a data attribute in the DOM to determine if the sidebar is currently expanded or collapsed and then call the collapse or expand function as appropriate. We’ll create this data attribute in the next section.

The collapse() function is responsible for collapsing the expanded sidebar, and it looks like this:

  
collapse() {
  this.sidebarContainerTarget.classList.remove("sm:w-1/5")
  this.iconTarget.innerHTML = `
  <svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 5l7 7-7 7M5 5l7 7-7 7" />
  </svg>
  `
  this.linkTargets.forEach(link => {
    link.classList.add("sr-only")
  })
  this.sidebarContainerTarget.dataset.expanded = "0"
}
  

  

Here we find the sidebar container, remove the width Tailwind class, swap out the icon for the expand icon, apply the Tailwind sr-only utility class to hide navigation link text from the page unless the user is accessing the site with a screenreader. Last, we update the sidebar’s expanded data attribute to 0, so that the next time the collapse/expand link is clicked the toggle function calls the expand function.

One of the wonderful things about Stimulus is that it provides a simple way to select elements from the DOM using targets. These targets take the place of selecting elements by class or id, instead relying on data attributes.

Using targets, we can keep our JavaScript and our HTML clear and readable, so that future developers don’t accidentally break our JavaScript by removing a class or changing an id.

The expand() function comes next, and it looks like this:

  
expand() {
  this.sidebarContainerTarget.classList.add("sm:w-1/5")
  this.iconTarget.innerHTML = `
    <svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
      <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11 19l-7-7 7-7m8 14l-7-7 7-7" />
    </svg>
  `
  this.linkTargets.forEach(link => {
    link.classList.remove("sr-only")
  })
  this.sidebarContainerTarget.dataset.expanded = "1"
}
  

  

Here we simply reverse the changes our collapse function makes:

Setting the sidebar width
Swapping the icon
Display link text to all users
Set the expanded data attribute to 1 so the next click on the icon routes to collapse

Finally, we need to tell our Stimulus controller about the targets we’re using by updating the targets definition at the top of the Stimulus controller:

  
static get targets() {
  return [ "sidebarContainer", "icon", "link" ]
}

With these changes in place, the <script> tag that adds Stimulus to our page looks like this:

  
<script>
  (() => {
    const application = Stimulus.Application.start()
  
    application.register("sidebar", class extends Stimulus.Controller {
      static get targets() {
        return [ "sidebarContainer", "icon", "link" ]
      }

      toggle() {
        if (this.sidebarContainerTarget.dataset.expanded === "1") {
          this.collapse()
        } else {
          this.expand()
        }
      }

      collapse() {
        this.sidebarContainerTarget.classList.remove("sm:w-1/5")
        this.sidebarContainerTarget.dataset.expanded = "0"
        this.iconTarget.innerHTML = `
        <svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
          <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 5l7 7-7 7M5 5l7 7-7 7" />
        </svg>
        `
        this.linkTargets.forEach(link => {
          link.classList.add("sr-only")
        })
      }

      expand() {
        this.sidebarContainerTarget.classList.add("sm:w-1/5")
        this.sidebarContainerTarget.dataset.expanded = "1"
        this.iconTarget.innerHTML = `
          <svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
            <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11 19l-7-7 7-7m8 14l-7-7 7-7" />
          </svg>
        `
        this.linkTargets.forEach(link => {
          link.classList.remove("sr-only")
        })
      }           
    })
  })()
</script>
  

  

If you make these updates, refresh the page and click on the collapse icon you’ll notice that absolutely nothing happens. That’s because our Stimulus controller isn’t connected to the DOM yet. We’ll finish up this guide by updating our HTML to make everything work.

Connecting to the DOM

For our Stimulus controller to work, we first need to add a controller data attribute somewhere in the HTML. In general, you want to attach the controller to the parent element for the part of the DOM you plan to change within the controller. Since the Sidebar controller makes changes to the sidebar and the elements inside of the sidebar, it makes sense to attach the controller to the top-level sidebar element:

  
<aside data-controller="sidebar" class="sm:w-1/5 bg-blue-500 min-h-screen">
  <!-- Sidebar content -->
</aside>

Next up, we need a data attribute to track the current state of the sidebar (expanded or collapsed):

  
<aside data-sidebar-target="sidebarContainer" data-expanded="1" class="sm:w-1/5 bg-blue-500 min-h-screen">
  <!-- Sidebar content -->
</aside>

The last update to the <aside> element is to tell our Stimulus controller that it is the sidebarContainer target:

  
<aside data-sidebar-target="sidebarContainer" data-expanded="1" data-controller="sidebar" class="sm:w-1/5 bg-blue-500 min-h-screen">
  <!-- Sidebar content -->
</aside>

Next up, our collapse icon container needs a target and an action:

  
<div data-action="click->sidebar#toggle" data-sidebar-target="icon" class="absolute right-2 top-2 cursor-pointer">
  <svg xmlns="http://www.w3.org/2000/svg" class="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor">
    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11 19l-7-7 7-7m8 14l-7-7 7-7" />
  </svg>
</div>

The data-action attribute is structured as “UserAction -> ControllerName#FunctionName”. For certain elements and actions, the UserAction component is optional but for a div, an action must always be provided. Read more about actions here.

With these attributes in place, each time the sidebar icon is clicked, the toggle() function defined in our Sidebar controller will be called. If you refresh the page now you’ll notice an error in your JavaScript console. One last addition to the HTML and we’ll be all set:

  
<nav>
  <ul class="flex flex-col overflow-hidden space-y-2">
    <li class="hover:text-gray-200 h-8">
      <a href="#" class="flex items-center h-8">
        <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1 inline-block" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
          <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M3 12l2-2m0 0l7-7 7 7M5 10v10a1 1 0 001 1h3m10-11l2 2m-2-2v10a1 1 0 01-1 1h-3m-6 0a1 1 0 001-1v-4a1 1 0 011-1h2a1 1 0 011 1v4a1 1 0 001 1m-6 0h6" />
        </svg>
        <span data-sidebar-target="link">
          Home
        </span>
      </a>
    </li>
    <li class="hover:text-gray-200 h-8">
      <a href="#" class="flex items-center h-8">
        <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
          <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M21 12a9 9 0 01-9 9m9-9a9 9 0 00-9-9m9 9H3m9 9a9 9 0 01-9-9m9 9c1.657 0 3-4.03 3-9s-1.343-9-3-9m0 18c-1.657 0-3-4.03-3-9s1.343-9 3-9m-9 9a9 9 0 019-9" />
        </svg>
        <span data-sidebar-target="link">
          Learn
        </span>
      </a>
    </li>
    <li class="hover:text-gray-200 h-8">
      <a href="#" class="flex items-center h-8">
        <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
          <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M16 7a4 4 0 11-8 0 4 4 0 018 0zM12 14a7 7 0 00-7 7h14a7 7 0 00-7-7z" />
        </svg>
        <span data-sidebar-target="link">
          About
        </span>
      </a>
    </li>
    <li class="flex items-center hover:text-gray-200 h-8">
      <a href="#" class="flex items-center h-8">
        <svg xmlns="http://www.w3.org/2000/svg" class="h-5 w-5 mr-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" aria-hidden="true">
          <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M20 13V6a2 2 0 00-2-2H6a2 2 0 00-2 2v7m16 0v5a2 2 0 01-2 2H6a2 2 0 01-2-2v-5m16 0h-2.586a1 1 0 00-.707.293l-2.414 2.414a1 1 0 01-.707.293h-3.172a1 1 0 01-.707-.293l-2.414-2.414A1 1 0 006.586 13H4" />
        </svg>
        <span data-sidebar-target="link">
          Contact
        </span>
      </a>
    </li>
  </ul>
</nav>
  

  

Here we added sidebar-target="link" to each of the span’s that wrap the link text. Recall that in our Stimulus controller we use linkTargets to toggle the sr-only class as needed. This ability to easily find and modify any number of elements without relying on classes that may or may not be changed in the future is one of the things that makes Stimulus so pleasant to work with.

this.linkTargets.forEach(link => {
  link.classList.add("sr-only")
})

With that last change in place, refresh the page one last time and see that everything works as expected.

Wrapping up

Today we learned how to use Tailwind CSS and Stimulus to create a simple sidebar that users can collapse and expand on demand.

While both a sidebar layout and collapsing/expanding content can be accomplished without any CSS or JavaScript frameworks, I hope this example has given you a taste for how readable, maintainable, and scalable Tailwind and Stimulus can be. A future developer revisiting code written with Stimulus and Tailwind can quickly deduce the purpose of each class and data attribute and can confidently make changes to implement new features to expand the site over time.

We’re just scratching the surface of what Tailwind and Stimulus can do. To dig in further, start with:

Tailwind’s excellent (really, excellent) documentation
The Stimulus Handbook
Better Stimulus for Stimulus patterns and best practices

Thanks for reading!

About the author:

My current project is Vestimonials, an async video communication tool to help companies collect and share video testimonials from their employees and customers. I’m available for Rails, product management, and strategy consulting. Get in touch if you have questions about this article or anything else that’s on your mind.

Remotely loading tabbed content with Ruby on Rails and Hotwire

2021-07-01T00:00:00+00:00

Hotwire is a set of tools for building web applications by sending HTML instead of JSON over the wire. Hotwire gives us a framework for making DOM updates without writing much, or any, JavaScript while delivering fast, modern-feeling web applications.

In today’s example, we’re building a interface that remotely retrieves a portion of the page content from an endpoint and replaces a targeted portion of the DOM with the response from the endpoint. We’ll build this without writing any JavaScript and with only minor additions to the standard Rails code you already know how to write.

Here is what it will look like when we are finished. We won’t be focused on styling today but our “tabs” will be fully functional and ready for you to add a nice looking Bootstrap or Tailwind skin.

To accomplish this, we will start with a new Rails 6.1 application, install Hotwire in the application, and then walk through the basics of adding Turbo Drive to our views.

I’m writing this assuming that you are comfortable with the basics of Ruby on Rails development and that you’ve never used Hotwire before.

You can find the complete source code for this tutorial on Github.

Let’s dive in.

Set up our project

To get started, we’re going to create a new Rails application and create the resources we need to start implementing our remotely fetched tabs. The application will be centered around a Person. A Person can have Awards and Credits and those awards and credits are what we’ll display in our tabs.

Let’s get started with the following commands in our terminal:

rails new hotwire_tabbing -T
cd hotwire_tabbing
rails g scaffold Person name:string
rails g model Credit name:string person:references
rails g model Award name:string person:references
rails db:migrate
rails s

Visit http://localhost:3000/people in your browser and create a person to use for testing and update the Person model to finish up the relationships with Awards and Credits.

  
class Person < ApplicationRecord
  has_many :awards
  has_many :credits
end

Finally, in the Rails console (rails c in your terminal), create a couple of Awards and Credits with the Person you created in the UI. We won’t be covering creation of these resources in this lesson.

  
Person.first.awards << Award.create(name: "Best Director")
Person.first.credits << Credit.create(name: "Star Wars - The good one")

Now our application shell is all set up.

Next up, we’ll install Hotwire and then we’ll start building.

Install Hotwire

Add Hotwire to your Rails application by adding hotwire-rails to your Gemfile and then running bundle install from your terminal. If you prefer, bundle add hotwire-rails from your terminal works too.

Once Hotwire is added to your Gemfile, install Hotwire by running rails hotwire:install from your terminal. Hotwire is now installed and you’re ready to start building.

Before moving on, be sure to restart your Rails server or you’ll encounter some undefined method errors as you work through the rest of this guide.

Setup awards and credits

Now that Hotwire is installed, our next step is to add controllers and views for Awards and Credits. These controllers and views will mostly look like standard Rails, with a few small adjustments to take advantage of the tools Hotwire provides.

First generate the controllers in your terminal:

rails g controller Awards
rails g controller Credits

And update the routes file to nest Awards and Credits endpoints under Person.

  
Rails.application.routes.draw do
  resources :people do
    resources :awards, only: %i[index]
    resources :credits, only: %i[index]
  end
end
  

  

Next, let’s fill in our Awards controller:

  
class AwardsController < ApplicationController
  before_action :set_person

  def index
    respond_to do |format|
      format.html { render partial: 'awards/list', locals: { awards: @person.awards, person: @person }}
    end
  end

  private

  def set_person
    @person = Person.find(params[:person_id])
  end
end
  

  

Here we’re using the person_id URL parameter to select the appropriate Person from the database, retrieiving that person’s Awards, and rendering a list partial that doesn’t yet exist.

Note that while we’ve chosen to use a partial here since that feels more comfortable for me, this guide would work fine using a regular old index view instead of a partial.

Since our list partial doesn’t exist yet, go ahead and create it now:

touch app/views/awards/_list.html.erb

And then fill it in:

  
<%= turbo_frame_tag "details_tab" do %>
  <div>
    <%= render partial: "shared/tabs" %>
    <div>
      <h3>Awards won by <%= person.name %></h3>
      <ul>
        <% awards.each do |award| %>
          <li><%= award.name %></li>
        <% end %>
      </ul>
    </div>
  </div>
<% end %>
  

  

Okay - now we’ve got a little bit of Hotwire code to review.

Our entire list partial is wrapped in a turbo_frame_tag which will turn into a <turbo-frame> HTML element when this view renders. The turbo frame helper provided by hotwire-rails requires an id argument (“details_tab” in our case) which Turbo uses to make DOM updates as needed.

Besides the turbo_frame helper, the rest of this view is standard Rails.

Note that the list partial relies on a shared/tabs partial that doesn’t exist yet. Create that now with:

mkdir app/views/shared
touch app/views/shared/_tabs.html.erb

The tabs partial will render our tabs like this:

  
<div>
  <%= link_to "Awards", person_awards_path(@person) %><br />
  <%= link_to "Credits", person_credits_path(@person) %>
</div>

These tabs are just simple HTML and we’ll render them in a couple of other places.

With the Awards controller and view built out, let’s do the same thing for Credits before we finish up by updating the Person show page to tie everything together.

First create the credits list partial.

touch app/views/credits/_list.html.erb

And fill in the credits list partial with HTML that will look pretty familiar:

  
<%= turbo_frame_tag "details_tab" do %>
  <div>
    <%= render partial: "shared/tabs" %>
    <div>
      <h3>Credits for <%= person.name %></h3>
      <ul>
        <% credits.each do |credit| %>
          <li><%= credit.name %></li>
        <% end %>
      </ul>
    </div>
  </div>
<% end %>
  

  

Then update the Credits controller:

  
class CreditsController < ApplicationController
  before_action :set_person

  def index
    respond_to do |format|
      format.html { render partial: 'credits/list', locals: { credits: @person.credits, person: @person }}
    end
  end

  private

  def set_person
    @person = Person.find(params[:person_id])
  end
end
  

  

Perfect.

Our last step is to update our Person show view to display the content of these tabs. Almost finished.

Tie everything together

The Person show page will be responsible for displaying the tabbed content that we built out through the Awards and Credits resources in the last section. Update app/views/people/show.html.erb like this:

  
<div>
  <h2><%= @person.name %></h2>
  <div>
    <%= render partial: "awards/list", locals: { person: @person, awards: @person.awards } %>
  </div>
</div>
  

  

Notice that here we don’t have a single line of nonstandard Rails code here.

The Show view simply renders the Person’s name and the awards list partial. The partial renders the tab navigation and the list of awards. All the heavy lifting is done behind the scenes by hotwire-rails by hooking into the details_tab <turbo-frame> that our list partials render.

In your browser, head to a show page for a Person with awards and credits and see that clicking the navigation links toggles the list content between Awards and Credits without requiring a full page turn.

Great work!

Wrapping up

This simple technique for remotely loading tabbed content with Hotwire reveals a powerful way of improving application performance and end user experience with minimal additional engineering overhead.

In a real application, moving requests for tabbed content that we don’t need right away into a separate request and eliminating the need for a full page turn reduces server load and speeds up page load times without major architecture changes. Small teams and solo developers can use Rails + Hotwire to provide modern, highly-performant web applications quickly and without getting into the heavy world of single page applications and heavy JavaScript frameworks.

Using Hotwire and Rails to build a live commenting system

2021-06-24T00:00:00+00:00

Today we’re exploring Hotwire, a new-old way of building web applications by sending HTML instead of JSON over the wire.

We’ll learn how Hotwire works with Ruby on Rails by building a simple application that allows users to leave comments on a project and see those comments in real-time, without requiring full page turns.

This guide presents an alternative approach to a guide I wrote last year which guides you through creating a similar experience using Stimulus. Either approach is valid and works well. The approach presented today presents a way of delivering the experience without writing any JavaScript code, and should feel very natural for any Ruby on Rails developer.

Here is what it will look like when we are finished. It won’t be pretty, but it will work and we won’t write a single line of JavaScript.

To accomplish this, we will start with a new Rails 6.1 application, install Hotwire in the application, and then walk through the basics of adding Hotwire to our views and controllers.

I’m writing this assuming that you are comfortable with the basics of Ruby on Rails development and that you’ve never used Hotwire before.

You can find the complete source code for this tutorial on Github.

Let’s dive in.

Set up our project

To get started we will create a new Rails application and scaffold up a Project resource to build our commenting system against. While you can work from an existing application of your own, you will find this guide easier to follow if you start from scratch along with me.

First, run these console commands to create and setup our Rails app:

rails new hotwire-comments -T
cd hotwire-comments
rails g scaffold Project name:string
rails g model Comment body:text project:references
rails db:migrate
rails s

In your browser, head to http://localhost:3000/projects and create a project. We’ll use this project as we build the commenting system.

Once that’s done, open the application’s code in your favorite editor and add has_many :comments to app/models/project.rb.

Next, display comments on the project show page by updating app/views/projects/show.html.erb with the following:

  
<!-- app/views/projects/show.html.erb -->

<p id="notice"><%= notice %></p>

<p>
  <strong>Name:</strong>
  <%= @project.name %>
</p>
<div>
  <% @project.comments.each do |comment| %>
    <div>
      <%= comment.body %>
    </div>
  <% end %>
</div>

<%= link_to 'Edit', edit_project_path(@project) %> |
<%= link_to 'Back', projects_path %>
  

  

And add a few comments in the rail console:

  
Project.first.comments << Comment.create(body: "Hotwire comments are coming soon!")

Wonderful.

If you’ve followed along successfully so far, when you visit http://localhost:3000/projects/1 you should see something that looks like this:

Now we’re ready to install Hotwire and start learning.

Installing Hotwire

To install Hotwire, first add the hotwire-rails gem to your Gemfile by running this commend in your console or by manually updating your Gemfile and running bundle install in your terminal.

bundle add hotwire-rails

Once the gem is added, run the installer from your terminal:

rails hotwire:install

After the installer runs, restart your rails server or you may encounter some undefined method errors later in this guide.

Now that we have Hotwire installed, let’s start with getting comments to render in real time.

Adding a comments stream

Hotwire as we’re using it relies on frameworks designed and maintained by the Basecamp team. One of those frameworks is Turbo. Turbo is a set of “complimentary techniques for speeding up page changes and form submissions”.

For our purposes, we’ll be interested in two key components of the Turbo package, Turbo Frames and Turbo Streams.

A Turbo Stream delivers page changes to the browser over Websocket. By adding the Hotwire Rails project to our application, we gain access to a set of helper methods that make working with Turbo Streams simple.

Let’s see how this works by updating our project show page as follows.

  
<!-- app/views/projects/show.html.erb -->

<p id="notice"><%= notice %></p>

<p>
  <strong>Name:</strong>
  <%= @project.name %>
</p>

<%= turbo_stream_from @project, :comments %>
<div id="<%= "#{dom_id(@project)}_comments" %>">
  <%= render @project.comments.order(created_at: :desc) %>
</div>

<%= link_to 'Edit', edit_project_path(@project) %> |
<%= link_to 'Back', projects_path %>
  

  

Let’s walk through the changes here one-by-one.

First, we add a turbostream that listens for broadcasts on the comments channel for our Project. This stream is how we subscribe to comment updates for our project. Note that because we’re working with a nested resource here (comments belong to a project), our stream subscribes to comments for just the current project.

We also add an id to the parent div of our comments list. This id is used to identify where to add broadcasted comments to the DOM. If this id is not present or does not match the id in the broadcast, no DOM updates will occur when comment changes are broadcast.

Finally, we’re using a _comment partial to render each comment. That partial doesn’t exist yet, let’s add it now:

mkdir app/views/comments
touch app/views/comments/_comment.html.erb

And then fill in the comment partial with the below content.

  
<!-- app/views/comments/_comment.html.erb -->

<div id="<%= dom_id(comment) %>">
  <%= comment.body %>
</div>

Note that here, we’re ensuring each comment has a unique id in the DOM, which ensures that new comments are properly inserted into the DOM.

Our last step towards getting our stream setup and comments prepending in real time is to add a callback to the comment model.

When this callback runs, the newly created comment will be broadcast on the project_comments stream that our project#show page is now subscribed to.

  
  # app/models/comment.rb
include ActionView::RecordIdentifier

after_create_commit { broadcast_prepend_to [project, :comments], target: "#{dom_id(project)}_comments" }

Here we are using the broadcast_prepend_to method provided by turbo-rails. There are a variety of methods that can be used to add, remove, and replace DOM content through callbacks triggered by model changes. The best place to find these methods is by reading the turbo-rails code.

Note that the target value passed to broadcast_prepend_to must exactly match the id set in the DOM earlier.

The dom_id method used to set the broadcast channel target is not normally available in models so we include the relevant ActionView helper class in the model. This simplifies our method of broadcasting to the appropriate channel. H/T to Chris Oliver at GoRails for this helpful trick.

With these changes in place, let’s test out our stream and see how it works.

Make sure you’ve got your project open in a web browser, and then open your Rails console and create a comment. If you’ve followed along successfully, immediately after you create the comment in the console, you should see the comment added to the project page open in your browser, like this:

Adding comments with a form

While our subscription is working great, users aren’t going to add content through the Rails console. Let’s finish up this tutorial by adding a form to add comments in the UI without requiring a page turn.

First, let’s create a partial to render our form:

touch app/views/comments/_form.html.erb

And add the below to the form:

  
<!-- app/views/comments/_form.html.erb -->

<%= turbo_frame_tag "comment_form" do %>
  <%= form_with model: comment, url: project_comments_path(@project) do |form| %>
    <% if comment.errors.any? %>
      <div id="error_explanation">
        <h2><%= pluralize(comment.errors.count, "error") %> prohibited this comment from being saved:</h2>

        <ul>
          <% comment.errors.each do |error| %>
            <li><%= error.full_message %></li>
          <% end %>
        </ul>
      </div>
    <% end %>
    <%= form.text_field :body %>
    <%= form.submit %>
  <% end %>
<% end %>

  

  

This is pretty close to a standard Rails form partial, with the added turbo_frame_tag that wraps the entire <form> element.

This frame_tag allows us to selectively replace the content of the form partial on form submission without writing any special Javascript.

Now we’ll need a controller to handle submissions from this form.

rails g controller Comments

Fill in the controller with the below content:

  
# app/controllers/comments_controller.rb

class CommentsController < ApplicationController
  def create
    @project = Project.find(params[:project_id])
    @comment = @project.comments.new(comment_params)

    respond_to do |format|
      if @comment.save
        format.turbo_stream { render turbo_stream: turbo_stream.replace('comment_form', partial: 'comments/form', locals: { comment: Comment.new }) }
        format.html { render partial: 'comments/form', locals: { comment: Comment.new }}
      else
        format.turbo_stream { render turbo_stream: turbo_stream.replace('comment_form', partial: 'comments/form', locals: { comment: @comment }) }
        format.html { render partial: 'comments/form', locals: { comment: @comment }}
      end
    end
  end

  private

  def comment_params
    params.require(:comment).permit(:body)
  end
end
  

  

Our comments controller create method is a fairly standard Rails controller with one twist: the create method responds to turbo_stream requests. When a turbo_stream request is sent to this endpoint, the controller responds with a turbo_stream response to replace the comment_form DOM element with the comments/form partial, both on a successful request and a failed request. Note that, as before, the “comment_form” id in our turbo_frame needs to match the target passed to the turbo_stream.replace method.

The only difference between the failure and success responses is using the existing @comment on failed responses so that errors can be rendered. On success, we set the comment local variable for the form partial to a new comment to clear out the body field on the form.

To test out the error path, you can update your Comment model to validate the presence of the body field.

Next, before we can create comments, we need to add the appropriate route to the routes.rb file:

  
# config/routes.rb

Rails.application.routes.draw do
  resources :projects do
    resources :comments, only: %i[create]
  end
end

Finally, let’s display the comment form on the project show page:

  
<!-- app/views/projects/show.html.erb -->
<p id="notice"><%= notice %></p>

<p>
  <strong>Name:</strong>
  <%= @project.name %>
</p>

<%= render partial: "comments/form", locals: { comment: Comment.new } %>
<%= turbo_stream_from @project, :comments %>
<div id="<%= "#{dom_id(@project)}_comments" %>">
  <%= render @project.comments.order(created_at: :desc) %>
</div>

<%= link_to 'Edit', edit_project_path(@project) %> |
<%= link_to 'Back', projects_path %>
  

  

Refresh your project page after you update the view.

If everything is set up correctly, you should see new comments added without a page turn. If you submit with the comment body left blank, you should see error messages added to the page.

Wrapping up

Combining Hotwire and Rails, with a sprinkling of Stimulus for client-side interactivity is a powerful way to build performant, scalable, developer-and-user friendly web applications.

While single page applications and independent frontends built with the latest JavaScript framework have their place, if you’re a small team building standard SaaS applications, I encourage you to spend time learning about Hotwire-powered applications to keep your team productive and focused on solving problems for your users, not fighting with your tech stack.

Helpful Hotwire resources:

Thanks for reading! As always, get in touch if you have any questions or if I can be of any help.

Building a horizontal slider with Stimulus and Tailwind CSS

2021-05-06T00:00:00+00:00

Today we’re building a component that is common but deceptively tricky to get right - a horizontal slider with a position indicator and navigation buttons.

We’ll have a list of items of an arbitrary length, and our slider will allow folks to scroll to see every item in the list. As they scroll, indicators below the slider will update to show which items are visible on the screen. Clicking on the indicators will scroll the corresponding item into view. The whole thing is pretty fancy.

Here’s what it will look like when we’re finished.

To accomplish this, we’ll start with a plain HTML file, pull in Tailwind CSS to make things look nice, and use Stimulus to build interactivity for our position indicators and navigation buttons.

I’m writing this assuming a solid understanding of HTML and CSS, and some level of comfort with JavaScript. If you’ve never seen Tailwind before, some of the classes we add for styling might feel a little odd. You don’t need any knowledge of how Stimulus works, but if you’re brand new you might want to read the Stimulus Handbook to help solidify some concepts.

Let’s dive in.

Project setup

For simplicity, we’re just going to use a plain old HTML file and pull in Tailwind and Stimulus from a CDN. In a real project, you should probably use a build system but we don’t need all that to demonstrate the concept!

Let’s start with our plain HTML. Go ahead and copy and paste the below into a file called slider.html or use a more exciting name. You’re the boss.

  
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Horizontal slider with Stimulus and Tailwind</title>
  </head>

  <body>
    <main>
      Here's where our slider will go, momentarily.
    </main>
  </body>
</html>
  

  

Now we’ll add in Stimulus and make Stimulus available through window.Stimulus. Add these script tags to the head tag, copied right from the Stimulus docs.

  
<script src="https://unpkg.com/stimulus/dist/stimulus.umd.js"></script>
<script>
  (() => {
    const application = Stimulus.Application.start()

    application.register("slider", class extends Stimulus.Controller {
      static get targets() {
        return [ "" ]
      }
    })
  })()
</script>
  

  

  
<link href="https://unpkg.com/tailwindcss@^2/dist/tailwind.min.css" rel="stylesheet">

Perfect, now when we open our slider.html we should be able to access window.Stimulus in the JavaScript console and the defaults applied by Tailwind should be visible on our placeholder text.

Let’s build the slider with Tailwind now.

Create our horizontal slider

We’ll start with the basic structure of the slider, with no Tailwind classes, and then we’ll add in the Tailwind classes to make everything function. Replace the text in <main> with the HTML below.

  
<div id="container">
  <h1>Our slider's title</h1>
  <div id="scrolling-content">
    <div>
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div>
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div>
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div>
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div>
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div>
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
  </div>
</div>
  

  

Open up slider.html and you’ll see some giant pictures of shoes. Not quite what we want, but a good starting point.

We’ll start with a flex container to hold our slider’s header, which will be static, and the slider itself, which will scroll horizontally. Update the content of <main> to include some basic container classes.

  
<div id="container" class="flex flex-col my-24">
  <h1 class="text-3xl text-gray-900 text-center mb-4">Our slider's title</h1>
  <div id="scrolling-content" class="flex overflow-x-scroll">
    <div class="w-96 h-64 px-4 flex-shrink-0">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
  </div>
</div>
  

  

The really important changes here are:

Adding flex overflow-x-scroll to the scrolling-content div. That sets the div to flex the child divs and adds the horizontal scrolling behavior we’re looking for with the CSS property overflow-x: scroll
Setting flex-shrink-0 to the individual image divs. This ensures the image divs don’t resize themselves to fit the viewport width using the CSS property flex-shrink: 0. Without this, the image divs would shrink automatically and the overflow-x-scroll property on the scrolling-content div wouldn’t do anything useful.

At this point, we’ve got a simple scrolling image gallery, nice work! Now we’ll get into JavaScript land by adding indicators that show the user which images are currently on screen and that function as navigation buttons to scroll the content to the clicked indicator.

Our indicators will be circles that change color based on whether they are in the active viewport or not. Again, we’ll start with our HTML. Add this HTML to the bottom of the container div.

  
<div class="flex mx-auto my-8">
  <ul class="flex justify-center">
    <!-- Note that we have one <li> for each image in our gallery -->
    <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500"></li>
    <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500"></li>
    <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500"></li>
    <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500"></li>
    <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500"></li>
    <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500"></li>
  </ul>
</div>
  

  

Now we’ve got some nice looking circles below our scrolling images, but they don’t serve any purpose. Next up is creating a Stimulus controller to make the dots come to life.

Bring the indicators to life with Stimulus

The Stimulus controller will be responsible for two things:

Updating the color of the indicator circles based on whether or not the corresponding image is currently visible to the user
Handling clicks on indicators and scrolling the container to the corresponding image

For the first task, we’ll rely on the IntersectionObserver API. This API is well-supported across modern browsers and is commonly used for tasks like lazy-loading images. In our case, we’re going to use it to change the color of the indicator circles. Let’s get started.

Update the Stimulus controller currently defined in our head with the following:

  
<script>
  (() => {
    const application = Stimulus.Application.start()
  
    application.register("slider", class extends Stimulus.Controller {
      static get targets() {
        return [ "scrollContainer", "image", "indicator" ]
      }
      initialize() {
        this.observer = new IntersectionObserver(this.onIntersectionObserved.bind(this), {
          root: this.scrollContainerTarget,
          threshold: 0.5
        })
        this.imageTargets.forEach(image => {
          this.observer.observe(image)
        })
      }

      onIntersectionObserved(entries) {
        entries.forEach(entry => {
          if (entry.intersectionRatio > 0.5) {
            const intersectingIndex = this.imageTargets.indexOf(entry.target)
            this.indicatorTargets[intersectingIndex].classList.add("bg-blue-900")
          }
          else { 
            const intersectingIndex = this.imageTargets.indexOf(entry.target)
            this.indicatorTargets[intersectingIndex].classList.remove("bg-blue-900")
          }
        })
      }
    })
  })()
</script>
  

  

There’s a lot here, let’s break it down a bit.

First, we add a few targets to our controller. We’ll use these to reference DOM elements that our controller cares about.

In the initialize method, we create a new observer using the IntersectionObserver constructor. The onIntersectionObserved callback function passed to the constructor is the function that will be called each time a visibility threshold is crossed.

In (closer-to) human terms: as you scroll the images left or right, the observer watches the visible part of the screen and fires the onIntersectionObserver function each time an image is more (or less) than half visible on the screen.

Also note that we bind this to the onIntersectionObserved function so that we can reference this and get back our Stimulus controller inside of the onIntersectionObserved function. Without binding this we would not be able to use Stimulus targets in this function and our JavaScript would be a bit more complicated.

At the end of the initialize method, we tell our observer which DOM elements it should watch over.

The onIntersectionObserved function simply loops over all of the watched DOM elements and adds a class if the element is more than half visible or removes that class if the element is not.

With this JavaScript added, refresh slider.html and see that nothing happens. To make this work, we need to update the HTML to connect the Stimulus controller to the DOM.

Let’s update our HTML as follows:

  
<div class="flex flex-col my-24" data-controller="slider">
  <h1 class="text-3xl text-gray-900 text-center mb-4">Our slider's title</h1>
  <div class="flex overflow-x-scroll" data-slider-target="scrollContainer">
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
  </div>
  <div class="flex mx-auto my-8">
    <ul class="flex justify-center">
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator"></li>
    </ul>
  </div>
</div>
  

  

The changes here are:

We added data-controller="slider" to our wrapper div to tell Stimulus that this div should be tied to our SliderController.
We added data-slider-target="scrollContainer" to the div that wraps our images and scrolls on the x-axis.
We added data-slider-target="image" to each of the image divs.
We added data-slider-target="indicator" to each of the indicator <li> tags

The addition of data-controller="slider" is mandatory - without adding this declaration our Stimulus code will never be executed. The targets are all technically optional and you could accomplish the same by adding classes or ids to the DOM but targets are a super helpful way to keep your code clean and concise and if you’re using Stimulus you should be using targets to reference DOM elements in most cases.

If you refresh slider.html again, you’ll see that the circles change color as we slide images in and out of view. Resize the browser, get wild with it if you want. One more step to go.

Now that we’ve got these nice navigation circles, the last step is to allow users to navigate between images by clicking on the corresponding circle. This can be accomplished with a new method in our Stimulus controller:

  
// Add this function alongside the existing initialize and onIntersectionObserved functions
scrollTo() {
  const imageId = event.target.dataset.imageId
  const imageElement = document.getElementById(imageId)
  imageElement.scrollIntoView({ block: "end", inline: "nearest", behavior: "smooth" })
}
  

  

This new function starts by identifying the target image and then uses Element.scrollIntoView() to scroll the parent container into the viewport, if it is not already visible.

To make this work, we need to add appropriate attributes to the images and indicators HTML, like this:

  
<div class="flex flex-col my-24" data-controller="slider">
  <h1 class="text-3xl text-gray-900 text-center mb-4">Our slider's title</h1>
  <div class="flex overflow-x-scroll" data-slider-target="scrollContainer">
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image" id="1">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image" id="2">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image" id="3">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image" id="4">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image" id="5">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
    <div class="w-96 h-64 px-4 flex-shrink-0" data-slider-target="image" id="6">
      <img src="https://images.unsplash.com/photo-1542291026-7eec264c27ff?ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8c2hvZXN8ZW58MHx8MHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=900&q=60" />
    </div>
  </div>
  <div class="flex mx-auto my-8">
    <ul class="flex justify-center">
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator" data-image-id="1" data-action="click->slider#scrollTo"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator" data-image-id="2" data-action="click->slider#scrollTo"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator" data-image-id="3" data-action="click->slider#scrollTo"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator" data-image-id="4" data-action="click->slider#scrollTo"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator" data-image-id="5" data-action="click->slider#scrollTo"></li>
      <li class="h-6 w-6 rounded-full mx-2 cursor-pointer bg-gray-500" data-slider-target="indicator" data-image-id="6" data-action="click->slider#scrollTo"></li>
    </ul>
  </div>
</div>
  

  

Note the changes here. Each image container div is given an id and each indicator is given a corresponding data-image-id. In the scrollTo function, we use data-image-id in a vanilla JavaScript document.getElementById call. The assigned ids are arbitrary - you could give each image a name or use a random string, as long as the image-id data attribute on the indicator matches the id on the image, you’re good to go.

After adding the ids, we also added data-actions to each indicator. The data-action attribute tells Stimulus which function to call when the click action occurs on the element. For more details on how data-action works, the Stimulus Handbook is a great starting point!

Refresh the page one more time and click on a circle for an image that isn’t on screen and your browser should scroll until that image is visible. Magic!

Improving scrollTo

While our scrollTo method works fine in isolation right now, if our slider element isn’t the only thing on the page, folks will have a fairly jarring experience - clicking on a dot will scroll the page horizontally (good!) and vertically (weird!).

This happens because scrollIntoView assumes you need to scroll both horizontally and vertically. You can’t only scroll horizontally with this function. This works great for full screen experiences where your slider is the only content on the page (like a full screen image gallery) but it fails when your slider has other content above and below it (like a gallery of product images on an ecommerce store listing)

To workaround this limitation, we can replace scrollIntoView with scrollTo. scrollTo allows us to scroll an element to a given x and y coordinate pair but, crucially, you can choose to only provide an x coordinate, eliminating any weird vertical scrolling.

Let’s update our scrollTo Stimulus function to use scrollTo instead of scrollIntoView:

  
scrollTo() {
  const imageId = event.target.dataset.imageId
  const imageElement = document.getElementById(imageId)
  const imageCoordinates = imageElement.getBoundingClientRect()
  this.scrollContainerTarget.scrollTo({ left: (this.scrollContainerTarget.scrollLeft + imageCoordinates.left), top: false, behavior: "smooth" })
}
  

  

Our new function has two key changes:

First, we extract the current position of our image relative to the viewport with getBoundingClientRect. This function returns, among other things, the x and y position of the element.
Next, we replace scrollIntoView with scrollTo. In the options, we set top to false to indicate we don’t want to change the vertical scroll and set left to the current left scroll position of the scrollContainer + the the image’s left (or x) position. Combining the current scroll position and the target element’s x position allows us to reliably scroll the container left and right programatically.

With this update in place, navigating the scroll container by clicking on the indicator circles no longer causes vertical scrolling.

Bonus round: Scroll behavior improvements

To finish up, let’s add a few more CSS rules to our slider to make it look and feel a little nicer.

First, we can add the hide-scroll-bar class to our scroll container. This built-in Tailwind CSS class hides the scroll bar, which looks a bit nicer and isn’t necessary with our indicators in place.

Next, we can prevent unwanted back navigation on mobile devices by adding the overscroll-x-contain class to the scroll container. Another built-in Tailwind class, this stops overscrolling in the scroll container (like swiping too aggressively to the left) from triggering scrolling on the whole page.

Finally, we’ll step outside of Tailwind for some scroll behavior CSS rules. Add a style tag to the head in slider.html and add the following CSS:

  
<style type="text/css">
  .gallery-item {
    scroll-snap-align: start;
  }
  .gallery {
    -webkit-overflow-scrolling: touch;
    scroll-snap-type: x mandatory;
  }
</style>
  

  

These rules instruct the browser to snap scrolling to each element with scroll-snap-type, adds momentum based scrolling on touch devices with -webkit-overflow-scrolling and tells the browser where to snap to for each gallery item with scroll-snap-align.

Add the gallery class to the scroll container and gallery-item to each image div and notice that scrolling the container now nicely snaps to each element when scrolling finishes.

Wrapping up and further reading

Some caveats to note before you use this code in production: at the time of this writing intersectionObserver and scrollTo are not implemented on IE11 and Safari does not support scrollTo options. You may wish to adjust the scrollTo function call to not pass in options or add polyfills for support on IE11, depending on your needs.

Special thanks goes to Joost Kiens who wrote an excellent article on using the intersectionObserver API to build an single element scroll container that served as a base for what we built today.

You can find the complete code for this guide on Github.

For questions or comments, you can find me on Twitter.

If you want to learn more about Tailwind or Stimulus, the official documentation for both is a great place to start. In particular, Tailwind’s documentation is some of the best on the internet and is highly recommended if you want to learn more about how Tailwind works.

Thanks for reading!

Building a video converter with Rails 6 and FFmpeg

2021-04-28T00:00:00+00:00

Today’s project pulls together a number of built-in Rails tools, plus a couple of incredibly useful open source projects to build a web application that can convert user-uploaded video files to MP4.

To demonstrate some of the modern parts of the standard Rail stack, we’re going to go further than just building a converter. We will enhance the user experience during the upload and conversion process with Active Storage to handle direct file uploads, Action Cable for real-time updates on the video conversion, and Stimulus to dynamically update the DOM without needing a heavy Javascript framework. When we’re finished, we’ll have an ugly but fully functional web application that allows a user to upload a video file. After the file is uploaded, we’ll convert the video to MP4 if needed, and then display that converted video to the user. It will look something like this:

This guide assumes that you are comfortable with Ruby on Rails and familiar with Active Storage and Stimulus, but you do not need to be an expert on any of these tools to get value from this guide. We’ll start with a brand new Rails 6 project, however, you can follow along with an existing Rails 6 project if you prefer.

If you use an existing project you may need to complete additional setup steps for Stimulus, Action Cable, and Active Storage that won’t be covered in this guide.

Project setup

To start, let’s create our Rails application using webpack and Stimulus, install Active Storage, add a User scaffold to work from, and migrate our database.

All standard Rails stuff here, hopefully nothing totally new yet!

  
    rails new upload_progress --webpack=stimulus --skip-coffee --database=postgresql -T
    cd upload_progress
    rails db:create
    rails g scaffold User name:string
    rails active_storage:install
    rails db:migrate
  

  

At this point, you can boot your rails server with rails s and visit localhost:3000/users to see the scaffold working as expected. Create a few users if you want. There are no rules here.

Now that the project is setup, let’s get into the fun stuff.

Uploading files with Active Storage

Our user will have a profile video, which we’ll use as the base for exploring video conversion later on in this guide.

To add a profile video, update the User model to add a profile_video backed by Active Storage.

  
    # app/models/user.rb

    class User < ApplicationRecord
      has_one_attached :profile_video
    end

Thanks to the power of Active Storage we can now just add the profile video to our controller and views and start uploading files right away.

Head to the users form and add a new form field for the video:

  
    <!-- app/views/users/_form.html.erb -->

    <%= form_with(model: user) do |form| %>
      <% if user.errors.any? %>
        <div id="error_explanation">
          <h2><%= pluralize(user.errors.count, "error") %> prohibited this user from being saved:</h2>

          <ul>
            <% user.errors.each do |error| %>
              <li><%= error.full_message %></li>
            <% end %>
          </ul>
        </div>
      <% end %>

      <div class="field">
        <%= form.label :name %>
        <%= form.text_field :name %>
      </div>

      <div class="field">
        <%= form.label :profile_video %>
        <%= form.file_field :profile_video %>
      </div>

      <div class="actions">
        <%= form.submit %>
      </div>
    <% end %>
  

  

Then update the UsersController to add profile_video to the existing user_params method.

  
    # app/controllers/users_controller.rb

    def user_params
      params.require(:user).permit(:name, :profile_video)
    end

Finally, we want to be able to see the video after we’ve uploaded it. Head to app/views/users/show and add a <video> element to the page.

  
    <!-- app/views/users/show.html.erb -->

    <video controls>
      <source src="<%= url_for(@user.profile_video) %>">
    </video>

Try it out and make sure that uploading videos works as expected by creating a new user and attaching a video file to the profile_video field.

Stunning. You’re a star.

While this works, we can make the upload experience a little better.

First - when the file upload dialog opens, the user can choose any type of file they like. Let’s add a guardrail to only allow video files to be selected in the upload dialog.

  
    <!-- app/views/users/_form_.html.erb -->

    <%= form.file_field :profile_video, accept: "video/*" %>

Note that in a real app accept is a nice user experience gain but it does not validate the user’s input. You should always validate the file type on the server! A malicious or curious user can easily change the accept attribute in the browser. For today we can get by with field-level validation on the client.

Adding direct uploads and an upload progress bar

When a user uploads a large file directly to your servers, they are going to find the request takes a long time and, in production, it may frequently time out, block other requests, and cause all kinds of headaches.

We can prevent all of those issues with the direct upload feature built in to Active Storage. Direct upload sends files directly from the user’s client to your chosen cloud storage provider. Direct uploads speed up the upload and keeping the request from timing out or backing up your servers.

Let’s add direct upload to the file field and, as a bonus, add an upload progress bar for the user.

First, confirm that you have ActiveStorage’s javascript added to your application.js file.

  
    // application.js

    // If you started with a new Rails project, these lines should already be in application.js. If not, add them
    import * as ActiveStorage from "@rails/activestorage"
    ActiveStorage.start()

Now, we need to tell our file field to use the direct upload javascript:

  
    <!-- app/views/users/_form_.html.erb -->

    <%= form.file_field :profile_video, accept: "video/*", direct_upload: true %>

Refresh your page and confirm that everything works exactly as it did before we added direct upload. Again, Rails makes things really easy.

While our file upload is working and we could leave it as it is, in the real world uploads can take a while, especially when you’re handling large video files. Giving the user feedback as their file uploads is a nice UX win. Let’s add a progress bar to track the file upload using a few sprinkles of Stimulus.

To track upload progress, we’ll listen for the direct-upload:progress event and, each time that event is emitted we’ll use the data in the event to update a progress bar in the UI.

We’ll start with our Stimulus controller, which looks like this:

  
    // app/javascript/controllers/upload_progress_controller.js

    import { Controller } from 'stimulus'

    export default class extends Controller {
      static targets = [ "progress", "progressText", "progressWidth" ]

      initialize() {
      }

      connect() {
        this.element.addEventListener("direct-upload:progress", this.updateProgress.bind(this))

        this.element.addEventListener("direct-upload:error", event => {
          event.preventDefault()
          const { id, error } = event.detail
          console.log(error)
        })
      }

      showProgress() {
        this.progressTarget.style.display = "block"
      }

      updateProgress() {
        const { id, progress } = event.detail
        this.progressWidthTarget.style.width = `${Math.round(progress)}%`
        this.progressTextTarget.innerHTML = `${Math.round(progress)}% complete`
      }

      disconnect() {
        this.element.removeEventListener("direct-upload:progress", this.updateProgress)
      }
    }
  

  

Let’s break this down a little bit - I’m assuming you’ve got a little bit of familiarity with Stimulus here, if you don’t, the Stimulus Handbook is a great starting point.

connect is used to setup our event listener and subscribe to the events we care about - in this case, direct-upload:progress and direct-upload:error

Our progress event listener calls updateProgress and then uses the progress data from the event to update the UI elements that make up our progress bar.

Next we’ll add our progress bar HTML and connect our Stimulus controller to the DOM.

  
    <!-- app/views/users/_form_.html.erb -->

    <%= form_with(model: user, html: { data: { controller: "upload-progress" } } ) do |form| %>

      <!-- snipped form fields -->

      <div style="display: none;" data-upload-progress-target="progress">
        <div>Uploading your video to our servers: <span id="progressText" data-upload-progress-target="progressText">Warming up...</span></div>
        <div id="progress-bar" style="background: gray; position: relative; width: 200px; height: 20px;">
          <div id="progressWidth" data-upload-progress-target="progressWidth" style="width: 0%; height: 100%; position: absolute; background: green;">
          </div>
        </div>
      </div>

      <div class="actions">
        <%= form.submit "Save", data: { action: "click->upload-progress#showProgress" } %>
      </div>
    <% end %>
  

  

There’s a lot here, let’s step through the updates individually.

First, we connect our Stimulus controller to the DOM by adding data: { controller: “upload-progress” } to our form_with

Then, we add our progress bar HTML. Don’t mind all the inline styles, you can use CSS if you like.

  
    <div style="display: none;" data-upload-progress-target="progress">
      <div>Uploading your video to our servers: <span id="progressText" data-upload-progress-target="progressText">Warming up...</span></div>
      <div id="progress-bar" style="background: gray; position: relative; width: 200px; height: 20px;">
        <div id="progressWidth" data-upload-progress-target="progressWidth" style="width: 0%; height: 100%; position: absolute; background: green;">
        </div>
      </div>
    </div>
  

  

The progress bar is hidden when the page loads and we use Stimulus targets on our progress bar container, progress text, and progress width elements. The Stimulus controller uses these targets to update the DOM as direct-upload:progress events are emitted.

Finally, on the form’s submit button, we listen for a click. When the form submit button is clicked, we call upload-progress#showProgress to remove the hidden style from the progress bar container.

Adding the video converter service

Now we’ve got our profile videos uploading nicely and we are informing users of the progress throughout.

Great progress, but we are accepting and displaying any video type and our goal is to build a video converter so that we have a predictable video type on profile videos. For this tutorial, let’s say that our requirements are that all uploaded videos must end up as MP4s.

Before we dive in to the details, let’s review what we want to accomplish at a high level. We want to:

Allow a user to upload a video file through the User Profile form
When a video file is uploaded, check the video’s content type
If the video is already an mp4 file, we don’t need to convert the video - our work is done
If the video is not an mp4 file, we want to transcode the video from its original type to .mp4

So, how do we accomplish step four? Converting a video on the fly sounds complicated, right?

Enter FFmpeg, an incredibly popular, open source solution for working with video (and audio, if that’s your thing).

While we can work with FFmpeg directly, we’re going to use a gem to make our interaction with FFmpeg a little simpler: streamio-ffmpeg

Let’s start by adding the gem to our project with bundle add streamio-ffmpeg and installing ffmpeg on our system with brew install ffmpeg on a Mac. Other installation options can be found here

With FFmpeg ready, let’s add a service to handle video conversion with mkdir app/services && touch app/services/video_converter.rb

Here’s what the service looks like:

  
    # app/services/video_converter.rb

    class VideoConverter
      def initialize(user_id)
        @user = User.find(user_id)
      end

      def convert!
        process_video
      end

      private

      def process_video
        @user.profile_video.open(tmpdir: "/tmp") do |file|
          movie = FFMPEG::Movie.new(file.path)
          path = "tmp/video-#{SecureRandom.alphanumeric(12)}.mp4"
          movie.transcode(path, { video_codec: 'libx264', audio_codec: 'aac' })
          @user.profile_video.attach(io: File.open(path), filename: "video-#{SecureRandom.alphanumeric(12)}.mp4", content_type: 'video/mp4')
        end
      end
    end
  

  

The important part of the service is the process_video method, so let’s zoom in there.

First, we open the existing profile video attached to the user, so we can access the video’s path.

@user.profile_video.open(tmpdir: "/tmp") do |file|

Next, we create a Movie object with the streamio-ffmpeg gem, using the original file uploaded by the user, which we’re going to transcode shortly.

movie = FFMPEG::Movie.new(file.path)

The path variable assignment is the location where we will create the new, transcoded video. Using this new path and the Movie object, we call the transcode method from the streamio-ffmpeg gem.

Finally, we attach the newly created video to the user, replacing the video that was previously uploaded. Easy!

@user.profile_video.attach(io: File.open(path), filename: "video-#{SecureRandom.alphanumeric(12)}.mp4", content_type: 'video/mp4')

Adding a background job

One more step before we can start converting uploaded videos. Working with video can be time and resource intensive. We don’t want to convert a video during a page turn or block our application servers with expensive video processing.

Let’s add a job that we can use to enqueue video processing jobs in the background. For this tutorial, we’ll use ActiveJob with the default :async adapter, but in production you’ll want to use a real background processor.

Add the job with rails g job convert_video

Our job receives a user id and calls the video converter service for that user, like this:

  
    class ConvertVideoJob < ApplicationJob
      queue_as :default

      def perform(user_id)
        VideoConverter.new(user_id).convert!
      end
    end

Converting uploaded videos

Now we’re ready to convert videos. In our UsersController, we’ll enqueue the background job in our create and update methods, after the user is saved.

  
    # app/controllers/users_controller.rb

    def create
      @user = User.new(user_params)

      respond_to do |format|
        if @user.save
          ConvertVideoJob.perform_later(@user.id)
        # snip response boilerplate
        end
      end
    end

    def update
      respond_to do |format|
        if @user.update(user_params)
          ConvertVideoJob.perform_later(@user.id)
        # snip response boilerplate
        end
      end
    end
  

  

Now when we upload a profile video, our VideoConverter service will convert the video to mp4 and replace the uploaded video with the newly converted video.

Try it out by uploading any non-mp4 video from the users form. In your server logs you should see output that looks like this if everything is working as expected:

INFO -- : Transcoding of /tmp/ActiveStorage-22-20210428-10611-1wvew4g.mov to tmp/video-9ZrvVFnAZZTJ.mp4 succeeded

But wait a minute. Transcoding can take a while and it happens in the background, so when I upload a video, the show page will render the non-transcoded video until after transcoding is complete. That’s going to cause all kinds of weird bugs, right?

Right.

Let’s get fancy with Action Cable, plus a few more sprinkles of Stimulus to track video conversion progress and render the converted video without a page turn.

Using Action Cable to broadcast video updates

First let’s talk through the problem again at a high level. We want to accomplish something like this:

When a video is uploaded, check to see if it is an mp4
If it isn’t, flag the video as needs conversion
If the video is flagged as needs conversion, don’t render the video element on the user’s show page
If a video is being processed, communicate that to the user
If the video has been processed, or did not need to be processed, render the video element on the show page

Let’s start with the logic we skipped in the last section to check if the video needs to be transcoded. Since there’s no point in converting an mp4 to an mp4, we should check the uploaded video’s content type before doing any processing.

If the content type is video/mp4, then we don’t need to transcode the video. If the content type is anything else, flag the video and start converting.

Before we write the logic code, let’s add a boolean value to our User model to track whether the profile video needs to be converted:

rails g migration AddConvertVideoToUsers convert_video:boolean

Migrate your database before moving on rails db:migrate

Now we can add the logic to our controller. In the real world, the controller probably isn’t the right place for this code, but we’re here to learn, so let’s stay focused.

  
    # app/controllers/users_controller.rb

    def create
      @user = User.new(user_params)
      respond_to do |format|
        if @user.save
          update_conversion_value
          ConvertVideoJob.perform_later(@user.id)
        # snip render logic
        end
      end
    end

    # PATCH/PUT /users/1 or /users/1.json
    def update
      respond_to do |format|
        if @user.update(user_params)
          update_conversion_value
          ConvertVideoJob.perform_later(@user.id)
        # snip render logic
        end
      end
    end

  private
    # snip
    def update_conversion_value
      return unless @user.profile_video

      needs_conversion = @user.profile_video.content_type != "video/mp4"
      @user.update_column(:convert_video, needs_conversion)
    end
  

  

The slightly clunky update_conversion_value method checks the content type of the profile video and assigns the appropriate value to the convert_video column.

Now that we’ve got this logic in place, we can head into our converter service and put that logic to use.

  
    # app/services/video_converter.rb
    def convert!
      return unless @user.convert_video?
      
      process_video
    end

This change ensures that our video converter doesn’t convert the video unless it has been flagged for conversion.

Next we want to update the value of convert_video after we’ve converted non-mp4 videos. We can do that with a new update_needs_conversion method in our service:

  
    # app/services/video_converter.rb

    class VideoConverter
      def convert!
        return unless @user.convert_video?

        process_video
        update_needs_conversion
      end

      private

      # Snip

      def update_needs_conversion
        @user.update_column(:convert_video, false)
      end
    end
  

  

Now when we upload a video, the converter will return without doing anything if the video is an mp4, otherwise it will convert the video and then update the convert_video flag on the user to false.

With this flag in place, we can add in our Action Cable and Stimulus magic to communicate a video’s conversion status to the user so they can see their uploaded video without refreshing the page.

Let’s remember our goal for this part of the project. We want to:

Check the video’s conversion status when we render the user show page
If the video is converted, display the video in a video element
If the video is not converted, display a placeholder for the video and communicate the video’s conversion status to the user
When the video finishes converting, automatically update the content of the user’s show page to display the video in a video element

Let’s dig in.

First, we need to generate an Action Cable channel to broadcast from. As usual, Rails comes with a built in converter for this. Use rails g channel VideoConversion to generate the new channel

Update the generated video_conversion_channel.rb file to look like this:

  
    class VideoConversionChannel < ApplicationCable::Channel
      def subscribed
        stream_from "video_conversion_#{params[:id]}"
      end
    end

This channel is responsible for broadcasting the progress of the video conversion process, updating a percentage complete element in the UI.

Next, let’s add the Stimulus controller that will listen for events from this channel and update the UI as it receives them.

First touch app/javascript/controllers/conversion_progress_controller.js

And then:

  
    // app/javascript/controllers/conversion_progress_controller.js

    import { Controller } from "stimulus";
    import consumer from "channels/consumer";

    export default class extends Controller {
      static targets = [ "progressText" ]
      
      initialize() {
        this.subscription = consumer.subscriptions.create(
          {
            channel: "VideoConversionChannel",
            id: this.element.dataset.id,
          },
          {
            connected: this._connected.bind(this),
            disconnected: this._disconnected.bind(this),
            received: this._received.bind(this),
          }
        );
      }

      _connected() {}

      _disconnected() {}

      _received(data) {
        this.updateProgress(data * 100)
      }

      updateProgress = (progress) => {
        let progressPercent = ''
        if (progress >= 100) {
          progressPercent = "100%"
        } else {
          progressPercent = Math.round(progress) + "%"
        }
        this.progressTextTarget.innerHTML = progressPercent
      }
    }
  

  

This controller contains a lot of Action Cable boilerplate, don’t let it overwhelm you.

The important parts are the channel we subscribe to in the initialize method and the _received method. When a new message is broadcast on the channel that the user is subscribed to (from the id in the initialize method), _received calls updateProgresswhich updates the DOM with the progress value broadcast by Action Cable.

Let’s wire this controller up to our HTML and start to bring it all together.

We want to subscribe to updates on a particular user, which means we need to update our show view to connect to the ConversionProgress Stimulus controller.

  
    <!-- views/users/show.html.erb -->

    <!-- snip -->
    <div data-id="<%= @user.id %>" data-controller="conversion-progress" style="max-width: 500px; max-height: 500px;">
      <% if @user.convert_video? %>
        <div>
          <p>We are converting your video. The video is currently <span data-conversion-progress-target="progressText">0%</span> processed</p>
        </div>
      <% else %>
        <video controls style="max-width: 100%; max-height: 100%;">
          <source src="<%= url_for(@user.profile_video) %>">
        </video>
      <% end %>
    </div>
    <!-- snip -->
  

  

Here we’ve added a data-id and a data-controller to the video’s parent video. data-id is used by the Stimulus controller to know which channel to subscribe to updates from, and the data-controller is used to connect the Stimulus controller to the DOM.

The other change here is adding logic to display the video as-is when the video has been converted, otherwise, we render text that our Stimulus controller will update as it receives updates from Action Cable.

After making these changes, if you upload a new non-mp4 video to a user and visit the show page, you’ll see the “We are converting your video” text but the percentage processed will never update. That is because we aren’t yet broadcasting the conversion progress to the Action Cable channel.

Fortunately, the FFmpeg gem makes broadcasting progress really simple. Let’s update the transcode call in our video converter service to broadcast the change:

  
    movie.transcode(path, { video_codec: 'libx264', audio_codec: 'aac' }) { |progress| ActionCable.server.broadcast("video_conversion_#{@user.id}", progress) }

Now we’ll see our progress text count up from 0% to 100% when we upload a new video that needs conversion. Depending on the video’s properties and your computer’s power, this might be a very fast process or it could take several minutes. Either way, you can now watch the progress in real time!

When the percentage gets to 100% you’ll notice one final issue blocker that we need to solve. Instead of replacing the video placeholder with the actual video when it reaches 100% conversion, the counter just stays there and the user has to refresh the page to see the video. We can fix that with a little more Action Cable, and a little more Stimulus.

First, add another Action Cable channel with rails g channel ConvertedVideo and update the subscribed method in the generated _channel.rb file:

  
    class ConvertedVideoChannel < ApplicationCable::Channel
      def subscribed
        stream_from "converted_video_#{params[:id]}"
      end
    end

Then add a new Stimulus controller for subscribing to the channel and managing updates with touch app/javascript/controllers/converted_video_controller.js

And add the code to subscribe and handle broadcasts on the ConvertedVideo channel.

  
    // javascript/controllers/converted_video_controller.js

    import { Controller } from "stimulus";
    import consumer from "channels/consumer";

    export default class extends Controller {
      static targets = ["videoContainer"];

      initialize() {
        this.subscription = consumer.subscriptions.create(
          {
            channel: "ConvertedVideoChannel",
            id: this.element.dataset.id,
          },
          {
            connected: this._connected.bind(this),
            disconnected: this._disconnected.bind(this),
            received: this._received.bind(this),
          }
        );
      }

      _connected() {}

      _disconnected() {}

      _received(data) {
        const videoElement = this.videoContainerTarget
        videoElement.innerHTML = data
      }
    }
  

  

This Stimulus controller is very similar to the last controller. In it, we subscribe to the ConvertedVideoChannel with the user id. When data is broadcast on the channel, the Stimulus controller looks for a videoContainer DOM element and replaces the content of that element with the data sent from Action Cable. We’ll see what that data looks like next.

Our goal here is to replace the container of the video placeholder element with the actual video once it has been processed. We can do this with Action Cable by taking advantage of the fact that we can render a view partial to a string and broadcast that string from Action Cable, making it easy to replace DOM content with HTML broadcast in a message.

To start, let’s add a partial that renders the video element, and add that to our the view. While we’re in the show view, we’ll also connect the new Stimulus controller to the DOM.

First run touch app/views/users/_profile_video.html.erb in your terminal and then:

  
    <!-- views/users/_profile_video.html.erb -->

    <video controls style="max-width: 100%; max-height: 100%;">
      <source src="<%= url_for(user.profile_video) %>">
    </video>

  
    <!-- views/users/show.html.erb -->

    <div data-id="<%= @user.id %>" data-controller="conversion-progress converted-video" data-converted-video-target="videoContainer" style="max-width: 500px; max-height: 500px;">
      <% if @user.convert_video? %>
        <div>
          <p>We are converting your video. The video is currently <span data-conversion-progress-target="progressText">0%</span> processed</p>
        </div>
      <% else %>
        <%= render "profile_video", user: @user %>
      <% end %>
    </div>
  

  

Now we’ve moved the video element to a partial, and updated our show view to render the partial when the video does not need to be converted.

Take note of the first line above. We’ve added the new converted-video Stimulus controller to the video container’s data-controller attribute. This connects the controller to the DOM and ensures that visitors to the show page are subscribed to the ConvertedVideo channel. We’ve also added a data-converted-video-target attribute to the same <div>. This target is used by the Stimulus controller to replace the progress text with the video element.

The last step is to update the VideoConverter service to broadcast a message containing the profile_video partial on the ConvertedVideo channel after the video has been converted.

    # app/services/video_converter.rb

    def convert!
      return unless @user.convert_video?

      process_video
      update_needs_conversion
      render_processed_video
    end

  private

    # Snip
    def render_processed_video
      partial = ApplicationController.render(partial: "users/profile_video", locals: { user: @user })
      ActionCable.server.broadcast("converted_video_#{@user.id}", partial)
    end

Here we’re calling a new render_processed_video method from our convert! method. This method renders a partial to a string and then broadcasts that string as data, to be picked up and used by our Stimulus controller. Magic.

Let’s see it in action.

Wrapping up

Thanks for making it through this guide! You can find the full source code for this guide on Github.

To recap, today we started with a fresh Rails 6 application. With the power of Stimulus, Active Storage, Action Cable and, most importantly, FFmpeg, we built an app that can convert a user-uploaded video file to mp4 when needed. While the file is converting, we communicate the progress to the user and display the converted user to them without asking them to reload the page.

To make the code in this tutorial production-ready, besides cleaning up the code and styling things, you should spend time validating file uploads from the user, add a real background job processor, and add error-handling and resilience to the video conversion service.

You might also add validation for file size to the videos, on both the client and server, and consider more efficient methods for converting videos to mp4 with more powerful FFmpeg functionality.

Further resources and contact info

If you’d like to see a production version of this project, take a look at the demo for Vestimonials, the product I’m building when I’m not writing technical articles like this one.

If you have questions or feedback on this article, you can find me on Twitter

Thanks for reading!

Quit Doing Stupid Shit

2021-01-27T00:00:00+00:00

At one point in my time at CareerPlug, the company adopted a phrase that I said in passing a few times - “Quit doing stupid shit” - as a company-wide mantra.

Was that a great phrase for the company to rally around? Probably not.

Most office workers do some amount of pointless busy work in their day (some more than others, perhaps), but saying that out loud can rub people the wrong way. No one wants to hear that their bosses think they’re doing stupid shit!

In spite of the insulting phrasing, there’s a lesson to learn from the mantra.

After leaving CareerPlug, I’ve been reflecting on the events that led me to the end of my time there. Chief amongst those reasons was intense burnout. After ten years of grinding to build the business, my brain and body needed a break, and they told me that by breaking down.

By the end of 2020, I was constantly sick, my cholesterol was through the roof, I was exhausted, and incapable of focusing on anything for more than a few minutes. I got diagnosed with two separate autoimmune disease in Q4. Both diseases are genetic and weren’t caused by the burnout; however, I believe the timing of the emergence of symptoms is directly related to the burnout I experienced.

What caused the burnout? Doing stupid shit, repeatedly.

I didn’t follow my own advice, even as I pushed for my team and the company to focus on what mattered most. I did all kinds of things that didn’t need to be done and put pressure on myself to do more and more, even when I knew that I was doing things that were unimportant.

Why didn’t I follow my own advice?

Focusing is hard. Especially in a bootstrapped business when resources are extremely tight. There’s always something else to do, and only so many people to delegate to. I took on tasks to avoid burning my team out, and paid the price.
Pushing back doesn’t come naturally to me. I like to help people. Other people realized that, and got into the habit of coming to me to solve things for them. I didn’t do a good enough job protecting my time, especially in the early years, leading to me inheriting many low value tasks that couldn’t be delegated.
I was too optimistic. I committed to things that were not realistic because I believed we would find a way to get them done. I set outrageous deadlines, missed them, and then tried to push harder to make up for the missed deadlines.
I was unhappy. The work the I love doing is building. CareerPlug was no longer primarily interested in building. The company got bigger, teams began competing for resources, new product work became harder and more fraught. Every change was mentally taxing. The joy in the work went away and I tried to grind harder in response, despite the bad feelings.

What did I learn from burnout land?

Focus is a company-wide effort. If you preach focus at an individual level, but everyone has a different focus, no one will focus on anything.
Work towards clear, measurable outcomes. If you don’t, you will end up doing all kinds of work that has no impact on the business without having any idea what work matters and what work is worthless. If you don’t know where you’re trying to go, you won’t get there.
Being a short-term pessimist has advantages. Optimism is necessary to succeed in the long term - big wins come from big bets. In the short term, when it comes to delivery, being a pessimist gives you space to deal with unseen problems without blowing a deadline. Stay positive about the future, be pessimistic about what you will deliver in the short term.
Loving your work can carry you through hard times, but you’re on thin ice. I worked long hours for years without feeling the pain, largely because the work was energizing. Once the work was no longer energizing, I was in an unhealthy place with a rapidly depleting gas tank.

Moving on to the next phase in my career, I’m hopeful I will be able to identify burnout-producing busy work more effectively, for myself and, more importantly, for my company. Tune out the noise, focus on what matters, and measure the outcomes your work produces. Easy.

Building a custom Stimulus generator for Rails

2020-11-30T00:00:00+00:00

If you’ve spent any time working with Ruby on Rails, you know that using generators can save you from a lot of console commands and copy/pasting from one file to the next. Need a new model? Just run rails g model MyNewModel name:string description:string and you’re set.

Rails covers most of the common use cases for you with built-in generators. If the built-in generators don’t meet your needs, you can get fancy by creating your own custom generators. Custom generators can save you time any time you’re finding yourself running lots of touch commands and copy/pasting boilerplate into new files.

Although Rails provides built-in generators for most things that Rails projects need, one common use case that’s missing is a generator for new Stimulus controllers. Stimulus , a modest JavaScript framework created by the team at Basecamp, is an increasingly common part of the Rails stack. Despite its growing popularity, Rails does not provide a generator out of the box (as of Rails 6.0). Without a generator, every new Stimulus controller requires a manual touch app/javascript/controllers/my_new_controller.js in the console and then copying from another controller, changing the class name, and otherwise doing stuff uninteresting stuff before you start building something new. No fun.

Today we’re going to solve this problem by learning how to build our own custom generator for Rails. Our custom generator will create new a new Stimulus controller in the app/javascript/controllers directory when we run rails g stimulus ControllerName.

Let’s get started.

Setup

I’m writing this assuming that you have a Rails 6 project and you’re working from that project in your terminal.

If you don’t have a Rails project handy, just run rails new stimulus_generator_project and then cd stimulus_generator_project to follow along.

Generating a generator

Rails is known for valuing developer experience, and Rails generators are a prime example of that love for developers. Rails provides a ton of generators out-of-the-box including a generator to generate new generators. Very meta stuff, and very convenient for us.

We’ll use the generator generator to generate a new Stimulus generator. Still with me? Great. Run this in your console and let’s move on from generating generators:

rails g generator stimulus

This applies Rails magic to create the bones of our generator, named stimulus, in the lib directory. You’ll notice that inside of the lib/generators/stimulus directory there are a few files and an empty templates directory. We’ll briefly talk about what each of these items is and then we’ll create our generator.

stimulus_generator.rb: This is the file that runs when you type rails g stimulus in your console. The class defined here will manage creating a new file each time the generator runs, and populating that file with the content from our template Stimulus controller.

USAGE: This file contains instructions for users of your generator. The contents of this file are printed when you type rails g stimulus in your console.

Templates: The templates directory contains any files that your generator will copy when it runs. In our case, we’ll just have one file. These template files can access and insert variables into the template, when needed.

stimulus_generator_test.rb: The generator generator helpfully creates a test file, which can be run (in our case) with rake test. Our generator is simple, but we’ll add a test to make sure it works as we expect.

Adding our code

As a reminder, we’re building a generator to create a new Stimulus controller. This generator will only accept a name as an argument and will always create the controller in the app/javascript/controllers directory when it runs.

Let’s first add a test to our stimulus_generator_test file. We’ll use this test to make sure our generator works without having to generator a Stimulus controller we don’t want.

In lib/generators/stimulus/stimulus_generator_test.rb add a test that runs our generator and validates that a new file is generated as expected:

  
require 'test_helper'
require 'generators/stimulus/stimulus_generator'

class StimulusGeneratorTest < Rails::Generators::TestCase
  tests StimulusGenerator
  destination Rails.root.join('tmp/generators')
  setup :prepare_destination

  test 'It generates a controller in the app/javascript/controllers directory' do
    run_generator ["Hello"]

    assert_file "app/javascript/controllers/hello_controller.js"
  end
end
  

  

When we run rake test the test will fail, which is expected since our generator hasn’t been built yet. Let’s work on making it pass.

First, create the template file that our generator will copy each time it runs. This file will live in the templates directory:

touch lib/generators/stimulus/templates/controller.js

In this new file, we’ll insert the content of our Stimulus controller:

  
import { Controller } from 'stimulus'

export default class extends Controller {
  static targets = []

  initialize() {
    // Called once, when the controller is first instantiated
  }

  connect() {
    // Called any time the controller is connected to the DOM
  }

  disconnect() {
    // Called any time the controller is disconnected from the DOM
  }
}
  

  

My template includes the basic structure of a Stimulus controller plus the lifecycle methods I find myself using most often, that’s what makes the most sense to me. You can include more lifecycle methods or just leave out everything but the class definition, you’re the boss of your own template file.

With our template file created, we can go into our generator file and tie the generator and template together:

  
class StimulusGenerator < Rails::Generators::NamedBase
  source_root File.expand_path('templates', __dir__)

  def create_controller
    template('controller.js', File.join("app/javascript/controllers/#{file_name}_controller.js"))
  end
end

Each time rails g stimulus Something is called, any method in this file will run. In our case, we just create a new file from our template using the template method (provided by Thor, if you’re interested in the inner workings of Rails generators). If we want to handle conflicting file names, make our file name generation smarter, process additional arguments or otherwise make our generator more powerful we can add additional methods to do so. For our purposes, we will keep things simple.

With our template and our generator class built, let’s see if our tests pass:

Perfect! We’ve got a working generator and we can run that generator with rails g stimulus CoolIdea.

If you’ve followed along with me through this tutorial, when we run the generator our new controller will be created at app/javascript/controllers/cool_idea_controller.js and it will be populated with the content from our controller template file.

Congratulations on making it this far! You’re a star.

Our last step is to update the USAGE file for our generator with useful information for others who encounter our generator in the future. Something like this will work fine:

Description:
Stubs out a new Stimulus controller.
This generator accepts one argument, the name of the controller, CamelCased or under_scored.
Example:
`rails generate stimulus Thing`
This will create a new Stimulus controller at `app/javascript/controllers/thing_controller.js`

Now when we run rails g stimulus our help text will be printed to the terminal. The generator is pretty simple today, but in the future we might add additional arguments or options. Starting with documentation from day one is always the better plan.

Wrapping up

Today we learned how we can use the magic of Rails to create our own generators and we used that knowledge to fill a gap in the Rails defaults by creating a simple generator to add new Stimulus controllers.

While our example was simple, the same technique can be applied to create sophisticated new generators for Service Objects, complex Stimulus controllers, or any other resource you find yourself needing. In addition to creating our own custom generators, we can also use this knowledge to change the default generators provided by Rails. Overriding the default model or controller generators can help your organization enforce code standards across an organization or just save you from copying boilerplate across your project.

For more reading on the power of Rails generators, start with the Rails guide on generators. Finally, if creating your own Stimulus generator isn’t a chore you’re looking forward to, I’ve published a gem you can use to add Stimulus generator just like the one in this tutorial to your Rails projects.

Thanks for reading!

Shape Up reflections: Volume 2

2020-11-28T00:00:00+00:00

This post is a lightly edited version of an internal update I sent to everyone at CareerPlug after the completing our second product cycle, modeled on Basecamp’s Shape Up method. I am making these updates public in hopes that others will find value in the lessons learned from a real team adopting a fairly radical new product development process.

The first update includes more information about our product team and our company, for context. Italicized text are extra context I’ve added for the public version of this update.

Let’s dig in. -David

Team - Cycle 2 is in the books! This was our second cycle following our new product development process and our first with 6 weeks of cycle time instead of 4. We made great progress on some key initiatives this cycle and the team learned a lot of lessons to make future cycles more effective. We’ll keep learning as we go, but we should feel proud of the learning we’ve done in the first two cycles and the work we’ve produced.

Below you’ll find an overview of the things that worked well, the things that didn’t work, and our lessons learned for future cycles. At the end of this reflection, I’ll look at some of the work we released in the first cycle and review the results so far.*

What worked

Six weeks is a better cycle length Extending cycle length from four to six weeks gave us more time to build impactful features and more space to explore options and pivot when needed. The biggest changes in this cycle would not have been possible to build in four weeks.

Small changes and open design space can go a long way In Cycle 2 we released a number of small changes that didn’t add more mental overhead for our users but still made their experience with the product better. This is possible because of the work we’ve done to make the product more focused. Over time, the product will get less focused and become harder to reason about without concerted effort to keep things as simple and as relevant to the user’s goals as possible.

We’re adapting quickly and learning from mistakes This is still a very new process for the product org and for the company. We don’t expect it all to just work right away. As we encounter new issues or something unexpected goes wrong, we’re identifying the problem and implementing solutions so we’re better prepared for next time. This will continue being a theme for the next few cycles - getting a little better each time is the goal.

An example of an in-cycle adjustment is that each project team now updates the status of their project with a red/yellow/green indicator so that we can identify off track projects and quickly review the status of all projects as needed. [David note: This small change was implemented as a new field in Jira, which engineering and QA use to track their in-cycle work. Hill charts would be more useful, but we haven’t adopted Basecamp as an organization.]

What didn’t work

Poorly written pitches A few pitches did not set the project team up for success. These were written by me and the miss on these was down to me not doing a good enough job in the shaping process.

The pitch for new account onboarding included way too many ideas, far more than could be achieved in a single six-week cycle. That pitch should have been scoped down to focus on a single impactful, achievable outcome and the rest of the work should have been included in a future pitch
The pitch for the new activation process needed to have the content of the page written and approved before the project began. Leaving marketing copy and pricing page layout copy out of the pitch made it difficult to build the right experience and led to way too much back and forth to agree on details that aren’t in the project team’s purview.

[David note: One of our major challenges has been finding the right level of detail to include in pitches. I left too many things open to interpretation in this past cycle and teams spent more time spinning their wheels than they should have. Finding the right balance is tough!]

Communication needs to be better Communication inside of the product org and to other teams was not as good as it should have been. This isn’t just a Shape Up challenge. Communicating clearly, thoughtfully, and in a timely manner is hard and we’re still learning how best to make it all tie together.

Inside the product org, there were several instances when a project team got off track on a project and did not raise the flag to their manager or the pitch owner quickly enough. This delay led to misses that could have been avoided with faster, more effective communication
Outside of the product org, several projects needed significant input or sign off from other teams and we did not communicate effectively with those teams, leading to delays in projects and missed requirements. [David note: Communication is hard and process is only part of the challenge. Our team is tight knit, kind, and supportive, but we also have some ingrained habits like not asking for help soon enough (because we don’t want to bother our peers who are busy) that will take a concerted effort to overcome. Shape Up isn’t a magic bullet and you’ve still got to do the work!]

Outcome tracking isn’t great Tracking outcomes is very manual and outcome definition is unclear. Since desired outcomes are just being tracked on the pitch, looking back at the work in a cycle and identifying how many projects achieved their desired outcome is difficult and time-consuming. This difficulty is compounded by the vague outcomes on some projects, particularly projects aimed at bringing in functionality to Voyager that exists in Classic but doesn’t have a clear business objective. /[David note: Voyager is the new version of our core software, Classic is the old version. We’ve migrated most of our user base to Voyager but the work to bring the rest of over has been difficult to fit into the Shape Up framework since the business value for some of these changes is quite low.]/

What we learned

Communication is (still) key If we write really clear product pitches, hand the pitch to the project team cleanly, and stay tightly connected to the pitch owner while we’re building, the work goes smoothly.

When we miss on one of those three key communication pieces, things get bumpy.

Pitches need more definition Through the first two cycles I intentionally left pitches open for more interpretation by the product teams. The reason for this is that our team, unlike the Basecamp team who created Shape Up, doesn’t have a huge amount of ingrained UX design experience on the selection committee. Instead of closing off the opportunities for our UX designers to do what they do best, I created pitches that left them room for research; however, I went too far in this direction and some of our pitches haven’t been clear enough to be really actionable and achievable in the appetite we’ve assigned.

In upcoming cycles we’ll need to find ways to include the UX team in the creation of pitches so that work is more clearly defined before it reaches the betting table or gets assigned to a project team.

What’s next

Cycle 3 is a holiday-shortened cycle. We’ve intentionally committed to less work in this cycle (no more than 4 weeks per project team) and we’re keeping the work more internally focused so that we can comfortably launch these changes when they’re ready even if we’re short staffed on the client-facing teams.

Cycle 1 early results

As mentioned above, outcome tracking is more manual than we would like right now, but we can identify the impact that some of the Cycle 1 launches have had so far: [David note: I removed the individual results here since they contain a lot of pretty confidential information about our client base. Each update was structured as a link to the project in our product management tool + a brief run down of the current state of the project vs. the expected outcome in the original pitch. In general, the work we released in our first cycle performed reasonably well, with a couple that underperformed expectations and a couple that exceeded expectations. Sharing outcomes internally has been very well received - our product organization’s results are harder to measure than other teams and the transparency into how our work is performing has helped build trust. Happy to share more details on the structure privately, for anyone interested.]

Building an instant search form with Stimulus.js and Rails

2020-11-15T00:00:00+00:00

August 5, 2021 update: Since publishing this post, Turbo was released. While the method described here for building a live search interface with Stimulus still works just fine, if you’re using Stimulus, you might be interested in the other half of Hotwire on the web. In August of 2021, I published a post describing how to build the same interface described here with hotwire-rails, instead of just Stimulus.

Today we’re going to build a search form that instantly filters results using Ruby on Rails and Stimulus, a “modest” JavaScript framework designed to add small bits of interactivity without a lot of heavy lifting.

When we’re finished, we’ll have a list of Players that users can search by name. As they type in the search box, we’ll make a request to the server for players that match and display the results on the frontend (almost) instantly. Here’s what the end result will look like:

I’m writing this assuming that you’ve already got Rails installed on your machine and that you are generally familiar with Rails and JavaScript. If you’re just getting started with either, you can still follow along, but you might find some things confusing!

I’m also not going to do a full overview of how Stimulus works, since the official docs do that well. Instead, our Stimulus work will be focused on building the desired behavior in Stimulus and connecting our Stimulus code to our HTML.

I’m going to walk through this step-by-step from a new Rails 6 project. If you want to follow along from an existing project that already has Stimulus and rails-ujs installed, skip slightly ahead to the Adding a search form section.

Let’s get started.

Setting up our project

First, we’ll create our Rails project, specifying that we want to include Stimulus:

rails new stimulus_live_search_demo --webpack=stimulus
cd stimulus_live_search_demo

Then, we’ll create our database and scaffold up a Player resource that our users will be soon be searching against:

rails db:create
rails generate scaffold player name:string
rails db:migrate

That’s it — that’s all the setup we need to do. Now we can dive into the fun stuff.

Our users will need a search form if we want them to be able to search. We’ll first create a partial for our search form:

mkdir app/views/shared
touch app/views/shared/_search_form.html.erb

And then update app/views/players/index.html.erb to render the search form:

  
<!-- app/views/players/index.html.erb -->
<p id="notice"><%= notice %></p>

<h1>Players</h1>
<div>
  <%= render partial: "shared/search_form" %>
  <div>
    <% @players.each do |player| %>
      <%= render partial: "player", locals: { player: player } %>
    <% end %>
  </div>
</div>

<br>

<%= link_to 'New Player', new_player_path %>
  

  

And add our player partial, which we’ll use later to render our search results:

touch app/views/players/_player.html.erb

  
<!-- app/views/players/_player.html.erb -->
<div>
  <%= player.name %>
</div>

Our search form partial looks like this:

  
<!-- app/views/shared/_search_form.html.erb -->

<%= form_with url: player_search_index_path, method: :get do |form| %>
  <div>
    <%= form.text_field :search, autocomplete: "off" %>
  </div>
<% end %>

You’ll notice that our search form url references a controller that we haven’t created yet, the PlayerSearch controller.

While we could build a search method in our PlayersController creating a separate controller lets us keep our controller actions RESTful and is a little more Rails-y, in my opinion.

Let’s add our controller now:

rails generate controller PlayerSearch index

  
class PlayerSearchController < ApplicationController
  layout false
  
  def index
    @players = Player.where("name LIKE ?", "%#{params[:search]}%")
  end
end

Notice the layout false which ensures that our index view will only render the HTML in app/views/player_search/index.html.erb. If we don’t set layout to false, every search request we make will render our index view inside of the layout defined in app/views/layouts/application.html.erb — we don’t want that since we’re planning to insert the search results into a piece of an existing page.

Next we’ll add the content of our PlayerSearch index view. Our view simply loops over an array of players and renders the player partial each iteration.

  
<!-- app/views/player_search/index.html.erb -->
<% @players.each do |player| %>
  <%= render partial: "players/player", locals: { player: player } %>
<% end %>

Finally, we’ll update our routes file so that we can access thePlayerSearch index method.

  
Rails.application.routes.draw do
  resources :players
  resources :player_search, only: [:index]
end

Now that we have the search form rendering and the controller action built, our next step is to implement the Stimulus search controller.

The Stimulus controller

First, let’s create our Stimulus controller:

touch app/javascript/controllers/search_controller.js

This controller has two jobs — submitting the search form as the user types and handling the response from the server each time a search query completes.

Let’s implement form submission first, in a function we’ll call search:

search() {
  Rails.fire(this.formTarget, 'submit')
}

Rails.fire is, a poorly documented function available through rails-ujs. It takes an element (formTarget, in this case, which we’ll attach add to our HTML shortly) and the name of an action to trigger (like click or submit) and then triggers that event on the element. Despite the poor documentation around this method, it is the preferred method of submitting forms programmatically in Stimulus controllers.

To access Rails.fire in our Stimulus controller, we need to assign Rails to the window. We can do that with a small update to app/javascript/packs/application.js.

  
window.Rails = require("@rails/ujs")
require("turbolinks").start()
require("@rails/activestorage").start()
require("channels")

import "controllers"
Rails.start()

Now we can submit the search form programmatically.

Next, let’s add the function to handle search results from the server. We’ll do this with a new function in our Stimulus controller:

  
handleResults() {
  const [data, status, xhr] = event.detail
  this.resultsTarget.innerHTML = xhr.response
}

This function grabs the inbound response from the server and replaces the contents of a (yet-to-be-defined) DOM element with the portion of the response we care about.

If the variable assignment looks mysterious to you, check out the Rails guide Working with JavaScript in Rails which describes the event handlers available in rails-ujs and the parameters that those event handlers include. In our case, we’ll be using the ajax:success event.

When finished, our complete Stimulus controller looks like this:

  
import { Controller } from "stimulus"

export default class extends Controller {
  static targets = [ "results", "form" ]

  connect() {
    console.log("Connected!")
  }

  search() {
    Rails.fire(this.formTarget, 'submit')
  }

  handleResults() {
    const [data, status, xhr] = event.detail
    this.resultsTarget.innerHTML = xhr.response
  }
}
  

  

Connecting the controller to the frontend

The Stimulus controller is ready — our last step is to connect our Stimulus controller and our HTML. To do so, we’ll need to update the index view in our Players view and the search_form partial we created earlier.

First, the updates to the the index view:

  
<!-- app/views/players/index.html.erb -->
<p id="notice"><%= notice %></p>

<h1>Players</h1>
<div data-controller="live-search">
  <%= render partial: "shared/search_form" %>
  <div data-target="live-search.results">
    <% @players.each do |player| %>
      <%= render partial: "player", locals: { player: player } %>
    <% end %>
  </div>
</div>

<br>

<%= link_to 'New Player', new_player_path %>
  

  

The data-controller attribute added to our parent div connects our Stimulus controller to the DOM. The data-target attribute on the results container div is used by handleResults in our Stimulus controller to render the results of our search when a request is submitted. If any of this syntax is confusing, the best place to start is the Stimulus handbook which explains how Stimulus uses these data attributes in detail.

Last step before we can start typing search terms — updating our search form partial:

  
<!-- app/views/shared/_search_form.html.erb -->
<%= form_with url: player_search_index_path, method: :get, data: { action: "ajax:success->live-search#handleResults", target: "live-search.form" } do |form| %>
  <div>
    <%= form.text_field :search, data: { action: "input->live-search#search" }, autocomplete: "off" %>
  </div>
<% end %>
  

  

We’re doing a few things here, let’s walk through it.

First, the data-action on our form element calls our handleResults method each time this form is submitted successfully. The data-target on the form element gives us an easy reference to the form in the Stimulus controller which we use in the search function to submit the form.

On the text field, we add a data-action that calls the search method each time the input event is fired on this input. Because using the browser’s autocomplete functionality on a search form isn’t usually desired, we also turn off autocomplete. You can skip turning off autocomplete if you like, it won’t bother me.

That’s it — as users type in the search field we automatically send a request to the server to find matching players and update the players list so users can see results in real time. Add a few players to your database and try it out!

Improving our search method

As you followed along with this tutorial, you might have noticed that our implementation submits the form every time the input event fires. This means that if the user types 10 characters in a row we’ll make 10 separate search requests to the server. In a toy application like ours that’s okay, but we can do a little better.

Instead of submitting the form each time an input event fires, we can instead wait for the user to stop typing before submitting the form. One way of doing this is by using a debounce function to delay form submission until after the user has stopped an action for a period of time. A simple implementation of debouncing that makes use of setTimeout and clearTimeout looks like this:

  
search() {
  clearTimeout(this.timeout)
  this.timeout = setTimeout(() => {
    Rails.fire(this.formTarget, 'submit')
  }, 200)
}
  

  

This change to delay the Rails.fire call until 200ms after the last input event fires.

In a real world application, instead of implementing your own debounce function, you should consider a resilient, well-architected solution from a library like Lodash. You can read more about debouncing here.

Wrapping up

Together we’ve built an instant search results with a very small amount of Stimulus and a few data attributes in our HTML. What’s really cool about this implementation is that it is reusable anywhere in your application that you want the same behavior. Since the Stimulus controller just submits a form and replaces data on the page, you could easily reuse the exact same Stimulus controller to build a search UX for any resource in your application!

From the base we’ve built in this demo application, we could add more complex search and filtering options and implement loading states to improve user experience on slower requests.

Thanks for reading!

Helpful resources

The full code for this repository can be found on Github
The Stimulus handbook
Working with JavaScript in Rails
The rails-ujs source code

Building a simple commenting form with Stimulus and Rails 6

2020-11-07T00:00:00+00:00

If you use Ruby on Rails, you’ve probably heard of Stimulus, a “modest” JavaScript framework for sprinkling in small bits of JavaScript to your mostly-HTML front-end. Instead of reaching for the monster JavaScript frameworks that have become very popular in recent years, Stimulus gives you the tools to add just enough interactivity while relying on plain-old Rails and HTML as often as possible.

To show you the power and simplicity of Stimulus we’re going to build a simple Stimulus-powered implementation of a commenting form. When we’re finished we’ll have a comment form that skips a page turn and neatly renders our new comments as we create them, without jQuery or JS partials — just a little bit of Stimulus-flavored JavaScript and HTML. I’m going to walk through this step-by-step from a new Rails 6 project. If you want to follow along from an existing project that already has Stimulus and rails-ujs installed, skip ahead to the Creating our comment form section.

I’m writing this assuming that you’ve already got Rails installed on your machine and that you are generally familiar with Rails and Javascript. If you’re just getting started with either, you can still follow along, but you might find some things confusing!

I’m also not going to do a full overview of how Stimulus works, since the official docs do that well. Instead, our Stimulus work will be focused on integrating Stimulus with Rails.

Before we start, a note: Since this tutorial was written, Stimulus 2 has been released. The below code will all work just fine with Stimulus 2; however, you’ll get some deprecation warnings in your web console. The companion repository for this tutorial has been updated for Stimulus 2 if you’d like to see what has changed in Stimulus 2.

The Setup

First, let’s create our Rails project and setup our database, from our terminal:

rails new stimulus_comments_demo --webpack=stimulus
cd stimulus_comments_demo
rails db:create

The --webpack=stimulus here is the important part. This will ensure that our new project is set up correctly and save us a lot of copy/pasting boilerplate later.

Our demo application will have Projects and each Project will have any number of comments.

Let’s setup the basics of that structure next:

rails generate scaffold project name:string
rails generate model comment body:text project:references
rails db:migrate

Now that our project has resources, let’s update our routes.rb file to set a root path:

  
Rails.application.routes.draw do
  resources :projects
  root to: 'projects#index'
end

Finally, our Projects have many comments — let’s add that association to our project model

  
class Project < ApplicationRecord
  has_many :comments
end

Whew — the Rails setup work is out of the way and we now have a project ready for our new stimulus comment form.

Verify everything is working by booting up your rails server and visiting localhost:3000 . You should see our unstyled projects page. Go ahead and add a new Project while you’re there since we’ll need that later.

Creating our comment form

With our Projects project setup, we can start building out comments. Let’s start by setting up our Rails controller for comments and adding views to support comments, from our terminal:

rails g controller Comments
touch app/views/comments/_show.html.erb
touch app/views/comments/_form.html.erb

Our Rails controller for comments will handle creation of new comments and, if you’ve ever handled a form POST in Rails, it will look pretty familiar.

  
class CommentsController < ApplicationController
  before_action :set_project

  def create
    comment = Comment.new(comment_params)
    comment.project = @project
    if comment.save
      # Instead of redirecting like we normally would, we render a partial for our Stimulus controller to handle

      # Note that we're just rendering HTML here, no need to render jbuilders or js partials

      render partial: 'show', locals: { comment: comment }, status: :ok
    else
      render partial: 'form', locals: { comment: comment }, status: :unprocessable_entity
    end
  end

  private

  def comment_params
    params.require(:comment).permit(:body)
  end

  def set_project
    @project = Project.find(params[:project_id])
  end
end
  

  

Before we move on to our views, let’s update our routes file so that our comments controller is accessible.

  
Rails.application.routes.draw do
  resources :projects do
    resources :comments, only: [:create]
  end
  root to: 'projects#index'
end
  

  

Next, we’ll add content to our comment view files, and update our project show view to render comments for the project.

  
<!-- app/views/projects/show.html.erb -->
<div class="comments" data-controller="comments">
  <div>
    <%= @project.name %>
  </div>
  <h2>Comments</h2>
  <div data-target="comments.commentList">
    <% @project.comments.each do |comment| %>
      <%= render partial: "comments/show", locals: { comment: comment } %>
    <% end %>
  </div>
  <div>
    <%= render partial: "comments/form" %>
  </div>
</div>

<!-- app/views/comments/_show.html.erb -->
<div>
  <%= comment.body %>
</div>

<!-- app/views/comments/_form.html.erb -->
<%= form_with model: Comment.new, url: project_comments_path(@project), data: { action: "ajax:success->comments#createSuccess" } do |form| %>
  <div>
    <%= form.text_field :body, data: { target: "comments.commentBody" }, placeholder: true %>
  </div>
  <div>
    <%= form.submit "Save" %>
  </div>
<% end %>
  

  

There are some important concepts in these views. Let’s walk through each one.

Our project show view connects to our Stimulus comments controller on line 2 with data-controller="comments" . The controller HTML element needs to wrap both our comment list and our comment form, otherwise the controller actions and targets on those elements won’t function.

Next, on line 7, we set our first Stimulus target with data-target="comments.commentList" . We’ll use this target in our Stimulus controller when we insert newly created comments into the DOM.

Finally, we connect our form to our Stimulus controller with data: { action: "ajax:success->comments#createSuccess on line 23.

This action listens for ajax:success events and calls the createSuccess method when an ajax:success event occurs. Remember that, by default, forms created with form_with are remote, which we want. Also note the data-target="comments.commentBody" on line 25, which we’ll use in our controller to clear the text field after submission.

At this point, if we visit a project’s show page in our Rails app and submit our comment form the form submission will work just fine, you will just have to reload the page before you’ll see each new comment.

Adding the Stimulus Controller

Our Stimulus controller is the last piece we’re missing, and it is simpler than you might imagine.

Let’s start by creating our Stimulus controller:

touch app/javascript/controllers/comments_controller.js

And add in the code we need to process the HTML that Rails sends us each time a comment is successfully POSTed to the server:

  
import { Controller } from "stimulus"

export default class extends Controller {
  // Read more about targets in the Stimulus reference guide: https://stimulus.hotwired.dev/reference/targets

  static targets = ["commentList", "commentBody"]

  // The connect lifecycle method runs each time our controller is connected to the DOM

  // Read more about lifecycle methods in the Stimulus reference guide: https://stimulus.hotwired.dev/reference/lifecycle-callbacks

  connect() {
    console.log("We're connected!")
  }

  createSuccess(event) {
    // This might look a little magical but its just deconstructing the standard rails-ujs event detail

    // Read more here: https://guides.rubyonrails.org/working_with_javascript_in_rails.html#rails-ujs-event-handlers

    const [_data, _status, xhr] = event.detail

    this.commentListTarget.innerHTML = xhr.response + this.commentListTarget.innerHTML
    this.commentBodyTarget.value = ''
  }
}

That’s it — that’s the whole Stimulus controller and now we’re handling comment creation without a page turn, like this:

How does this all just work? The magic of Rails is doing a lot of heavy lifting behind the scenes so that we can listen for ajax events and respond to them in our Stimulus code.

Specifically, rails-ujs adds event handlers for us, making it easy to handle ajax events in our JavaScript code. With rails-ujs notifying us of ajax events, Stimulus kicks in through the data-action set on the form so that we can respond to the ajax event(s) we care about and run JavaScript to process the HTML generated by our Rails server. Mostly though, magic.

If all you were looking for is how to use rails-ujs and Stimulus to handle form POSTs in Rails, you’ve reached the end of the line. Read on for a bonus guide on Tailwind CSS setup and animating our comments.

Bonus: making things fancy with Tailwind CSS and animations

While adding commenting is nice, everything is more fun when it looks and feels nicer. Let’s add a little style to our project with Tailwind CSS and, for fun, look at one way to animate the entry of our comments into the DOM, instead of having them barge in unannounced.

Adding Tailwind to our project

Tailwind is a wonderful CSS framework that is rapidly gaining popularity in the web development world. While we could add styles without a framework, setting up Tailwind is worth the effort.

First, let’s install Tailwind and add it to our project, from our terminal:

  
yarn add tailwindcss
mkdir app/javascript/stylesheets
yarn tailwind init app/javascript/stylesheets/tailwind.js
touch app/javascript/stylesheets/application.scss

Next, in our application.scss file, we need to pull in Tailwind’s base styles:

  
@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";

Finally, we need to tell webpack about our application.scss file by adding import "stylesheets/application" to app/javascripts/packs/application.js.

Since we haven’t seen it before, let’s look at the full application.js file:

  
// This file is automatically compiled by Webpack, along with any other files

// present in this directory. You're encouraged to place your actual application logic in

// a relevant structure within app/javascript and only use these pack files to reference

// that code so it'll be compiled.

require("@rails/ujs").start()
require("turbolinks").start()
require("@rails/activestorage").start()
require("channels")

// Uncomment to copy all static images under ../images to the output folder and reference

// them with the image_pack_tag helper in views (e.g <%= image_pack_tag 'rails.png' %>)

// or the `imagePath` JavaScript helper below.

//

// const images = require.context('../images', true)

// const imagePath = (name) => images(name, true)

import "controllers"

import "stylesheets/application"

Everything in this file is boilerplate generated by Rails when we setup our project except for line 21, which we just added.

Almost there — now let’s update our postcss file to tell it about our Tailwind configuration file by adding require('tailwind.css')('app/javascript/stylesheets/tailwind.js') to postcss.config.js. Again, let’s take a look at this file in full to make sure we’ve got it setup correctly.

  
let environment = {
  plugins: [
    require('tailwindcss')('app/javascript/stylesheets/tailwind.js'),
    require('autoprefixer'),
    require('postcss-import'),
    require('postcss-flexbugs-fixes'),
    require('postcss-preset-env')({
      autoprefixer: {
        flexbox: 'no-2009'
      },
      stage: 3
    }),
  ]
}

module.exports = environment
  

  

Yours might be a little different — the Rails default is to update module.exports directly instead of assigning the configuration to a variable first. Either way, just add the content of line 3 to the plugins array and you’re all set.

Next, fire up your webpack-dev-server with bin/webpack-dev-server from your terminal so that changes get compiled as you work.

Okay — last step before we can start styling things. We need to update the application layout to tell it about the new stylesheet pack that webpack is creating for us by replacing the stylesheet_link_tag in the default layout with stylesheet_pack_tag.

Here’s what your application layout should look like:

  
<!DOCTYPE html>
<html>
  <head>
    <title>StimulusComments</title>
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>

    <%= stylesheet_pack_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
    <%= javascript_pack_tag 'application', 'data-turbolinks-track': 'reload' %>
  </head>

  <body>
    <%= yield %>
  </body>
</html>
  

  

Now we’re all set to use Tailwind to add a little style to our project. Tailwind’s documentation is exceptional so I’m not going to cover how Tailwind works. Instead, I’ll just add the markup I used, all of which is just standard Tailwind classes.

Here’s our styled Project show markup:

  
<div class="container mx-auto max-w-lg border border-gray-200 p-4 mt-4 rounded">
  <div>
    <h2 class="text-2xl mb-4"><%= @project.name %></h2>
  </div>
  <div data-controller="comments">
    <h3 class="text-xl mb-2">Comments</h3>
    <div data-target="comments.commentList" class="mb-4">
      <% @project.comments.order(created_at: :desc).each do |comment| %>
        <%= render partial: "comments/show", locals: { comment: comment } %>
      <% end %>
    </div>
    <div>
      <%= render partial: "comments/form" %>
    </div>
  </div>
</div>
  

  

And our comment show partial:

  
<div class="mb-2 border-b border-gray-200 p-2">
  <%= comment.body %>
</div>

And, finally, our form:

  
<%= form_with model: Comment.new, url: project_comments_path(@project), data: { action: "ajax:success->comments#createSuccess" } do |form| %>
  <div class="mb-4">
    <%= form.text_field :body, class: "shadow appearance-none border rounded w-full py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline", data: { target: "comments.commentBody" }, placeholder: "Type your comment..." %>
  </div>
  <div>
    <%= form.submit "Save", class: "bg-transparent hover:bg-blue-500 text-blue-700 font-semibold hover:text-white py-2 px-4 border border-blue-500 hover:border-transparent rounded cursor-pointer" %>
  </div>
    <% end %>
  

  

All put together, we get a Project show page that looks like this:

Animating comment creation

While our comments get added to the DOM right away, they appear instantly, which can feel a little jarring. Let’s wrap up by animating the entry of each new comment into the DOM by first making some small updates to the createSuccess method in our comments_controller.js file.

  
createSuccess(event) {
  const [data, status, xhr] = event.detail
  const target = event.target

  // Create a wrapper div to hold our newly created comment

  const newComment = document.createElement("div")
  
  // Add our animation class to the wrapper div and set the innerHTML to the comment rendered by the server

  newComment.classList.add("fade-in-left")
  newComment.innerHTML = xhr.response

  // Add our wrapped comment to the beginning of the comment list

  this.commentListTarget.prepend(newComment)
  this.commentBodyTarget.value = ''

  // After the animation finishes (in 0.6 seconds), remove our animation class

  // If we don't remove the animation class, adding another comment will cause every comment added since our initial page load to animate together

  setTimeout(() => {
    newComment.classList.remove("fade-in-left")
  }, 600);
}

Our fade-in-left class isn’t functional yet because we haven’t added any CSS to it. We’ll add that next, with some help from animista.net so we don’t have to write our own animation.

Let’s add the following to our application.scss file, so we don’t need to create a new file as we near the end of our time together:

  
.fade-in-left {
  -webkit-animation: fade-in-left 0.6s cubic-bezier(0.390, 0.575, 0.565, 1.000) both;
          animation: fade-in-left 0.6s cubic-bezier(0.390, 0.575, 0.565, 1.000) both;
  }
  
  /* ----------------------------------------------
  * Generated by Animista on 2020-10-15 23:32:49
  * Licensed under FreeBSD License.
  * See http://animista.net/license for more info.
  * w: http://animista.net, t: @cssanimista
  * ---------------------------------------------- */
  
  /**
  * ----------------------------------------
  * animation fade-in-left
  * ----------------------------------------
  */
  @-webkit-keyframes fade-in-left {
          0% {
            -webkit-transform: translateX(-50px);
                    transform: translateX(-50px);
            opacity: 0;
          }
          100% {
            -webkit-transform: translateX(0);
                    transform: translateX(0);
            opacity: 1;
          }
        }
        @keyframes fade-in-left {
          0% {
            -webkit-transform: translateX(-50px);
                    transform: translateX(-50px);
            opacity: 0;
          }
          100% {
            -webkit-transform: translateX(0);
                    transform: translateX(0);
            opacity: 1;
          }
        }
  

  

With those changes in place, each newly added comment now politely animates into the DOM:

Wrapping up

Together, we looked at how you can use the power of rails-ujs and Stimulus to add a little bit of interactivity to your Rails apps without needing to rely on bulky front-end frameworks or clunky JS partials and jQuery solutions. Stimulus + Rails can deliver simple, clean, and interactive experiences for your users, while retaining all of the joys that come with building standard Rails apps.

Here’s a preview of what you can build with just a little bit more Stimulus than what we worked on today— all of the interaction you see in the gif below is built with Stimulus + rails-ujs:

Thanks for reading!

Additional Resources:

The full code for this repository can be found on Github .
The Stimulus official docs
Working with JavaScript in Rails
BetterStimulus.com for various Stimulus patterns and best practices

Shape Up reflections: Volume 1

2020-09-28T00:00:00+00:00

Below you’ll find a lightly edited version of an internal update I sent to everyone at CareerPlug after the completing our first product cycle, modeled on Basecamp’s Shape Up method.

Since the target audience for these updates is the company first, I’ll add inline commentary and footnotes as needed to add context for readers that aren’t intimately familiar with CareerPlug’s internal workings.

My intent in sharing these kinds of internal documents is to help demystify the inner workings of product organizations — or at least the one product organization I can demystify.

As an introduction to our org, CareerPlug has ~50 employees in total, with a product team of 13 people. Today, our product team is made of up two UX designers, eight engineers, and two QA engineers, plus me.

Our core product, launched in 2012, is an Applicant Tracking System (ATS) used by more than 12,000 businesses in the United States and Canada. Our core clients are brick-and-mortar businesses (think retail, restaurants, and home services) with limited professional HR resources. Our ATS covers everything from distributing job postings to a variety of online job boards (think Indeed) through pre-hire assessments, interviews, offer letters, and background checks. It is a big, fairly complex piece of software, but we do our best to make the end user experience simple.

Let’s dig in. -David

Hi Team,

Our first product cycle went pretty well!

We launched new features and learned lessons about how to do better in the next cycle. In the next two cycles we want to keep up the velocity (launching new stuff) and the learning. We know we’ve still got plenty of improvements to make.

Speaking of learning, in between each cycle I will write reflections on what happened in the previous cycle, what we’ve learned, and what we’re trying next. I’ll share these reflections with the company, for anyone who is interested in knowing about how the product team works.

Before we start, a note on the content of this update, and what will change in future updates: There isn’t a section below talking about how we’re progressing towards our targeted outcomes. Since none of the work in cycle one has been out in the world long enough to measure the impact, we’re not ready to dig into results.

In all subsequent versions of these reflections, a section will be dedicated to reviewing the results of work released in previous cycles. We’ll review the expected outcome of each pitch and see if we achieved that outcome. With that out of the way, here’s this cycle’s David novel 🙂

What worked

We committed to the right amount of work, and we built the right processes to deliver those commitments.

Of the projects we committed to in this cycle, we completed all but one.

Delivering almost everything we wanted to in our first cycle is a big win, and I’m proud of how well we performed as a team. We pushed ourselves to find ways to solve problems in the allotted time, made hard decisions about scope, and mostly succeeded in producing what we set out to produce.

I’ll talk more about learnings from the project we didn’t deliver in just a bit.

Product-team communication was strong We’ve talked before about rebuilding the airplane while we’re still in the air - we did that successfully in this cycle. Engineers and designers had to decide on and implement new ways to communicating with each other as we moved from high-fidelity, finished mockups to “here’s a problem and some words about a solution, let’s figure out what works.” That’s hard to do.

[David note: Our pitches are a little different than what is outlined in Shape Up. Mostly, our pitches leave more space available for UX and engineering to design the solution. My version of fat marker sketches is words, most of the time.]

Each project team handled the communication problem differently but they all arrived at a solution that worked. They kept each other informed, solved problems, got feedback, and released high-quality work under tight deadlines.

Beyond the project team level, multiple people on the product team flagged process gaps and problems that need to be fixed in the next cycle.

Cross-team feedback happened We got more feedback and insights from CX and sales than we’ve ever gotten before. The projects we completed this cycle are better due to that participation. This feedback was possible thanks to a concerted effort from project teams to connect with other teams and keep them involved.

A lot of work went into trying new ways of communicating with other teams. Connecting when schedules are full and case loads are high is difficult. What we tried was mostly successful, compared to what we’ve done before.

There’s more work to do to keep these channels open and make sure everyone who wants to has a chance to share their thoughts, but we should celebrate our progress here.

[David note: A major driver in switching to Shape Up was seeing an opportunity to open up the pitch process for feedback across the company and to push UX and engineering to work more directly with other teams. In our previous process UX was getting business requirements and getting some feedback during design but engineering would often change implementation details during development. These surprise changes made our feedback processes feel insufficient. Shape Up isn’t really a required thing for this, but I saw a way to adjust the process a bit to meet this need for my team.]

The investment we made in Voyager is paying dividends Bear with me on this one, I know that we’ve still got plenty of work to do on Voyager, especially on the settings pages.

[David note: Voyager is the name of our redesigned ATS. We made a decision ~18 months ago that we needed to give ourselves a better base to build from, and Voyager is what we’ve built. Voyager has been in use by clients since March, and the bulk of our client base uses Voyager exclusively. Some clients still use the original version of our ATS, which we call Classic. In this section, I spend time reminding the team why we took on this work — which has been contentious at times due to mistakes I’ve made along the way — because repetition helps with important messages like this.]

I’m going to briefly recap the Why of Voyager to frame up what I mean by paying dividends.

We invested in Voyager because we knew the ATS had reached a point where we could no longer innovate on the product in meaningful ways. It worked well enough and people who knew how to use it got by just fine, but that was about it. The most obvious sign that something was wrong was that most of our core features were used by a small percentage of our users.

The years we spent searching for our identity as we grew beyond just serving State Farm led to a lot of unused features, confusing experiences, and a UX that couldn’t accommodate any new workflows. Now that we finally knew exactly who the product was for, we had to get the product into a state where it could actually be made for those people.

On the technical side, we did an admirable job keeping the code base clean and the product performant with the resources we had in the early years. [David note: We had one engineer for the first few years of our life, and 2 for a few years after that.] We built automations and wrote tests when we could, and we survived. Things were okay, and new developers were able to get up and running without /too/ much pain. Just like for our clients, our developers and designers experience with the ATS was, at best, fine.

Despite all of our hard work, a decade is a very long time in web development years, and we have learned a lot about how to build fast, scalable products in the last ten years. We knew our architecture needed to change to get where we wanted to go. [David note: On the technical side we primarily needed to rip out much of our frontend architecture, which was pretty out of date. We also upgraded to Rails 6 and started using Stimulus.js heavily]

Some of the features and experiences we want to build might have been possible on Classic, but they wouldn’t be the best they could be. Staying on Classic would have locked us into innovating slowly on a less reliable, slower product. With a stronger technical foundation, we would have the ability to innovate faster while making the product faster, more reliable, and easier to understand.

The right long term investment was building Voyager. We know who our product is for, and how to get it to them, and with Voyager, we knew we’d have a product ready for long term growth. In the short term, that meant a whole lot of pain and, until now, not much to show for the work.

This product cycle, Voyager began to pay us back. We have a strong technical foundation, a well-maintained codebase that follows internal best practices and agreed upon patterns, and a library of components we can use to quickly implement new features. The feature work we did in this cycle all moved faster because of those benefits, none of which could have existed in Classic.

On top of the speed of innovation, we also saw the benefit of having space to add new features and functionality. There’s room to grow, thoughtfully, which means we can build the things our users need. Even better, having space means we can show our users that these new features exist and educate them on how they work. Classic was (and is) so full of stuff — much of it unused— that we didn’t have room to educate our users and help them succeed. New features in Classic just fell into the abyss.

We aren’t done with Voyager — and the ATS will never be “done” — but this cycle was the culmination of eighteen months of hard work, and the early results are exactly what we envisioned when we started building.

What didn’t work

Four week cycles and one week projects When we decided to move to this new cycle process, I made the call to make cycles four weeks long. I made this decision because moving straight from two week sprints to six week cycles felt like too big of a leap. I was too cautious here. We should have just ripped off the bandaid and gone straight to six weeks.

Four week cycle length was the wrong decision because four weeks is too cramped. Projects aren’t just engineering work - research, UX design, prototyping, and feedback all have to happen alongside engineering and QA. Doing all that well inside of a four weeks is always going to be very hard, and sometimes it will be impossible.

Even worse, four week cycles led us to include several one week (and two half week) projects. One week is way too tight for proper research, design, engineering, and QA, even for small projects.

We made it work this cycle because of a lot of hard work from the team, but four week cycles and one week projects don’t allow us to bring our best.

Feedback from clients was too light We didn’t get much client feedback on our larger projects. This is a function of two things:

We don’t have a strong bench of readily available clients to give feedback without a lot of advance notice. Scheduling a call two weeks out doesn’t work in our new world where we’re designing and developing concurrently and releasing quickly.
Four week cycles don’t leave much time for feedback at all. We need to start building really quickly. With six weeks, we’ll have a little more time early on to get organized and connect with clients. Without more direct client feedback on larger projects, we’ll miss the mark more often than we should.

What we learned

This new process can work! Launching a new thing that changes how a whole company works together is hard. There’s always a risk that it won’t work, even if it sounds great on paper.

In this case, it went pretty well. We worked together, we moved the product forward, and we learned how we can do even better next time. We know the core philosophy is sound, and that we can do good work within our new constraints.

We need to be better partners — and ask for more from our partners Integration work that follows our new process rules (fixed, non-negotiable timeframes) is going to be tough. We had two point of hire integrations slated for this cycle and only one of them got done. The integration we missed on was identified as a risk on day one. We know they would likely be slow to respond, and we tried to get ahead of it by reaching out proactively very early on, but we didn’t reach the finish line.

[David note: We have a variety of integrations with external systems (background check providers, payroll systems, time tracking systems). We’re still learning how to best work with these partners in a way that doesn’t cause a lot of extra operational overhead.]

We need to be better partners by communicating openly about how our process works and what we require from partners before and during development cycles. Misalignment of expectations with an integration partner is very costly for us because we bear all of the development cost. It doesn’t cost the partner anything if a project drags on, which means our risk going into a new integration partnership is much higher than the partner’s risk.

We shouldn’t commit until we’ve gotten buy-in that the partner is there to support us from the start, including a commitment to have a technical contact who can be available within 24 hours when building and testing the integration.

What’s next?

New cycle and project lengths In the next cycle, we’ll learn how six week cycles work for us. Six week cycles means more time for research, testing, and feedback, and will give our ideas a little more room to grow. Each six week cycle will be followed by a two week cooldown - one extra week for devs to improve our infrastructure, tooling, and dive into creative ways to be more efficient and deliver a great product. Cooldown means plenty of bug fixing time too.

We’re coupling six week cycles with a minimum project length of two weeks. If we’re truly following our new process, even small development projects need time for research, testing, and feedback, along with writing code and doing QA. One week isn’t enough time to do all of that well for anything more than the smallest of features. Two weeks feels right, right now.

A minimum of two weeks for small batch projects means we’ll have to be more thoughtful about how we construct pitches for two week projects. There are always lots of incremental improvements that can be made to a product, and that’s especially true of our ATS since we’re still completing a major redesign. Classic has plenty of good ideas that have yet to make it to Voyager, and those good ideas are often very small tweaks that, on their own, aren’t two weeks of work.

Our job will be to find the themes in the small tweaks that we need to make so that they can be combined into coherent, measurable small batch projects. These themes (say, making filtering on the applicants page better) will help us make the small, incremental changes we need to make without letting small batch projects devolve into free-for-alls that can’t be reflected on or measured.

Retain You’ll see the first work on Retain happening in this cycle. By the end of the cycle, you’ll be able to see the first pieces of Retain and use them yourselves.

While that work happens, Clint and I will continue to refine the vision for Retain and recruit advocates for Retain from outside of the company to be our early testers and to serve as champions for Retain when it is ready for the world to see.

[David note: Retain is a new product we are building. It will release in 2021]

Communication We’re still learning how best to communicate — with each other, with the company, and with clients — and the next cycle will be another opportunity to grow. Expect more opportunities to give preview in-progress work, more visible product metrics, and better insight into upcoming pitches. We’ll also continue working on building our bench of go-to clients for feedback and testing. Longer cycles will give us more time to do that, rather than constantly being heads down trying to hit a delivery deadline.

Product launches Now that we know who we’re building for, and we’re building things that benefit all of our users, we can begin talking more publicly about /what/ we’re launching and /why/ people should care. We’ll be experimenting with new ways to educate users and prospects about what’s changed in CareerPlug, and how what we’re building will benefit their business.

We’re proud of the work we’re doing, and we know it provides enormous value for thousands of companies. We’re going to start bragging about it a little more, and pushing people to use everything in the system, not just the stuff we’re not scared to tell them about.

That’s all for cycle one. Thanks for reading.

Product managers: your job is not to say no

2020-07-13T00:00:00+00:00

One of the most damaging pieces of advice you can receive as a new product manager is that your job is to say no. This advice, well-meaning and oft-repeated, isn’t wrong but when taken at face value it is incredibly damaging.

Your job as a product manager is not saying no; your job is saying yes to the right things.

This might seem like a silly distinction — in either case, you’re saying no a lot, right? — but your mindset makes all the difference.

How you perceive the idea

When you define your core function as saying no, you approach every new idea closed off, ready to fire off a no, especially if what you are hearing doesn’t align with the plans you already have. When your job is to say no, you look for reasons to throw an idea onto the no pile so you can check the box — “Great job, you did your job and protected the product!” — without giving each new piece of information the right level of consideration. Each time you say no, you should be able to articulate why you’re saying no — to yourself and to anyone around you.

If you can’t articulate why it’s a no, you are not ready to say no.

Forcing yourself to articulate why you’re saying no will help you build better products. You never know where the next great idea will come from. If you are tuning in to a variety of directional inputs you are going encounter new ideas constantly but these new ideas are useless if your mental model doesn’t allow you to approach each idea with an open mind.

This shift in perspective — from starting at no to starting at why not — will have a profound impact on how you perceive new ideas. Each new idea becomes an opportunity to learn and expand your understanding of your product’s place in the market instead of another distraction from your goals.

How others perceive you

“I know you’re going to say no to this but I have to ask you anyway.” A CSM started a conversation with me by telling me what I was going to say before I said it. That sentence opened my eyes to how my approach to internal conversations was making others feel. Focusing on saying no — protecting the roadmap instead of hearing and considering each new idea — caused others to feel unheard. Despite wanting to give everyone an opportunity to make their voice heard, my approach to conversations with my internal team made them feel left out.

When people come to you, the product manager, they’re bringing problems and ideas that matter to them and they want to know that they matter to you too. Leading with no means that you won’t engage deeply with the idea in front of you, and others will pick up on that energy. They’ll get tired of hearing no — even if you let them talk before you say no — and eventually they’ll be too discouraged to bring new ideas to you since they expect you to shut their idea down.

If you start each conversation by looking for the right things to say yes to, you create an environment where others feel heard. They’ll feel you engaging with the idea, digging in deeper, asking why, and encouraging them to think through the idea themselves.

This shift makes a big difference in the tone of a conversation. More questions get asked as you take the time to learn more about the problem they need to solve. You interrogate your current assumptions and share those assumptions with the person you’re talking to. They learn from you as you talk to them about how the idea they have might fit into the overall product strategy, and you learn from them as they bring you new information. If the conversation ends in a no, the right mindset can soften the blow that comes with hearing no.

This might sound a lot like being a good listener which, yes — true. But product managers with great listening skills don’t always deploy those skills equally. Some PMs will use their strong listening skills when talking with someone who has a high level of influence in the organization or on the roadmap. When those same PMs are talking with someone with little or no influence, their great listening skills disappear, replaced by a race to say no and move on.

Good ideas can come from anywhere, not just from the top. If you approach each product conversation as a chance to learn, grow, and maybe uncover a great new idea, you’ll build stronger relationships and a better product.

Thoughtfully saying no

So you’re saying I’m supposed to say yes to more things? No (ha). You are still going to say no a lot. I’m not arguing that saying no is bad, instead, I’m encouraging you to think about the impact your mindset has on you, your users, your peers, and your product.

Remember that your job is to say yes to the right things. You are tasked with deploying your limited resources to delivering the most value to your customers. Each time you encounter a new idea, look at that as an opportunity to learn and approach the conversation with a goal of discovering if the idea is the right thing for your product. If it isn’t, thoughtfully say no by keeping these key points in mind:

Be clear — if the answer isn’t yes, the answer is no. Not “not right now” or “maybe later”. Just say no.
Explain your reasoning — why isn’t this the right thing to work on right now? What are we saying yes to and why? Give context so everyone understands why the answer is no.
Acknowledge the impact — the idea got to you because someone cared about it. Hearing no is disappointing at best and in the worst cases, it can cost you business. Have empathy for the people who have to hear (and deliver) the no.

Thanks for reading!

Do less: A strategy for building better products

2020-07-05T00:00:00+00:00

Product managers love building new things. It is a driving force behind why many of us decide on this career path; we get to spend all day figuring out what cool new thing to build next.

This love of building stuff serves us well in the early days of a product. Everything is new, you don’t have many users, and you are missing core components of your product’s value proposition. Delivering new features quickly helps you find problems that your users want you to solve before you run out of cash.

Eventually, if you build the right things and all of the pieces fall into place, you get past the early stage of searching for product-market fit and find that you have a growing base of happy users.

Reaching this point is a wonderful achievement — and a signal that your product strategy needs to change.

New feature addiction

Many product teams fail to recognize the strategic shift they need to make once they reach product-market fit and so they continue churning out new feature after new feature for years without knowing why. Often times this happens out of inertia. We’re used to building new things and building new things is comfortable, so we keep building new things.

Features will be released, fail to catch on, and drift off into the abyss as your team pivots to the next new idea. This inertia, left unchecked, will lead to a confusing product that lacks vision and fails to deliver on its promise.

All of these new features, each of which was probably a nice idea in isolation, will begin to wear on users. They bought your product because it solved a problem for them and each new feature release is another thing they have to learn, a new tab, a new button, a new pop-up announcing the latest release that they don’t need.

New feature addiction will eventually produce outcomes that you aren’t going to like:

Your core feature set will age poorly. As you chase new features to expand your product and solve new problems, the features that got you to product-market fit will start to feel less valuable, less innovative. Your competitors will learn what made you successful and they’ll copy and improve on the good parts, without all of your baggage.
The debt built up from years of new features that didn’t quite hit the mark will make your product hard to use, hard to explain, and weigh down your engineers. Bugs will crop up more frequently, your support team will struggle to articulate how your product works, and your velocity will slow down.

Avoid these outcomes by intentionally doing less.

Do less

Do less doesn’t mean stop working on your product. Instead, it is a call to refocus your team on the right work to keep your product healthy for the long term. A less controversial way to think about the concept of doing less is doing less work that people see.

What I really mean when I encourage product teams to “do less” is to make your product better by focusing on the hard, often invisible work of optimizing what you already have. If you do this right your users will /feel/your work more than they’ll see your work.

Doing less can feel uncomfortable, frustrating, or confusing for teams that are used to feature-based roadmaps and cultures that praise output and splashy new releases.

How can you do less and still deliver a great product? Let’s find out.

Know who you are

The first step to doing less is to create a product vision that clearly defines what your product is, who it is for, and what problems it solves. This vision will form the boundaries within which changes to the product should live. If a feature idea comes through that sounds valuable but doesn’t align with the vision, that feature should not be built.

Before you achieve product-market fit, you should have a product vision, but that vision likely won’t have enough details to keep new feature addiction in check. As you turn the corner to a more mature product, refine your vision, get buy-in from your team, and build processes to hold yourself accountable to the vision.

Do less by defining your vision and boxing out the work that distracts you from achieving that vision.

Go deep, not wide

Besides inertia and a love of the new, another reason we build new features is that new features are easy to build. Every new feature is a Greenfield full of hopes and dreams for what it could be in — all the hard stuff is still somewhere out ahead.

Working to improve a feature after it is out in the world is more difficult. You have to go deep into how people use the feature, what value they get from it, and how it could be improved. This type of work uncovers more warts than you’ll find in Greenfield work and it carries with it more risk of upsetting your users if you miss the mark.

Despite the difficulty of the work, going deeper into the features that your users already value will produce a higher return in the long run. Your customers didn’t buy 100 features, they bought a handful of really valuable, easy-to-use features. Continuous improvements to a small set of high-value features will always win out over a constant barrage of new features that are tangential to your product’s core job.

Do less by going deep into your core features, instead of playing around on the margins.

Focus on quality

No matter what your software does and who your users are, there are two things that your customers want:

Your product should be fast
Your product should work

Those two things are universally true in software. No one wants to use your slow product, and they don’t want to deal with bugs. Let’s tackle speed first. We know that faster page loads produce better conversion rates — people don’t want to waste time staring at a blank screen or a loading icon.

Speed typically isn’t a concern in the early days of most SaaS products. Everything is fast because no one uses your product. Problematic database queries are masked behind tiny tables, edge cases haven’t been discovered yet, and your single Heroku instance can handle 100x your peak traffic.

Left unattended, your product will slow down. Database tables grow, edge cases become commonplace, and your eyes are on launching a new feature while your users spend 3 seconds waiting for every single page turn. Time spent optimizing performance will almost always be a higher return on investment than time spent building nice-to-have features. Don’t wait until your users tell you things are slow — push your engineering team to proactively seek out and eliminate poor performance early and often.

The other side of quality is fixing bugs. Software is never bug-free and that should not be the aim ; however, a perception that your product is “buggy” will kill your growth faster than nearly anything else. No one wants to spend their time and money on something that doesn’t work — error pages, downtime, and unexpected behavior all add up to a terrible experience. Ignoring bugs will encourage your users to move on to a competitor that can promise a more reliable solution, even if your product has more features.

The cost of unresolved bugs compounds over time. A bug is a crack in your product’s foundation — the best time to fix that crack was before it existed. The second best time is now. Find and fix bugs before they reach your users, and build a product that is focused, testable, and easy to understand, or find yourself paying the cost for a buggy product.

Do less by prioritizing a fast, reliable product experience.

Doing less doesn’t mean stop innovating

Does doing less mean you never build new features?

No. There will be times when an opportunity exists that is worth investing in. If you stay focused on doing less, you’ll be more likely to be in a position to capitalize on new opportunities because your existing product will be more focused and more prepared to integrate a new core feature than an unfocused product full of unfinished ideas.

How do you know if a feature is worth investing in? Start with your product vision. Does the new feature closely align with your vision? Do the users you serve have the problem you want to solve and do they trust you to solve it? Do you have enough input signals to validate that the idea is worth pursuing? If you can answer yes to these questions, and your core features are fast, reliable, and well-optimized, then it might be time to incorporate something new.

Wrapping up

Doing less is hard — I’ve struggled at various times in my career with wanting to push new features ahead of the less exciting and less visible work that comes with committing to doing less.

The strategies I’ve outlined above have helped me keep my teams focused on the long term, and they’ve been developed in direct response to getting off course by chasing after new things instead of digging deeper.

The work of doing less requires discipline, focus, and clarity of purpose. As with most really valuable things, you won’t get there overnight but the results will be worth it.

If you’re struggling with where to start, try to identify just one area where you can start to change your habits:

If you’re dealing with a buggy or unreliable product, work with your engineering team to understand what’s causing the bugs and divert resources away from new feature development to resolve outstanding bugs
Look for an area of your core product that you haven’t touched in a while. Use that feature as if you were a new user and take notes on what you could improve about the experience. Spend time reviewing your directional inputs to find patterns related to that core feature. Develop a plan to make that one feature a bit better than it is today and see how that work impacts your KPIs.

Starting small, changing what you can, and looking for evidence that this approach works for your product, is, as usual, the best approach.

Directional inputs for your product

2020-06-29T00:00:00+00:00

The world of product management is noisy. Your clients are asking for new features. Your sales team needs a magical new feature to close their next deal. Your success team heard from 50 people yesterday about gaps in your product. Noise is coming in from every direction, all of the time. Learning to process a constant barrage of information from disparate sources into a coherent product vision is vital to success as a product manager.

One mistake that product managers make is becoming too focused on a single input channel. Most often, this happens when a single source is the loudest — the squeaky wheel gets the new feature. To break out of this habit, it can help to identify all of the different input channels that you should be monitoring and clarify the role each channel plays in the product management process.

External conversations

Let’s talk about external conversations with users and prospects first. This is the lowest hanging fruit — we all know we’re supposed to be talking to people, right? Right.

Early on in a product’s lifecycle direct conversations with the people who use your product and the people who might use a product like yours are your lifeblood. If you haven’t achieved product-market fit, you’ll never get there without talking to your users and learning from them about how to make your product better. You don’t yet have enough users to have reliable or useful data so you must make judgment calls based on your deep understanding of the individuals who use your product. Trusting your gut is a lot easier if you have a bench of people who can provide honest feedback for you as you shape the early direction of your product.

As your user base grows, the weight you give to each individual opinion from a user must change. In the early days, you may only have 100 users — you can talk to 10 of those people and feel pretty confident you know how your user base feels about a particular topic. Once you have thousands or millions of users, counting on one-to-one conversations as your primary source of truth becomes less useful.

External conversations will always be a key input channel as a product manager. No matter the size of your user base you can use these conversations to rapidly identify problems and new opportunities and to get quick, honest feedback on your product. External conversations also play an important role in humanizing your users — telling a story about a real person when you are pitching an idea is a great way to build a case to a reluctant internal team.

Internal conversations

Internal conversations with your frontline teams help you identify trends, pressure test ideas and multiply your access to real stories. Your support team members are going to know your product better than you, especially the quirks that confuse or frustrate your users. Your sales team will know what features really get people excited and which are falling flat in demos. Talking to these frontline teams regularly will help you identify the areas of your product that need the most focus. Building trust and two-way communication with these frontline teams is one of the best things you can do to keep your product heading in the right direction.

Conversations with your engineering team are part of this channel as well. Keep an open line of communication with your key engineering resources and leave room for them to share honest feedback with you on what technical debt needs to be addressed and when.

Internal conversations can be a dangerous input channel for a product manager because of the dreaded HiPPO trap. The pressure to move a pet project along from an executive can be intense. It can sometimes feel easier to just prioritize a project to eliminate the pressure, even when you know that it isn’t the right direction for your product. Managing expectations and avoiding the trap is a topic for another day — for now just keep in mind that when you approach internal conversations with executive teams, your job as a product manager is to deliver a great product and it is your role to speak up and push back when steps are being skipped to ease internal pressure.

Conversations, whether they’re with internal or external folks, will be happening daily and this input channel should always be the biggest block on your calendar. Avoid spending all of your time in meetings and on the phone with users and internal stakeholders— this channel can be incredibly time consuming and you will find that the return on most individual conversations is quite low.

Research

Research is an input channel that encompasses all of the stuff that product managers do that looks like searching Google all day. Competitor research, industry research, market research, and, depending on your company’s definition of product management, technical research all get broadcast on this channel. This channel must be actively sought out — your competitors aren’t going to bang down your door to tell you about a new feature and you are probably not so central to your industry that the news all runs through you.

The most obvious input in this channel is competitor research. Be cautious about relying too much on competitor research — in many cases, you’ll find it is of limited value. Don’t ignore what your competitors are doing but if what they are doing is driving your roadmap you will always deliver a product that feels a little bit behind. Instead, keep an eye on what they’re doing and know how you compare but build what works best for your users and your own long term goals.

Because you have to actively seek out this channel, it is easy to ignore it in favor of channels that demand your attention. To avoid this, I recommend carving out dedicated time to focus on researching what’s happening in your space and writing up notes to structure your thinking. The right cadence for this type of work is determined by your working style and the maturity of your product and your market — a few hours a week works best for me but you might find you need more or less time on this channel.

A handy way to save yourself some time and energy when doing research is to let other people do some of the work for you — subscribe to a service like Owler to rollup news on industries or competitors you care about, find the important newsletters and journalists covering your industry and subscribe to them, and let the research show up in your inbox.

Feedback data

Feedback data is our first of two channels that involve that famous product management task of staring at a spreadsheet for a few hours and wondering why no one likes the feature everyone told you they had to have.

Feedback data is aggregated information on what clients are saying about your product. We’re firmly in qualitative data land with this channel and most of the value in this channel will be in reading words that people wrote about your product.

This type of data can be sourced from places like:

Client surveys (such as NPS)
Support cases from your ticketing system
Search trends on your online help center
Feedback on a community portal.

The feedback that you have available will depend on the size and maturity of your business, how transparent individual business units are, and what proactive work you’ve done to build these channels.

Feedback helps you apply a new lens to individual conversations — if you hear a problem in a one-on-one conversation your next step should often be referencing aggregated feedback to look for validation this the problem is relevant across your user base.

Feedback can also help you identify trends that you might not see in a few individual conversations. NPS surveys and other sources of direct, structured feedback on your product from a wide range of users can identify the most acute problems your product has and, with the right analysis, can help surface potential solutions to those problems.

In my experience, NPS data is often the most valuable input in this channel. NPS surveys are a constant source of high-value ideas and, as your product matures, you’ll find that negative NPS feedback about your product will focus on very similar things. If you aren’t running NPS surveys for your product, consider partnering with your client-facing teams to start an NPS process as soon as you can. Every zero you receive is an opportunity to have a conversation about what your product can do better.

If you have access to it, support case data can be high impact too. Reading email/chat cases directly is the ideal for this, but even higher level trends on the types of questions people ask can be a useful source for understanding what areas of your product you should focus on.

The right cadence for reviewing and analyzing feedback will depend on the data that you can access and how much of it you have. In most cases, a weekly review of incoming data will work. In organizations with longer cycles or lower volume, a monthly deep-dive may be sufficient.

Engagement data

This is where the spreadsheets can get really intense (this is how I learned Excel has a row limit of 1,048,576) and where you can get very, very lost if you aren’t careful.

Engagement data moves away from softer, qualitative feedback data into direct quantitative data. Forget what people are saying about your product — how are they actually using it?

Engagement data plays two important roles in helping you shape your product’s direction.

First, engagement data allows you to validate what you are hearing through conversations and feedback inputs with how people are actually using your product. Is everyone telling you that they love a new feature? That’s great! How many of them have used it more than once?

Whenever possible, back up decisions that you make based on conversational input with usage data.

Second, the vast majority of your users are silent. You never hear from them and you never will. They aren’t going to get on the phone with you, contact your support team, or fill out a survey. They’ll just use your product in silence. The only way you can give these users a voice is to understand how they use your product. Which features are popular and which are lagging? Which features are most likely to predict renewal? Which features never get used but feature prominently on your marketing homepage?

In addition to deep dives into the world of SQL queries, CSV dumps, and pretty charts and graphs generated from product usage data, I include data from sales and marketing in this channel. Understanding what leads are coming in and which are moving through your conversion funnel helps you build a better product. This raw data can be a great way to identify new market opportunities or ways to tighten up the funnel with a stronger product.

Reviewing and analyzing engagement data should be part of your daily routine. As your product grows the amount of data you have about how your product is used can quickly get overwhelming. Stay on top of the data and build good processes for yourself to automate reporting on key metrics to make this channel more valuable.

Bringing it together

Learning to balance all of these inputs takes time, experience, and plenty of false starts. Your personality and your background will naturally attract you to spending more time on certain channels — if you love talking to people you might overemphasize conversational channels. If talking to people takes a lot of energy for you (that’s me) then you may find yourself leaning into data and spreadsheets and missing out on important conversations.

Ensure that you are using the full array of input channels by building routines that force you to consider each channel, and evaluate the source of the items that make it onto your roadmap to hold yourself accountable to staying engaged with each channel.

Finally, remember that tuning into all of these different input channels is a privilege that most people at your company won’t get to have — think about ways to surface the insights that you glean from spending so much time processing this feedback to the rest of your company. As product managers, we have an opportunity to help propagate a deep understanding of our products, how people feel about them, and how people use them throughout our whole company. Invest in that opportunity and you may be surprised at your returns.

Colby.so

Searching and filtering with Turbo 8

Setup

Searching

Automatically submit the search form

Search and sort together

Filter by team

Wrapping up

Building a sortable table with Turbo 8's page refreshes

Setup

Sorting the table

Sort both directions

Wrap up

Turbo 8 morphing refreshes on Rails

Application Setup

Refreshing posts

Let’s see how it works.

Broadcast refreshes

Pros and cons of Turbo Refreshes

Building a modal form with Turbo Stream GET requests and custom stream actions

Application setup

Adding new cards in a modal

Creating a custom Turbo Stream action

Wrapping up

Everyone GET in here! Infinite scroll with Rails, Turbo Streams, and Stimulus

Application setup

Paginating a resource with Pagy

Manual “Load more” button

Infinite scroll with Stimulus controller

Wrapping up

Interactive charts with Ruby on Rails, StimulusReflex, and ApexCharts

Setup

Adding a static chart

Add job name filter to chart

Wrapping up

Hotwiring Rails Newsletter - August 2022

Table of contents

Articles and guides

Fly Reactive Rails by Julian Rubisch

Vite-lizing Rails by Vladimir Dementyev

Using Turbo Streams with Kredis by Mansa Keïta

Booleans don’t exist in Ruby by Mike Burns

Video | Subdomain Routing in Rails by CJ Avila

Galaxy Brain CSS Tricks with Hotwire and Rails by Matt Swanson

Video | Naming things: rails new by Joel Drapper and Kasper Timm Hansen

New and interesting PRs and releases

My recent writing

The best things I read recently

Until next time

Rails is back isn't enough

What do we do?

Writing effective coding tutorials

Write for a specific audience

Make a promise and keep it

Focus on what you’re teaching

Create visual checkpoints

Build incrementally

Explain the code at the right level

Check your work

A simple formula to get started

Introduction

Setup the application

Build the application

Conclusion

Closing thoughts

Hotwiring Rails Newsletter - April 2022

Table of contents

Articles and guides

Distributed Systems In Rails series by Nate Hopkins

The Problem That ViewComponent Solves For Me by Jason Swett

Hotwire Handbook Part 1 by Phil Reynolds

Import Maps Under The Hood In Rails 7 by Paweł Dąbrowski

Dynamically Re-use and Lazy-load Pages Using Hotwire

New and interesting PRs and releases

My recent writing

Lessons learned from publishing on Gumroad

Background

Writing process and pre-launch

Beta readers

Publishing the book as an app