Persist Messages to the Database

  • Creating a Message model
  • Querying database
  • Persist messages to database
  • Unix timestamp to DateTime

To keep our app simple with the functionality most people expect, we're going to persist all messages to our Postgres database. In order to do this, we need to create a Message model so our database knows what kind of data it's receiving.

Later on, we may decide that we don't need to store these messages in a database or we may need to cache our messages in memory for better performance. If we later decide to go that route, we can use ETS. No need for premature optimization.

Creating the Message model

We're going to use the built-in mix generator to create the boilerplate for a new migration.

$ mix phoenix.gen.model Message messages body from room \
timestamp:datetime user_id:references:users

This will create three new files, the names of which should have logged in your terminal.

The first thing we should check out is our migration file to update our database with the new data type. This file should already have all the relevant fields we are interested in and we don't have to change anything.

The last thing we're going to do before we persist our messages to the database is add a function that gives us the ability to limit the number of messages we receive from the database (with a default of 10).

Ecto allows us to define our queries as functions using from/2, which is a really nice feature, so we're going to create a new function called latest_room_messages that finds all messages from a particular room, then orders them by the most recent, and limits the number of messages.

defmodule PhoenixChat.Message do
use PhoenixChat.Web, :model
schema "messages" do
field :body, :string
field :timestamp, Ecto.DateTime
field :room, :string
field :from, :string
belongs_to :user, PhoenixChat.User
timestamps
end
@required_fields ~w(body timestamp room)
@optional_fields ~w(user_id from)
@doc """
Creates a changeset based on the `model` and `params`.
If no params are provided, an invalid changeset is returned
with no validation performed.
"""
def changeset(model, params \\ :empty) do
model
|> cast(params, @required_fields, @optional_fields)
end
@doc """
An `Ecto.Query` that returns the last 10 message records for a given room.
"""
def latest_room_messages(room, number \\ 10) do
from m in __MODULE__,
where: m.room == ^room,
order_by: [desc: :timestamp],
limit: ^number
end
end

At this point, you should run your migration.

$ mix ecto.migrate

Updating our RoomChannel

Now we need to update our RoomChannel to start using our new model.

The first thing to change is our join/3 function. Rather than simply checking the authorization status and returning the socket, we're going to reply with a payload of message records. This way, when a client joins the channel, she automatically receives the message history.

We start this function with the room_id and pass it into the Message.latest_room_messages/2 function we created earlier. This creates an Ecto query based on the room_id, which we can them pass to Repo.all, which finds all of our data that matches the query.

Then we use Enum.map to format our messages in the way our client expects to receive them (explained in greater detail below the code block). Then we reverse the order to show the newest results first.

Finally, we return the messages that we just created along with our socket. join/3 expects one of three things as its return value: {:ok, Phoenix.Socket.t}, {:ok, map, Phoenix.Socket.t}, or {:error, map} We were previously sending the first value, but now that we have a map to send along, we're sending the second value.

We also created a private function called message_payload, which takes the message and formats it before returning it.

defmodule PhoenixChat.RoomChannel do
use PhoenixChat.Web, :channel
require Logger
alias PhoenixChat.{Message, Repo}
def join("room:" <> room_id, payload, socket) do
authorize(payload, fn ->
messages = room_id
|> Message.latest_room_messages
|> Repo.all
|> Enum.map(&message_payload/1)
|> Enum.reverse
{:ok, %{messages: messages}, socket}
end)
end
defp message_payload(message) do
from = message.user_id || message.from
%{body: message.body,
timestamp: message.timestamp,
room: message.room,
from: from,
id: message.id}
end
...
end

Note the syntax used to call message_payload. Using an ampersand (&) along with the arity (number of arguments) is a shorthand way of writing anonymous functions. The two functions below are equivalent.

|> Enum.map(&message_payload/1)
|> Enum.map(fn x -> message_payload(x) end)

You will see this syntax a lot in Elixir. For more, see the docs.

The last thing we should do before actually writing these to our database is check if the id passed to UserSocket is an administrator id or an anonymous user id.

We first pull the id out of the params passed to connect/2. Then, if that value exists and we can find that user in the database, then we assign that to the value user.

Then we assign values to socket depending on whether or not user passed the two criteria above (exists and is in the database). If the user does exist, we assign the user_id, username, and email to the socket. If user does not have a value, we assign the user_id to nil and give it the uuid for the anonymous user.

alias PhoenixChat.{Repo, User}
...
def connect(params, socket) do
user_id = params["id"]
user = user_id && Repo.get(User, user_id)
socket = if user do
socket
|> assign(:user_id, user_id)
|> assign(:username, user.username)
|> assign(:email, user.email)
else
socket
|> assign(:user_id, nil)
|> assign(:uuid, params["uuid"])
end
{:ok, socket}
end

Now we can persist our messages to the database. The most logical place to do this is within our handle_in function in the RoomChannel, since this is where we handle incoming messages. Rather than simply broadcast the message as we did previously, we're going to add the message to the database, then broadcast it.

Updating handle_in

Once we receive an incoming connection, we add two parameters to our payload from socket.assigns using Map.put, which adds the named key-value pair to a map: user_id and from.

Then we create a changeset with the %Message{} model and the payload we just updated in preparation for our database update.

Then we use a case statement to check if Repo.insert accepts our changeset. If it does, it will return {:ok, message} and we should format our message with message_payload and broadcast as we we did before. In the event of an error, it returns an error.

The message payload sent to all client subscribers to the "message" event and includes a from field. This from is used to identify who sent the message and is either a user_id or a uuid. If a uuid, then user is anonymous; if a user_id, the user is an admin.

def handle_in("message", payload, socket) do
payload = payload
|> Map.put("user_id", socket.assigns.user_id)
|> Map.put("from", socket.assigns[:uuid])
changeset = Message.changeset(%Message{}, payload)
case Repo.insert(changeset) do
{:ok, message} ->
payload = message_payload(message)
broadcast! socket, "message", payload
{:reply, :ok, socket}
{:error, changeset} ->
{:reply, {:error, %{errors: changeset}}, socket}
end
end

Unix timestamp to DateTime

Since we're receiving all of our timestamps from JavaScript, the time is going to be in milliseconds since January 1, 1970, which is a different format than Ecto.DateTime expects. Because of this disparity, we're going to create a small module to handle this for us.

$ touch lib/phoenix_chat/date_time.ex
defmodule PhoenixChat.DateTime do
@behaviour Ecto.Type
def type(), do: :datetime
def cast(milliseconds) when is_integer(milliseconds) do
with {:ok, datetime} <- DateTime.from_unix(milliseconds, :milliseconds),
{:ok, ecto_datetime} <- Ecto.DateTime.cast(datetime),
do: {:ok, ecto_datetime}
end
def cast(value) do
Ecto.DateTime.cast(value)
end
def load(value) do
Ecto.DateTime.load(value)
end
def dump(value) do
Ecto.DateTime.dump(value)
end
end

Then we need to change our Message model to use our PhoenixChat.DateTime module in message.ex.

...
schema "messages" do
field :body, :string
field :timestamp, PhoenixChat.DateTime
field :room, :string
field :from, :string
belongs_to :user, PhoenixChat.User
timestamps
end
...

And now we have message persistence to our Postgres database. You'll also notice that when write messages in your phoenix-chat component, the messages will appear on the correct side. That's because we're now passing in the from field, which our component is using to determine on which side the messages displays.

Previous: Use Presence to List Active UsersNext: Create Anonymous User Model