I’m working through Hypermedia Sytstems using django and htmx in public. This post starts applying the patterns from Chapter 6: More Htmx Patterns. In the last post, I finished working through Chapter 5. If you haven’t read the prior posts, they’re linked from the introductory one. The code can be found on sr.ht. To pick up where this post does, start from the tag pagination.

Now that paging through our table lines up better with what users expect in 2023, it’s time to improve the search experience. Currently, users type into a search box, click search, and get a whole page back. I would like to preserve that behavior, but also make the table update, live, as the user types into the search box.

First, I’ll improve the display a bit by applying the tool-bar class from missing.css and render the search field more manually so that hx- attributes can be added:

<form method="get" action="{% url 'contacts:contacts' %}" class="tool-bar">
    {{ search_form.query.label_tag }}
    <input
            id="{{ search_form.query.id_for_label }}"
            {% if search.form.query.value %}
            value="{{ search_form.query.value }}"
            {% endif %}
            name="{{ search_form.query.html_name }}"
            maxlength="255"
    >
    <input type="submit" value="Search">
</form>

widget_tweaks offers a less manual way to do this; I’m trying this out to see which I prefer.

As a first attempt at active search, I’ll have this field automatically send the get request while typing and replace the table body:

<form method="get" action="{% url 'contacts:contacts' %}" class="tool-bar">
    {{ search_form.query.label_tag }}
    <input
            id="{{ search_form.query.id_for_label }}"
            {% if search.form.query.value %}
            value="{{ search_form.query.value }}"
            {% endif %}
            name="{{ search_form.query.html_name }}"
            maxlength="255"
            hx-get="{% url 'contacts:contacts' %}"
            hx-trigger="search, keyup delay:200ms changed"
            hx-target="tbody"
            hx-swap="outerHTML"
            hx-select="tbody"
    >
    <input type="submit" value="Search">
</form>

That was almost perfect. Adding hx-push-url="true" got exactly the behavior I wanted. It’s even a nice progressive enhancement. On an app this simple, I’d probably usually stop here. But it seems worth following along with the book and making a back-end change to get the server to send only the part of the HTML I’d like to replace instead of selecting that on the client side using hx-select.

The django-htmx middleware I installed earlier makes it easy to use the headers sent by htmx on the client side to distinguish an htmx-originated search request and just send a fragment. To do that, it’s easier if I separate the part of the contact_list.html template that renders the rows into its own file, contact_list_rows.html:

<tbody>
{% for contact in page_obj %}
    <tr>
        <td>{{contact.first_name}}</td>
        <td>{{contact.last_name}}</td>
        <td>{{contact.phone_number}}</td>
        <td>{{contact.email}}</td>
        <td>
            <a href="{% url 'contacts:edit_contact' contact.id %}">Edit</a>
            <a href="{% url 'contacts:contact' contact.id %}">View</a>
        </td>
    </tr>
{% endfor %}
{% if page_obj.has_next %}
    <tr>
        <td colspan="5" style="text-align: center">
            <span hx-target="closest tr"
                  hx-swap="outerHTML"
                  hx-select="tbody > tr"
                  hx-trigger="revealed"
                  hx-get="?query={{ query }}&page={{ page_obj.next_page_number }}">
                Loading more contacts...
            </span>
        </td>
    </tr>
{% endif %}
</tbody>

and include that from contact_list.html:

{% extends "base.html" %}
{% block content %}
    <form method="get" action="{% url 'contacts:contacts' %}" class="tool-bar">
        {{ search_form.query.label_tag }}
        <input
                id="{{ search_form.query.id_for_label }}"
                {% if search.form.query.value %}
                value="{{ search_form.query.value }}"
                {% endif %}
                name="{{ search_form.query.html_name }}"
                maxlength="255"
                hx-get="{% url 'contacts:contacts' %}"
                hx-trigger="search, keyup delay:200ms changed"
                hx-target="tbody"
                hx-swap="outerHTML"
                hx-push-url="true"
        >
        <input type="submit" value="Search">
    </form>
    <table>
        <thead>
            <tr>
                <th>First Name</th>
                <th>Last Name</th>
                <th>Phone Number</th>
                <th>Email Address</th>
                <th></th>
            </tr>
        </thead>
        {% include 'contacts/contact_list_rows.html' %}
    </table>
    <div>
        <a href="{% url 'contacts:add_contact' %}">Add new contact</a>
    </div>
{% endblock content %}

Once that’s separate, it’s a small tweak to the view function to notice the search trigger from HTMX and send only the table body:

def contacts(request):
    contacts_list = Contact.objects.all()
    search_form = SearchForm()
    query = request.GET.get("query", "")
    if query:
        search_form = SearchForm(initial={"query": query})
        contacts_list = contacts_list.filter(
            Q(first_name__icontains=query)
            | Q(last_name__icontains=query)
            | Q(email__icontains=query)
        )
    paginator = Paginator(contacts_list, 25)
    page_obj = paginator.get_page(request.GET.get("page", None))
    ctx = {
        "page_obj": page_obj,
        "search_form": search_form,
        "query": query,
    }
    if request.htmx and request.htmx.trigger_name == "query":
        return render(request, "contacts/contact_list_rows.html", ctx)
    return render(request, "contacts/contact_list.html", ctx)

With a real remote database, this could cause some disconcerting lag since there’s not yet any loading indicator. So as a last step, adding an hx-indicator element can offer the user some feedback:

{% extends "base.html" %}
{% load static %}
{% block content %}
    <form method="get" action="{% url 'contacts:contacts' %}" class="tool-bar">
        {{ search_form.query.label_tag }}
        <input
            id="{{ search_form.query.id_for_label }}"
            {% if search.form.query.value %}
                value="{{ search_form.query.value }}"
            {% endif %}
            name="{{ search_form.query.html_name }}"
            maxlength="255"
            hx-get="{% url 'contacts:contacts' %}"
            hx-trigger="search, keyup delay:200ms changed"
            hx-target="tbody"
            hx-swap="outerHTML"
            hx-push-url="true"
            hx-indicator="#spinner"
        >
        <input type="submit" value="Search">
    </form>
    <img src="{% static 'img/spinning-circles.svg' %}" id="spinner" class="htmx-indicator" alt="Loading...">
    <table>
        <thead>
            <tr>
                <th>First Name</th>
                <th>Last Name</th>
                <th>Phone Number</th>
                <th>Email Address</th>
                <th></th>
            </tr>
        </thead>
        {% include 'contacts/contact_list_rows.html' %}
    </table>
    <div>
        <a href="{% url 'contacts:add_contact' %}">Add new contact</a>
    </div>
{% endblock content %}

These are small modifications that feel very satisfying to me.

Lazy Loading

It can improve some apps' UX to defer the loading of some sections of a page.

Though that’s unlikely to be a concern with a contacts app like this, it isn’t hard to simulate the concern for the sake of exploration by adding a function that returns the count of all contacts in the database. Since that’s the idea that the book uses to motivate lazy loading, I’ll start by adding a custom count function that takes a very long time to return the number of contacts in a queryset:

def count_contacts(queryset):
    sleep(1)
    return queryset.count()

Then I’ll wreck my app’s UX by adding that to the listing page, first by having the view add it to the template context:

def contacts(request):
    contacts_list = Contact.objects.all()
    search_form = SearchForm()
    query = request.GET.get("query", "")
    if query:
        search_form = SearchForm(initial={"query": query})
        contacts_list = contacts_list.filter(
            Q(first_name__icontains=query)
            | Q(last_name__icontains=query)
            | Q(email__icontains=query)
        )
    paginator = Paginator(contacts_list, 25)
    page_obj = paginator.get_page(request.GET.get("page", None))
    ctx = {
        "page_obj": page_obj,
        "search_form": search_form,
        "query": query,
        "total": count_contacts(contacts_list),
    }
    if request.htmx and request.htmx.trigger_name == "query":
        return render(request, "contacts/contact_list_rows.html", ctx)
    return render(request, "contacts/contact_list.html", ctx)

and finally by rendering it in the template:

{% extends "base.html" %}
{% load static %}
{% block content %}
    <form method="get" action="{% url 'contacts:contacts' %}" class="tool-bar">
        {{ search_form.query.label_tag }}
        <input
            id="{{ search_form.query.id_for_label }}"
            {% if search.form.query.value %}
                value="{{ search_form.query.value }}"
            {% endif %}
            name="{{ search_form.query.html_name }}"
            maxlength="255"
            hx-get="{% url 'contacts:contacts' %}"
            hx-trigger="search, keyup delay:200ms changed"
            hx-target="tbody"
            hx-swap="outerHTML"
            hx-push-url="true"
            hx-indicator="#spinner"
        >
        <input type="submit" value="Search">
    </form>
    <img src="{% static 'img/spinning-circles.svg' %}" id="spinner" class="htmx-indicator" alt="Loading...">
    <table>
        <thead>
            <tr>
                <th>First Name</th>
                <th>Last Name</th>
                <th>Phone Number</th>
                <th>Email Address</th>
                <th></th>
            </tr>
        </thead>
        {% include 'contacts/contact_list_rows.html' %}
    </table>
    <div>
        <a href="{% url 'contacts:add_contact' %}">Add new contact</a>
    </div>
    <div><span class="color info">({{ total }} Total Contacts)</span></div>
{% endblock content %}

As expected, a browser test at this point slows page loads down quite visibly.

To fix this, it’s time to move the count into its own view function. Since I don’t want to just copy and paste the query, first I add a custom manager for the Contact model that knows how to do a text search:

class ContactManager(models.Manager):
    def text_search(self, query_string):
        if not query_string:
            return self.all()
        return self.filter(
            Q(first_name__icontains=query_string)
            | Q(last_name__icontains=query_string)
            | Q(email__icontains=query_string)
        )


class Contact(models.Model):
    first_name = models.CharField(max_length=255, blank=True)
    last_name = models.CharField(max_length=255, blank=True)
    phone_number = models.CharField(max_length=32, blank=True)
    email = models.EmailField(unique=True)

    objects = ContactManager()

    def __str__(self):
        return f"{self.first_name} {self.last_name} <{self.email}>"

Then I use that in the list view:

def contacts(request):
    search_form = SearchForm()
    query = request.GET.get("query", "")
    contacts_list = Contact.objects.text_search(query)
    paginator = Paginator(contacts_list, 25)
    page_obj = paginator.get_page(request.GET.get("page", None))
    ctx = {
        "page_obj": page_obj,
        "search_form": search_form,
        "query": query,
        "total": count_contacts(contacts_list),
    }
    if request.htmx and request.htmx.trigger_name == "query":
        return render(request, "contacts/contact_list_rows.html", ctx)
    return render(request, "contacts/contact_list.html", ctx)

After confirming that I didn’t break anything, I move the expensive call into a new view function:

def contacts(request):
    search_form = SearchForm()
    query = request.GET.get("query", "")
    contacts_list = Contact.objects.text_search(query)
    paginator = Paginator(contacts_list, 25)
    page_obj = paginator.get_page(request.GET.get("page", None))
    ctx = {
        "page_obj": page_obj,
        "search_form": search_form,
        "query": query,
    }
    if request.htmx and request.htmx.trigger_name == "query":
        return render(request, "contacts/contact_list_rows.html", ctx)
    return render(request, "contacts/contact_list.html", ctx)


@require_safe
def contacts_count(request):
    query = request.GET.get("query", "")
    contacts_list = Contact.objects.text_search(query)
    ctx = {
        "total": count_contacts(contacts_list),
    }
    return render(request, "contacts/contact_total.html", ctx)

and a corresponding trivial template contact_total.html

<div><span class="color info">({{ total }} Total Contacts)</span></div>

then add a route for the new view:

urlpatterns = [
    path("", views.index, name="index"),
    path("contacts", views.contacts, name="contacts"),
    path("contacts/new", views.add_contact, name="add_contact"),
    path("contacts/<int:pk>", views.contact_handler, name="contact"),
    path("contacts/<int:pk>/edit", views.edit_contact, name="edit_contact"),
    path("contacts/<int:pk>/delete", views.delete_contact, name="delete_contact"),
    path("contacts/<int:pk>/email", views.check_email, name="check_contact_email"),
    path("contacts/email", views.check_email, name="check_new_email"),
    path("contacts/count", views.contacts_count, name="count"),
]

and change the old template to have htmx lazy load the new view:

    <div hx-get="{% url 'contacts:count' %}" hx-trigger="load"><span class="color info"></span></div>

I chose to trigger on load instead of on reveal because the item is at the bottom of the table, and I don’t like the UX when the infinite scroll interacts with it.

That fixes the performance problem on load, but it’s confusing to have only the total display when there’s a live search filtering the results list. This view is easily reused for a filtered count, which can be triggered as the results get updated using custom events.

I added a new status field to the search form:

<form method="get" action="{% url 'contacts:contacts' %}" class="tool-bar">
    {{ search_form.query.label_tag }}
    <input
        id="{{ search_form.query.id_for_label }}"
        {% if query %}
            value="{{ query }}"
        {% endif %}
        name="{{ search_form.query.html_name }}"
        maxlength="255"
        hx-get="{% url 'contacts:contacts' %}"
        hx-trigger="search, keyup delay:200ms changed"
        hx-target="tbody"
        hx-swap="outerHTML"
        hx-push-url="true"
        hx-indicator="#spinner"
    >
    <input type="submit" value="Search">
    <div id="search_status"
            {% if query %}
                hx-get="{% url 'contacts:count' %}?query={{ query }}"
                hx-trigger="load"
            {% endif %}
    ></div>
</form>

I used custom events because sending it down with the updated table in an element marked hx-swap-oob="true" didn’t work the way I expected it to. It sounds like HTMX might be doing something special with the <tbody> node when it’s swapping things in. So I updated the live search to send an event containing the query string:

def contacts(request):
    search_form = SearchForm()
    query = request.GET.get("query", "")
    contacts_list = Contact.objects.text_search(query)
    paginator = Paginator(contacts_list, 25)
    page_obj = paginator.get_page(request.GET.get("page", None))
    ctx = {
        "page_obj": page_obj,
        "search_form": search_form,
        "query": query,
    }
    if request.htmx and request.htmx.trigger_name == "query":
        response = render(request, "contacts/contact_list_rows.html", ctx)
        return trigger_client_event(
            response, "updated-query", {"query": f"?query={query}"}
        )
    return render(request, "contacts/contact_list.html", ctx)

and wrote just a little bit of javascript to act on that:

        document.addEventListener("updated-query", (evt) => {
            status_elt = document.getElementById("search_status")
            status_elt.setAttribute("hx-get", "{% url 'contacts:count' %}" + evt.detail.query)
            status_elt.setAttribute("hx-trigger", "update-query-count")
            status_elt.textContent = ""
            htmx.process(status_elt)
            status_elt.dispatchEvent(new Event("update-query-count"))
        })

That javascript listens for the updated query to come from the server. When it does, it updates the status field above to have an hx-get URL that includes the new query string, sets its trigger to a new event, tells htmx about the change, then sends the new event. That’s not as simple as I’d like it to be, my complaint is more with verbosity here than any real complexity. I’ll revisit this if anyone on the htmx discord has some insight.

Inline Deletion

At the moment, deleting a contact is a little bit cumbersome. You have to click edit in order to get to a delete button. It requires relatively little to make that accessible from the table view itself.

First, we need to add a link to each row:

    {% for contact in page_obj %}
        <tr>
            <td>{{contact.first_name}}</td>
            <td>{{contact.last_name}}</td>
            <td>{{contact.phone_number}}</td>
            <td>{{contact.email}}</td>
            <td>
                <a href="{% url 'contacts:edit_contact' contact.id %}">Edit</a>
                <a href="{% url 'contacts:contact' contact.id %}">View</a>
                <a href="#"
                   hx-delete="{% url 'contacts:contact' contact.id %}"
                   hx-confirm="Really delete {{ contact }}?"
                >
                    Delete
                </a>

            </td>
        </tr>
    {% endfor %}

Even hx-boost-ed, though, that’s not a great UX. It’d be better if the row just disappeared without reloading the entire contact list. That’s easily done by having the server send an empty string, adding an hx-target attribute, and having htmx swap the outerHTML:

<a href="#"
    hx-delete="{% url 'contacts:contact' contact.id %}"
    hx-confirm="Really delete {{ contact }}?"
    hx-target="closest tr"
    hx-swap="outerHTML"
>

But in order to preserve the behavior of the form, the server will need to see what triggered the deletion before sending the empty string; the edit form should still redirect.

@require_http_methods(["DELETE"])
def delete_contact(request, contact):
    contact.delete()
    messages.info(request, f"Contact {contact} deleted.")
    if request.htmx and request.htmx.trigger_name == "delete-contact-button":
        return HttpResponseSeeOther(resolve_url("contacts:contacts"))
    return HttpResponse("")

With that, deletion works from both places, but it’s so fast that it’s almost hard to follow. Adding a little bit of styling helps a great deal with this. An inline style sheet the top of the contact list template:

<style>
tr.htmx-swapping {
    opacity: 0;
    transition: opacity 1s ease-out;
}
</style>

along with an update to the delete link:

hx-swap="outerHTML swap:1s"

will cause the row to fade out in a way that makes it easy to see. That makes inline deletion work, but the flash messages don’t show until we have an actual page load. This time, a new view and a custom event with no new javascript will take care of it. First, I need to move the notifications from the base template to their own template and include them from the base:

notifications.html

<div id="notifications">
    {% if messages %}
        {% for message in messages %}
            <div class="box {{ message.level_tag }}">{{ message }}</div>
        {% endfor %}
    {% endif %}
</div>

base.html

<body hx-boost="true" hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'>
    <header>
        <nav>
            <h1><a href="{% url 'contacts:index' %}">Contacts.app</a></h1>
        </nav>
        {% include 'notifications.html' %}
    </header>
    <main>
        {% block content %}
            <h1>Content goes here!</h1>
        {% endblock content %}
    </main>
</body>

Then write a view function that just sends notifications

def notifications(request):
    return render(request, "notifications.html", {})

and add a URL for it:

app_name = "contacts"
urlpatterns = [
    path("", views.index, name="index"),
    path("notifications", views.notifications, name="notifications"),
    path("contacts", views.contacts, name="contacts"),
    path("contacts/new", views.add_contact, name="add_contact"),
    path("contacts/<int:pk>", views.contact_handler, name="contact"),
    path("contacts/<int:pk>/edit", views.edit_contact, name="edit_contact"),
    path("contacts/<int:pk>/delete", views.delete_contact, name="delete_contact"),
    path("contacts/<int:pk>/email", views.check_email, name="check_contact_email"),
    path("contacts/email", views.check_email, name="check_new_email"),
    path("contacts/count", views.contacts_count, name="count"),
]

With those in place, attributes are all that are necessary to tell the app to fetch notifications in response to server events:

<div id="notifications"
     hx-get="{% url 'contacts:notifications' %}"
     hx-trigger="update-notifications from:body"
>
    {% if messages %}
        {% for message in messages %}
            <div class="box {{ message.level_tag }}">{{ message }}</div>
        {% endfor %}
    {% endif %}
</div>

and then that update needs to be triggered by the empty delete view:

@require_http_methods(["DELETE"])
def delete_contact(request, contact):
    contact.delete()
    messages.info(request, f"Contact {contact} deleted.")
    if request.htmx and request.htmx.trigger_name == "delete-contact-button":
        return HttpResponseSeeOther(resolve_url("contacts:contacts"))

    response = HttpResponse("")
    return trigger_client_event(response, "update-notifications")

Bulk Delete

The last section of Chapter 6 has us adding the ability to delete contacts in bulk by checking a box next to each one. Most of what’s necessary is already here.

Before starting this, it makes sense to move the add link to the top of the table, turn it into a button, and move the contact count to the top as well. Otherwise it gets annoying to test because of the infinite scroll on the table.

First, there needs to be an extra column on the table:

So contact_list.html grows:

    <table>
        <thead>
            <tr>
                <th></th>
                <th>First Name</th>
                <th>Last Name</th>
                <th>Phone Number</th>
                <th>Email Address</th>
                <th></th>
            </tr>
        </thead>
        {% include 'contacts/contact_list_rows.html' %}
    </table>

and contact_list_rows.html does also:

{% for contact in page_obj %}
    <tr>
        <td><input type="checkbox" name="selected_contact_ids" value="{{ contact.id }}"></td>
        <td>{{contact.first_name}}</td>
        <td>{{contact.last_name}}</td>
        <td>{{contact.phone_number}}</td>
        <td>{{contact.email}}</td>
        <td>
            <a href="{% url 'contacts:edit_contact' contact.id %}">Edit</a>
            <a href="{% url 'contacts:contact' contact.id %}">View</a>
            <a href="#"
                hx-delete="{% url 'contacts:contact' contact.id %}"
                hx-confirm="Really delete {{ contact }}?"
                hx-target="closest tr"
                hx-swap="outerHTML swap:1s"
            >
                Delete
            </a>
        </td>
    </tr>
{% endfor %}

The easiest way to collect the row ids is to add the whole table to a form along with a delete button:

<form>
    <div class="tool-bar">
        <a class="<button>" href="{% url 'contacts:add_contact' %}">Add new contact</a>
        <button
                hx-delete="{% url 'contacts:contacts' %}"
                hx-confirm="Are you sure you want to delete these contacts?"
                hx-target="body"
        >
            Delete selected contacts
        </button>
        <div id="total-display" hx-get="{% url 'contacts:count' %}" hx-trigger="load"><span class="color info"></span></div>
    </div>
    <table class="table">
        <thead>
            <tr>
                <th></th>
                <th>First Name</th>
                <th>Last Name</th>
                <th>Phone Number</th>
                <th>Email Address</th>
                <th></th>
            </tr>
        </thead>
        {% include 'contacts/contact_list_rows.html' %}
    </table>
</form>

For the server side, to maintain RESTful appearences, first I break the contacts URL into a dispatcher and a view for each verb:

def contacts_handler(request):
    match request.method:
        case "GET" | "HEAD":
            return contacts(request)
        case "DELETE":
            return bulk_delete(request)
    raise BadRequest()

and point the url map there:

    path("contacts", views.contacts_handler, name="contacts"),

The bulk_delete function is only interesting because django doesn’t provide the same convenience methods for DELETE requests as it does for POST and GET:

def bulk_delete(request):
    contact_ids = list(
        map(int, QueryDict(request.body).getlist("selected_contact_ids"))
    )
    records = [get_object_or_404(Contact, pk=cid) for cid in contact_ids]
    count = len(records)
    for c in records:
        c.delete()
    messages.info(request, f"Deleted {count} contacts")
    return contacts(request)

It might be interesting to attempt to preserve any filter the user has set here, but for today, this is good enough.

The next post will pick up with Chapter 07: A Dynamic Archive UI.