The examples here are written in Elm, the original language of the RemoteData pattern. Like the pattern itself, the thinking can be generalized to other languages.
I. What is a code meme?
I’m not certain that code memes exist. I have a hard time defining the idea and unlike Justice Stewart, I am uncomfortable asserting the reality of something that escapes definition.1 “Code meme” is a label I give to some bits of programming I encounter on Twitter. The technical content of the code meme is a readymade response to a common experience, a macro of thought, rigidly copied from one context to the next. Like any meme, a code meme suddenly seems to be everywhere at once until suddenly it is nowhere at all.
Not every use of a programming pattern is mimetic. Programmers often adapt a pattern to a problem. Then the pattern informs but does not dictate the result. Code memes defy adaptation. Perhaps the most well-known code meme is the Hello World program. Hello World is a readymade solution to the problem of what program to write when beginning to learn a new language. The thought expressed by the code is unmodified. Only details like language or implementation change.
II. RemoteData is a code meme
The code meme Slaying a UI Antipattern in… refers to the RemoteData pattern. 2, 3, 4, 5, 6 To invoke the meme as “Slaying a UI Antipattern in Elm” means to show an Elm implementation of RemoteData. Like the Hello World program, the implementation is used to sample the flavor of the language, specifically the flavor of that language’s union types.
The RemoteData pattern and the UI Antipattern are both approaches to modeling the state of data generated by a REST request. User interfaces generally present remote data in states like not loaded, loading, error, and success. The UI Antipattern represents these states implicitly. The current state is derived from a model with roughly this shape:
RemoteData improves on this approach by using a union type to make these states explicit. The state of the data is encoded into the data’s type:
Making the state explicit has two benefits. First, the state is universally consistent and clear. There is no risk that consumers of the data will incorrectly derive an implicit state. Second, consumers of the data are required to handle every state:
This code will not compile unless the patterns of the case statement cover every possible state. The result is a robust system.
III. RemoteData does not model all possible states
RemoteData models a stateful cache of data in terms of a stateless
transfer of data, REST.
The current state of a
RemoteData value preserves none of its prior state.
For example, the transformation of
Success b to
Loading means that the
b is lost.
To carry that information from
Success b to
Loading requires us to update
Loading constructor to
Loading b or
Loading (Maybe b).
Here we must recall the old adage: user interfaces are state machines.
Many common features demand a continuity of context.
For example, infinite scroll is a combination of
Success a and
By breaking this continuity, RemoteData makes it impossible to model many
These states include:
- empty, general error, and request pending
- empty, general error, and request pending for a subset of the data
- empty, error for a subset of the data, and request pending
- empty, error for a subset of the data, and request pending for a subset of the Data
- data cached and general error
- data cached and error for a subset of the data
- data cached and request pending
- data cached and request pending for a subset of the data
- data cached, general error, and request pending
- data cached, general error, and request pending for a subset of the data
- data cached, error for a subset of the data, and request pending
- data cached, error for a subset of the data, and request pending for a subset of the data
We can attempt to salvage the RemoteData pattern by adding nullable error and
data parameters to the
This enables us to model states that are a mix of loading, error, and data. For example, the “data cached, general error, and request pending” state would look like this:
The “empty, general error, and request pending” state would look like this:
We can use a similar approach to associate the loading and error states with a
subset of the data.
Here we assume a collection of items where each item is uniquely identified by a
The relationship between the state and an item in the collection is made by
adding a nullable string parameter to the
But this approach only enables us to associate the error state or the loading
state with one item at a time.
The approach does not enable us to relate a mix of states with multiple items in
Nor can we relate multiple states to the same item at the same time.
Loading (Maybe String) (Maybe a) (Maybe b) must become
Loading (Maybe String) (Maybe String, Maybe a) (Maybe b) to associate both the
error and the loading state with an item in the collection.
We could continue to cram more information into the
RemoteData type but I
don’t recommend it.
This path leads directly back to implicit states.
Any code that consumes a value of the
RemoteData type will explode with the
nested case statements required to derive the current state.
IV. Inefficient-meme hypothesis
The efficient-market hypothesis states that prices always reflect all available information. I propose an inefficient-meme hypothesis: a meme never reflects all available information. As an utterance becomes more and more memetic, it is a map corresponding less and less to the territory.7
Edsger Dijkstra famously defined programming as “the art of organizing complexity, of mastering multitude”.8 In organizing complexity, we cannot make our programs less complex than they need to be. As programmers, we should be wary of code memes. The inefficient-meme hypothesis suggests that the code meme masters multitudes by leaving some of those multitudes out.
V. An alternative
It is tempting to sign off here, at the conclusion of my critique. But the critique does not solve the problem of modeling the states of a remote data cache. I work a lot on user interfaces. Whether or not I use the remote data pattern, I still want to solve that problem. So here is an alternative approach to modeling these states. This is an experimental approach. I’m not sure that it is a sane approach. I am always suspicious that there is a simpler way and glad to hear when there is.
The core of this problem is an insufficient number of states.
Even a relatively simple user interface requires many more than the four states
There are two kinds of missing state.
The first kind of missing state is compound state.
Even a relatively simple user interface displays states that are a mix of
“empty”, “loading”, “error”, and “success”.
Let’s start by defining those states.
The second kind of missing state is specific state.
The state of a collection and the state of an item in that collection are not
always the same.
For example, an application might load a list of data first and then later make
a request for details about one item in that list.
Then only the item, not the collection, should be in the
General and specific states are achieved by creating a cache of caches.
Imagine that we are fetching data from a Person API.
Person, we’ll create a
Then we’ll store each
PersonCache in a
Dict keyed by
Finally, we’ll create a
Cache for this
General states are managed by the outer cache.
Specific states are managed by the inner caches.
All possible states are represented.
But when will the cache change its current state?
The cache will change its state in response to a
Our cache will respond to three events: the start of a request to the remote
data source and resolutions of that request to an error or data.
VI. A remote data cache is a state machine
Every combination of cache state and cache event results in a new cache state.
All states and all events are known, and all state changes are knowable too.
For example, a cache in the
Empty state should change to
Error a event occurs.
Here is a table that shows how each state should change in response to each
State | Sync | Error | Update ----------------------------------------------------------------------------------- Empty | EmptySyncing | EmptyInvalid a | Filled b EmptyInvalid a | EmptyInvalidSyncing a | EmptyInvalid a | Filled b EmptySyncing | EmptySyncing | EmptyInvalid a | Filled b EmptyInvalidSyncing a | EmptyInvalidSyncing a | EmptyInvalid a | Filled b Filled b | FilledSyncing b | FilledInvalid a b | Filled b FilledSyncing b | FilledSyncing b | FilledInvalid a b | Filled b FilledInvalid a b | FilledInvalidSyncing a b | FilledInvalid a b | Filled b FilledInvalidSyncing a b | FilledInvalidSyncing a b | FilledInvalid a b | Filled b
We know what the states are, when the states change, and how the states change. We know what we want to happen. Now we need a system that will make it happen. In fact, there is already a name for such a system. By describing the remote data cache, we have specified a finite state machine.
A finite state machine has three components: states, events, and transitions. An event prompts the finite state machine to transition from its current state to a new state. The finite state machine makes the correct transition by testing conditions at the time of the event. In our case, these conditions are all the combinations of states and events found in our state change table.
A simplified Elm
update function closely resembles a simple state machine:
Msg is like an event, the
Model is like a state, and each branch of the
case expression is a transition from the current state to the next state.
A major difference between this
update function and a finite state machine is
update produces an infinite number of states.
Without upper and lower bounds,
update can return any member of the infinitely
large set of integers.
We can write a similar update function that changes the state of the remote
This is the heart of the state machine.
updateCache takes a
CacheEvent event and a current
Cache state, and
returns a new
The state will change according to the rules defined in the state change table
There is no prescribed implementation of a remote data cache as a finite state machine. States, events, and transitions should reflect the needs of the application. And state machines themselves come in several varieties. My principal aim is to make the conceptual connection. Nonetheless, it is not enough to say “just use a finite state machine”. Implementing a state machine for this purpose is not trivial, so I want to touch on a few of the details.
VII. Polymorphic transitions without polymorphism
updateCache, we must remember that this cache system
should be compatible with different types of data.
There is one problem here.
When the state of the cache changes, it’s sometimes necessary to change the
For example, the
Update b event might require us to merge new data with
existing data if the state of the cache is
How the data changes is probably specific to the type of data.
So the cache must do some type-specific transformations while remaining
uncoupled to the type.
The solution is to pass transitions into the state machine. These transitions are hooks for moments in the process of changing the state. The state machine knows how and when to call these hooks but does not know what they do. This keeps type-specific logic out of the cache while enabling type-specific transformations.
For the moment, let’s provide two of these transitions.
This first is
updateEmpty, a transition that is called when an empty cache
changes to a filled cache.
updateEmpty is called with data from the
Update a cache event.
The second is
updateFilled, a transition that is called when a filled cache
changes to a filled cache.
updateFilled is called with data from the
Update a cache event and data that
is already in the cache.
VIII. General and specific state changes
Recall that our Person cache example is structured as a cache of caches. As we move forward with the implementation, we realize that there is no way to update the inner caches. Updating the inner cache should begin as an update to the outer cache. But how can the outer cache know that the event and the resulting state change are specific to one of the inner caches? In our system, it cannot.
Again, the cache should not know the structure of the cached data.
This means that the cache should not even know about the inner caches.
Any cache just needs to know the difference between a change to all of the
cached data and a change to a piece of the cached data.
Events are the medium of communication with the state machine.
Update b event signals a change to all of the cached data, we need to
add a new event that signals a change to a subset of the data.
We’ll call this event
Changing the data in response to
Update b is different from changing the data
in response to
For example, merging two lists of data in response to
Update b is one
Merging two copies of one item in that list in response to
Patch c is a second
To account for this difference, we’ll add a
This transition is called with data from the
Patch c event and all of the
patchFilled, not the state machine, is responsible for selecting the subset of
data to transform.
To update the state of an inner cache, we’ll send an event through the outer
cache to the inner cache.
Patch CacheEvent is used to carry the inner cache event through the outer
patchFilled receives that event and coordinates the update.
This is the part of the approach that I am the least comfortable with. The concept is undoubtedly hard to understand here and it was tricky to implement for the proof of concept.
IX. Mediating state explosion
All consumers of the remote data cache are required to handle every cache state. Depending on the pattern matching capabilities of the language, explicit states result in verbose code. The code becomes especially noisy when many states are handled in the same way.
View functions are prone to suffer from this noisiness. Many of these view functions have a binary relationship to the state of the cache. A loading icon is displayed when the cache is in a loading state. Otherwise, no loading icon is displayed. An error message is displayed when the cache is in an error state. Otherwise, no error message is displayed. View functions responsible for these features do not need to know all varieties of cache state. It is advisable to reduce the many states of the cache to a set of fewer states for the view.
Then each kind of view defines its own reduction of
For example, an error view might reduce all cache error states to
Show Error and all other cache states to
This approach is enhanced by a higher-order function that handles the
The view itself is a composition of these functions. This keeps large case statements out of the function that produces the html.
X. Unfinished maps
There is a lot of information here and I’m not convinced that any of it is good. This is, emphatically, an experiment. While the RemoteData pattern was too simple, this approach is in danger of becoming too complicated. We are accustomed to doubt the correctness of a messy abstraction because it is messy. I am inclined to also doubt tidy abstractions because they are tidy. The lesson of the code meme is that messiness and tidiness of abstraction can be the same kind of incorrectness. Our world is both messy and tidy. Our abstractions crystallize our understanding of that world. So we can expect our abstractions to be messy and tidy too. Pronounced messiness and pronounced tidiness might both be signs of incomplete understanding. We began with a tidy abstraction and now end with a messy abstraction. Please let me know when you find the middle path.
The usefulness of the code meme, like any code, depends on how accurately it represents the problem:
A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness.— Alfred Korzybski
^ Edsger Dijkstra (1970) "Notes On Structured Programming" via Wikiquote:
The art of programming is the art of organizing complexity, of mastering multitude and avoiding its bastard chaos as effectively as possible.