Hands on With a Solid Web App

Background Information

In 1989, Sir Tim Berners Lee invented the World Wide Web. A year later, he released the first browser. At the time, it was a means for researchers at CERN and other research institutions to share information. It rapidly exploded in the following years and now dominates the world economy. But the web we use now is not what Sir Berners Lee had envisioned. While there is widespread sharing of information, as was the original intention, it is controlled by a few big tech companies acting as gatekeepers. They make profit by selling the user’s data. The users do not have control of their data, having limited ability to retrieve, delete or control who can access it. Legislation, such as GDPR in the EU, has been slow to adapt and often falls short of resolving the fundamental issues at hand.

Today, Sir Berners Lee is leading the charge to push the web back towards the vision he had had at the beginning. Giving the users the control without losing the power of the apps which have been developed in the past decades to improve our lives.

‍

How to Fix it

The key step is to invert the idea of where data is kept. Currently, most web services have their own databases which store all the relevant information about you. The key idea of Solid is to put all that data into a single “Pod” which all apps can access (given appropriate permission). This pod would be under the user’s direct control so that they can control the permissions as they wish. If they come to disagree with a company’s policies, they can immediately cut off access, without that company’s involvement. If the user moves to a new house or changes phone number, they have only a single location to change it, rather than having to remember which of the dozens of services need to be updated.

With application data being stored in the user’s control, rather than the apps’, this would also allow alternative apps to work with the same dataset — preventing vendor lock-in.

An added benefit is that since most app backends are there to provide an interface to the backend and control access rights, which many apps can do entirely without one, as these two functions are replaced by the pod provider.

‍

Our Experience With Developing Solid

Everyone in our Discovery group was quite technical. Our specializations are mixed, but we all know our way around web development quite well. But working with linked data - as is used in the Solid protocol - was a hurdle. We’re very familiar with relational databases (e.g. SQL), or document databases (e.g. mongoDB), and all the layers of abstraction built over the top of them to make working with them easier. Linked data is not yet as mature as these other technologies, and the tools are not yet as sophisticated as what we are accustomed to. So, our initial progress was slow.

To demonstrate, here is the same data, shown in both JSON and turtle (a linked data format used in Solid). First, the old school JSON:

‍

  “name”: “Stuart Heap”, 
  “email”: { 
    “work”: “fake.email@mail.com” 
  }, 
  “address”: { 
    “country”: “Germany”, 
    “locality”: “Munich”, 
    “postal-code”: “xxxxx”, 
    “region”: “Bavaria”, 
    “street-address”: “123 Fake Street” 
  }, 
  “organization-name”: “Motius GmbH”, 
  “role”: “Fullstack Developer” 
} 

‍

And now, the turtle (some fields have been removed for simpler presentation):

  @prefix : <#>. 
  @prefix pro: <./>. 
  @prefix n: . 
  @prefix n0: . 
  @prefix schem: .pro:card a n0:PersonalProfileDocument; n0:maker :me; n0:primaryTopic :me.:id1614167342660 a n:Work; n:value .:id1614174549144 
      n:country-name “Germany”; 
      n:locality “Munich”; 
      n:postal-code “xxxxx”; 
      n:region “Bavaria”; 
      n:street-address “123 Fake Street”.:me 
      a schem:Person, n0:Person; 
      n:fn “Stuart Heap”; 
      n:hasEmail :id1614167342660; 
      n:organization-name “Motius GmbH”; 
      n:role “Fullstack Developer”. 

‍

Now let’s demonstrate how one would get the user’s name in each example.

Traditional:

const getName = async () => { 
  const user = await getUser() 
  return user.name 
}

‍

Solid:

const getName = async () => { 
  const dataset = await getSolidDataset(webId) 
  if (dataset) { 
    const profile = getThing(dataset, webId) 
    if (profile) { 
      return getStringNoLocale(profile, VCARD.fn) || ‘Unknown’ 
    } 
  } 
  return ‘Unknown’ 
} 

‍

The increased line count is one thing, but not really the key point. It is pretty clear what user.name means. The developer knows immediately what is going on and can move on without thinking. However, getThing says basically nothing. Indeed, the Thing it returns could represent any form of data. In our case, where we wrote the data ourselves, there will be a Person stored, and that that person has a VCARD.fn (formatted name) value stored as a string. That however is not necessarily true. With a traditional architecture, with a specific database, I can ensure that every user will have a name, and that name will be a string. The frontend code does not need to be careful (that’s not to say exception handling isn’t needed, but let’s not get off topic here). In Solid, the user owns the data. And if there’s one thing which is true in the life of a web developer, it’s that users will do weird things. You can’t assume anything about the data, and so a Solid app will need much more care.

Given well written vocabularies (essentially schemas for linked data), it is imaginable that we could have tools which take a dataset and return an easily usable JSON object, or at least a similarly easy to use format. But during our short time with this technology, we found no such tools.

This is perhaps not a truly fair example. As it’s ignoring the fact that in most traditional apps, there would be a backend involved which would have to interface with the database to produce the JSON. One might argue that all we’ve done is move some of the backend complexity to the frontend, which would be a lateral move at worst. However, many modern databases simply store data in JSON (or something very much like it) already (e.g. MongoDB), and there are many tools which abstract this conversion away, and given a schema, will produce an interface to retrieve JSON (e.g. Prisma).

When the data gets more complicated, the added complexity here scales too. And it gets even worse when writing data to a pod. Here is the function from my beer recipe app for saving a new beer with a given name:

‍

const addBeer = async (name: string) => { 
  if (beerDataset.value) { 
    const id = `#${uniqid()}` 
    const newBeer = createThing({ 
      url: storageLocation.value + BEER_STORAGE + id, 
    }) 
    const typedBeer = setUrl( 
      newBeer, 
      RDF.type, 
      BEER_RECIPE.BeerRecipe.value, 
    ) 
    const namedBeer = setStringNoLocale( 
      typedBeer, 
      BEER_RECIPE.name.value, 
      name, 
    ) 
    const ownedBeer = setUrl( 
      namedBeer, 
      BEER_RECIPE.writtenBy.value, 
      webId.value, 
    ) 
    beers.value.push(ownedBeer) 
    beerDataset.value = setThing(beerDataset.value, ownedBeer) 
    saveSolidDatasetAt( 
      storageLocation.value + BEER_STORAGE, 
      beerDataset.value, 
      { fetch: session.value.fetch }, 
    ) 
  } 
} 

‍

For some context — it’s a Vue app, meaning there is data stored on the component in refs, hence all the .values you see littered around.

First we have to create a Thing. At this point it’s a completely abstract thing, the only distinguishing feature is where it will be located. We then start assigning properties to it one at a time. We say that it is of type BeerRecipe (a custom type published in my pod), and give it a name, id, and an author (each property needing its own function call). We then set that Thing in the dataset, and save it back to the pod.

For context, if we were using a traditional backend which accepted HTTP requests, the equivalent might look something like this:

‍

fetch(‘/beers’, { 
  method: ‘POST’, 
  headers: { 
    ‘Content-Type’: ‘application/json’, 
  }, 
  body: { 
    name, 
    author: user.id, 
  }, 
}) 

‍

Again, the argument could be made that the traditional example is hiding backend complexity. This is true, but we’re also skipping over writing the vocabulary for the beer recipe data. Which with my current understanding is a much more difficult task than handling the request on the backend and storing it in SQL, even if I were to forgo all the tools to make that process easier.

The bottom line here is that the fundamental CRUD operations are significantly more cumbersome to write when using the current Solid libraries, when compared to what we’re used to. If we want to produce features as quickly as we’re used to, with a nice developer experience, we’re going to need to improve the tooling. But that’s really no surprise: people have been iteratively developing the tooling to work with traditional technologies for decades. Solid has a lot of catching up to do.

‍

What Could the Future Look Like?

Solid is clearly designed with the end user in mind. And the benefits for the end user are necessary and valuable. From the developer’s side, there are apparent tradeoffs, but it is a community task to build the tools to make working with this technology easier and more accessible. With these tools in place, I think working with Solid will end up being easier than with a traditional backend. But for the time being, it makes things harder.

However, for the companies building apps, and hoping to make money out of them (ultimately, where most apps come from), I think there are more downsides than upsides. I think this is going to make widespread adoption of this technology an uphill battle. The entrenched position of established companies, especially with them already owning much of their users’ data, and holding it hostage, makes undercutting them a difficult proposition - even if the deal for the user is better. There is little to no incentive for companies to give up the huge economic benefit granted to them from having unrestricted access to user data.

If you found this in depth article about Solid web apps interesting, then go and check out the recorded clip of the live presentation from our Discovery Conference!