Effigy, a distributed social data layer

DRAFT

Payloads have different types, and ultimately reference larger blobs in the network.

Referencing other entries are to its hash, which contains a pointer to the identity and the synod. Both the identity and the synod can be referenced with the blob request mechanism. Once these are loaded, the system then uses the [identity,head] from the signalling server or connected clients to reconstruct the feed, whose entries are validated using the signatures.

References are not to the feed itself, but to an entry on the feed. So it's possible to find references to an entity, locate the last version of its feed, but not find the original entity on it. This would be in the scenario where a user deleted an entry for their feed, and all the subsequent items needed to be rewritten and republished. This will break references to that entity and all following entity so change the past with care.

Invalid feeds don't propogate

1. Feeds that are too large for the network to propagate. (e.g. you can post a link to a movie but not the movie itself)
2. Feeds that contain invalidly signed entries
3. Feeds that contain unencryted images with exif data
4. Feeds that have a revoked public key (i.e. last valid id was x everything after that is wrong)

Each node keeps track of identities and heads, which is the latest entry. When a node connects, it announces all of the identities that are tracked, and what latest head it has. Nodes can then share and request heads, and if they have the feed read keys they can trace things backwards to get to the root.

Every bit of data in the system is stored as a key pair, with the key being the cryptographic hash of the data. This blob could be an entities which contains metadata to describe itself; a signature and a link to a blob containing the public key signing it, and a link to a synod which is the account that is associated with the identity.

When a client connects, it notifies the other side which blobs it's interested in, and other other side says which blobs it has.

Signalling is a way to broadcast to other nodes the your latest [identity,head] pair and as a way to coordinate WebRTC connections. If two clients are connected over Websockets to the same singalling server they can exchange messages directly, which allows for offer and answer SDP messages to make initial contact with a node that doesn't allow incoming connections (i.e. web browsers) and then will be able to coordinate the connection through a STUN server.

Providing this service requires a DNS name and an externally accessible IP address, though probably not a huge amount of data (unless it's also caching and storing a lot of blobs). Additional directory and other services could be worth paying for.

Relaying moving data through this system if the two nodes can't connect directly. I'm envisioning having a coturn instance acting as a TURN server that understands the identity as authentication.

Providing this service requires bandwidth and an externally accessible IP, so it's logical to charge money for this.

This node receives messages for another user for situations where the recipient has no knowledge of the sender and therefore no reason to be tracking their identity. The next time the recipient connects to this signalling server, it will see the message.

Signalling services that the recipient has requested as a dropbox are published as part of their synod, so you'll need that in order to direct a message to them.

Data leasing is a quest for the node to store an identities data for at least a certain period of time, to make it accessible to the network when the original device is offline. This is similar to a pinning service in IPFS.

One scenario is that all of the identities in a synod provide mutual data leasing, which means that your data is backed up on all of your devices. You could lose everything less 1 and still be able to recreate the graph.

Another scenario is that you provide leasing to trusted friends, the sort of people you give spare house keys to, so that if either of you have a catastrophic failure you can reconstruct everything. (Some provision for recovering lost synod private keys would need to be thought out.)

A third example is providing data leasing as a service, which would be something that you could charge for.

scale 800 width
clienta->clientb: wantservice(sessiontoken)
clientb->clienta: providesservice(sessiontoken)
clienta->clientb: call(service,datalease,head)

If both identities are mutually trusted and on a permitted contact list then they can use the WebRTC mechanisms to have real time communication. I'm not sure how well push notifications really work over progressive web apps, but seems worth exploring since we already are connecting over WebRTC.

When a client starts up, it connects to the clients it knows about and is able to reach. It first announces the service that it provides to the network, in the case it has a list of identity heads and can store blobs.

scale 800 width

client1 -> client2 : sessionrequest[identity1,head1,challenge1]
client2 -> client1 : sessionrequest[identity2,head2,challenge2]
client1 -> client2 : sessionauth[signed(identity1,challenge2)]
client2 -> client1 : sessionauth[signed(identity2,challenge1)]

If either of the signatures don't match, then the session is considered unauthenticated.

If the signature matches, then the head announced is considered to be the head of the respective identity. Note that it not a requirement that a head tracking service downloads and verifies the head

After authentication the clients exchange the list of services they are willing to provide to the other.

scale 800 width

client1 -> client2 : provides[service_list]
client2 -> client1 : provides[service_list]

TODO There needs to be a way for the client to request access to an additional service.

Here one side sends a get_heads request for its identity watch list. The other returns with a list of [identity,head] pairs for identities that it knows about.

scale 800 width

client1 -> client2 : getHeads([identities])
client2 -> client1 : lastKnownHeads([identities,heads])

Signalling is tracking presence and helping to coordinate connections between devices (i.e. start of the WebRTC handshake).

scale 800 width

client -> signalserver : online
signalserver -> otherclients : online(identity,head)
signalserver -> client : presence_list(identities)
client -> signalserver : <disconnect>
signalserver -> otherclients : offline(identity)

scale 800 width

client1 -> signalserver : message_to(identity2,payload)
signalserver -> client2 : message_from(identity1,payload)

Clients periodically send iwant lists to each other, returning the blobs that it has or is willing to share (based on perhaps bandwidth, if the client is operating on a battery, or over a metered celluar connection).

TODO Clients should track incoming blob requests and outgoing blog bandwidth to create a leech ratio that takes into account sharing reciprocity. Credit is created by sharing more or perhaps by purchasing bandwidth from the remote server. The request is signed by the identity so the clients can tell who is asking for what.

scale 800 width

clienta->clientb: iwant(blobs)
clientb->clienta: ihave(blobs)
clienta->clientb: getblob(blob,maxsize)
clientb->clienta: blob

Once the client is connected to a system, signal or client, it sends a list of the blobs that it wants. Since these are content addressable and signed by the identity, it doesn't matter where they come from.

A signal server may or may not have blobs – it's a regular client that presumably is free of firewall and NAT messiness, and had the additional feature of being able to relay requests.

The client looks through its list of identities, and all of the head announcements that it's received. For each of these it tries to get the blob associated with the head. Note that synod are also identities so fall into this logic as well.

scale 800 width

(*) --> "Receive identity,head message"

If "tracking identity" then
  -->[Yes] "get blob"
else
  -->[No] "ignore"
Endif

"get blob" If "has blob" then
  -->[Yes] If "decrypt blob" then
    -->[get parent] "get blob"
  else
    -->[No] "ignore"
  Endif

else
  ->[No] "add to iwantlist"
Endif

--> (*)