Playing with cabal

Cabal is a “experimental p2p community chat platform”. It’s fully distributed over the dat protocol. When you create a new chat area – something like a Slack – it allows anyone with the same key to post and view messages everywhere. When you post a message, everyone gets it and shares it with everyone else, so even when your computer drops off there will still be a coherent view of the data.

/Updated 2020-07-14 for cabal-core 5.0/

Local computer

You’ll need at least two computers to really play with this. But we can simulate the situation by running a docker container as well as a local instance of the cabal cli client. We can then move that container (or at least the image) around to run it on a remote server if you don’t want to mess with anything on the remote machine. Lets first install it locally:

On the terminal

If you just want to play around you can use npx, or install the cabal cli command using npm install --global cabal. Either way, lets start up a new cabal chat.

1
npx cabal --new

Take a note of the cabal URI on the top right of the screen, we’ll use that to connect with later. Keep this window running and start up a new Terminal window. Type in a test message such as “this is the first message” (or something less banal). There are also slash commands available, /help will list them. Change your display nickname with /n local node or something like that.

Dockerizing

This is optional, you can just repeat the same steps on the remote node if you want. I have this in a Docker container because I don’t want to install node on my remote server, but if you have you computers and aren’t anal about it its super simple to just install node and just run npx cabal on them also.

I like keeping things in Docker containers when playing around so that no unnecessary packages get installed on the machine and the data files are nicely cleaned up when I exit out.

This is a very basic node-based Dockerfile. Cabal/Dat uses port 3282 to communicate, so we’ll need to expose and open that.

Dockerfile

1
2
3
4
5
6
7
8
9
FROM node:11
MAINTAINER Will Schenk <wschenk@gmail.com>

EXPOSE 3282

USER root
RUN npm install -g cabal

CMD cabal

Then build like so:

1
docker build . -t $USER/cabal

Create a simple script to make it easier to run in the future. (Note I’m using --network host here because I was having trouble getting dat to punch through the Docker networking/NAT stuff.)

1
2
3
#!/bin/bash

docker run -it --rm --name cabal --network host $USER/cabal cabal "$@"

This script is for running one off commands, and it throws away the container each time so you’ll lose the history. If you really wanted to use this for something more than an experiment you should mount the internal /root/.cabal directory to a docker volume and/or not make it a temporary container.

Push the container to the repository using docker push $USER/cabal, and then move everything to a different computer. Then run:

1
bash run.sh cabal://f6e83732c84fe310515fe7162376e5420c6f4a27d015094ccc22be658b62d3c8

If all goes well, after a few moments of network dancing you should be able to see the original message you sent. Type away to see if things show up!

Using `cabal-core` in code

That’s super cool already, but lets go a little deep and start writing our own handlers to see what we can do with cabal programmaticly. cabal isn’t secure at all so we probably shouldn’t use this for anything real, but its a nifty way to communicate and discover without having to setup very much at all!

1
2
npm init
npm i add cabal-core tmp delay

Printing out everything that is in a swarm

First we create a Cabal instance with a storage directory and our key, using cabel = Cabal(directoy,key)
Then we connect it to the swarm to start replicating using swarm(cabal)
Once the local database has been loaded and processed, the cabal.ready() callback is called. Events received after this are new.
We can watch peers join and disconnect using cabal.on('peer-added') and cabal.on('peer-dropped')
We can listen for new messages and topics using cabal.messages.events.on('message')
(We can also listen for messages just in a specific channel, not shown below)
We can listen for new channels cabal.channels.events.on( 'add' )
We can get a current channel list using cabal.channels.get()
We can listen for new nicknames using cabal.users.events.on('update')

Here’s it in code form dump.js:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
const Cabal = require('cabal-core');
const swarm = require('cabal-core/swarm.js');
const tmp = require('tmp');

if( process.argv.length != 3 ) {
  console.log( "Usage" )
  console.log( "node dump.js cabalkey")
  process.exit(1)
}

// Strip out the awesome
const key =  process.argv[2].replace('cabal://', '').replace('cbl://', '').replace('dat://', '').replace(/\//g, '')
console.log( "Connecting to", key)

// For testing to keep things clean
const storage_dir = process.env.CABAL_DIR || tmp.dirSync().name;
console.log( "Storing our database in", storage_dir);

// Create a cabal instance
const cabal = Cabal( storage_dir, key, {maxFeeds:1000} )

console.log( "Connecting to swarm")
swarm(cabal);

// This is called after the local database is loaded
cabal.ready( () => {console.log( "Database is ready!")})

// Print out connect and disconnect messages
cabal.on('peer-added', (key) => console.log( "Peer added",  key ) );
cabal.on('peer-dropped', (key) => console.log( "Peer dropped", key) );

// Prints messages as they come in
cabal.messages.events.on('message', (message) => console.log("Got a message", message))

// Print user messages as they come in
cabal.users.events.on('update', (key, message) => console.log( "user message", message ))

// Print out channel list
cabal.channels.get( (err,channels) => console.log( "Channels", channels))
cabal.channels.events.on( 'add', (channel) => console.log( "new channel", channel))

// Process will hang waiting for events

Now lets restart everything and see what happens! When I start up node dump.js key on my machine, and then start up the cabal-cli client on the other machine and:

Type hello everyone
Type /n Will - change my nickname to Will
Type This is my new name
Type /join new channel
Type lets talk about this instead
Type /topic Serious Stuff
Type /quit

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
Connecting to f6e83732c84fe310515fe7162376e5420c6f4a27d015094ccc22be658b62d3c8
Storing our database in /tmp/tmp-5029E7feUNe8UiyB
Connecting to swarm
Database is ready!
Channels []
Peer added 6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097
new channel default
Got a message { key:
   '6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097',
  seq: 0,
  value:
   { type: 'chat/text',
     content: { channel: 'default', text: 'hello everyone' },
     timestamp: 1556023954404 } }
user message { key:
   '6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097',
  seq: 1,
  value:
   { type: 'about',
     content: { name: 'Will' },
     timestamp: 1556023961901 } }
Got a message { key:
   '6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097',
  seq: 2,
  value:
   { type: 'chat/text',
     content: { channel: 'default', text: 'This is my new name' },
     timestamp: 1556023967911 } }
new channel new channel
Got a message { key:
   '6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097',
  seq: 3,
  value:
   { type: 'chat/text',
     content:
      { channel: 'new channel', text: 'lets talk about this instead' },
     timestamp: 1556023978198 } }
Got a message { key:
   '6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097',
  seq: 4,
  value:
   { type: 'chat/topic',
     content: { channel: 'new channel', text: 'Serious Stuff' },
     timestamp: 1556023983080 } }
Peer dropped 6a4ed8b325acf3cb4a7c47625d0e66610ebc9be75b775e934efdb5440bb09097

Publishing a message

Publishing a message involves appending to your local feed and then synchronizing to the swarm. Which means waiting for the swarm to connect, a peer to join, and then them downloading everything. We don’t really know how long it will take, and there’s no real way to guarantee that it will show up. But we want a CLI tool that will eventually return, and not sit around forever. Here’s the logic we’ll use:

Initialize cabal and try to connect to the swarm
Post the message to the local store
Once we get a peer connect message, wait 2 seconds for syncing and return success
If we don’t get a peer connect in 10 seconds, then exit with error code.

Of course, if you just want to publish a message and serve the swarm forever, we could just take out the timeouts.

publish.js:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
const Cabal = require('cabal-core');
const swarm = require('cabal-core/swarm.js');
const delay = require('delay');
const tmp = require('tmp');

if( process.argv.length != 5 ) {
  console.log( "Usage" )
  console.log( "node publish.js cabalkey channel message")
  process.exit(1)
}

// Strip out the awesome
const key =  process.argv[2].replace('cabal://', '').replace('cbl://', '').replace('dat://', '').replace(/\//g, '')
console.log( "Connecting to", key)
const channel = process.argv[3]
const message = process.argv[4]

// For testing to keep things clean
const storage_dir = process.env.CABAL_DIR || tmp.dirSync().name;
console.log( "Storing our database in", storage_dir);

// Create a cabal instance
const cabal = Cabal( storage_dir, key, {maxFeeds:1000} )

console.log( "Connecting to swarm")
swarm(cabal);

cabal.on('peer-added', (key) => {
  console.log( "Peer added",  key );
  delay( 2000 ).then( () => {
    console.log( "Quitting 2 seconds after peer connect" )
    process.exit(0)
  })
})

let messageStored = false
function publishCallback() {
  messageStored = true;
  console.log( "Message stored locally" )
}

// This is called after the local database is loaded
cabal.ready( () => {
  console.log( "Database is ready!")

  cabal.publish( {type: "chat/text", content: { text: message, channel}}, publishCallback );
})

delay( 10000 ).then( () => {
  if( messageStored ) {
    console.log( "10 second time before swarm connect, stored locally");
    process.exit(1)
  } else {
    console.log( "10 second timeout before message was stored");
    process.exit(1);
  }
})

Then try running this code with a running cabal node on the remote host and without one. You’ll notice that the messages that were stored locally will eventually be delivered once a connection is made.

Next steps

One thing that I’m interested in playing around with is using this as a message bus of sorts, to each have remote devices report in on what they are doing (raspberry pi’s for example) or be able to script it, to have a bot listening that takes commands in and run actions on the server and then pushes the results out (to IPFS for example.) But that’s for another post.

References