flextag-protocol

General websocket protocol using flextags

The idea here is that client and server software have a conversation using natural language statements which the other end might be expecting (aka flextags):

• Client: Please tell me the sum of 3 and 22
• Server: Sure, the sum of 3 and 22 is 25.
• Client: What's your CPU temperature
• Server: My CPU temperature is 59 C

(This is example3 in the Repo.)

Why would we want to do this?

1. Dogfood. Testing out flextags as a data serialization. Of course you could use JSON for this, but do flextags really work okay like this, too? Let's find the limits.

2. Documentation. Websocket code can get pretty confusing, trying to figure out the semantics of each of the parameters in the messages sent each direction. With flextags, the documentation tends to be pretty close to the code. The human-conversation metaphor may also help think through how protocols should work.

3. Modularity. Parts of the client and parts of the server can chat without interfering with each, since they wont match each other's text (assuming good unambiguous flextag design).

4. Versioning. Servers can have modules answering older versions of the protocol in parallel with new ones. Messages are routed based on text matching, so as long as the patterns are disjoint it should be fine. (todo: warn if overlapping patterns are ever declared.)

Example

Here is example 4, which is the sums part of example 3 :

const { startServer } = require('flextag-protocol')
 
function sums (conn) {
  conn.preSend('Sure, the sum of ?x and ?y is ?sum')
  conn.onMatch('Please tell me the sum of ?x and ?y', b => {
    const x = parseFloat(b.x)
    const y = parseFloat(b.y)
    const sum = x + y
    conn.send({x, y, sum})
  })
}
 
startServer({talkers: [sums]})

We can talk to the server from the command line using a tool like wscat:

$ wscat -c ws://localhost:8080
connected (press CTRL+C to quit)
> What's your CPU temp
< My CPU temperature is 60 C
>

Or use the example client:

const { startClient } = require('flextag-protocol')
 
function sumCheck (conn) {
  conn.send('Please tell me the sum of ?a and ?b', {a: 3, b: 22})
  conn.onMatch('Sure, the sum of ?a and ?b is ?c', ({a,b,c}) => {
      console.log(`Server says ${a} + ${b} = ${c}`)
      conn.mgr.stop() // conn.mgr is the client object; this will end process
      console.log('Stopping')
  })
}
 
startClient(process.env.WSADDR, [sumCheck])

which we might run like

$ DEBUG=flextag-protocol WSADDR=ws://localhost:8080 node example4-client

Usage

Start by making one or more "talker" functions. They are passed a Connection object (called "conn" above) whenever a new connection becomes active. This can happen many times on the server as different clients connect. It can happen may times on the client, if the connection is lost and re-established, but there should never be more than one at once (per call to startClient). startClient will keep trying to establish and re-establish a connection until stop() is called.

Methods and properties of the Connection:

• conn.close() closes the connection. Note that Client code attempts to re-open a connection whenever it is closed, so you probably want client.stop() instead.

• conn.mgr is the Client or Server object created, depending which side you're on. Needed for some operations like conn.mgr.stop(); see client.stop() and server.stop() below.

• conn.log is like console.log but it logs to debug() and it distinguish which connection is making this output

• conn.send(string) just sends the string down the websocket pipe as the next message.

• conn.send(string, obj) uses flextag-mapper.unparse to fill in the variables in the string with values from obj

• conn.send(obj) uses flextag-mapper.unglueWalk to send all the flextags necessary to describe the obj, if possible. This can handle cyclical connected graphs of objects, if sufficient output patterns have been declared. TODO: some way to use flextag-mapper.glue at the receiving end of this.

• conn.preSend(pattern) declare another output pattern which later calls to conn.send(obj) can use. The set of patterns is shared among all the talkers on that connection, but not between connections.

• conn.onMatch(pattern, callback) call the callback when a statement is received matching the pattern. Any variables in the pattern will be bound as properties of the bindings argument passed to the callback.

• conn.hijack = f, set a function to handle just the next message coming across the websocket stream. Useful for handling binary messages by prefixing them with a statement about how to handle them.

• conn.on('close', cb) call cb when the connection closes, from either end, giving you a chance to clean up any attached resources.

For startServer(options), the options object is passed on to appmgr for things like the port number. We just use the "talkers" field, which should be an array of talker functions to call on each new connection being establish. The call returns the Promise of the Server object (an AppMgr), which is also passed to talker as conn.mgr. On the Server object:

• server.stop() shut down the server, closing any websocket connections. Should allow the process to exit, or a new server to be started on the same port when it resolves.

• server.app the express instance we're using. You can set up your own routes if you want the http web server to be doing something. We pre-define one route, /flextag-protocol as a simple web form giving RESTful access to the talkers.

• server.siteurl is like "http://example.org". This is correctly set to include the port even if the port is dyanmically assigned via PORT=0, eg http://localhost:37597

• server.sitewsurl is like "ws://example.org". This is correctly set to include the port even if the port is dyanmically assigned via PORT=0.

The function startClient(wsaddr, talkers) resolves to the Client object, which is also available as conn.mgr as passed to talkers.

• client.stop() shut down the connection and the connection's watchdog, which otherwise attempts to restart the connection when it closes. This should allow the process to exit.