Hello World

The original hello.c program was first published in "THE C PROGRAMMING LANGUAGE" in 1978. The authors wrote in the "Getting Started" section:

The only way to learn a new programming language is by writing programs in it.

Similarly, we believe that in order to learn a new protocol you have to write specifications (schemas) and programs that build and parse the messages in the schemas. You may have access to an environment that generates the encoders and decoders for you. Otherwise, you will have to code them by hand.

A simple Blink schema:

Hello/1 ->
 string Greeting

A message in Blink tagged text format:

@Hello|Greeting=Hello world

In Blink Compact binary format:

0d 01 0b 48 65 6c 6c 6f 20 77 6f 72 6c 64

In Blink Compact binary format with comments:

0d                      // MsgSize: 13
01                      // MsgType: 1 (Hello)
0b                      // Greeting: 11 bytes
48 65 6c 6c 6f 20 77 6f // "Hello "
72 6c 64                // "world"

In Blink Native binary format:

1c 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 0b 48 65 6c 6c 6f 20 77 6f 72 6c 64

In Blink Native binary format with comments:

1c 00 00 00             // MsgSize: 28
01 00 00 00 00 00 00 00 // MsgType: 1 (Hello)
00 00 00 00             // Extension offset: N/A
00 00 00 00             // Greeting: offset
                        // -- data area --
0b                      // Greeting: 11 bytes
48 65 6c 6c 6f 20 77 6f // "Hello "
72 6c 64                // "world"

The schema annotated with a maximum size for the Greeting field:

Hello/1 ->
string (12) Greeting

In Blink Native binary format (with schema annotation):

19 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
0b 48 65 6c 6c 6f 20 77 6f 72 6c 64 00

In Blink Native binary format with comments:

19 00 00 00             // MsgSize: 25
01 00 00 00 00 00 00 00 // MsgType: 1 (Hello)
00 00 00 00             // Extension offset: N/A
0b                      // Greeting: 11 bytes
48 65 6c 6c 6f 20 77 6f // "Hello "
72 6c 64 00             // "world" (1 pad byte)

A Slightly Longer Example

A Blink schema may have a namespace specified.

namespace Tutorial

The following schema defines a few messages that allow a client to insert and cancel orders, and a server to acknowledge that an order has been inserted or canceled.

InsertOrder/1 ->
  string Symbol,
  string OrderId,
  u32 Price,
  u32 Quantity

CancelOrder/2 ->
  string OrderId

OrderInserted/3 ->
  string OrderId

OrderCanceled/4 ->
  string OrderId

This schema has all information a Blink codec needs to encode any of the four messages defined.

An InsertOrder message in Blink tagged text format:

@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000

In Blink Compact binary format:

0f 01 03 49 42 4d 06 41 42 43 31 32 33 7d a8 0f

In Blink Compact binary format with comments:

0f                      // MsgSize: 15 bytes
01                      // MsgType: 1 (InsertOrder)
03                      // Symbol:  3 bytes
49 42 4d                //          "IBM"
06                      // OrderId: 6 bytes
41 42 43 31 32 33       //          "ABC123"
7d                      // Price:   125
a8 0f                   // Quantity: 1000

An OrderInserted message in Blink tagged text format:

@Tutorial:OrderInserted|OrderId=ABC123

In Blink Compact binary format:

08 03 06 41 42 43 31 32 33

In Blink Compact binary format with comments:

08                      // MsgSize: 8 bytes
03                      // MsgType: 3 (OrderInserted)
06                      // OrderId: 6 bytes
41 42 43 31 32 33       //          "ABC123"

An InsertOrder With Overlong Price

Some applications use ILP (Integrated Layer Processing) to increase performance. For example: a response message may be created ahead of time and one or a few fields are then updated to match the exact scenario. In the case of an InsertOrder it could be used to send a hedge order to the market in response to a received execution message.

Overlong fields are legal in Blink Compact binary format so it is easy to prepare a message and just fill in one field to minimize the encoding overhead in the critical path.

The value 0 (zero) can be represented in an overlong form:

c4 00 00 00 00

Using the InsertOrder example above, a prepared encoding could be:

13 01 03 49 42 4d 06 41 42 43 31 32 33 c4 00 00 00 00 a8 0f

In Blink Compact binary format with comments:

13                      // MsgSize: 19 bytes
01                      // MsgType: 1 (InsertOrder)
03                      // Symbol:  3 bytes
49 42 4d                //          "IBM"
06                      // OrderId: 6 bytes
41 42 43 31 32 33       //          "ABC123"
c4 00 00 00 00          // Price:   0 (overlong)
a8 0f                   // Quantity: 1000

The application can then replace the overlong zero price with another overlong price at byte offset 13 instead of having to rebuild the entire message.

An InsertOrder With Optional Fields

Let's extend the schema definition of InsertOrder to include two optional fields at the end of the message.

InsertOrder/1 ->
  string Symbol,
  string OrderId,
  u32 Price,
  u32 Quantity,
  string ExecInst?,
  string ClearingAccount?

An InsertOrder message with absent optional fields:

@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000

In Blink Compact binary format:

0f 01 03 49 42 4d 06 41 42 43 31 32 33 7d a8 0f c0 c0

In Blink Compact binary format with comments:

11                      // MsgSize: 17 bytes
01                      // MsgType: 1 (InsertOrder)
03                      // Symbol:  3 bytes
49 42 4d                //          "IBM"
06                      // OrderId: 6 bytes
41 42 43 31 32 33       //          "ABC123"
7d                      // Price:   125
a8 0f                   // Quantity: 1000
c0                      // ExecInst: NULL (absent)
c0                      // ClearingAccount: NULL (absent)

The Blink encoding rules allow trailing absent optional fields to be optimized away.

In Blink Compact binary format:

0f 01 03 49 42 4d 06 41 42 43 31 32 33 7d a8 0f

In blink Compact binary format with comments:

0f                      // MsgSize: 15 bytes
01                      // MsgType: 1 (InsertOrder)
03                      // Symbol:  3 bytes
49 42 4d                //          "IBM"
06                      // OrderId: 6 bytes
41 42 43 31 32 33       //          "ABC123"
7d                      // Price:   125
a8 0f                   // Quantity: 1000

If you compare this InsertOrder example with the first example given further above, you will find that the two absent fields at the end of the message do not result in any extra overhead.

Absent optional fields that are not located at the end of a message will obviously have to be encoded with a NULL.

A Schema Exchange Example

Blink schemas can be encoded in Blink format. The Blink schema for Blink schemas is defined in the Blink Schema Exchange Specification, and can also be downloaded separately here.

Given the schema for InsertOrder:

InsertOrder/1 ->
  string Symbol,
  string OrderId,
  u32 Price,
  u32 Quantity

The schema in Blink Compact binary format:

4f                      // Msg size=79
81 fa                   // Msg Blink:GroupDef, id=16001
c0                      // Annotations: NULL
                        // Name: ->
c0                      //   Ns: NULL
0b                      //   Name: 11 bytes
49 6e 73 65 72 74 4f 72 //         "InsertOrder"
64 65 72                // ...
01                      // Id: 1
04                      // Fields: [4]
                        //   0 ->
c0                      //     Annotations: NULL
06                      //     Name: 6 bytes
53 79 6d 62 6f 6c       //           "Symbol"
c0                      //     Id: NULL
                        //     Type: ->
04                      //       DynGrp size=4
86 fa                   //       DynGrp Blink:String, id=16006
c0                      //       Annotations: NULL
c0                      //       MaxSize: NULL
00                      //     Optional: false
                        //   1 ->
c0                      //     Annotations: NULL
07                      //     Name: 7 bytes
4f 72 64 65 72 49 64    //           "OrderId"    
c0                      //     Id: NULL
                        //     Type: ->
04                      //       DynGrp size=4
86 fa                   //       DynGrp Blink:String, id=16006
c0                      //       Annotations: NULL
c0                      //       MaxSize: NULL
00                      //     Optional: false
                        //   2 ->
c0                      //     Annotations: NULL
05                      //     Name: 5 bytes
50 72 69 63 65          //           "Price"
c0                      //     Id: NULL
                        //     Type: ->
03                      //       DynGrp size=3
8e fa                   //       DynGrp Blink:U32, id=16014
c0                      //       Annotations: NULL
00                      //     Optional: false
                        //   3 ->
c0                      //     Annotations: NULL
08                      //     Name: 8 bytes
51 75 61 6e 74 69 74 79 //           "Quantity"
c0                      //     Id: NULL
                        //     Type: ->
03                      //       DynGrp size=3
8e fa                   //       DynGrp Blink:U32, id=16014
c0                      //       Annotations: NULL
00                      //     Optional: false

An InsertOrder encoded in Blink Native format

We can reuse the the schema definition of InsertOrder without any additional annotations.

InsertOrder/1 ->
  string Symbol,
  string OrderId,
  u32 Price,
  u32 Quantity,
  string ExecInst?,
  string ClearingAccount?

An InsertOrder message in Blink tagged text format:

@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000

In Blink Native binary format:

3b 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
18 00 00 00 19 00 00 00 7d 00 00 00 e8 03 00 00
00 00 00 00 00 00 00 00 00 00 03 00 00 00 49 42
4d 06 00 00 00 41 42 43 31 32 33

In Blink Native binary format with comments:

3b 00 00 00             // MsgSize: 59 bytes
01 00 00 00 00 00 00 00 // MsgType: 1 (InsertOrder)
00 00 00 00             // Extension offset: N/A
1a 00 00 00             // Symbol offset: 26
1d 00 00 00             // OrderId offset: 29
7d 00 00 00             // Price: 125
e8 03 00 00             // Quantity: 1000
00                      // ExecInst: absent
00 00 00 00             // ExecInst offset: N/A
00                      // ClearingAccount: absent
00 00 00 00             // ClearingAccount offset: N/A
                        // -- data area --
03 00 00 00             // Symbol:  3 bytes
49 42 4d                //          "IBM"
06 00 00 00             // OrderId: 6 bytes
41 42 43 31 32 33       //          "ABC123"

Another InsertOrder encoded in Blink Native format

We can specify a fixed capacity in the schema for strings in InsertOrder.

InsertOrder/1 ->
  string (8) Symbol,
  string (12) OrderId,
  u32 Price,
  u32 Quantity,
  string ExecInst?,
  string ClearingAccount?

An InsertOrder message in Blink tagged text format:

@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000

In Blink Native binary format:

38 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
03 49 42 4d 00 00 00 00 00 06 41 42 43 31 32 33
00 00 00 00 00 00 7d 00 00 00 e8 03 00 00 00 00
00 00 00 00 00 00 00 00

In Blink Native binary format with comments:

38 00 00 00             // MsgSize: 56 bytes
01 00 00 00 00 00 00 00 // MsgType: 1 (InsertOrder)
00 00 00 00             // Extension offset: N/A
03                      // Symbol:  3 bytes
49 42 4d 00 00 00 00 00 //          "IBM"
06                      // OrderId: 6 bytes
41 42 43 31 32 33 00 00 //          "ABC123"
00 00 00 00             //          (6 pad bytes)
7d 00 00 00             // Price: 125
e8 03 00 00             // Quantity: 1000
00                      // ExecInst: absent
00 00 00 00             // ExecInst offset: N/A
00                      // ClearingAccount: absent
00 00 00 00             // ClearingAccount offset: N/A
                        // -- data area --