If I look here: https://docs.python.org/2/library/struct.html at section 7.3.2.2. Format Characters, there are no Format letter for a Python Type Integer with standard size 6. I have tried '6p' or even '3H' but that doesn't work for instance in:
import struct
struct.calcsize('!6p')
Sorry if this sounds dumb... I am learning how to convert strings as packed binary data for the first time for a string message that has a length of 6 with integer as a value. So what is the proper letter format to get a standard size 6 for an integer?
UPDATE
My task is to convert the binary messages contained in NASDAQ TotalVIEW-ITCH 5.0 specification document, here
So for instance, if you look at Section 4.1: System Event Message (Page 3), I would convert the message of type 'S' as follow:
class SystemEventMessage(Message):
type = 'S'
description = "System Event Message"
message_size = struct.calcsize("!HH6pc") + 1
def __init__(self, message):
(self.locate,self.tracking,self.timestamp, self.code) = struct.unpack("!HH6pc", message[1:])
def pack(self):
return struct.pack("!cHH6pc", self.type,self.locate,self.tracking, self.timestamp, self.code)
I get this error:
struct.error: unpack requires a string argument of length 11
So I am assuming my error has to do with the '6p' since I can't find any integer with standard size 6.
UPDATE 2
So I use the below suggestion of using 6s rather than 6p, for the following message type "I" in the documentation Section 4.6: Net Order Imbalance Indicator (NOII) Message, I do:
class NoiiMessage(ITCH41MarketMessage):
type = 'I'
description = "NOII Message"
message_size = struct.calcsize("!HH6sQQc8sIIIcc") + 1
def __init__(self, message):
(self.locate,self.tracking,self.timestamp, self.pairedShares, self.imbalance,
self.imbalanceDirection, self.stock, self.farPrice, self.nearPrice,
self.currentRefPrice, self.crossType, self.priceVariationbsindicator
) = struct.unpack("!HH6sQQc8sIIIcc", message[1:])
def pack(self):
return struct.pack("!cHH6sQQc8sIIIcc", self.type,self.locate,
self.tracking, self.timestamp,
self.pairedShares, self.imbalance,
self.imbalanceDirection, self.stock,
self.farPrice, self.nearPrice,
self.currentRefPrice, self.crossType,
self.priceVariationbsindicator)
I get this error:
struct.error: unpack requires a string argument of length 49
Now that's strange since !HH6sQQc8sIIIcc is of length 49...
Everyone thank you for the help!
NASDAQ TotalView-ITCH 5.0
[S0099123456Q]________________________________ wireline SEQ _____________
| | | ||
| | | |+---------------[[ Event Code ]]
| | | +----------------[[ Timestamp ns ]]
| | +----------------------[[ Tracking NUM ]]
| +------------------------[[ Stock Locate ]]
+--------------------------[[ Message Type ]]
Name | Offset | Length | Value | Notes
---------------|--------|--------|---------|-------------------------------------------------------------
Message Type | 0 | 1 | “S” | System Event Message.
Stock Locate | 1 | 2 | Integer | == 0 Always
Tracking Number| 3 | 2 | Integer | NASDAQ OMX internal tracking number
Timestamp | 5 | 6 | Integer | Nanoseconds since midnight.
Event Code |11 | 1 | Alpha | == { 0 | S | Q | M | E | C } See System Event Codes below.
In order to get NASDAQ_Timestamp,
grab 6-uchar-s using 6B in the mask or char[] 6s
and postprocess them into int() only if needed on the consumer side
which defers latency and unblocks destreaming engine wireline performance.
+ also enjoy the Python 7.3.2.1. notes on BigEndian / network ordering once using "!" mask-prefix
>>> struct.pack( ">c2H6sc", "S", 0, 99, "123456", "Q" )
'S\x00\x00\x00c123456Q'
>>> struct.unpack( ">c2H6sc", "S\x00\x00\x00c123456Q" )
('S', 0, 99, '123456', 'Q')
| | | | |
| | | | +---------------[[ Event Code ]]
| | | +--------------------[[ Timestamp ns ]]
| | +-----------------------------[[ Tracking NUM ]]
| +---------------------------------[[ Stock Locate ]]
+-------------------------------------[[ Message Type ]]
"!HH6sQQc8sIIIcc" _
+||-||||-||||||----------------------------1 |_|_ "!" a "Network"-Byte-order
+|-||||-||||||----------------------------2 |_| H as a 2 Byte unsigned short
| |||| |||||| 3 |_|_
+-||||-||||||----------------------------4-|_| H as a 2 Byte unsigned short
|||| |||||| 5 |_|_
+|||-||||||----------------------------6-|_| 6s as a 6 Byte char[]
||| |||||| 7 |_|
||| |||||| 8 |_|
||| |||||| 9 |_|
||| |||||| 10 |_|
||| |||||| 1 |_|_
+||-||||||--------------------------- 2-|_| Q as a 8 Byte unsigned long long
|| |||||| 3 |_|
|| |||||| 4 |_|
|| |||||| 5 |_|
|| |||||| 6 |_|
|| |||||| 7 |_|
|| |||||| 8 |_|
|| |||||| 9 |_|_
+|-||||||---------------------------20-|_| Q as a 8 Byte unsigned long long
| |||||| 1 |_|
| |||||| 2 |_|
| |||||| 3 |_|
| |||||| 4 |_|
| |||||| 5 |_|
| |||||| 6 |_|
| |||||| 7 |_|_
+-||||||----------------------------8-|_|_ c as a 1 Byte char
+|||||----------------------------9-|_| 8s as a 8 Byte char[]
||||| 30 |_|
||||| 1 |_|
||||| 2 |_|
||||| 3 |_|
||||| 4 |_|
||||| 5 |_|
||||| 6 |_|_
+||||----------------------------7-|_| I as a 4 Byte unsigned int
|||| 8 |_|
|||| 9 |_|
|||| 40 |_|_
+|||----------------------------1-|_| I as a 4 Byte unsigned int
||| 2 |_|
||| 3 |_|
||| 4 |_|_
+||----------------------------5-|_| I as a 4 Byte unsigned int
|| 6 |_|
|| 7 |_|
|| 8 |_|_
+|----------------------------9-|_|_ c as a 1 Byte char
+---------------------------50-|_|_ c as a 1 Byte char
Format | C-type | Python-type | Standard size
========|======================|====================|===============
x | pad byte | no value |
c | char | string of length 1 | 1
b | signed char | integer | 1
B | unsigned char | integer | 1
? | _Bool | bool | 1
h | short | integer | 2
H | unsigned short | integer | 2
i | int | integer | 4
I | unsigned int | integer | 4
l | long | integer | 4
L | unsigned long | integer | 4
q | long long | integer | 8
Q | unsigned long long | integer | 8
f | float | float | 4
d | double | float | 8
s | char[] | string |
p | char[] | string |
P | void * | integer |