Codec

From Teltonika Telematics Wiki

Introduction

A codec is a device or computer program for encoding or decoding a digital data stream or signal. Codec is a portmanteau of coder-decoder. A codec encodes a data stream or a signal for transmission and storage, possibly in encrypted form, and the decoder function reverses the encoding for playback or editing.


Codec ID table

Codec 8 Codec 8 extended Codec 12 Codec 13 Codec 16
0x08 0x8E 0x0C 0x0D 0x10

Codec 8

  • AVL data packet

Because the smallest information amount that can be written is one bit, there can be some bits left unused when result is byte array. Any unused bits should be left blank.

Below table represents AVL data packet structure.


4 zeros Data field length Codec ID Number of Data 1 AVL Data Number of Data 2 CRC-16
4 bytes 4 bytes 1 byte 1 byte 30-147 bytes 1 byte 4 bytes

Number of data – number of encoded data (number of records). Codec ID is constant 08.

Data field length is the length of bytes [codec id, number of data 2]. Number of data 1 should always be equal to number of data 2 byte. CRC-16 is 4 bytes, but first two are zeroes and last two are CRC-16 calculated for [codec id, number of data 2] Minimum AVL packet size is 45 bytes (all IO elements disabled). Maximum AVL packet size for one record is 783 bytes.


  • AVL data
Timestamp Priority GPS Element IO element
8 bytes 1 byte 15 bytes 6-123


Timestamp – difference, in milliseconds, between the current time and midnight, January 1, 1970 UTC

  • Priority


0 Low
1 High
2 Panic


  • GPS Element


Longtitude Latitude Alitude Angle Stellites Speed
4 bytes 4 bytes 2 bytes 2 bytes 1 bytes 2 bytes

X Longitude

Y Latitude1

Altitude In meters above sea level1

Angle In degrees, 0 is north, increasing clock-wise 1

Satellites Number of visible satellites1

Speed Speed in km/h. 0x0000 if GPS data is invalid1

Longitude and latitude are integer values built from degrees, minutes, seconds and milliseconds by formula

d - Degrees

m - Minutes

s - Seconds

ms - Milliseconds

p - Precision (10000000)

If longitude is in west or latitude in south, multiply result by –1. To determine if the coordinate is negative, convert it to binary format and check the very first bit. If it is 0, coordinate is positive, if it is 1, coordinate is negative.

Example:

Received value: 20 9c ca 80 Converted to BIN: 00100000 10011100 11001010 10000000 first bit is 0, which means coordinate is positive Convered to DEC: 547146368 For more information see two‘s compliment arithmetics.


  • IO Element
Event IO ID 1 byte
N of Total IO 1 byte
N1 of One Byte IO 1 byte
1’st IO ID 1 byte
1’st IO Value 1 byte
...
N1’th IO ID 1 byte
N1’th IO Value 1 byte
N2 of Two Bytes 1 byte
1’st IO ID 1 byte
1’st IO Value 2 bytes
...
N2’th IO ID 1 byte
N2’th IO Value 2 bytes
N4 of Four Bytes 1 byte
1’st IO ID 1 byte
1’st IO Value 4 byte
...
N4’th IO ID 1 byte
N4’th IO Value 4 bytes
N8 of Eight Bytes 1 byte
1’st IO ID 1 byte
1’st IO Value 8 bytes
...
N8’th IO ID 1 byte
N8’th IO Value 8 bytes

Event IO ID – if data is acquired on event – this field defines which IO property has changed and generated an event. If data cause is not event – the value is 0.

N total number of properties coming with record (N=N1+N2+N4+N8)

N1 number of properties, which length is 1 byte

N2 number of properties, which length is 2 bytes

N4 number of properties, which length is 4 bytes

N8 number of properties, which length is 8 bytes


Example

Received data:

Treciad.png

00000000 4 zeroes, 4 bytes

0000008c data length, 4 bytes

08 – Codec ID

0- Number of Data (1 record)

1’st record data

0000013feb55ff74 – Timestamp in milliseconds (1374042849140)

GMT: Wed, 17 Jul 2013 06:34:09 GMT

00 – Priority

GPS Element

0f0ea850 – Longitude 252618832 = 25,2618832º N

209a6900 – Latitude 546990336 = 54,6990336 º E

0094 – Altitude 148 meters

0 – Angle 214º

0 – 12 Visible sattelites

0 – 0 km/h speed

IO Element

00 – IO element ID of Event generated (in this case when 00 – data generated not on event)

1e – 30 IO elements in record (total)

09 – 9 IO elements, which length is 1 Byte

0 – IO element ID = 01

0 – IO element’s value = 0

02 – IO element ID = 02

0 – IO element’s value = 0

03 – IO element ID = 03

0 – IO element’s value = 0

4 – IO element ID = 04

0 – IO element’s value = 0

16 – IO element ID = 22 (dec)

0 – IO element’s value = 1

47 – IO element ID = 71 (dec)

03 – IO element’s value = 3

F0 – IO element ID = 240 (dec)

0 – IO element’s value = 0

15 – IO element ID = 21 (dec)

04 – IO element’s value = 0

C8 – IO element ID = 200 (dec)

0 – IO element’s value = 0

0C – 12 IO elements, which value length is 2 Bytes

09 – IO element ID = 9 (dec)

0073 – IO element’s value

0a – IO element ID = 10 (dec)

0046 – IO element’s value

0b – IO element ID = 11 (dec)

0050 – IO element’s value

13 – IO element ID = 19 (dec)

0046 – IO element’s value

43 – IO element ID = 67 (dec)

06d7 – IO element’s value

1 – IO element ID = 68 (dec)

0 – IO element’s value

B5 – IO element ID = 181 (dec)

000b – IO element’s value

B6 – IO element ID = 182 (dec)

0007 – IO element’s value

42 – IO element ID = 66 (dec)

2e9f – IO element’s value

2 – IO element ID = 24 (dec)

0 – IO element’s value

cd – IO element ID = 205 (dec)

3 – IO element’s value

CE – IO element ID = 206 (dec)

0 – IO element’s value

07 – 7 IO elements, which value length is 4 Bytes

C7 – IO element ID = 199 (dec)

0 – IO element’s value

f1 – IO element ID = 241 (dec)

0000601a – IO element’s value

46 – IO element ID = 70 (dec)

00000134 – IO element’s value

48 – IO element ID = 72 (dec)

00000bb8 – IO element’s value

4 – IO element ID = 73 (dec)

00000bb8 – IO element’s value

4a – IO element ID = 74 (dec)

00000bb8 – IO element’s value

4c – IO element ID = 76 (dec)

1 – IO element’s value

02 – 2 IO elements, which value length is 8 Bytes

4e – IO element ID = 78 (dec)

0 – IO element’s value

cf – IO element ID = 207 (dec)

0 – IO element’s value

0 – Number of Data (1 record)

00003fca - CRC-16, 4 Bytes (first 2 are always zeros)

  • SENDING DATA OVER TCP/IP

First when module connects to server, module sends its IMEI. First comes short identifying number of bytes written and then goes IMEI as text (bytes).

For example IMEI 356307042441013 would be sent as 000f333536333037303432343431303133

First two bytes denote IMEI length. In this case 000F means, that imei is 15 bytes long.

After receiving IMEI, server should determine if it would accept data from this module. If yes server will reply to module 01 if not 00. Note that confirmation should be sent as binary packet. I.e. 1 byte 0x01 or 0x00.

Then module starts to send first AVL data packet. After server receives packet and parses it, server must report to module number of data received as integer (four bytes).

If sent data number and reported by server doesn’t match module resends sent data.


Example:

Module connects to server and sends IMEI:

000f333536333037303432343431303133

Server accepts the module:

01

Module sends data packet:

AVL data packet header AVL data array CRC
Four zero bytes,

‘AVL data array’ length – 254

CodecId – 08,

NumberOfData – 2. (Encoded using continuous bit stream.

Last byte padded to align to byte boundary)

CRC of ‘AVL data array’
00000000000000FE 0802...(data elements)...02 00008612

Server acknowledges data reception (2 data elements): 00000002


  • SENDING DATA OVER UDP/IP


  • UDP channel protocol

UDP channel is a transport layer protocol above UDP/IP to add reliability to plain UDP/IP using acknowledgment packets. The packet structure is as follows:

UDP datagram
UDP channel packet x N Example 2 bytes Packet length (excluding this field) in big ending byte order
Packet Id 2 bytes Packet id unique for this channel
Packet Type 1 byte Type of this packet
Packet payload m bytes Data payload


Packet Type
1 Data packet requiring acknowledgment

Acknowledgment packet should have the same packet id as acknowledged data packet and empty data payload. Acknowledgement should be sent in binary format

Acknowledgment packet
Packet length 2 bytes 0x0003
Packet id 2 bytes same as in acknowledged packet
Packet type 1 byte 0x02


  • Sending AVL data using UDP channel

AVL data are sent encapsulated in UDP channel packets (Data payload field).

AVL data encapsulated in UDP channel packet
AVL packet id (1 byte) Module IMEI AVL data array

AVL packet id (1 byte) – id identifying this AVL packet Module IMEI – IMEI of a sending module encoded the same as with TCP AVL data array – array of encoded AVL data


Server response to AVL data packet
AVL packet id (1 byte) Number of accepted AVL elements (1 byte)


AVL packet id (1 byte) – id of received AVL data packet

AVL packet id (1 byte) – id of received AVL data packet

Number of AVL data elements accepted (1 byte) – number of AVL data array entries from the beginning of array, which were accepted by the server.

Scenario:

Module sends UDP channel packet with encapsulated AVL data packet (Packet type=1 or 0). If packet type is 0, server should respond with valid UDP channel acknowledgment packet. Since server should respond to the AVL data packet, UDP channel acknowledgment is not necessary in this scenario, so Packet type=1 is recommended.

Server sends UDP channel packet with encapsulated response (Packet type=1 – this packet should not require acknowledgment)

Module validates AVL packet id and Number of accepted AVL elements. If server response with valid AVL packet id is not received within configured timeout, module can retry sending.


Example: Module sends the data:

UDP channel header AVL packet header AVL data array
Len – 253,

Id – 0xCAFE, Packet type – 01 (without ACK)

AVL packet id – 0xDD,

IMEI – 1234567890123456

CodecId – 08,

NumberOfData – 2. (Encoded using continuous bit stream)

00FDCAFE01 DD000F3133343536373839303132333435 0802…(data elements)…02


Server must respond with acknowledgment:

UDP channel header AVL packet acknowledgment
Len – 5,

Id – 0xABCD, Packet type – 01 (without ACK)

AVL packet id – 0xDD,

NumberOfAcceptedData – 2

0005ABCD01 DD02


Codec 8 Extended

  • Protocols overview


Difference between codec8 and codec8 extended

Codec 8 Codec 8 Extended
Codec ID 0x08 0x8E
AVL Data IO element length 1 Byte 2 Bytes
AVL Data IO element total IO count length 1 Byte 2 Bytes
AVL Data IO element IO count length 1 Byte 2 Bytes
AVL Data IO element AVL ID length 1 Byte 2 Bytes
Variable size IO elements Does not include include Includes variable size elements

Table 2. Codec 8 and 8 Extended differences

Main differences between are shown in above table. AVL data element sizes in codec 8 extended protocol was increased to 2 byte length and new variable type added. For more detailed description look in codec 8 and codec 8 extended chapters.


  • Codec 8 extended protocol sending over TCP


AVL data packet

Below table represents AVL data packet structure.

4 zeros Data field length Codec ID Number of

Data 1

AVL Data Number of

Data 2

CRC-16
4 Bytes 4 Bytes 1 Byte 1 Byte 38-768

Bytes

1 Byte 4 Bytes

Table 3. AVL data packet structure


Number of data – number of encoded data (number of records). Codec ID is constant 0x8E.

Data field length is the length of bytes [codec id, number of data 2]. Number of data 1 should always be equal to number of data 2 byte.

CRC-16 is 4 bytes, but first two are zeroes and last two are CRC-16 calculated for [codec id, number of data 2]

Minimum AVL packet size is 53 bytes (all IO elements disabled).


  • AVL Data
Timestamp Timestamp Priority GPS Element IO Element
8 Bytes 1 Byte 15 Bytes 14 - 744

Table 4. AVL data structure


  • Priority
0 Low
1 High
2 Panic

Table 5. Priority element values


  • GPS Element
Longitude Latitude Altitude Angle Satellites Speed
4 Bytes 4 Bytes 2 Bytes 2 Bytes 1 Byte 2 Bytes

Table 6. GPS element structure


  • IO Element
Event IO ID 2 bytes
N of Total IO 2 bytes
N1 of One Byte IO 2 bytes
1'st IO ID 2 bytes
1'st IO Value 1 bytes
N1'th IO ID 2 bytes
N1'st IO Value 1 bytes
N2 of Two Byte IO 2 bytes
1'st IO ID 2 bytes
1'st IO Value 2 bytes
N2'th IO ID 2 bytes
N2'st IO Value 2 bytes
N4 of Four Byte IO 2 bytes
1'st IO ID 2 bytes
1'st IO Value 4 bytes
N4'th IO ID 2 bytes
N4'st IO Value 4 bytes
N2 of Eight Byte IO 2 bytes
1'st IO ID 2 bytes
1'st IO Value 8 bytes
N8'th IO ID 2 bytes
N8'st IO Value 8 bytes
NX of X Byte IO 2 bytes
1'st IO ID 2 bytes
1'st IO Length 2 bytes
1'st IO Value defined by length
NX'st IO ID 2 bytes
NX'st IO Length 2 bytes
NX'st IO Value defined by length

Table 7. IO element structure

N - total number of properties coming with record (N=N1+N2+N4+N8+NX)

N1 - number of properties, which length is 1 byte

N2 - number of properties, which length is 2 bytes

N4 - number of properties, which length is 4 bytes

N8 - number of properties, which length is 8 bytes

NX - number of properties, which length is defined by length element


  • Communication with server

Communication with server is the same as with codec 8 protocol, except in codec8 extended protocol codec id is 0x8E.

Example:

Module connects to server and sends IMEI:

000f333536333037303432343431303133

Server accepts the module:

01


Module sends data packet

AVL data packet header AVL data array CRC
Four zero bytes,

‘AVL data array’ length – 254

Codec Id – 8E, NumberOfData – 02.

(Encoded using continuous bit stream.

Last byte padded to align to byte boundary)

CRC of ‘AVL data array’
00000000000000FE 8E02...(data elements)...02 00008612

Table 8. IO element structure

Server acknowledges data reception (2 data elements): 00000002

  • Example

Received data:

Ggg.png

00000000 4 zeros, 4 bytes

0000005F data length, 4 bytes

8E – Codec ID

01- Number of Data (1 record)

1’st record data

0000015FBA40B620– Timestamp in milliseconds (1510658324000)

GMT: Tuesday, November 14, 2017 11:18:44 AM

00 – Priority

GPS Element

0F0DCDE4 – Longitude 252562916 = 25, 2562916º N

20959D30 – Latitude 546676016 = 54,6676016 º E

008A – Altitude 138 meters

0000 – Angle 0º

06 – 6 Visible sattelites

0000 – 0 km/h speed

IO Element

0000 – IO element ID of Event generated (in this case when 0000 – data generated not on event)

0006 – 6 IO elements in record (total)

0001 – 1 IO elements, which length is 1 Byte

00EF – IO element ID = 239 (dec)

00 – IO element’s value

0001 – 1 IO elements, which length is 2 Byte

0011 – IO element ID = 17 (dec )

001E – IO element’s value

0001 – 1 IO elements, which length is 4 Byte

0010 – IO element ID = 16 (dec )

0000CBDF – IO element’s value

0002 – 2 IO elements, which length is 2 Byte

000B – IO element ID = 11 (dec )

000000003544C875 – IO element’s value

000E – IO element ID = 14 (dec )

0000000029BFE4D1 – IO element’s value

01 – Number of Data (1 record)

0000D153 – CRC-16, 4 Bytes (first 2 are always zeros


Codec 8 extended protocol sending over UDP

  • AVL data packet

AVL data packet is the same as with codec 8, except codec ID is changed to 0x8E.


Example:

Module sends the data:

UDP channel header AVL packet header AVL data array
Len – 253,

Id – 0xCAFE, Packet type – 01

AVL packet id – 0xDD,

IMEI – 1234567890123456

Codec Id – 8E, NumberOfData – 02.

(Encoded using continuous bit stream

00FDCAFE01 DD000F3133343536373839303132333435 8E02…(data elements)…02

Table 9. Example packet send to server


Server must respond with acknowledgment:

UDP channel header AVL packet acknowledgment
Len – 5,

Id – 0xCAFE, Packet type – 01

AVL packet id -0xDD

Number of Accepted Data -2

0005CAFE01 DD02

Table 10. Example packet server response


Example

Server received data:

Penkt.png

Data length: 00a1 or 161 Bytes (not counting the first 2 data length bytes)

Packet identification: 0xCAFE 2 bytes

Packet type: 01

Packet id: 1b

Imei length: 000f

Actual imei: 333536333037303432343431303133

Codec id: 8E

Number of data: 01

Timestamp: 0000013febdd19c8

Priority: 00

GPS data: 0f0e9ff0209a718000690000120000

Codec 12 Nebaigta

  • About Codec12

Codec12 is original Teltonika protocol for device-server communication over GPRS messages.

Codec12 GPRS commands can be used for sending configuration, debug, digital outputs control commands or other (special purpose command on special firmware versions). This protocol is also necessary for using FMB63/FM63/FM5300/FM5500/FM4200 features like: Garmin, LCD communication, COM TCP Link Mode.

Note: GPRS commands are supported only through TCP connection

  • FM firmware requirements

Supported GPRS commands on each device depend on firmware version. For available GPRS commands on each device, please refer to Table1 FM firmware requirement “SMS over GPRS” means that all standard SMS commands text can be sent to device via GPRS in Codec12 format.

Device SMS over GPRS SMS over GPRS via UDP Special Codec12 GPRS commands
FM11YX Available since base firmware 01.11.XX Since base firmware

1.26.00

Available in FM1100 and FM1110 special firmware version 12.XX.XX*
FM12YX Available since base firmware 01.03.XX - Available in FM12YX special firmware 09.XX.XX*
FM10YX Available since base firmware 00.03.XX - Available since base firmware 00.06.XX and later versions**
FM3400 Available since base firmware 01.01.XX - -
FM36YX Available in base firmware Since base firmware 01.06.01 -
FM5300 Available in all firmware versions - Available in all firmware versions**
FM5500 Available in all firmware versions - Available in all firmware versions**
FM2200 Not available - Special firmware version 07.XX.XX*
FM4200 Not available - Special firmware version 42.XX.XX*
FM63YX Available in all firmware versions Since base firmware 00.02.19 Available in all firmware versions**
FMB9YX Available in base firmware Available in base firmware -
FMB0YX Available in base firmware Available in base firmware -
FMB1YX Available in base firmware Available in base firmware -
FMB6YX Available in all firmware versions Since base firmware 00.02.19 Available in all firmware versions**
FMA1YX Available in base firmware Since base firmware 1.26.00 Available in special firmware version 12.XX.XX*
FMA2YX Available in base firmware Available in base firmware -

Table 1. FM firmware requirement

'*' - supported special commands are listed on special firmware description. Please contact Your Teltonika sales manager for more details;

'**' - supported special commands are listed on device user manual, chapter “GPRS Commands”


  • GPRS command session

Following figure shows how GRPS command session is started over TCP. wq.png

First FM opens GPRS session and sends AVL data to server (refer FM protocols). Once all records are sent and correct sent data array acknowledgment is received by FM then GPRS commands in Hex can be sent to FM.

The ACK (acknowledge of IMEI from server) is a one byte constant 0x01. The acknowledgement of each data array send from FMXXXX is four bytes integer – number of records received.

Note, that GPRS session should remain active between FM and server, while GPRS commands are sent. For this reason active datalink timeout (global parameters in FMXXXX configuration) is recommended to be set to 259200 (maximum value).


  • General Codec12 message structure

The following diagram shows basic structure of Codec 12 messages.


Command message structure

0x00000000 Data size 0x0C Command quantity 0x05 Command size Command Command quantity CRC
4 bytes 4 bytes 1 byte 1 byte 1 byte 4 bytes X bytes 1 byte 4 bytes


Response message structure

0x00000000 Data size 0x0C Command quantity 0x06 Command size Command Command quantity CRC
4 bytes 4 bytes 1 byte 1 byte 1 byte 4 bytes X bytes 1 byte 4 bytes


Structure explanation:

The message starts with preamble field - four zero bytes. Then goes four bytes data size field (size is calculated from 0x0C field to the second command or response quantity field). Then follows one byte Codec ID field (in Codec 12 it is always 0x0C). Then goes command or response quantity field (it is ignored when parsing the message). After that goes one byte message type field. It can be 0x05 to denote command or 0x06 to denote response. Then follows four bytes command or response size field. After it follows the command or response field itself. After that goes the second command or response quantity field. At the end there’s four bytes CRC field.

Note that difference between commands and responses is message type field: 0x05 means command and 0x06 means response.

The algorithm to calculate CRC is CRC-16 (also known as CRC-16-IBM). All the fields from codec ID to second command/response quantity field are used to calculate CRC. The algorithm of how to calculate


CRC is shown in Figure 3 CRC calculation algorithm.

qqq.png Figure 3 CRC calculation algorithm


  • Command coding table

Command has to be convert from ASCII characters (char) to hexadecimal (Hex)

Table 2 ASCII conversion table

asa.png


  • Command parsing example

Hexadecimal stream of command and answer in this example are given in hexadecimal form. The different fields of message are highlighted in gray and yellow for better readability and command is converted in ASCII for better understanding.

Server command

Hexadecimal stream:

sx.png

Parsed:

Preamble: 0x00000000

Packet Length: 0x00000018

Codec: 0x0C

Quantity of commands: 0x01

Command type: 0x05

Command size: 0x00000010

Command in ASCII after conversion: #GET DATAORDER<CR><LF>

Quantity of commands: 0x01

CRC: 0x00004990


Device answer

Hexadecimal stream:

sds.png

Parsed:

Preamble: 0x00000000

Packet Length: 0x00000016

Codec: 0x0C

Quantity of commands: 0x01

Command type: 0x06

Command size: 0x0000000E

Command response is in ASCII after conversion: #DATAORDER=1<CR><LF>

Quantity of commands: 0x01

CRC: 0x00000095


  • Codec12 GPRS commands examples

The example commands given in hexadecimal form separated by dollar signs are suitable to be sent from TCP server during data exchange session between FMXXXX device and server (for more details see Figure 1 Command session) The command can be sent from a terminal program such as Hercules (in TCP server mode). Simply write command as explained below into Hercules Send field and click Send button. The TCP server must be listening on specified port (see field Port and button Listen in Figure 4).

he.png

Figure 4 Hercules terminal GUI


  • SMS over GPRS in Codec12 examples

In case of sending SMS commands over GPRS, do not use SMS logins set during configuration or do not leave empty spaces before command. Devices and firmware versions that support SMS over GPRS are listed in Table1.

Codec 13 Nebaigta

  • About Codec13

Codec13 is original Teltonika protocol for device-server communication over GPRS messages. This protocol is necessary for using following FM features: COM TCP Link Mode (binary/ASCII/binary buffered/ASCII buffered) if message timestamp parameter is enabled in device configuration. Codec13 messages are one way only (Codec 13 is used for FM->Server sending).

  • FM firmware requirements

Codec13 availability depends on device and firmware version.

Codec 16 Nebaigta

  • Description and example

Records to server will be sended as shown in table bellow. The main difference between CODEC8 and CODEC16 is CODEC ID which will be 0x10 instead of 0x08, AVL ID‘s in AVL data is sent in 2 bytes, instead of 1 byte.

Also new parameter – Generation type is adeed.

By receiving 0x10 codec ID server must know that AVL data record will be parsed different.

Codec16 is supported from firmware – 00.03.xx and newer. ll AVL ID‘s which are higher then 255 will can be used only in CODEC16 protocol.