PTP

Files CCore/inc/net/PTPBase.h CCore/src/net/PTPBase.cpp

Files CCore/inc/net/PTPExtra.h CCore/src/net/PTPExtra.cpp

Files CCore/inc/net/PTPClientDevice.h CCore/src/net/PTPClientDevice.cpp

Files CCore/inc/net/PTPServerDevice.h CCore/src/net/PTPServerDevice.cpp

PTP is the Packet Transaction Protocol. This is a packet-based, reliable, transactional, parallel point-to-point protocol. It is best suited to implement an asynchronous call-type client-server interaction. It works atop a packet-based communication layer. There is no security features in the protocol.

Protocol description

PTP defines rules for two endpoints, one is the Server, another is the Client. These endpoints exchange raw data packets (byte packets). Client issues call requests, Server takes call data, processes it and returns some resulting data. From the Client perspective, it makes a function call. Function arguments is a byte range. Server "evaluates" the function and returns a result — another byte range. The meaning of data is out of scope PTP protocol, it is defined by an upper protocol level. Usually, Server may serve multiple Clients. From the protocol perspective all transactions are parallel and independent.

Protocol defines the following constants:


const unsigned MaxPacketLen = 1472 ;
const unsigned DeltaInfoLen =   32 ;
const unsigned MaxInfoLen   = 1440 ; // MaxPacketLen-DeltaInfoLen
   
const unsigned MinTimeout   =   300 ; // msec, 0.3 sec   
const unsigned MaxTimeout   = 10000 ; // msec, 10  sec  
const unsigned DeltaTimeout =   100 ; // msec, 0.1 sec 

MaxPacketLen — maximum packet length. This value is chosen as the maximum UDP data length fits a standard Ethernet frame length.

DeltaInfoLen — maximum delta between packet length and information length.

MaxInfoLen — maximum information length.

MinTimeout — minimum timeout value.

MaxTimeout — maximum timeout value.

DeltaTimeout — timeout increment. Each time a timeout value is used, it is incremented by DeltaTimeout up to MaxTimeout.

Protocol defines the following data types:


using uint8 = ??? ;
   
using uint16 = ??? ;
   
using uint32 = ??? ;
   
using uint64 = ??? ;


using PacketType = uint32 ;

using SlotId = uint32 ;
   
using RecallNumber = uint32 ;
 
struct TransId
 {
  uint64 number;
  uint64 clock;
 };

struct Info
 {
  uint32 len : len<=MaxInfoLen ; 
  uint8 data[len];
 };

uint8, uint16, uint32, uint64 are usual arithmetic unsigned integral types. For the protocol purpose they are transported using the big-endian representation.

PacketType is used to represent a packet type.

SlotId is used to represent a slot id. Slot is a processing resource on both sides. It contains data, related with a single transaction.

RecallNumber is used to represent a recall number.

TransId is 128-bit value, it is generated by Client and serves as the unique transaction id. It's recommended, that the number is incremented with each transaction and the clock is a high-frequency clock or another random input. A random mask can be applied to TransId for the better diversity.

Info is a call or return information. It is sent as the length, followed by the information bytes.

There are 8 packet types, 3 with attached information and 5 are short. Each packet starts with a fixed format header. Info packets are ended with Info.


struct Packet_CALL // info, Client to Server
 {
  PacketType type = 1 ;
  TransId tid;
  SlotId client_slot;
     
  Info client_info;
 };
 
struct Packet_RET // info, Server to Client
 {
  PacketType type = 2 ;
  TransId tid;
  SlotId client_slot;
  SlotId server_slot;
     
  Info server_info;
 };
 
struct Packet_RECALL // info, Client to Server
 {
  PacketType type = 3 ;
  TransId tid;
  SlotId client_slot;
     
  RecallNumber number;

  Info client_info;
 };

struct Packet_ACK // short, Client to Server
 {
  PacketType type = 4 ;
  TransId tid;
  SlotId client_slot;
  SlotId server_slot;
 };
    
struct Packet_NOINFO // short, Server to Client
 {
  PacketType type = 5 ;
  TransId tid;
  SlotId client_slot;
  SlotId server_slot;
 }; 
    
struct Packet_RERET // short, Server to Client
 {
  PacketType type = 6 ;
  TransId tid;
  SlotId client_slot;
  SlotId server_slot;
 }; 
    
struct Packet_SENDRET // short, Client to Server
 {
  PacketType type = 7 ;
  TransId tid;
  SlotId client_slot;
  SlotId server_slot;
 }; 
    
struct Packet_CANCEL // short, Server to Client
 {
  PacketType type = 8 ;
  TransId tid;
  SlotId client_slot;
  SlotId server_slot;
 }; 

To start a transaction, Client allocates a slot, prepares client Info, generates TransId and client SlotId. Then it prepares and sends the CALL packet.


Client                ->   CALL   -> Server

TransId tid;
SlotId client_slot;
Info client_info;

Server accepts transaction by allocating a server slot. It may replay with RET, CANCEL, NOINFO or RERET packets.


Client                <-   RET    <- Server

TransId tid;                         TransId tid;
SlotId client_slot;                  SlotId client_slot;
Info client_info;                    SlotId server_slot;
                                     Info client_info; // "arguments"
                                     Info server_info; // "result"

Client completes transaction by the ACK packet, it cleanup the processing slot. Server cleanup the processing slot upon reception of this packet.


Client                ->   ACK    -> Server

TransId tid;                         TransId tid;
SlotId client_slot;                  SlotId client_slot;
SlotId server_slot;                  SlotId server_slot;
Info client_info;                    Info client_info;
Info server_info;                    Info server_info;                 
                                     

The "good" case if a transaction is finished by these 3 packets. Everything else is to make the protocol reliable.

Once Client is sent the CALL packet, it setup a timer to count timeout. The initial timeout value is the MinTimeout. If this timer expires before an expected reply, the RECALL packet is sent. number starts from 1 and is incremented with saturation per each RECALL packet. The timeout value is incremented each time it is expired.


Client                ->  RECALL  -> Server

TransId tid;
SlotId client_slot;
Info client_info;

RecallNumber number = 1 ;
bool no_info = false ;

Server may send the NOINFO packet instead of RET packet to confirm it has received the CALL packet.


Client                <-  NOINFO  <- Server

TransId tid;                         TransId tid;
SlotId client_slot;                  SlotId client_slot;
Info client_info;                    SlotId server_slot;
                                     Info client_info;

If Client gets this packet it sets the flag no_info in the processing slot. If this flag is set, then Client sends the SENDRET packet at a timeout expiration instead of RECALL.


Client                ->  SENDRET -> Server

TransId tid;                         TransId tid;
SlotId client_slot;                  SlotId client_slot;
SlotId server_slot;                  SlotId server_slot;
Info client_info;                    Info client_info;

bool no_info = true ;

To cancel a transaction Server may send the CANCEL packet. Client responds with the ACK packet and cleanup the processing slot.

Server may also send the RERET (Ready Return) packet. In response, Client sends the SENDRET packet.

Server starts a transaction processing upon CALL or RECALL packet. If there is no available slots the packet is discarded. If the packet is CALL and the transaction id is used, then the packet is discarded. If Server serves multiple Clients, Client id must be accounted for the transaction identification. The CALL packet starts the transaction processing. If the RECALL packet has the new transaction id, it also starts the transaction processing. Otherwise one of the packet RET, CANCEL or NOINFO is returned to the Client. RET or CANCEL is returned if the slot processing is finished, NOINFO if the slot processing is in progress.

When Server completes the slot processing and has the processing result, it sends the RET packet or the CANCEL packet, if the processing is canceled by the execution entity. Once it is done, the Server setup a timer to count timeout. When timer is expired, Server sends RERET packet. Server sends RET or CANCEL packet as the answer on SENDRET packet from Client. And only ACK packet finally cleanup the processing slot.

If Server gets SENDRET packet and cannot find the correspondent processing slot, it responds with CANCEL packet.

If Client gets any packet and cannot find the correspondent processing slot, it responds with ACK packet.

The file txt/cpp/PTP.txt.cpp contains a precise protocol description in pseudo-C++ code.

The file net/PTPBase.h contains this protocol definition in C++ in the namespace PTP.

PTPClientDevice

PTPClientDevice is an implementation of PTP client protocol as a device class. It is located in the PTP namespace (::CCore::Net::PTP).


class ClientDevice : public ObjBase
 {
   ....

  public:
   
   static const Unid TypeUnid;

   // constructors
  
   explicit ClientDevice(StrLen ep_dev_name,ulen max_slots=100);
   
   virtual ~ClientDevice();
   
   // methods
   
   PacketEndpointDevice * getEPDevice() const;
   
   void start(Packet<uint8,TransExt> packet);
   
   void abortAll();

   using StatInfo = ClientStatInfo ;
   
   void getStat(ClientStatInfo &ret);
   
   ulen getMaxOutboundInfoLen(); // always > 0 , <= MaxInfoLen
   
   ulen getMaxInboundInfoLen(); // always > 0 , <= MaxInfoLen
   
   // generic transactions
   
   template <class Ext>
   void start(Packet<uint8,Ext> packet,const typename Ext::InputType &input);
   
   template <class Ext>
   void start_format(Packet<uint8,Ext> packet,const typename Ext::InputType &input);
   
   struct FormatResult
    {
     PacketFormat format;
     bool too_short;
     
     FormatResult(NothingType) : too_short(true) {}
     
     FormatResult(PacketFormat format_) : format(format_),too_short(false) {}
     
     bool operator ! () const { return too_short; }
     
     bool noRoom() const { return format.max_data==0; }
    };
   
   template <class Ext>
   static PacketFormat GetFormat(); 
   
   template <class Ext>
   static FormatResult GetFormat(ulen max_outbound_info_len); 
   
   template <class Ext>
   FormatResult getFormat();
   
   template <class Ext>
   PacketFormat getFormat_guarded(const char *name); 
   
   // support service
   
   void setSeed(PTPSupport::SeedExt *ext);
   
   bool setLengths(PTPSupport::LenExt *ext);

   void support(Packet<uint8,PTPSupport::LenExt> packet);
   
   void support(Packet<uint8,PTPSupport::SeedExt> packet);
   
   void support(Packet<uint8,PTPSupport::SessionExt> packet);
   
   FormatResult getEchoFormat();
   
   void support(Packet<uint8,PTPSupport::EchoExt> packet,uint32 delay_msec);
   
   void support(Packet<uint8,PTPSupport::ErrorDescExt> packet,ServiceIdType service_id,FunctionIdType function_id,ErrorIdType error_id);
   
   void support(Packet<uint8,PTPSupport::ExistExt> packet,ServiceIdType service_id,FunctionIdType function_id);
   
   // initial interaction
   
   bool support(PacketSet<uint8> &pset,MSec timeout=DefaultTimeout);
   
   void support_guarded(MSec timeout=DefaultTimeout);

   // connection events

   void attach(PacketEndpointDevice::ConnectionProc *proc);

   void detach();
 };

The first constructor argument is the PacketEndpointDevice object name. The second is the number of transaction slots. When all slots are active, an extra transaction is put into a wait queue. It is activated as soon as a slot becomes available.

getEPDevice() returns the PacketEndpointDevice this object is attached to.

getMaxOutboundInfoLen() and getMaxInboundInfoLen() are info length limits, these values are calculated based on the PacketEndpointDevice properties, they are always positive.

abortAll() aborts all active transactions.

getStat() returns the protocol statistic counters.

start(Packet<uint8,TransExt> packet) starts a transaction. This is the most general way to perform a transaction. You must prepare a packet, fill it with client info and call this method. TransExt constructor argument is a flag, set this flag to MovePacketBuf if you require the packet buffer movement with server info. When transaction is finished, the packet is completed. In the completion function you will see the server info, it is the field server_info of the TransExt. You must use the operator ! to check if the transaction is failed. If you required the packet buffer movement, the packet will have the packet buffer with the server info attached. Otherwise, the server_info disappears after the completion function is returned.


struct TransExt
 {
  TransResult result;              // output
  PtrLen<const uint8> server_info; // output
  
  // constructors
  
  explicit TransExt(MoveFlagType move_flag);
  
  // methods
  
  bool operator ! () const { return result!=Trans_Ok; }

  ....
 };

Another way to perform a transaction is to define a special structure Ext like this:


struct Ext : ExtBase<ServiceId,FunctionId>
 {
  using InputType = .... ;
  using OutputType = .... ;
  
  void done(const OutputType &output)
   {
    ....
   }
 };

ServiceId is the service id and FunctionId is the function id. See the recommended service/function encoding. Serialized InputType is used as the function argument and OutputType is used to deserialize the function result.

Then two methods can be used to start a transaction.

start(Packet<uint8,Ext> packet,const typename Ext::InputType &input) starts a transaction, it uses the given input object to prepare a function argument in the packet buffer. When the transaction is successfully finished, the method done() is called. It can transport data from the output object to the Ext structure. Errors are reported in result and error_id fields of the base class. The method reset() resets them.


template <ServiceIdType ServiceId_,FunctionIdType FunctionId_,MoveFlagType MoveFlag=MoveNothing>
struct ExtBase
 {
  // consts
  
  static const ServiceIdType  ServiceId  = ServiceId_ ;
  static const FunctionIdType FunctionId = FunctionId_ ;
  
  // data
  
  TransResult result;
  ErrorIdType error_id;
  
  // constructors

  ExtBase(); // if( MoveFlag!=MovePacketBufByRequest ) 
  
  explicit ExtBase(MoveFlagType move_flag); // if( MoveFlag==MovePacketBufByRequest )

  // methods
  
  bool isOk() const { return !result && !error_id ; }
  
  void reset();

  ....
 };

Another method start_format() can be used, if you need to place additional data after the input object. You have to preplace data to the packet buffer. To do this you need a packet format. You can get it using the following methods:

getFormat<Ext>() returns the FormatResult result.

getFormat_guarded<Ext>() is a guarded version, it throws an exception, if the outbound info length is too short.

GetFormat<Ext>(ulen max_outbound_info_len) works the same way as the getFormat<Ext>(), but it is a static method and it uses the given max_outbound_info_len.

GetFormat<Ext>() uses the "infinite" max_outbound_info_len.

The Ext::InputType must define the static constant Ext::InputType::MaxLen to cap the extra data length.

If you select MoveNothing flag, then packet data buffer will be lost after the completion function is finished. But if you set it to the MovePacketBuf, then during the completion processing the packet buffer with resulting data is moved to the transaction packet. And if you select MovePacketBufByRequest, then you have to specify a required behavior giving the argument to the ExtBase constructor.

The family of methods are built using this approach to perform support service functions. Below are the correspondent Ext structures.


/* struct PTPSupport::LenExt */  

struct PTPSupport::LenExt : ExtBase<PTPSupport::ServiceId,PTPSupport::FunctionId_Len>
 {
  using InputType = LenInput ;
  using OutputType = LenOutput ;
  
  ulen max_outbound_info_len;
  ulen max_inbound_info_len;
  
  void done(const OutputType &output);
 };
 
/* struct PTPSupport::SeedExt */  

struct PTPSupport::SeedExt : ExtBase<PTPSupport::ServiceId,PTPSupport::FunctionId_Seed>
 {
  using InputType = SeedInput ;
  uisng OutputType = SeedOutput ;
  
  uint64 seed1;
  uint64 seed2;
  
  void done(const OutputType &output);
 };
 
/* struct PTPSupport::SessionExt */ 

struct PTPSupport::SessionExt : ExtBase<PTPSupport::ServiceId,PTPSupport::FunctionId_Session>
 {
  using InputType = SessionInput ;
  uisng OutputType = SessionOutput ;
  
  void done(const OutputType &);
 };
 
/* struct PTPSupport::EchoExt */ 

struct PTPSupport::EchoExt : ExtBase<PTPSupport::ServiceId,PTPSupport::FunctionId_Echo,MovePacketBuf>
 {
  using InputType = EchoInput ;
  using OutputType = Tailed<EchoOutput> ;
  
  PtrLen<const uint8> data;
  
  void done(const OutputType &output);
 };
 
/* struct PTPSupport::ErrorDescExt */ 

struct PTPSupport::ErrorDescExt : ExtBase<PTPSupport::ServiceId,PTPSupport::FunctionId_ErrorDesc,MovePacketBuf>
 {
  using InputType = ErrorDescInput ;
  using OutputType = Tailed<ErrorDescOutput> ;
  
  PtrLen<const uint8> desc;
  
  void done(const OutputType &output);
 };
 
/* struct PTPSupport::ExistExt */  
 
struct PTPSupport::ExistExt : ExtBase<PTPSupport::ServiceId,PTPSupport::FunctionId_Exist>
 {
  using InputType = ExistInput ;
  using OutputType = ExistOutput ;
  
  bool exist;
  
  void done(const OutputType &output);
 };

There are two methods for the initial interaction with the server. Both methods performs three functions: Session, Seed and Len. Seeds are used for transaction id generation, input/output info lengths are corrected according Len function.

support() returns true, if the interaction was successful. You have to supply some packet set for this function.

support_guarded() throws an exception in case of failure.

setSeed() can be used to seed the client from the PTPSupport::SeedExt fields.

setLengths() can be used to reduce inbound/outbound lengths from the PTPSupport::LenExt fields. It returns true, if both lengths are non-null. Otherwise it returns false and does nothing.

You can attach/detach some PacketEndpointDevice::ConnectionProc interface to handle connection lost events, using the methods attach()/detach().

PTPServerDevice

PTPServerDevice is an implementation of PTP server protocol as a device class. It is located in the PTP namespace (::CCore::Net::PTP).


class ServerDevice : public ObjBase
 {
   ....

  public: 
   
   static const Unid TypeUnid;

   // constructors
  
   explicit ServerDevice(StrLen mp_dev_name,ulen max_slots=100);
   
   virtual ~ServerDevice();
   
   // methods
   
   PacketMultipointDevice * getMPDevice() const;
   
   ulen getMaxOutboundInfoLen() const;
   
   ulen getMaxInboundInfoLen() const;
   
   void cancelAll();

   void cancelFrom(XPoint point);

   using StatInfo = ServerStatInfo ;
   
   void getStat(ServerStatInfo &ret);
   
   void send_info(TransIndex idx,Packet<uint8> proc_packet,PtrLen<const uint8> server_info);
   
   void send_cancel(TransIndex idx);
   
   void attach(ServerProc *proc);
   
   void detach();
   
   void waitActiveComplete();
   
   bool waitActiveComplete(MSec timeout);
   
   bool waitActiveComplete(TimeScope time_scope);
 };

The first constructor argument is the PacketMultipointDevice object name. The second is the number of available transaction slots.

getMPDevice() returns the PacketMultipointDevice this object is attached to.

getMaxOutboundInfoLen() and getMaxInboundInfoLen() are info length limits, these values are calculated based on the PacketMultipointDevice properties, they are always positive.

cancelAll() cancels all active transactions.

cancelFrom() cancels all transactions with the given endpoint.

getStat() returns the protocol statistic counters.

To implement a server functionality some server processor must be attached.

attach() attaches the server processor.

detach() detaches the server processor and waits while it is in use.


struct ServerProc : InterfaceHost
 {
  static const Unid TypeUnid;

  virtual void inbound(XPoint point,TransIndex idx,Packet<uint8> packet,PtrLen<const uint8> client_info)=0;
  
  virtual void tick()=0;
 };

Server processor interface has two methods. inbound() is called to process a transaction. The first argument is a client address, the second is a some transaction index, then packet follows and client info range from this packet. tick() is a network tick. When processor finished with transaction, it must call either send_info() or send_cancel() methods. Each of them starts with transaction index. send_info() takes also a packet and a server info data range from this packet. The original client packet may be reused, otherwise it must be completed.

Server processor may have an optional co-interface PacketMultipointDevice::ConnectionProc to handle connection events. Its methods will be called by the ServerDevice during processing the same events from the multipoint device.

waitActiveComplete...() methods can be used to wait until all processing slots are free (no in-progress transactions).