# Target Communication Framework Specification

Copyright (c) 2007, 2008 Wind River Systems, Inc. Made available under the EPL v1.0

Direct comments, questions to the dsdp-tm-dev@eclipse.org mailing list

# Table of Contents

- Overview
  - Goals
  - Definitions
  - Requirements
  - Syntax Rules Notation
- Framework Software Design Considerations
  - Concurrency
  - Reflection
  - Message ordering
- Transport Layer
- Communication Protocol
  - Commands
  - Results
  - Events
  - Flow Control
  - Examples
- API
- JSON - Preferred Marshaling
  - JSON - Examples
- Locator Service
  - Peer Attributes
  - Locator Service Commands
  - Locator Service Events
  - Locator Service API

# Overview

Today almost every device software development tool on the market has its own method of communication with target system. Communication methods often require individual setup, configuration and maintenance, impose unnecessary limitations. Target Communication Framework goal is to establish common ground in the area of communication protocols between development tools and embedded devices.

The goal is a single protocol used to communicate between all tools and targets:

## Goals

- Universal, simple, lightweight, vendor agnostic framework for tools and targets to communicate for purpose of debugging, profiling, code patching and other device software development needs.
- Single configuration per target (not per tool per target as today in most cases), or no configuration when possible.
- Minimal overhead and footprint on target side.

## Definitions

Peer:
  communication endpoint. Both hosts and targets are called peers. A peer can act as a client or a server depending on services it implements.
Service:
  group of related commands, events and semantic define a service. A service can be discovered, added or removed as a group at communication endpoint.
Message:
  a packet of data, formatted according to framework specification and transmitted over communication channel.
Channel:
  communication link connecting two endpoints (peers). A single channel may be used to communicate with multiple services. Multiple channels may be used to connect the same peers, however no command or event ordering is guaranteed across channels.
Command:
  command is a message sent to remote peer in order to request some predefined action there.
Result:
  result is a message sent as a response to a command.
Event:
  event is a message sent to all interested parties in order to notify them about state changes.

## Requirements

- Simple and extensible protocol.
- Small footprint on the target.
- Fully asynchronous, message based communication.
- Two ways of message routing:
  - Point to point request/response (command/result) communication.
  - Subscription based broadcast of notifications (events).
- Full duplex, symmetric communication: both host and target should be able to send commands and events at same time, though ability to establish communication channel can be limited to host only.
- For each communication channel between two peers, the framework should preserve order of commands, results and events.
- Support for slow and high latency connections.
- Transport protocol agnostic. The framework should work well, at least, on top of: TCP/IP, UDP, USB, RS232 and JTAG.
- The framework should support multiplexing, that is, single target device shared between multiple tools at same time. To reduce footprint on the target, multiplexing can be implemented on host if needed.
- Dynamic discovery of participating targets and hosts. No configuration when possible.
- Dynamic discovery of available services (high level protocols, command sets). Clients can query for available services.
- Services can be added and removed dynamically.
- Framework should define a set of common high level interfaces (services). For example: flow control, memory access, registers access, up-load mechanism, kernel awareness, run control, target file system, console, flash programming. Implementation of these interfaces is optional, but if provided it will support much wider compatibility with various tools.
- Framework should be layered in such a way so it is possible to use different transport medias (e.g. TCP/IP, RS232, USB, etc) without any changes to individual services. In other words, transport implementation should be services agnostic, and services implementation should be transport agnostic.
- Each service defines how marshalling is done for command, result and event arguments. This allows existing target agents to remain unchanged.
- Framework should define a preferred marshalling mechanism that new services can use.
- The definition of services (groups of related commands and events) is separate from the definition of the framework itself. The framework provides unified communication mechanism, while services use it to communicate with its clients.
- Anybody (including 3rd parties) can add services without having to modify communication protocol or framework software.
- The framework should support tunneling through a proxy. Proxy may be used, for example:
  - to bridge different transport protocols, like TCP and RS232;
  - to make a RS232 or USB target connection accessible from multiple hosts;
  - to access targets behind firewalls or otherwise not directly accessible
- A proxy should be able to provide services in addition to those implemented by a target. Such distribution of services allows target services to be implemented on a host, thereby reducing the footprint on the target. For example, debug information, stack back trace or OS awareness can be implemented by a proxy on a host. To provide this functionality, proxy services would typically use low-level target services, like memory access.
- Supports of concurrent requests. Maximum number of concurrent requests (window size) can be limited on target side. Simple agents only have to support window size of 1. Framework should maintain a queue of additional requests, so tools don’t need to know the window size. This may only be relevant for certain transport protocols e.g. UDP.
- Events can be broadcasted at any time, i.e. no polling should be required.
- Protocol should support a standard mechanism of sending data larger than MTU.

## Syntax Rules Notation

Format of the protocol messages is defined by syntax rules. Syntax is described using a simple variant of Backus-Naur Form. In particular:

- Italic lower case words in a courier font, enclosed into angular brackets, are used to denote syntactic categories, for example: <token>. Category name can be followed by colon and a text, which explains semantics of the category, for example: <int: error code> has same meaning as <int>, but denotes that the integer number used to indicate an “error code”.
- A syntax rule consists of a category designation followed by one or more syntax definitions for the category. The category name and each definition are placed on separate lines, bullets are used to denote definitions, for example:

              <chars>
                  Ø <char>
                  Ø <chars> <char>

- Spaces are added for readability only and they are not part of the syntax.
- All text in the category definition, other then categories and spaces, is UTF-8 based representation of a message bytes.
- The symbol ‘•’ designates a zero byte.

# Framework Software Design Considerations

The framework will be packaged, distributed and installed on a host as separate product. It should be installed as system service and require no configuration for most common case – target connected over TCP or UDP on a local network. For more complicated setup, framework should have easily accessible and user friendly GUI with all relevant configuration options.

Framework should use a dynamic discovery protocol to locate targets and other hosts running instances of the framework when possible, and maintain a dynamic list of available communication endpoints, as well as lists of services available at each endpoint. Host discovery is needed to locate hosts able to proxy communications for targets, which are not accessible otherwise - for example, targets connected with RS232 or JTAG to a remote host. It should also be possible to add target configuration manually. Development tools will access this data through the Locator Service API and use it, for example, to present a user a list of available targets that have capabilities needed by a particular tool.

Framework should provide software libraries to be used by tools and target agents developers. The libraries should be available at least for ANSI C and Java. On host side, at least Windows, Solaris and Linux must be supported. Libraries will provide APIs for low-level communication protocol, Locator Service, preferred marshaling and predefined common services.

The proposed target communication protocol is text-based. It allows extensions, which define messages with blocks of binary data, but it is not a recommended data formatting, and its usage is supposed to be limited. Text-based protocols have both advantages and disadvantages in compare with binary protocols.

Advantages:

- The software for text-based protocols is easier to develop and debug since they use a relatively human-friendly communication.
- It is possible to use huge selection of existing tools and library routines to view, edit, validate, and transform text-based data.
- Text based definition is in line with current trend in Internet protocols: most popular protocols such as SMTP and HTTP are text-based.

Disadvantages:

- Text-based protocols usually need more bytes to store numerical data than binary protocols do.
- Parsing of text-based data is not efficient compared to parsing of binary data since text-based data is usually not stored in a way similar to how it is stored in computer memory.
- It is seldom possible to read only part of a text-based message since the exact byte offset to a data item is generally not known.

A possible alternative to consider is binary, variable length encoding like BaseStream.

## Concurrency

Concurrent asynchronous communication is much faster then synchronous, because it alleviates communication channel latency and allows better bandwidth utilization. But it also requires proper design of framework software. Concurrency, in general, implies multithreading. However, systems developed with global multithreading, are often unreliable and prone to different kinds of thread synchronization problems, which are often very difficult to locate and resolve. We therefore strongly recommend that the software is designed to follow the compartment threading model, which simplifies thread synchronization and promotes reliable software design. In this model each thread execution path is strictly contained in predefined subset of code (compartment), and no code, except for reentrant libraries, is executed by multiple threads. Each compartment has a message queue and other threads communicate with the compartment thread by posting messages to the queue.

Framework APIs are designed to be compatible with the compartment threading model. Hence the API functions do not contain any thread synchronization primitives to protect against multiple threads using the functions. All framework APIs belong to a single compartment and should be used by a single thread. The same thread is used to dispatch events and command results. Concurrency is achieved by declaring API functions to be asynchronous. Asynchronous functions do not have any return value, and returns immediately, most of the time before the intended job is done. They take additional arguments to specify a callback function and callback data. In object-oriented languages such as Java, this is typically done by a single callback object argument containing both the data and the function. The result listener is called asynchronously when the job is done. This approach is commonly known as asynchronous, event-driven or callback-based programming.

One important characteristic of an asynchronous code is that the methods defined by the user will often be called from within the framework itself, rather than from the user's application code. The framework often plays the role of the main program in coordinating and sequencing application activity. This phenomenon is called Inversion of Control (also known as the Hollywood Principle - "Don't call us, we'll call you").

## Reflection

Communication between development tools and embedded devices must allow a host to collect target side data and build a reflection of target state. Reflection is usually incomplete – a subset of all remote data. Reflection is always delayed – it represents a remote peer state in the past. Reflection can be updated by polling for data changes or by listening to events (event is communication message that is sent asynchronously by a peer to notify others about state change). Reflection is correct if it represents a state that actually did happen on remote peer.

Reflection is coherent if it is exactly equal to subset of peer state at a single moment of time and that moment of time is not too far in the past. Non-coherent reflection can have parts of data representing peer state at different moments of time. Coherent reflection is more valuable for a user, because non-coherent reflection can have logically conflicting data if that data was collected at different time.

Traditionally, debuggers would ensure coherence of state reflection by collecting data only while target is suspended, and flushing all (or most) reflection data (reducing observed subset to zero) when target is resumed. This approach does not work well for multithreaded, multicore or real time targets. Maintaining correctness and coherence of a non-empty reflection while target is running requires additional support from target agent, communication software and debugger itself.

Since remote peer state is changing over time, coherent reflection can be built only if:

- Observed subset of state is properly selected and dynamically re-selected. Observing too much can overflow communication channel. Observing too little has little value for a user.
- Observer is listening to all relevant events.
- Events are coming in exactly same order as corresponding changes happen.
- Events are properly ordered relative to other messages that carry state data.
- All changes in observed subset of peer state are reported by events.
- All event messages must either contain a complete description of a change or they all should not contain any state data at all. If client is getting some data from events and required to retrieve new values of other changed data by using other means (commands), the reflection will not be coherent at least until such retrieval is complete. And if such periods of data retrieval overlap, the reflection will never be coherent. Sending deltas with events is usually more efficient then using data retrieval commands to update reflection.

## Message ordering

The transmission order of commands, results and events is important, it coveys valuable information about target state transitions and it should be preserved when possible. Consider an example:

Client transmits:

        Command X=2

Then, as result of some activity of another client or the target itself, X is assigned value 3.

Target transmits:

        Event X=3
        Result X=2

Now client has to show value of X to a user. If the order of messages is preserved, the client will know that command was executed after X was assigned 3, the last message contains last known value of X and 2 is the correct value to show. If the target is allowed to transmit events and results in arbitrary order, the client will have no clue what to show – 2 or 3. In fact, the client will have to make a tough decision about each message it receives: either trust message data as correct last known target state, or assume the message came in out-of-order and ignore it, or re-request the information from the target.

Note that re-requesting data from the target, in general, does not solve the problem of interpretation of messages when order is not preserved. For example, after sending a request to read value of X, X could change at about the same time, and client could receive:

        Event X=2
        Result X=3
        Event X=4

If order is not preserved, it is still impossible to tell which value of X is the last one. A client could assume value of X unknown every time it receives a notification of X change, and then re-request the data again. But this is expensive and, if events coming in frequently, client can end up in infinite loop re-requesting the data again and again, and it will never have trustworthy data about current target state.

Developers should be careful when using multithreading or multiple queues in software design – it can easily cause message reordering.

The framework itself is required to preserve message order. However, if for whatever reason a target agent cannot preserve message order, the result will be that clients of the service can receive messages in the wrong order. When this is the case it should be well documented, so tools developers are aware and can make the best of the situation. In most cases it will not cause any trouble, but there is no perfect way to restore actual sequence of events and maintain data coherency after ordering was lost, and in some cases it can severely impact tool functionality and user experience.

# Transport Layer

Tools are required to be transport protocol agnostic, so most of the layer functionality is used internally by framework and is not exposed to clients. This layer maintains a collection of transport protocol handlers. Each handler is designed to provide:

- Enumeration of available peers, including both automatically discovered and manually configured peers. Handler fires notification events when peers are added or removed. Enumeration can be implemented by scanning JTAG chain, by broadcasting special UDP packet and waiting for responses, by communicating with ICE hardware, or by any other suitable means.
- Bidirectional point-to-point communication of data packets. Packets are arrays of bytes of arbitrary size. Transport handler and underlying protocol are responsible for adding all necessary control data, headers, error checking bits, addresses, fragmentation/defragmentation, flow control, transmission retries and whatever necessary to ensure lossless, order-preserving delivery of packets.
- Configuration UI should allow user to inspect and modify properties of both manually configured and automatically discovered peers, setup new peers, view connections status and statistics.

Existing service discovery protocols can be used together with the framework, for example:

- Zero Configuration Networking (Zeroconf), see http://www.zeroconf.org;
- Service Location Protocol (SLP), developed by the IETF;
- Jini, which is Sun’s Java-base approach to service discovery, see http://www.sun.com/jini;
- Salutation, developed by an open industry consortium, called the Salutation Consortium;
- Microsoft’s Universal Plug and Play (UPnP), see http://www.upnp.org;
- Bluetooth Service Discovery Protocol (SDP).

Service discovery protocols, as well as transport protocols will be supported by framework plug-ins, they are not part of framework code itself, and they can be developed by 3rd parties. Note that existing discovery protocols define term “service” differently - as an independent communication endpoint (usually a TCP/IP port). In this document it is called “peer” (host, target, communication endpoint), and a peer can provide multiple services over single communication channel.

Using of standard discovery protocols should be optional, because it can potentially cause conflict or interference between development tools and application being developed over a use of same standard protocol – devices software often includes implementation of service discovery protocols as part of application code to support their main functions.

# Communication Protocol

The communication protocol defines data packets properties and roles common for all services. The communication protocol API provides functions for opening and /closing of the communication channel for a particular peer, and for sending and receiving data packets. The protocol define contents of a part of a packet, the rest of the packet is treated as array of bytes at this level. The communication protocol implementation also provides:

- Multiplexing – opening multiple channels per peer.
- Proxy – packet forwarding in behalf of other hosts.

Protocol defines three packet types: commands (requests), results (responses), and events. Each packet consists of several protocol defined control fields followed by byte array of data. Binary representation of control fields is a sequence of zero terminated ASCII strings. Format of data array depends on a service. We recommend using framework preferred marshaling for data formatting.

Syntax:

    <message>
        Ø <command>
        Ø <result>
        Ø <event>
        Ø <flow control message>

## Commands

    <command>
        Ø C • <token> • <service name> • <command name> • <byte array: arguments>

Command packets start with string “C”.

    <token>
        Ø <chars>

Token is unique string generated by framework for each command. It is used to match results to commands.

    <service name>
        Ø <chars>

Service name is used to identify a service that handles the command, it is same string as returned by Service.getName().

    <command name>
        Ø <chars>

Command name interpretation depends on a service.

A command should always be answered with result packed. Result does not have to be positive – it can include an error code, but there always must be one. Since client cannot detect that a response is missing, if for some reasons peer is not able to answer a command, it should consider such situation a fatal communication error and it must shutdown the communication channel. It is not necessary to wait for result before sending next command. In fact, sending multiple commands in a burst can greatly improve performance, especially when connection has high latency. At the same time, clients should be carefully designed to avoid flooding the communication channel with unlimited number of requests, since this will use resources in forms of memory to store the requests and time to process them.

## Results

    <result>
        Ø R • <token> • <byte array: result data>
        Ø P • <token> • <byte array: result data>

Result packets start with string “P” for intermediate result and “R” for final result. Receiving of “R” result concludes execution of corresponding command. There should be exactly one “R” result for each command. In addition, command execution can produce any number of intermediate “P” results. “P” results can be sent before “R”, and it can serve, for example, as command execution progress report when execution takes long time.

    <token>
        Ø <chars>

Token should match token field of one of the pending commands that produced the result.

## Events

    <event>
        Ø E • <service name> • <event name> • <byte array: event data>

Event packets start with string “E”.

    <service name>
        Ø <chars>

Service name identifies a service that fired event, same string as returned by Service.getName().

    <event name>
        Ø <chars>

Event name meaning depends on a service.

Events are used to notify clients about changes in peer state. Services should provide sufficient variety of events for clients to track remote peer state without too much of polling. Clients, interested in a particular aspect of the target state, should have a “reflection” (or “model”) of that state and update the reflection by listening for relevant events. If a service implements a command that changes a particular aspect of peers state, then, normally, it should also generate notifications event when that same part of the state changes and it should provide a command to retrieve current value of the state – to be used by clients to initialize the reflection. Service events are defined statically, together with commands. The framework does not do any event processing besides delivering them to clients, however a service can define additional event related functionality if necessary, for example, commands for event filtering, enabling, disabling, registration, etc. Care should be taken when designing events for a service - if events are sent too frequently, they will cause flooding of the communication channels and degrade performance. However, too few events will force clients to poll for changes and can also degrade performance. A balanced approach is the best.

## Flow Control

It often happens that one side of communication channel produces messages faster then they can be transmitted over the channel or can be consumed by another side. This will cause channel traffic congestion (flooding). Framework will deal with the problem and slow down transmitting side by blocking execution inside sendEvent(), sendCommand() and sendResult() functions when message buffers are full. However, in many cases, it is not the best way to handle congestion. For example, it can make a tool UI appear locked for prolonged time or it can break target software if it is designed to work in real time. Clients can use flow control events to implement advanced techniques to handle traffic congestion, for example, message coalescing, switching to less detailed messages, etc.

    <flow control message>
        Ø F • <int: traffic congestion level> •

Traffic congestion level value is in range –100..100, where –100 means no pending messages (no traffic), 0 means optimal load, and positive numbers indicate level of congestion. When a peer receives flow control message with congestion level > 0 it should try to reduce its transmition speed.

## Message Examples

Examples use simplified command arguments and result data. See service description for actual data formats.

Executing suspend command from RunControl service:

    Send   :      C 1 RunControl suspend “Thread1”
    Receive:      E RunControl suspended “Thread1”
    Receive:      R 1 “Success”

Same command, but target was already suspended:

    Receive:      E RunControl suspended “Thread1”
    …
    Send   :      C 2 RunControl suspend “Thread1”
    Receive:      R 2 “Already suspended”

Same command, but target was suspended (by another client) after sending the command, but before command was executed:

    Receive:      E RunControl running “Thread1”
    …
    Send   :      C 3 RunControl suspend “Thread1”
    Receive:      E RunControl suspended “Thread1”
    Receive:      R 3 “Already suspended”

## Framework API

    /**
     *
     * Class Protocol provides static methods to access Target Communication Framework root objects:
     * 1. the framework event queue and dispatch thread;
     * 2. local instance of Locator service, which maintains a list of available targets;
     * 3. list of open communication channels.
     */
    public class Protocol {

        private static IEventQueue event_queue;

        /**
         * Before TCF can be used it should be given an object implementing IEventQueue interface.
         * The implementation maintains a queue of objects implementing Runnable interface and
         * executes run methods of that objects in a sequence by a single thread.
         * The thread in referred as TCF event dispatch thread. Objects in the queue are called TCF events.
         * Executing run method of an event is also called dispatching of event.
         *
         * Only few methods in TCF APIs are thread safe - can be invoked from any thread.
         * If a method description does not say "can be invoked from any thread" explicitly -
         * the method must be invoked from TCF event dispatch thread. All TCF listeners are
         * invoked from the dispatch thread.
         *
         * @param event_queue - IEventQueue implementation.
         */
        public static void setEventQueue(IEventQueue event_queue);

        /**
         * @return instance of IEventQueue that should be used for TCF events.
         */
        public static IEventQueue getEventQueue();

        /**
         * Returns true if the calling thread is TCF dispatch thread.
         * Use this call to ensure that a given task is being executed (or not being)
         * on dispatch thread.
         * This method is thread-safe.
         *
         * @return true if running on the dispatch thread.
         */
        public static boolean isDispatchThread();

        /**
         * Causes runnable to have its run
         * method called in the dispatch thread of the framework.
         * Runnables are dispatched in same order as queued.
         * If invokeLater is called from the dispatching thread
         * the runnable.run() will still be deferred until
         * all pending events have been processed.
         *
         * This method can be invoked from any thread.
         *
         * @param runnable the Runnable whose run
         * method should be executed asynchronously.
         */
        public static void invokeLater(Runnable runnable);

        /**
         * Causes runnable to have its run
         * method called in the dispatch thread of the framework.
         * Calling thread is suspended util the method is executed.
         * This method is thread-safe.
         *
         * @param runnable the Runnable whose run
         * method should be executed on dispatch thread.
         */
        public static void invokeAndWait(Runnable runnable)
            throws InterruptedException;

        /**
         * Get instance of the framework locator service.
         * The service can be used to discover available remote peers.
         *
         * @return instance of ILocator.
         */
        public static ILocator getLocator();

        /**
         * Return an array of all open channels.
         * @return an array of IChannel
         */
        public static IChannel[] getOpenChannels();

        /**
         * Interface to be implemented by clients willing to be notified when
         * new TCF communication channel is opened.
         */
        public interface ChannelOpenListener {
            public void onChannelOpen(IChannel channel);
        }

        /**
         * Add a listener that will be notified when new channel is opened.
         * @param listener
         */
        public static void addChannelOpenListener(ChannelOpenListener listener);

        /**
         * Remove channel opening listener.
         * @param listener
         */
        public static void removeChannelOpenListener(ChannelOpenListener listener);

        /**
         * Transmit TCF event message.
         * The message is sent to all open communication channels – broadcasted.
         */
        public static void sendEvent(String service, String name, byte[] data);

        /**
         * Call back after TCF messages sent by this host up to this moment are delivered
         * to their intended target. This method is intended for synchronization of messages
         * across multiple channels.
         *
         * Note: Cross channel synchronization can reduce performance and throughput.
         * Most clients don't need cross channel synchronization and should not call this method.
         *
         * @param done will be executed by dispatch thread after communication
         * messages are delivered to corresponding targets.
         */
        public static void sync(Runnable done);
    }

    /**
     * IChannel represents communication link connecting two endpoints (peers).
     * The channel asynchroniously transmits messages: commands, results and events.
     * A single channel may be used to communicate with multiple services.
     * Multiple channels may be used to connect the same peers, however no command or event
     * ordering is guaranteed across channels.
     */
    public interface IChannel {

        /**
         * Channel state IDs
         */
        static final int
            STATE_OPENNING = 0,
            STATE_OPEN = 1,
            STATE_CLOSED = 2;

        /**
         * @return channel current state, see STATE_*
         */
        int getState();

        /**
         * Send command message to remote peer for execution. Commands can be queued
         * locally before transmission. Sending commands too fast can fill up
         * communication channel buffers. Calling thread will be blocked until
         * enough buffer space is freed up by transmitting pending messages.
         */
        IToken sendCommand(IService service, String name, byte[] args,
            ICommandListener done);

        /**
         * Command listener interface. Clients implement this interface
         * to receive command results.
         */
        interface ICommandListener {

            /**
             * Called when progress message (intermediate result) is received
             * from remote peer.
             */
            void progress(byte[] data);

            /**
             * Called when command result received from remote peer.
             */
            void result(byte[] data);
        }

        /**
         * Send result message to remote peer. Messages can be queued locally before
         * transmission. Sending messages too fast can fill up communication channel
         * buffers. Calling thread will be blocked until enough buffer space is
         * freed up by transmitting pending messages.
         */
        void sendResult(IToken token, byte[] results);

        /**
         * Get current level of outbound traffic congestion.
         *
         * @return integer value in range –100..100, where –100 means no pending
         * messages (no traffic), 0 means optimal load, and positive numbers
         * indicate level of congestion.
         *
         * Note: inbound traffic congestion is detected by framework and reported to
         * remote peer without client needed to be involved.
         */
        int getCongestion();

        /**
         * Channel listener interface.
         */
        interface IChannelListener {

            /**
             * Notifies listeners about congestion level changes. When level > 0
             * client should delay sending more messages.
             */
            void congestionLevel(int level);
        }

        /**
         * Subscribe a channel listener. The listener will be notified about changes of
         * outbound traffic congestion level.
         */
        void addChannelListener(IChannelListener listener);

        /**
         * Remove a channel listener.
         */
        void removeChannelListener(IChannelListener listener);

        /**
         * Command server interface.
         * This interface is to be implemented by service providers.
         */
        interface ICommandServer {

            /**
             * Called every time a command is received from remote peer.
             */
            void command(IToken token, String name, byte[] data);
        }

        /**
         * Subscribe a command server. The server will be notified about command
         * messages received through this channel for given service.
         */
        void addCommandServer(IService service, ICommandServer listener);

        /**
         * Remove a command server.
         */
        void removeCommandServer(IService service, ICommandServer listener);

        /**
         * A generic interface for service event listener.
         * Services usually define a service specific event listener interface,
         * which is implemented using this generic listener.
         * Service clients should use service specific listener interface,
         * unless no such interface is defined.
         */
        interface IEventListener {
            void event(String name, byte[] data);
        }

        /**
         * Subscribe an event message listener for given service.
         */
        void addEventListener(IService service, IEventListener listener);

        /**
         * Unsubscribe an event message listener for given service.
         */
        void removeEventListener(IService service, IEventListener listener);

        /**
         * @return IPeer object representing local endpoint of communication channel.
         */
        IPeer getLocalPeer();

        /**
         * @return IPeer object representing remote endpoint of communication channel.
         */
        IPeer getRemotePeer();

        /**
         * @return map of services available on local peer. It maps service names to
         * IService implemetations.
         */
        Map<String, IService> getLocalServices();

        /**
         * @return map of services available on removte peer. It maps service names to
         * IService implemetations.
         */
        Map<String, IService> getRemoteServices();

        /**
         * Close communication channel.
         */
        void close();

        /**
         * Close channel in case of communication error.
         * @param error
         */
        void terminate(Throwable error);

        /**
         * Redirect this channel to given peer using this channel remote peer
         * locator service as a proxy.
         * @param peer_id
         */
        void redirect(String peer_id);
    }

    /**
     * Object implemeting IToken interface is created by framework for every
     * command sent over communication channel. It is used to match command to its
     * results, and also can be used to cancel commands.
     */
    public interface IToken {

        /**
         * Try to cancel a command associated with given token. A command can be
         * canceled by this method only if it was not transmitted yet to remote peer
         * for execution. Successfully canceled command does not produce any result
         * messages.
         *
         * @return true if successful.
         */
        boolean cancel();
    }

# Preferred Marshaling

TCF messages data format is service specific. Since services specifications are separate from protocol specification, a service designer can choose any data format that suits the service requirements best. However, to promote better compatibility and to simplify service design and implementation, we recommend to use JSON for data formatting.

JSON (pronounced like the English given name Jason), which stands for "JavaScript Object Notation", is a lightweight, text-based, language-independent computer data interchange format. JSON is a subset of the object literal notation of JavaScript but its use does not require JavaScript.

JSON represents data with the same basic types that programming languages use. JSON's basic types are:

- Number (integer, real, or floating-point)
- String (double-quoted with backslash escapement)
- Boolean (`true` and `false`)
- Array (an ordered sequence of values)
- Object (collection of key/value pairs)
- `null`

The structures used in most programming languages easily map directly onto JSON's structures, and back again.

JSON maps data onto Unicode string. Then the string is mapped onto array of bytes using UTF-8 encoding.

JSON specification:

    <object>
        Ø {}
        Ø { <members> }

    <members>
        Ø <string> : <value>
        Ø <members> , <string> : <value>

    <array>
        Ø []
        Ø [ <elements> ]

    <elements>
        Ø <value>
        Ø <elements> , <value>

    <value>
        Ø <string>
        Ø <number>
        Ø <object>
        Ø <array>
        Ø <boolean>
        Ø null

    <boolean>
        Ø true
        Ø false

    <string>
        Ø ""
        Ø " <chars> "

    <chars>
        Ø <char>
        Ø <chars> <char>

    <char>
        Ø <any Unicode except " or \ or control>
        Ø \"
        Ø \\
        Ø \/
        Ø \b
        Ø \f
        Ø \n
        Ø \r
        Ø \t
        Ø \u <four-hex-digits>

    <number>
        Ø <int>
        Ø <int> <fraction>
        Ø <int> <exponent>
        Ø <int> <fraction> <exponent>

    <int>
        Ø <digit>
        Ø <digit 1-9> <digits>
        Ø - <digit>
        Ø - <digit 1-9> <digits>

    <fraction>
        Ø . <digits>

    <exponent>
        Ø <e> <digits>

    <digits>
        Ø <digit>
        Ø <digits> <digit>

    <e>
        Ø e
        Ø e+
        Ø e-
        Ø E
        Ø E+
        Ø E-

See www.json.org for more details.

## Examples

This is a JSON array containing two objects:

       [
           {
              "Precision": "zip",
              "Latitude":  37.7668,
              "Longitude": -122.3959,
              "City":      "SAN FRANCISCO",
              "State":     "CA",
              "Zip":       "94107",
              "Country":   "US"
           },
           {
              "Precision": "zip",
              "Latitude":  37.371991,
              "Longitude": -122.026020,
              "City":      "SUNNYVALE",
              "State":     "CA",
              "Zip":       "94085",
              "Country":   "US"
           }
       ]

# Locator Service

Locator Service uses transport layer to search for peers and to collect data about peer’s attributes and capabilities (services). Discovery mechanism depends on transport protocol and is part of that protocol handler. Targets, known by other hosts, are added to local list of peers. Security? Automatically discovered targets require no further configuration. Additional targets can be configured manually.

All TCF peers must implement Locator service. The implementation is part of the framework itself. It is the only required service, all other services are optional and, formally, not part of the framework.

## Peer Atributes

<object: peer data> is collection of peer attributes. It should, at least, contain member "ID" : <string>. It can also contain a number of components describing peer properties and capabilities. Predefined attributes are:

- `"ID" : <string>` - ID of the peer.
- `"Name" : <string>` - human readable peer name.
- `"OSName" : <string>` - peer OS name, if applicable.
- `"TransportName" : <string>` - name of a trasport protocol to use to connect to this peer, for example: TCP.
- `"Host" : <string>` - peer host name, if transport is TCP or UDP.
- `"Aliases" : <string>` - peer host name aliases, if transport is TCP or UDP.
- `"Addresses" : <string>` - peer IP addresses, if transport is TCP or UDP.
- `"Port" : <string>` - peer port number, if transport is TCP or UDP.

Most clients dont need to know peer attributes other then ID and Name. Clients are expected to call IPeer.openChannel() method and let the framework to check peers attributes and create appropriate communication cahnnel that is best suited for communication with the peer. After a channel is established, a client can learn peer capabilities by looking at services it implements (use IChannel.getRemoteServices() method to get a map of services).

## Locator Service Commands

### redirect

    C • <token> • Locator • redirect • <string: peer ID> •

The command redirects the channel to become connected to given peer. Locator service starts acting as a proxy.

Reply:

    R • <token> • <error report> •

### sync

    C • <token> • Locator • sync •

Sync command does nothing and simply returns back an empty result. The command is used for cross channel synchronization. Since commands are executed in order they were issued, by waiting for sync result a client makes sure that all commands, that were issued before sync, are fully processed.

Reply:

    R • <token> •

## Locator Service Events

    E • Locator • Hello • <array: service names> •
    E • Locator • peerAdded • <object: peer data> •
    E • Locator • peerChanged • <object: peer data> •
    E • Locator • peerRemoved • <string: peer ID> •

Hello
  is the first message sent by the framework after establishing a communication channel. The message lets other side of the channel to know capabilities of this peer. Message data consists of an array of service names that are provided by the peer. Service names list is a complete and unambiguous declaration of peer's capabilities. To avoid ambiguity, different services (even slightly different, like versions of same service) must have different names. Framework delays all other communications between peers until exchange of Hello messages is complete.
peerAdded
  is sent when the service discovers a new peer.
peerChanged
  is sent when peer attributes change.
peerRemoved
  is sent when the service deletes information about a peer.

## Locator Service API

    /**
     * Base interface for all service interfaces.
     * A client can get list of available services by
     * calling IPeer.getLocalServices or IPeer.getRemoteServives
     */
    public interface IService {

        /**
         * Get unique name of this service.
         */
        String getName();
    }

    /**
     * Both hosts and targets are represented by objects
     * implementing IPeer interface. A peer can act as host or
     * target depending on services it implements.
     * List of currently known peers can be retrieved by
     * calling ILocator.getPeers
     */
    public interface IPeer {

        static final String
            ATTR_ID = "ID",
            ATTR_NAME = "Name",
            ATTR_OS_NAME = "OSName",
            ATTR_TRANSPORT_NAME = "TransportName",
            ATTR_IP_HOST = "Host",
            ATTR_IP_ALIASES = "Aliases",
            ATTR_IP_ADDRESSES = "Addresses",
            ATTR_IP_PORT = "Port";

        /**
         * @return map of peer attributes
         */
        Map<String, String> getAttributes();

        /**
         * @return peer unique ID, same as getAttributes().get(ATTR_ID)
         */
        String getID();

        /**
         * @return peer name, same as getAttributes().get(ATTR_NAME)
         */
        String getName();

        /**
         * Same as getAttributes().get(ATTR_OS_NAME)
         */
        String getOSName();

        /**
         * Same as getAttributes().get(ATTR_TRANSPORT_NAME)
         */
        String getTransportName();

        /**
         * Open channel to communicate with this peer.
         * Note: the channel is not fully open yet when this method returns.
         * It’s state is IChannel.STATE_OPENNING.
         * Protocol.Listener will be called when the channel will be opened or closed.
         */
        IChannel openChannel() throws IOException;
    }

    /**
     * ILocator service uses transport layer to search for peers and to collect data about
     * peer’s attributes and capabilities (services). Discovery mechanism depends on
     * transport protocol and is part of that protocol handler. Targets, known by other
     * hosts, are added to local list of peers.
     * Automatically discovered targets require no further configuration. Additional targets
     * can be configured manually.
     *
     * Clients should use Protocol.getLocator() to obtain local instance of ILocator,
     * then ILocator.getPeers() can be used to get of available peers (hosts and targets).
     */
    public interface ILocator extends IService {

        static final String NAME = "Locator";

        /**
         * Auto-configuration command and response codes.
         */
        static final int
            CONF_REQ_INFO = 1,
            CONF_PEER_INFO = 2;

        /**
         * @return Locator service name: "Locator"
         */
        String getName();

        /**
         * Get map (ID -> IPeer) of available peers (hosts and targets).
         * The method return cached (currently known to the framework) list of peers.
         * The list is updated according to event received from transport layer
         */
        Map<String,IPeer> getPeers();

        /**
         * Redirect this service channel to given peer using this service as a proxy.
         */
        IToken redirect(String peer_id, DoneRedirect done);

        interface DoneRedirect {
            void doneRedirect(IToken token, Exception error);
        }

        /**
         * Call back after TCF messages sent to this target up to this moment are delivered.
         * This method is intended for synchronization of messages
         * across multiple channels.
         *
         * Note: Cross channel synchronization can reduce performance and throughput.
         * Most clients don't need channel synchronization and should not call this method.
         *
         * @param done will be executed by dispatch thread after communication
         * messages are delivered to corresponding targets.
         *
         * This is internal API, TCF clients should use {@code org.eclipse.tm.tcf.protocol.Protocol}.
         */
        IToken sync(DoneSync done);

        interface DoneSync {
            void doneSync(IToken token);
        }

        /**
         * Add a listener for locator service events.
         */
        void addListener(Listener listener);

        /**
         * Remove a listener for locator service events.
         */
        void removeListener(Listener listener);

        interface Listener {
            void peerAdded(IPeer peer);

            void peerRemoved(IPeer peer);

            void peerChanged(IPeer peer);
        }
    }