Skip to content

PostCaptureMerge

Toby Schneider edited this page May 26, 2016 · 6 revisions

Post CAPTURE 2011 changes merge (mostly all goby-acomms)

This is a rather long merge with the changes warranted after the CAPTURE 2011 cruise. Since 2.0 is in pre-release stage, hopefully the many breaking API changes will be forgiven. The various acomms APIs are getting close to what I'd like to see for release, so please get your feedback to be soon.

I have tried to summarize the salient points at the beginning with detail below.

Overview of changes

  • Structural
    • All libraries under goby/2.0/src/acomms are compiled into a single libgoby_acomms.so (or .a) instead of libgoby_dccl.so, etc.
    • All libraries under goby/2.0/src/util are compiled into a single libgoby_util.so (or .a) instead of libgoby_logger.so, etc.
    • Directories named "lib*" are now just "*". For example, goby/src/acomms/libdccl => goby/src/acomms/dccl
    • All Goby custom Google Protobuf option extensions are now a single global message defined in goby/src/protobuf/option_extensions.proto. This is a Google-assigned id (1009) to be compatibility with other groups using custom option extensions. This causes some syntax changes (see below).
    • All Protobuf messages prefixed by the Goby module to which they belong. For example, modem_message.proto => acomms_modem_message.proto.
    • Unit tests moved into a separate parallel tree (goby/src/test) instead of embedded with libraries (e.g. goby/src/acomms/dccl/test/dccl1 => goby/src/test/acomms/dccl1)
    • Tools / applications moved into a separate parallel tree (goby/src/apps) instead of embedded with libraries (e.g. goby/src/util/tools/serial2tcp_server => goby/src/apps/util/serial2tcp_server). Tests will not be built unless enabled (cmake . -Denabled_testing=ON)
  • Modemdriver: see new_mm_driver_test_log1.txt and new_mm_driver_test_log2.txt for a run log of the new unit test (test/mm_driver1) that gives a nice overview of the major functionality and changes.
    • Protobuf messages heavily reworked. Basically there is just one message now (ModemTransmission) that takes the place of the old ModemDataTransmission, ModemRangingRequest, ModemRangingReply, ModemDataInit, and ModemDataRequest. The message can be extended by various drivers (e.g. MMDriver) to add driver specific functionality (such as LBL pings, diagnostics information, etc.).
    • Support for Narrowband LBL ranging ($CCPNT)
    • Support for User mini-packet ($CCMUC)
    • Support for CST statistics message in place of $CATOA (plus much more information). CACST is set on by default unless the user explicitly requests CST,0. The MMDriver send / receive logic has been reworked to flush messages upon receipt of $CACST (unless disabled), causing a signal_receive for all acoustic events. This is especially useful for folks doing OWTT with synchronous clocks.
    • Support for XST Transmit statistic message via a new signal (signal_transmit_result)
    • Added unit test for MMDriver (src/test/mmdriver1) that runs two modem tests (requires modem test box or two modems in a bucket) for all 5 major modem transmission types.
  • AMAC
    • More or less completely rewritten to be a subclass of a std::list<ModemTransmission>. If you think about what a TDMA cycle is, it's a bunch of Transmissions plus a timer (thanks for this insight Chris).
    • New custom option extension of ModemTransmission (the new "Slot") allowing for a unique id to be assigned to a given slot. This lets various slots be identified for specific data payload purposes. The extension is goby.acomms.protobuf.unique_id defined in acomms_amac.proto.
    • Removed MAC_AUTO_DECENTRALIZED mode until I can further test its proper functioning.
  • Queue
    • Modified to work properly with changes to modemdriver
  • DCCL
    • Ability to use a custom DCCL ID codec instead of the default (1 or 2 byte self-delimiting integer) for a given message or messages. This was handy for encoding Micro-Modem mini-packets using a custom 3 bit DCCL ID instead of the default.
    • Encode / decode API changed slightly to allow embedded messages to be decoded without copying.
  • as family of conversion functions
    • string.h renamed as.h
    • as specialized to work correctly with enumerations.
  • Time
    • All Protobuf messages using uint64 microseconds since UNIX as the time format.
    • goby_time() has a template form now that understands: goby_time<uint64>() (microseconds since UNIX), goby_time<double>() (seconds and fractional seconds since UNIX), goby_time<boost::posix_time::ptime>(). It also still has the non-template form goby_time() that returns boost::posix_time::ptime as before.
    • goby::util::as<boost::posix_time::ptime>() understands how to convert from uint64 (assuming microseconds since UNIX) and double (assuming seconds since UNIX). Similarly, goby::util::as<double> and goby::util::as<uint64> work as expected for a boost::posix_time::ptime.

Details / motivation of specific changes

AMAC / MACManager rewrite

  • Use MACManager as a std::list, that's what it is now. So, use the std::list insert, push_back, delete, etc. to manipulate "slots". This makes the MACManager API much cleaner. To keep MACManager happy, one needs to call a new method "update()" after changing any slot parameters. (Technically only if the changes invalid std::list iterators, but it's best to be safe and call it after any slot changes.
  • The "Slot" Protobuf object has thus been replaced by ModemTransmission (defined in goby/src/protobuf/acomms_modem_message.proto), with a custom option extension (goby.acomms.protobuf.slot_seconds, defined in acomms_amac.proto) allowing for the length of a slot. This of course means that MACManager can initiate anything that the given modem can transmit. For the WHOI micro-modem this means $CCCYC (DATA), $CCMPC (MICROMODEM_TWO_WAY_PING), $CCMUC (MICROMODEM_MINI_DATA), $CCPDT (MICROMODEM_REMUS_LBL_RANGING), $CCPNT (MICROMODEM_NARROWBAND_LBL_RANGING).
  • The MAC_AUTO_DECENTRALIZED peer discovery mode will likely reappear at some point as a layer built above MACManager ("DynamicMAC"?). Third-party polling (MAC_POLLED) and "fixed" decentralized ("plain-old-TDMA") are still implemented. Since MACManager is just a fancy std::list with a timer now, it's very easy to manipulate slots.

See unit tests (goby/2.0/src/test/acomms/amac*) and examples (goby/2.0/share/examples/acomms/amac/*) to see the new changes in action.

Global option extension for Goby

Reworked dccl and queue options extensions to use a single embedded message field for each type of option extension (DCCLFieldOptions / QueueFieldOptions for field option extensions, DCCLMessageOptions/QueueMessageOptions for message option extensions).

This reduces the number of option extensions taken by Goby to one, instead of a potentially large number and makes only minor syntactical changes. This allows Goby to "play nicely" with other users of Google Protocol Buffers option extensions (see discussion under "Custom Options" at http://code.google.com/apis/protocolbuffers/docs/proto.html#options).

I have registered with Google and received extension 1009 for use with all types

Message using old version:

import "goby/protobuf/option_extensions.proto";

message Simple
{
  option (dccl.id) = 1;
  option (dccl.max_bytes) = 32;

  required string telegram = 1 [(dccl.max_length)=10];
}

Message using new syntax (proposed merge)

import "goby/protobuf/option_extensions.proto";

message Simple
{
  option (goby.msg).dccl.id = 1;
  option (goby.msg).dccl.max_bytes = 32;

  required string telegram = 1 [(goby.field).dccl.max_length=10];
}

In the newer versions of Protobuf (2.4+), you can write this simpler

import "goby/protobuf/option_extensions.proto";

message Simple
{
  option (goby.msg) = { dccl { id: 1  max_bytes: 32 } };
  required string telegram = 1 [(goby.field).dccl.max_length=10];
}