Skip to content

StandaloneDccl

Jump to bottom
Toby Schneider edited this page May 26, 2016 · 3 revisions

discussion for the blueprint https://blueprints.launchpad.net/goby/+spec/dccl-breakout-api

Notes for making DCCL separate library from Goby and having Goby use it as a dependency

  1. split and rename projects to 3.0. done: see https://launchpad.net/dccl

  2. logging Requirements:

    1. Looks like or is a std::ostream done, is a std::ostream
    2. Can pass in custom class for where output goes can pass function object for various verbosities
    3. Supports "low cost" switch (no recompilation) between lots of debug and no debug output (minimal runtime overhead). is() function with short-circuiting operator &&.
    4. Optional thread safety with minimal overhead

  3. Run hooks stuff: Useful set of hooks: no hooks

  4. singleton / normal class for DCCLCodec: use normal class.

  5. wiki (still part of Goby wiki? http://libdccl.org/wiki)

  6. webpage (gobysoft.org/dccl? http://libdccl.org):

  7. documentation (doxygen and/or other?)

Ugly API Draft 0

namespace dccl {
 // Interface TBD - derived from FlexOstream
 class Logger : public std::ostream {
 public:
     Logger();
     ~Logger();
 };
 // TypeRegistry wraps/replaces the functionality of DynamicProtobufManager and
 // FieldCodecManager and the like. Perhaps they still exist, but this is the
 // public API.
 class TypeRegistry {
 public:
     TypeRegistry(Logger& logger);
     ~TypeRegistry();
     // Pass dccl_load() a pointer to this type registry, not a codec.
     void load_shared_library_codecs(void* dl_handle);
     template<typename FieldCodecType>
     void register_field_codec(const std::string& name);
     template<typename ProtobufMessage> void register_message();
     void register_message(const google::protobuf::Descriptor& desc);
     template<typename ProtobufMessage> void info(std::ostream& os) const;
     void info(const google::protobuf::Descriptor& desc, std::ostream& os) const;
     void info_all(std::ostream& os) const;
 };
 class MessageCodec {
 public:
     MessageCodec(TypeRegistry&);
     ~MessageCodec();
     // ID Codec is any field codec in the type registry with type uint32.
     configure(Logger& logger, const TypeRegistry& reg,
               const std::string& idCodec);
     // If crypto string is empty, don't use encryption.
     // There are some limitations to DCCL Crypto and support may change
     // in the future? This is a reasonable place to start.
     void set_crypto_passphrase(const std::string& passphrase);
     // Return the message size in bytes
     unsigned size(const google::protobuf::Message& msg);
     void encode(std::string* bytes, const google::protobuf::Message& msg);
     void decode(const std::string& bytes, google::protobuf::Message* msg);
     template <typename ProtobufMessage> unsigned id() const;
     unsigned id(const google::protobuf::Descriptor* desc) const;
     unsigned id(const std::string& encoded_bytes);
 };
}

TES (10/1/12)

  • MessageCodec looks good, in general. I'm not sold on discarding the DCCLConfig protobuf message, but would consider having a simpler overload for configure (like you have above) without it.

  • As a point of style, I prefer to use pointers instead of non-const reference. This makes it clear from reading the calling code that your object is going to be modified. See encode / decode methods above.

  • TypeRegistry needs a way to load a *.proto file directly. I think DynamicProtobufManager and FieldCodecManager are still separate concepts. Perhaps rather than having TypeRegistry at all, we put the reduced (simplified) loading of messages & codecs in MessageCodec?

  • if we're getting rid of "decode_repeated", decode needs a way to be called recursively, either by returning the remaining part of the unused byte string or taking a string pointer and removing the used bytes. That is,

    std::string decode(const std::string& bytes, google::protobuf::Message* msg); // returns remainder
// or
void decode(std::string* bytes, google::protobuf::Message* msg); // "eats" the part of `bytes` used to decode

CAM (10/5/12)

  • Pointers v. non-const reference is fine - I'll probably forget, but I will try to conform (and have no problem with it). I just try to be good about const vs. non-const references to indicate mutability.
  • I think we need an encode/decode repeated on further reflection, though I'd love to just use a std::list<Message*> rather than the templating stuff. I understand why you're doing it (smart pointers) so we can talk about this.
  • I like the idea of a simplified set of API calls for loading messages/codecs, but I don't know if it's practical. If you have two MessageCodec's, how do they share the set of field codecs? Are they static? Do you register once per instance? Neither of those seemed palatable based on our previous discussions, but I'm all ears if you have an idea.
  • I took another stab at a "two-registry" solution - ProtobufRegistry may just be DPM and that's fine. I'm not stuck on names at all.
class FieldCodecRegistry {
public:
    FieldCodecRegistry(Logger& logger);
    ~FieldCodecRegistry();
    void load_shared_library(const std::string &shared_lib_path);
    template<typename FieldCodecType>
    void register_codec(const std::string& name);
};

class ProtobufRegistry {
public:
    ProtobufRegistry() { }
    ~ProtobufRegistry() { }
    gp::Descriptor* find_descriptor(const std::string &protobuf_type_name);
    Message* new_message(const std::string &protobuf_type_name);
    Message* new_message(const google::protobuf::Descriptor *desc);
    void load_file(const std::string &file_path);
    void load_path(const std::string &path);
    void load_shared_library(const std::string &shared_lib_path);
    void register_message(const google::protobuf::Descriptor& desc);
    template<typename ProtobufMessage> void register_message() {
        register_message(ProtobufMessage::descriptor());
    }
    void info_all(std::ostream& os) const;
    void info(const std::string& name, std::ostream& os) const;
    void info(const gp::Descriptor& desc, std::ostream& os) const;
    template<typename ProtobufMessage> void info(std::ostream& os) const {
        info(ProtobufMessage::descriptor(log));
    }
};

TES (10/8/12)

  • On the subject of templates and pointers: perhaps we make two API classes, a basic API (e.g. MessageBasicCodec) that inherits from the advanced API (e.g. MessageCodec). The advanced API can have calls like

    template<typename GoogleProtobufMessagePointer>
    void encode_repeated(std::string* encoded, const std::list<GoogleProtobufMessagePointer>& msgs)

    whereas the basic API uses :

    void encode_repeated(std::string* encoded, const std::list<google::protobuf::Message*>& msgs)
{ MessageCodec::encode_repeated(encoded, msgs); }

  • I think we should think of the reasons one would use more than one MessageCodec. The one I can think of is using one for Micro-Modem mini-packets and one for normal data (using different ID codecs). In this case I don't think they would share any of the same messages, and thus each MessageCodec could have its own FieldCodecRegistry and ProtobufRegistry.

Logging

I think we can make this pretty simple by defining just our own custom streambuf, and doing away with FlexOstream and the various manipulators/color stuff completely. I left the name FlexOstreamBuf in there so you know what I'm talking about but I think we do away with that name too (maybe alternatively use DCCLLogBuffer?):

// .h
namespace dccl {
  extern FlexOStreamBuf dlogbuf;
  extern std::ostream dlog;
  bool log_is(Verbosity verbosity)
  {
    if(!dlogbuf.contains(verbosity))
    {       
       return false;
    }
    else
    {    
       dlogbuf.set_verbosity(verbosity);   
       return true;
    }
  }

  namespace logger
  {
     enum Verbosity {
           NONE = 0,
           WARN = 0x01,
           INFO = 0x02,
           DEBUG1 = 0x04,
           DEBUG2 = 0x08,
           DEBUG3 = 0x10,
           ALL = WARN | INFO | DEBUG1 | DEBUG2 | DEBUG3
     };
  }
  class FlexOStreamBuf : public std::streambuf
  {
          public:
            FlexOStreamBuf();
            ~FlexOStreamBuf();
            /// virtual inherited from std::streambuf.
            /// Called when std::endl or std::flush is inserted into the stream
            int sync();
            /// virtual inherited from std::streambuf. Called when something is inserted into the stream
            /// Called when std::endl or std::flush is inserted into the stream
            int overflow(int c = EOF);
            /// connect a signal to a slot (function pointer or similar)
            template<typename Slot>
               void connect(int verbosity_mask, Slot slot)
            {
               enabled_verbosities_ |= verbosity_mask;
               if(verbosity_mask & WARN)
                    warn_signal.connect(slot);
               if(verbosity_mask & INFO)
                    info_signal.connect(slot);
               // ...
            }
            /// connect a signal to a member function
            template<typename Obj, typename A1, typename A2>
                void connect(int verbosity_mask, Obj* obj, void(Obj::*mem_func)(A1, A2))
            { connect(verbosity_mask, boost::bind(mem_func, obj, _1, _2)); }

            /// sets the verbosity level until the next sync()
            void set_verbosity(Verbosity verbosity)
            { current_verbosity_ = verbosity; }

            bool contains(Verbosity verbosity)
            { return verbosity & enabled_verbosities_; }
          private:
            void display(const std::string& s);

          private:
            int enabled_verbosities_; // mask of verbosity settings enabled
            boost::signals2::signal<void(const std::string& log_string, Verbosity verbosity)> warn_signal;
            boost::signals2::signal<void(const std::string& log_string, Verbosity verbosity)> info_signal;
            boost::signals2::signal<void(const std::string& log_string, Verbosity verbosity)> debug1_signal;
            boost::signals2::signal<void(const std::string& log_string, Verbosity verbosity)> debug2_signal;
            boost::signals2::signal<void(const std::string& log_string, Verbosity verbosity)> debug3_signal;
     };    
}

// .cpp
dccl::FlexOStreamBuf dccl::dlogbuf;
std::ostream dccl::dlog(&dccl::dlogbuf);
// sync() and overflow() same as before
// display becomes
void dccl::FlexOStreamBuf::display(const std:string& s)
{
  if(current_verbosity_ & WARN)
    warn_signal.connect(slot);
  if(current_verbosity_ & INFO)
    info_signal.connect(slot);
  // ...
}

// DCCL library internal usage
using dccl::log_is;
using dccl::dlog;
using namespace dccl::logger;
// this works because dlog inherits from std::ios which defines operator void*() which allows implicit bool cast
// and C++ is short-circuiting on the &&
log_is(WARN) && dlog << "some warning" << std::endl;

// DCCL library user usage
void log_warnings(const std::string& log_message, dccl::logger::Verbosity verbosity)
{
   std::cerr << log_message << std::endl;
}
void log_info(const std::string& log_message, dccl::logger::Verbosity verbosity)
{
   printf("%s", log_message.c_str());
}
dlogbuf.connect(dccl::logger::WARN, &log_warnings);
dlogbuf.connect(dccl::logger::INFO | dccl::logger::DEBUG1, &log_info);

I'd probably also like to add convenience functions for also logging straight to standard ostreams (std::cout, std::cerr, etc.)

Clone this wiki locally