Watch 2

Star 1

C++ 97.6%

Makefile 1.4%

C 0.6%

Shell 0.4%

HTTPS

Download ZIP Download TAR.GZ Download BUNDLE Open with VS Code Open with VSCodium Open with Intellij IDEA

Please don't upload to GitHub

json2ccsds

Version 2025年10月04日 15:45 CEST

Watch filesystem (inotify) events for changes to a set of CSV files, and convert these into CCSDS telemetry packets, using a JSON description of the CSV format. Send the CCSDS encoded telemetry to one of two configurable destinations, based on AOS/LOS status.

Notes

(none)

Latest changes

For older changes, please refer to the historical change log below.

2025年10月04日 lookup list decoders transcoding support

• increased json2ccsds version to 1.0
• increased json2mdb version to 0.4
• JSON TM description format: lookup list decoder value list now supports a "purpose" attribute:
  • "purpose":"all" (default) value will be used by json2ccsds for encoding TM and by json2mdb for a calibration definition
  • "purpose":"encode" value will be excluded from json2mdb calibration definition
  • "purpose":"decode" value will be excluded from json2ccsds encoding lookup

2025年09月23日 json2ccsds formatBandwidthStatistics: thread and memory safety patches

• increased json2ccsds version to 0.9
• bandwidthTools formatBandwidthStatistics: made tempString non-static to avoid multithreading conflict
• bandwidthTools formatBandwidthStatistics: prevent buffer overrun by use of snprintf
• ipv6_utilities ipv4FromString: replaced deprecated use of gethostbyname with getaddrinfo
• updated Makefile to support use of pkg-config for newer versions of OpenXLSX that have sub-dependencies. Relevant settings:
  • set USE_PKG_CONFIG=no (default) when include directories shall be configured per json2ccsds Makefile
  • set USE_PKG_CONFIG=yes if OpenXLSX include directories and libs shall be determined by pkg-config
  • set LINK_OPENXLSX_DEPENDENCIES=no (default) when linking against OpenXLSX static library (including dependencies)
  • set LINK_OPENXLSX_DEPENDENCIES=yes if libpugixml and libzip shall be linked directly

2025年08月06日 json2ccsds and tcp-server-bridge: major bugfixes in IPv4 fallback implementation

• To fix a couple of remaining mismatches between IPv4 and IPv6 structures, remaining network functions that work with IP version sensitive structures have been changed to work with a SocketConfigurationIpv6 instead:
  • int openAndConnectIpv4TcpSocket( struct SocketConfigurationIpv6 *sockConf, uint32_t timeoutMS = 0, bool silent = false )
  • int openAndConnectIpv4Socket( bool tcp, struct SocketConfigurationIpv6 *sockConf, int sockPurpose, uint32_t timeoutMS = 0, bool silent = false )
  • int openAndConnectIpv6TcpSocket( struct SocketConfigurationIpv6 *sockConf, uint32_t timeoutMS = 0, bool silent = false )
  • int openAndConnectIpv6Socket( bool tcp, struct SocketConfigurationIpv6 *sockConf, int sockPurpose, uint32_t timeoutMS = 0, bool silent = false )
• Accordingly, functions in json2ccsds that were invoking these functions had to be modified. This only concerns the SendBuffer class, which now establishes a SocketConfigurationIpv6 according to the IPv6 vs. IPv4 situation in the constructor, and proceeds to use that in the sender_thread
• BUGFIX SendBuffer constructor: upon failure, throw exception before spawning sender_thread
• json2ccsds: wrapped an additional exception handler around SendBuffer instantiation for a cleaner shutdown
• setupIpv4UdpSocket and setupIpv6UdpSocket no longer overwrite SocketConfigurationIpv6 address information for sockPurpose == UDP_SEND, as the address information is expected to be used for the sendto-destination
• openAndConnectIpv4Socket and openAndConnectIpv6Socket now consistently return -1 on failure, in line with other functions

2025年07月29日 json2ccsds, json2mdb, tcp-server-bridge, slowRead: bugfix in logging framework IPv4 fallback (testIpv6Sockets())

• when DISABLE_IP6 was set, testIpv6Sockets() would issue another log message which triggered the old bug that a message is issued while trying to initialize the messaging framework. Removed this message.
• added function activateIpv4Fallback() to permanently (for the runtime) force fallback of all IPv6 functions to IPv4
• added function ipv4FallbackIsActive() to check whether activateIpv4Fallback() has been used
• added function notifyAboutIpv4Fallback() to emit a log message exactly once about IPv4 fallback being in use

2025年07月24日 json2ccsds IPv4 fallback

• increased json2ccsds version to 0.8
• added support for fallback to IPv4 in case IPv6 sockets are not available on the host OS

2025年07月24日 tcp-server-bridge IPv4 fallback

• increased tcp-server-bridge version to 0.2
• added support for fallback to IPv4 in case IPv6 sockets are not available on the host OS
• added an outer exception handler in the main routine

2025年07月24日 json2ccsds & json2mdb & slowRead & tcp-server-bridge: logging framework bugfix

• logging framework: bugfix for a deadlock on logging mutex during initialization
• increased json2mdb version to 0.3
• increased slowRead version to 0.3

2025年07月24日 Makefile

• added option FORCE_IP4=no (can be set to FORCE_IP4=yes to test the IPv4 fallback functionality)

2025年04月22日 yamcs-displays folder

• housekeeping.par: added missing low-floats2--2 through low-floats2--7

TODO

json2ccsds

N/A (feature-complete)

json2mdb

N/A (feature-complete)

TODO TBD (low priority)

• json2shared.cpp loadDecoders: add exception for valueListPurposeTag == valueListCommentTag

json2ccsds

• on parse error, discard 1 more line, and re-try

Dependencies

• Building json2mdb requires
  • boost::program_options (e.g. libboost-program-options1.83-dev on debian testing / trixie)
  • the OpenXLSX library sources, and Makefile OPENXLSX_PATH to be set correctly to the OpenXLSX sources
• Converting the .xlsx telemetry database generated by json2mdb to an .xls Excel 97 Workbook requires either LibreOffice or MS Office installed and needs to be done as a manual step
• the telemetry generated by json2ccsds can be decoded by Yamcs using the telemetry database generated by json2mdb
• installing Yamcs with the "Getting Started" instructions requires:
  • a working installation of maven - Java software project management and comprehension tool (specifically, the mvn command invokes maven)
  • openjdk-##-jdk (e.g. openjdk-21-jdk on debian testing / trixie) - otherwise the maven build will fail

Why use it?

to be written

License

Copyright 2022 Lars Uffmann

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Scope

This repository contains

• json2ccsds telemetry encoding & streaming application. This application is intended for use as flight software on a spacecraft scientific payload.
• json2mdb application that generates an XLSX spreadsheet telemetry database for Yamcs. This application is only needed to prepare / update the mission database when the JSON definition changes.
• slowRead application that will read lines from an existing comma-separated-values (CSV) file and print them to stdout, which can be redirected to a file that json2ccsds is monitoring, to simulate live telemetry. This application is only needed for validation testing.

Purpose

This software trivializes the task of generating CCSDS compliant telemetry from a scientific payload that stores telemetry as CSV files.

Terminology

Term	Explanation
JSON	JavaScript Object Notation
CSV	comma-separated values
AOS	acquisition of signal (= live link)
LOS	loss of signal (= no link, data buffering needed)
TBW	to be written (this list is ...)

How to use

Creating a mission database and configuring Yamcs

• create a working JSON definition that describes your CSV format(s) - one packet definition per CSV file format is required (user manual with format description will follow eventually - for now refer to the example definitions in the data/ folder)
• create an XLSX mission database by running ./json2mdb -d path/to/your-tm-definition.json
• open the generated .xlsx files in an office application
• save them in Excel 97-2003 (.xls) format
• copy these new (.xls) files into your Yamcs MDB folder
• modify the mdb section of your Yamcs project yaml configuration to the following settings (replace your-tm-definition.xls per your configuration):

mdb:
 # Configuration of the active loaders
 # Valid loaders are: sheet, xtce or fully qualified name of the class
 - type: "sheet"
 spec: "mdb/ccsds-headers.xls"
 subLoaders:
 - type: "sheet"
 spec: "mdb/your-tm-definition.xls"

• (optional) if you have an existing, compatible ccsds-headers definition, you can just add the last two lines for a new sub loader
• (optional) delete from the yaml configuration file the default entry

  - type: xtce
   args:
   file: mdb/xtce.xml
  

Notes for the Yamcs quickstart setup

• the Yamcs MDB folder is the repository subfolder target/yamcs/mdb/
• the Yamcs project yaml configuration is stored in src/main/yamcs/etc/yamcs.myproject.yaml

Starting Yamcs and monitoring telemetry

• after saving the config file, start Yamcs with mvn yamcs:run
• open a web browser and navigate to (http://localhost:8090)
• click on myproject -> Mission database --> Parameters and verify that your defined parameters show up there (as per the .xls spreadsheet)
• edit json2ccsds.conf to contain the desired default settings (specifically, destination-ip and destination-ip-LOS)
• start json2ccsds with the destination port towards the default incoming TM port of Yamcs: ./json2ccsds -p 10015
• start writing one of your CSV files that contain the described format to the location where you configured json2ccsds to look for updates:
  • Example: ./slowRead source-data/housekeeping-sample.csv > telemetry/housekeeping-tm.csv
• in the Yamcs web browser window, navigate to Telemetry -> Packets and verify that you have incoming data

You can now navigate to Telemetry -> Displays and generate simple telemetry displays based on your incoming data. Congratulations, you have a working setup.

Default (test) displays

You can find displays for the sample telemetry-description.json in the folder yamcs-displays, those displays can be uploaded into a running Yamcs web interface. Playback commands for this sample telemetry are available in play-syslog.sh and play-housekeeping.sh.

Additional remarks

Please note that - for the time being - playback of a file that has no string delimiters (in this example the syslog is interpreted by a regular expression), requires the slowRead application to be used with the --max-lines 1 setting to force it to write to stdout each line without trying to understand string delimiters.

How it works

to be written

Contact

You can reach me for questions at Contact information

Historical change log

2025年04月22日 json2ccsds & json2mdb

• json2shared: moved exception for missing document-header to be thrown before missing packets node

2025年04月22日 slowRead

• renamed outfile -> infile in help output for usage

2025年04月17日 json2mdb

• stringToLongDouble updated for hex numbers to try converting to unsigned if signed conversion throws std::out_of_range

2025年04月17日 json2ccsds

• stringToLongDouble updated for hex numbers to try converting to unsigned if signed conversion throws std::out_of_range
• protected tailCSVLines against a race when source data is written faster than the function can keep up - will give up eventually and accept current read position
• protected handleAction for ACTION_READMORE against a race when source data is written faster than json2ccsds can keep up - this now triggers skipping to end of source data file
• modified initial read of existing data source to trigger an IN_MODIFY event for inotify_action_thread, instead of directly processing & potentially entering a race with a fast writer

2025年04月17日 slowRead, tcp-server-bridge and inotifyHandlerTest

• no changes, but dependencies (utilities, inotifyHandler) were modified, requiring a new version date

2025年04月16日 json2ccsds & json2mdb

• all modules: added function header comments, some tabs-to-spaces reformatting

2025年04月13日 json2ccsds & json2mdb

• json2shared minor BUGFIX: initialize ZOE_BIT_OFFSET to -1 (was 0, which in practice did not trigger a bug)
• encoder module: removed using std::string from header file, replaced use of string with std::string across module
• encoder node::parseCSVFromVector and node::encodeCCSDS: for the instance string (used in fieldInstanceName for logging), implemented wrapping when parent already passes down a non-empty instance string (from [04] to [0,4])

2025年04月11日 json2ccsds & json2mdb

• config: removed default value for input-description

2025年04月07日 json2ccsds & json2mdb

• json2shared loadDecoders: added a check for duplicate keys, aligned some log messages

2025年04月07日 generic: data/telemetry-description.json

• described motor1State and motor2State fields with lookup
• corrected repeat of low-floats2 from 7 to 8

2025年04月04日 json2ccsds & json2mdb

• aligned parameter checks and error messages while working on validation testing

2025年04月04日 tcp-server-bridge

• changed default ports to 10017 and 10018

2025年04月03日 json2ccsds & json2mdb

• recurseJSON downwards compatibility with older jsoncpp library versions: do not use find( std::string ), instead call find( char const *, char const * )

2025年04月03日 json2ccsds

• permit disabling syslog by config file even when syslog-port is configured

2025年04月02日 json2ccsds & json2mdb

• changed all occurrences of Yamcs to correct spelling (upper-case first letter)

2025年03月31日 json2mdb

• re-enabled configuration errors on unknown command line parameters

2025年03月31日 json2ccsds & json2mdb

• added an error message output to loadConfigFromFile for unrecognized options
• renamed parameter input-definition to input-description to reflect purpose of JSON
• cleaned up json2shared module use of string literals and namespace std
• aligned comments to reflect the naming of JSON as "telemetry description" (except for CCSDS headers, which is considered a definition because it describes exact fields)

2025年03月29日 json2ccsds

• re-enabled an EventQueue overflow warning

2025年03月29日 json2ccsds & json2mdb

• aligned log and error messages in shared modules

2025年03月29日 generic

• aligned log / error messages
• ipv6_utilities: renamed configureReceiveBuffer -> configureReceiveBufferSize

2025年03月28日 json2ccsds

• encoder storeVariantValue: added a check that a valid ISO date could be encoded
• added AppName to SendBuffer log & error messages

2025年03月28日 json2ccsds & json2mdb

• json2shared getPacketInfo: added ranges checks on apid and packet-id, replaced atoi with stoi (because atoi returns 0 on error)
• encoder: changed a non-string decoder property from a warning to an error

2025年03月28日 slowRead

• moved remaining status / log messages to cerr to prevent interfering with data output on stdout

2025年03月26日 json2ccsds

• editorials on log/error messages
• SendBuffer: only attempt reconnects upon broken connection when buffer has data
• SendBuffer: limit reconnect (socket reinitialization) to TCP protocol
• pushLinkStatusNotification BUGFIX: added re-arming of ONLINE and OFFLINE notifications

2025年03月24日 json2ccsds

• consistent using declaration of namespace std::literals::string_literals, use of string literal for concatenations
• fixed an invocation of a nominal log message with logError and a warning message with logMessage

2025年03月24日 json2mdb

• consistent using declaration of namespace std::literals::string_literals, use of string literal for concatenations
• fixed an exception message (removed redundant AppName)

2025年03月24日 slowRead

• changed a nominal log message from cerr to cout
• added initialization of logging framework for module functions that use it

2025年03月24日 tcp-server-bridge

• code cleanup - removed disabled logging messages

2025年03月24日 inotifyHandlerTest

• added initialization of logging framework for module functions that use it

2025年03月21日 json2ccsds & json2mdb

• when TM description uses regex, permit configuration of delimiter, escape character and string qualifier, which will be inherited by subfields
• when TM description uses regex, ensure use of default delimiter, escape character and string qualifier for subfields
• added a check on change log entry version field to begin with a decimal digit because Yamcs requires this

2025年03月21日 json2mdb

• auto-generate a single change log entry if telemetry-description doesn't contain a change log

2025年03月19日 json2mdb

• added JSON change log to resulting MDB

2025年03月19日 json2ccsds & json2mdb

• recurseJSON now takes as first parameter a bool ccsdsHeaders to tell it whether it is parsing a CCSDS headers definition (true) or a TM description (false)
• added JSON specification versioning

2025年03月19日 generic

• readISODate BUGFIX: only correct for the timezone after validity checks against date components read from string
• readISODate BUGFIX: timeOffset must be substracted from timestamp, not added (+01:00 means 1 hour ahead of UTC)

2025年03月18日 json2mdb

• enhanced functionality of timestampString to include timezone information
• added author, creation & modification date and description document properties to created XLSX files

2025年03月18日 json2ccsds and json2mdb

• allow configuration of timestamp epoch for ISO date fields with the "epoch" property (GPS, POSIX), defaults to GPS

2025年03月18日 json2ccsds and slowRead: drop carriage return at end of line

• csvTools getCSVRecord: auto-remove trailing carriage return from input lines for compatibility with DOS-like text files

2025年03月18日 generic

• data/telemetry-description.json: changed syslog-time field definition to newly available "epoch":"POSIX"

2025年03月17日 inotifyHandlerTest

• added versioning and trivial help output

2025年03月14日 json2ccsds and json2mdb

• renamed tmDefinitionJSON to tmDescriptionJSON everywhere, and telemetry-definition.json to telemetry-description.json in default parameter
• replaced remaining cout / cerr calls with logMessage / logError

2025年03月14日 generic

• moved findDelimiter function from utilities module to encoder module

2025年03月13日 json2ccsds

• implemented previously forgotten use of link-check-interval config parameter
• enabled and tested syslog capability via UDP (to syslog and a separate log-port, if desired)

2025年03月12日 json2mdb

• removed help output for --detach option which is not applicable to json2mdb

2025年03月11日 json2ccsds and json2mdb

• utilities module: added isFile function for checking whether a path exists and is a file or symlink
• load CCSDS header definition from ~/.json2ccsds/ccsds.json, otherwise relative to executable location (./ or ./data/) even when launched from another directory
• interpret relative path to input-definition from current working directory (was previously interpreted from application executable directory)
• legacy compatibility for input-definition and JSON packet data-source: support "//" initial characters as wildcard for a path relative to executable path, e.g. "//data/tm.json"
• if in the telemetry definition JSON, a packet's data-source is not provided as an absolute path, then treat it as a path relative to the current working directory

2025年03月11日 slowRead

• replaced use of string with std::string

2025年03月06日 json2ccsds

• modified tailCSVLines to read file from the back (to avoid long processing for large files)
• replaced SIGIO signal handling with use of poll on inotifyQueueFD
• simplified class EventQueue with the understanding that a read on an inotify event queue will never return partial events and that the maximum event size is limited to sizeof( struct inotify_event ) + NAME_MAX + 1, with NAME_MAX being defined in limits.h
• renamed inotifyWatchFD to inotifyQueueFD

2025年03月06日 slowRead

• added license & help output to slowRead application
• main: updated sleep logic for --fast, --faster, --even-faster, added parameter --record-delay

2025年03月05日 generic

• removed using namespace std statement from project header files

2025年03月04日 json2ccsds

• replaced signal handler for SIGIO with a blocked signal and synchronous handling within inotify_action_thread
• inotifyHandler module: created class EventQueue for encapsulation of event extraction from inotify file descriptor raw data
• removed obsolete global "using namespace std" to avoid naming conflicts

2025年02月26日 json2ccsds

• validateJson2CcsdsConfig: added safe-guard against flight-software generating too much traffic - bandwidthCap is limited to MAX_CONFIGURABLE_BW
• LargeFifoBuffer MAJOR BUGFIX: in memory-only mode, LargeFifoBuffer maxPages are linked to cachePageCount (before, FreeSpace() check could succeed, when no new page could be loaded for writing)
• handleAction & los_sender_thread: added / modified tmBuffer corruption checks as a safe-guard for undiscovered bugs
• los_sender_thread BUGFIX: added code to avoid a race condition for a plausibility check on packet data read from tmBuffer
• tested for memory leaks with valgrind - no more leaks found (post bugfix)
• minor BUGFIX main: delete [] for freeing packet.packetBuf to match new [] (was delete)

2025年02月25日 json2ccsds

• SendBuffer: prevent blocking send in case TCP destination goes offline and doesn't respond
• checked dynamicSizing logic when last (string) field repeats -> node::createFixstringWLen disallows dynamic sizing if repeat != 1

2025年02月25日 generic

• requiredDataType: changed check of precision to use valgrind-compatible std::numeric_limits< double >::min() instead of long double
• BUGFIX freeDecoders: timestamp and timestampPrecise params (struct timestamp) were not freed
• utilities: replaced C-style casts with static_cast & reinterpret_cast, replaced NULL with nullptr

2025年02月21日 slowRead

• validateRecord: added tolerance for leading spaces, tabs and a single string qualifier before evaluating first field

2025年02月21日 json2ccsds

• parseCSVFromVector: search for subfields in unquoted rawValue instead of (potentially) decoded value (because decoded timestamp might have unexpected syntax)
• BUGFIX parseCSVFromVector: changed return code of findDelimiter for "not found" or error to std::string::npos (was 0 and bug was inconsequential only due to luck)

2025年02月21日 generic

• reintroduced whitespace as allowed field delimiter (helpful for subfield parsing)

2025年02月20日 json2ccsds

• node::createMDB: implemented logic for fixedPaddingBits
• BUGFIX main: moved inotifyWatchFD initialization back before recurseJSON is called - this bug was introduced in commit on 2025年02月07日

2025年02月25日 json2mdb

• smoke-tested for memory leaks with valgrind - no more leaks found (post bugfix)

2025年02月20日 json2mdb

• writeToSpreadsheet: implemented logic for fixedPaddingBits
• encoder UniqueParameters: added functionality to modify auto-generated (spare) field names to avoid naming conflicts
• writeToSpreadsheet: spare field names now use subfieldNameSeparator, preventing conflicts with user-defined field names

2025年02月20日 generic

• encoder node::widen / node::align: refactored node alignment logic (simplified & bugs fixed)
• added code-snippets/telemetry-with-padding.json as a test case

2025年02月07日 json2ccsds

• encoder createMDB: bugfixes for unique field name generation (previous version failed with exceptions on repeating fields with subfields)
• encoder createMDB: aligned output (field names) with json2mdb
• main: moved SendBuffer (AOS & LOS) creation and inotify_init into scope in which Json2CcsdsConfig.printPacketDefinitions is false

2025年02月07日 json2mdb

• writeToSpreadsheet: bugfixes for unique field name generation (previous version failed with exceptions on repeating fields with subfields)
• upgraded version number to 0.2

2025年02月05日 json2ccsds

• refined bandwidth control, upgraded version number to 0.7

2025年01月27日 json2ccsds

• implemented tcp-server-bridge to allow json2ccsds (as a client) to connect to Yamcs (which also acts as a TCP client)
• verified TCP link functionality - json2ccsds can connect as TCP client to tcp-server-bridge, to which Yamcs connects as a second TCP client on a separate port, and receives good telemetry

2025年01月27日 tcp-server-bridge

• implemented tcp-server-bridge as a new appplication to allow json2ccsds - acting as a TCP client - to connect to Yamcs (which also acts as a TCP client)

2024年12月07日 json2ccsds

• ipv6_utilities: implemented TCP client connect with timeout
• BUGFIX: install SIG_IGN for SIGPIPE to prevent application crash on TCP connection close from remote
• verified TCP link functionality

2024年12月05日 json2ccsds

• implemented demo-dass BW control bugfix (rounding error) for future finer BW control
• discard TM packets when bandwidth cap is exceeded (separate send buffers for AOS and LOS)

2024年11月28日 json2ccsds

• bandwidthTools formatting & replaced some C-style casts with static_cast

2024年11月13日 json2ccsds

• move non-trivial tasks (packet parsing & sending) into a thread & remove them from signal handler sigHandlerInotify
• main: shut down threads before freeing resources (new inotify_action_thread needs the resources to function)

2024年10月23日 json2ccsds

• set ccsds-zone-of-exclusion flag for packets towards LOS destination & recompute checksum
• BUGFIX computeSimpleChecksum: when adding 2-byte values, must compare wordPos + 1 vs. length
• verified CCSDS (simple) checksum algorithm

2024年10月22日 json2ccsds

• enforce unique parameters names across all packets: Yamcs will not separate by packet parameters of identical name CAUTION: for json2ccsds, this only throws an exception if called with --packet-definitions parameter - because encoding valid CCSDS packets does not rely on unique parameter names. For json2mdb, duplicate parameter names always cause an exception.

2024年10月22日 json2mdb

• changed field references in containers condition column accordingly according to modified ccsds packet parameter naming scheme
• enforce unique parameters names across all packets: Yamcs will not separate by packet parameters of identical name

2024年10月22日 generic

• changed data/ccsds.json parameter naming scheme to ccsds-* to avoid conflicts with user-defined parameters

2024年10月18日 json2mdb

• sort DataTypes by bitwidth, then engineering type, then type name

2024年10月17日 json2ccsds

• ipv6 support completed
• started removing using namespace ... code from header files (mostly namespace std and std::literals::string_literals)

2024年10月17日 generic

• removed udpTools and tcpTools modules from project, as they are superseded by the ipv6_utilities module

2024年10月15日 json2mdb

• added system endianness detection for float data types & reflected system endianness in float parameter type definitions
• added a precise timestamp definition with subseconds to timestamp fields (by duplicating subseconds field and using timestamp seconds as offset)

2024年10月15日 json2ccsds

• updated with generic change to utilities module, mentioned above

2024年10月15日 generic

• utilities module: renamed a too-generic macro define INVALID to LOG_LEVEL_INVALID to avoid naming clashes

2024年10月15日 yamcs-displays folder

• added new precise timestamps to syslog.par and housekeeping.par

2024年10月14日 json2ccsds

• cleaned up some logging messages in json2shared module

2024年10月14日 json2mdb

• cleaned up logging, added some instructions for establishing a working Yamcs configuration to --log-level LOG_INFO
• split TM definition into two separate files: ccsds-headers.xls and a file named using the stem of the file indicated by input-definition + .xlsx (e.g. telemetry-definition.xlsx)

2024年10月14日 yamcs-displays folder

• updated yamcs-displays to support new structure of generated spreadsheets (separation into CCSDS headers and TM packet definitions)

2024年10月12日 json2ccsds

• BUGFIX encoder.cpp node constructor from JSON object: ensure that decoders collection is not accessed by decoderName before existence is verified
• using the below exception, fail to parse record if (non null) column value is not found in a configured lookup decoder list
• added a std::range_error exception to fieldDecoders.cpp decodeLookup so that parser can fail if a (non null) lookup key is not found
• added meaningful error messages to json2shared getPacketInfo for bad delimiter, escapeCharacter or stringQualifier
• added explicit parse error when delimiter or escapeCharacter or stringQualifier are configured with whitespace. TBD: this could cause unwanted limitations in parsing subfields
• replaced all occurrences of hardcoded whitespaces " \t" with a string constant WHITESPACES (defined in utilities.h)
• BUGFIX utilities.cpp unquote: whitespace checks were always failing when whitespaces.length() > 1 due to an impossible check that a character had to match all whitespace characters

2024年10月10日 json2ccsds

• ipv6 support for logging framework implemented
• integrated ipv6_utilities into project
• bugfix: initLogApplicationName is back to using a module global fixed-size character array, because apparently std::string behavior is undefined

2024年10月10日 play-syslog.sh, play-housekeeping.sh

• modified scripts so that they echo the replayed TM to console (using tee)

2024年10月07日 json2ccsds

• timestamp fields are now encoded as GPS epoch (was POSIX), so that Yamcs will correctly account for leap seconds
• sequence counters are now shared by packets on the same APID
• handleAction with ACTION_READMORE is now invoked upon startup for existing files, allowing to generate TM from the last N records of a source data file as configured for a packet in the tail-records property
• moved code from sigHandlerInotify to new function handleAction
• BUGFIX: moving a dataSource file away from the monitored location now correctly triggers an ACTION_READMORE and the new ACTION_CLOSE_AT_EOF
• introduced ACTION_CLOSE_AT_EOF that allows to perform ACTION_READMORE with a subsequent close at end of file
• BUGFIX: removed IN_MOVED_FROM trigger for handleDelete (because it's handled by handleMove)
• minor BUGFIX sigHandlerInotify: corrected a LOG_DEBUG message to use logMessage instead of logError
• disabled some debug logging in encoder.cpp node::parseCSVFromVector

2024年10月04日 json2ccsds

• on AOS->LOS transition, los-buffer-size seconds of buffered TM will be replayed to the LOS destination to prevent data gaps
• added parameter link-check-interval (seconds) to allow configuration of intervals at which destinations will be pinged
• tested automatic TM destination switch based on AOS/LOS status
• altered parameters:
  • renamed create-mdb to packet-definitions (because json2mdb creates an MDB, json2ccsds can only echo the binary packet definition)
  • renamed bandwidthCap to bandwidth-cap to fit parameter naming scheme
  • bandwidth-cap now defaults to none, not to max, deemed easier to understand (Note: currently not well implemented anyways, bandwidth-cap setting would block and cause a build-up of TM if source data exceeds the cap)
• integrated tmBuffer and added LOS sender thread
• editorials on formatting, moved some log messages to LOG_DEBUG / LOG_INFO
• integrated LargeFifoBuffer module into project & constructed tmBuffer variable
• renamed module utilityFunctions to utilities

2024年10月02日 json2ccsds

• added property tail-records to JSON packet definition format: how many records to read from a file with multiple records that is newly moved into the monitored data-source location
• tolerance to input data errors: improved, bug-fixed & (smoke-)tested
• added property record-max-lines to JSON packet definition format

2024年10月02日 json2mdb

• field calibration posix_timestamp replaced with gps_timestamp using epoch GPS

Back to top of page