What is EARS?
An Answer to a problem
The EU FP7 Projects Eurofleets and Eurofleets2 are an European wide alliance of marine research centers that aim to share their research vessels (RVs), to improve information sharing on planned, current and completed cruises, on details of ocean-going research vessels and specialized equipment, and to durably improve cost-effectiveness of cruises.
Within this context logging of information on how, when and where anything happens on board of the vessel is crucial information for data users in a later stage. This forms a primordial step in the process of data quality control as it could assist in the understanding of anomalies and unexpected trends recorded in the acquired data sets. In this way completeness of the metadata is improved as it is recorded accurately at the origin of the measurement.
At the time that the Eurofleets project started, every institution and country had adopted different strategies and approaches, which complicated the task of users that need to log general purpose information and events on-board whenever they access a different platform loosing the opportunity to produce this valuable metadata on-board.
One of the tasks of EUROFLEETS 2 is the development of “event log software”, the Eurofleets Automatic Reporting System (EARS). EARS enables scientists and operators to record what happens during a survey.
Events generated automatically by acquisition instruments will also be handled, enhancing the granularity and precision of the event annotation. The adoption of a common procedure to log survey events and a common terminology to describe them is crucial to provide a friendly and successfully metadata on-board creation procedure for the whole European fleet. The possibility of automatically reporting metadata and general purpose data, following the OGC standards, will simplify the work of scientists and data managers with regards to data transmission. To make metadata interoperable and fill the semantical gap between organisations, international vocabularies are used. Terms that are not present in vocabularies are created and governed by Eurofleets as part of the developments of Eurofleets 1 and 2.
An improved accuracy and completeness of metadata is expected when events are recorded at acquisition time. This will also enhance multiple usages of the data as it allows verification of the different requirements existing in different disciplines.
Concept and Basics on Eurofleets Automatic Reporting System (EARS)
The underlying idea of EARS is that, during a survey, all underway data and occurring events should be recorded in a database so that later, relevant information can be gathered semantically upon domain specific criteria.
- In the case of underway data various kinds of sensors with different wiring standards are bridged to a private Ethernet network where software can isolate datagrams and parse them. Most of the incoming datagrams are very likely to be in the NMEA format, but the system can handle any kind of format thanks to a datagram parser editor, which allows to customize the software for practically any kind of set-up.
- The events occurring during a scientific cruise can be anything from instrumental anomalies, to observations not recorded by a dedicated sensor. Also the collection of any type of material, a ‘sample', prior to analysis is considered as an event. Events cover a wide range of domains, and generally refer to a human input. With this in mind, a specific Graphic User Interface was developed, but events can also be generated by any automatic system that detects an interesting fact of interest which needs logging.
Figure 1 shows the concept of an event, and the special semantic conception of it adopted in EARS.
Technicians and scientists in different disciplines are the potential users of the tool, but crew members can be also be responsible to generate common-purpose events related to navigation, such as the departure or arrival to a specific station. Logging of information on how, when and where an event happens on board of the vessel is crucial information for data users in a later stage.
In summary, EARS is built on top of the adoption of three basics: a common event logging procedure, the adoption of standards, formats and semantics, and a set of event access and retrieval procedures.
- The common event logging procedure
For the event recording procedure, EARS has been developed with two different strategies:
- In order to record data that provides the boundary conditions for the rest of the observations (e.g. navigation, weather data, sea water basic properties) or that are recorded by means of unattended (no human interaction) systems, EARS provides a logging system based on broadcasting these diverse data in the form of NMEA sentences and storing them in a relational database.
- For recording events triggered by an human interaction (any observation or annotation that any scientist or system operator wants to have recorded), EARS provides a Graphical User Interface.
- Standard formats and semantics
It is easy to understand how important it is that the meaning of information is clear both across, and even within domains. EARS' approach is to formalize 'meaning', using shared resources such as controlled vocabularies that can be linked to/from/within the information itself. This way, information carries its own definition.
This approach was followed throughout all aspects of the development of EARS 1 and 2. Underway data and events are always linked to external controlled vocabularies. At the same time new terms can be created and made available to the whole community. An ontology is developed allowing the storage of pre-defined relations between these terms. Querying this ontology presents users a configuration for their scientific cruise. A common approach for storing events, using common terms, facilitates cross-domain data discovery.
Once all information is collected in the database, this can be retrieved on board with the same tool that has been used to ingest manual events. Accessing the event log from outside the vessel is also interesting for many purposes. The production of event summaries could be done in many ways, from automatically compiled or by demand.
In order to offer a summary of the event log to users in a human-readable format, EARS has adopted a standardized way to provide this information to ensure interoperability with other information systems (other EUROFLEETS services or Data Centre services). Thus each EARS platform exposes the event summary by using a specific language from the Open Geospatial Consortium (OGC) called SensorML that allows to produce fine description of any observation procedure, sensor or instrument, as well its history of events associated.
A set of Web Service specific interfaces are prepared in order to get from the vessels different information, including the last 24h event registry. For example a Ship Summary Report (SSR) is available that gathers the most important information of the last 24 hours, or the vessel track. This is not only accessed directly by end users, but also by the EUROFLEETS EVIOR portal. This is a service that adds further value by providing to a number of infobases an overview of the underway profile and activities based on the EARS web services access. EVIOR retrieves and harmonizes information acquired from multiple vessels at the same time and therefore allows to easily provide a snapshot of their activities.
EARS IN DETAIL
EARS V1 is composed by a set of software designed and developed during the EUROFLEETS1 project, but also uses existing elements that have been either designed and used previously by the partners, as well as instrumentation that are commonly present on board. The main elements that compose EARS V1 are shown in Figure 2, with the arrows showing the flow of information.
A NMEA GPS is the basic instrumentation that EARS needs to work, as it provides the information about the location where an event is generated. Additionally EARS can work with any unattended instrument that is able to generate information using the NMEA183 standard in order to provide the set of general purpose information commonly referred as "underway data". Underway data is used complementary to better understand the boundary conditions where the main data acquisition activity occurs.
- SERIAL TO ETHERNET CONVERTERS
EARS is designed to work with data and services using Internet Protocols running on the vessel's Ethernet Local Area Network. For that reason EARS uses specific devices (commercial) to convert NMEA Rs232 messages produced by GPS and Instruments to TCP/UDP datagrams.
Casino+ is an Ifremer software application designed to store the events and the information that come (UDP datagrams) from instruments. The relational database that EARS V1 uses to store all the information, is the Casino+DB (a MySQL instance).
The Configuration GUI is a piece of software specifically designed in EARS to configure CASINO+ system to work in EARS environment. The GUI facilitates the ingestion of new NMEA instruments and facilitates the usage of the system for a non experts users.
Events are the key piece of information managed in EARS and they are designed as a combination of a set of elements (subject, actor, tool, action, properties) that are under a relational controlled vocabulary. This means that for storing an event that describes "any fact that happens during a survey which has a potential interest to be logged" it is necessary to use specific terms combined and ordered in a specific way. By using controlled vocabularies, EARS consolidates the way how an event should be created and increases the interoperability of the final metadata. As the potential combination of terms are quite large, in EARS V1 the vocabulary is organized in what we call "personal or vessel trees" that are the most commonly used terms used in a specific acquisition scenario (geophysics, oceanography, fisheries, etc.) or the whole vessel.
The next figure shows at the left side an example of this vocabulary personalization in form of "personal tree", in this case showing the potential terms and their relations ready to be used for a specific "mission" (cruise) to build up events (right side).
This GUI allows a user to create time-tagged events that describe any possible fact related to the navigation of the vessel, the operation of a specific instrument or observation, or the course of the survey. The GUI also allows the user to set up the primary information of the survey (name and acronyms, chief scientist, ...) and modify and retrieve the event log already entered into the system.
An advanced usage of the Manual Event Entry GUI is to set up the personal or vessel tree (the set of terms that the user will use to build up an Event).
- SHIP SUMMARY REPORT ENGINE
This automatic piece of software produces Ship Summary Report ("SSR") files. This XML file contains basic information about the vessel (name, last navigation info), the current cruise on board, and also the last 24 hours of navigational data in GML format and the last event report in SensorML format.
SSR files are populated, sent to shore if satellite communications are available, and provided to the vessel's operator site and to the SeaDataNet site through the EVIOR portal.
EARS 2 in detail
EARS 1 has been released on the 1st EUROFLEETS2 User Workshop that took place on the 21st-22nd October 2015 in Ostend. At this moment (October 2016), EARS2 is ready for a first release. It will be made public during the 2nd EUROFLEETS2 workshop, 3rd and 4th november 2016 in Vigo, Spain.
The functionalities of EARS 1 are still available in EARS2, but are made available with different tools. For instance, the GUI is still available (in a new version) and performs the same role (creating events, managing the vocabulary and performing the cruise and program setup). Casino+ is no longer used in EARS2 and has been replaced by a dedicated acquisition module. Exposing the sensor devices (via a MOXA) on the vessel's network is still a prerequisite.
General architecture of EARS 2
The EARS 2 system is a natural evolution from EARS 1. EARS 2 has an architecture based fundamentally on two new components:
- Use of a more sophisticated vocabulary engine that, contrary to the one in EARS 1, provides meaning to the terms used in the system. A set of ontologies and infrastructures (hidden to the user) enables EARS users to use a more powerful vocabulary management mechanism with which "vocabulary trees" can be built.
- Use of a Web Service Layer to integrate the different EARS elements. On top of a formal specification (Eurofleets Web Service White Book), a set of Web services has been implemented as a way to integrate all of EARS elements (applications, Data Acquisition modules, Databases and SSR engine). This element, also transparent to the user, will facilitate the implementation of the system in a very high variety of vessel system information infrastructures facilitating the "EARS" implantation in the European Research Fleets.
There have been small changes to the EARS 1 database architecture; this component still stores the navigation and basic underway data, and the events.
There are two other distinct components in EARS 2, namely the Acquisition prototype and the EARS 2 GUI.
The different scopes and use cases of the EARS ontology system
The semantical backbone consists of three ontology documents plus their infrastructure and functions as the knowledge storage system of EARS. It is a repository that stores information on external "static" vocabularies (lists of research vessels, sea areas, harbours, organisations,…), and provides a meaning to and meaningful combinations for the terms hitherto used in EARS, like tools, tool categories, actions and processes.
The ontology has received some refinements compared to EARS1. An ontology is a specification of a conceptualization: it predefines what entities are in play, what their properties are how they might come into being, how they will behave and interact, how they should be mapped to each other etc.
The EARS 1 terms subject, actor, tool, action, properties have been renamed to tool category, tool, action+process and property. The combination of an EARS2 action and process constitutes an EARS1 action. For instance "End Tow" becomes "Towing-End", which allows for more flexibility and meaningfulness. Also the predefinition of events like it was available in EARS1 has been made more formal.
The EARS 1 concepts of a personal or vessel tree have been made more explicit. A third type, the base scope has been added. This scope provides gives the users a basic set of concepts that are based on the international SeaDataNet vocabularies and on the concepts (processes, actions and properties and their combinations) that have been defined and refined during EF1 and 2. The vessel tree provides a basic template for any user (program) of the vessel and is a derivative of the base tree, that is very large and unwieldy for practical use. It can subsume terms and relationships of the basic tree and can define new terms and relationships. The program scope corresponds to the personal tree and provides a unified list of EARS concepts for the responsibles of the program. All scopes are an ontology in themselves.
When a vessel operator or principal investigator of a program creates a new term, he/she introduces this into the EARS system. The term receives a unique identifier and is propagated to the higher level scopes as a new term by making use of governance. Governance means that the term is reviewed and added to the Eurofleets corpus if it is interesting.
The EARS front-end application makes direct use of this ontological backbone and provides the user the possibility to create new terms etc.
The web services play an unifying role in the EARS 2 stack. In order to provide clients a unified and simple interface, and in order to hide complex configuration issues, a web service has been specified. This specification documents how the information (in what format, i. e. XML, and how this information is composed) in the EARS data structures is shown to users, and where it can be retrieved (via which url verbs and url arguments). Web services make accessing (meta)data very easy as no special configuration of software packages are needed. In fact given the specification of the webservice with the central Eurofleets objects and their relationships it is possible to provide one's own implementation of the web service and plug it into existing cruise management systems.
The web service layer retrieves and stores information on the manual events, the scientific program, the cruise and the basic navigation.
The acquisition prototype is the replacemenet of CASINO+ and the EARS configurator of EARS1. It is termed a prototype as it provides a template for vessel operators to expose the real-time datagrams from their devices in Java code. It foregoes the use of XML templates in order to transform device datagrams but enables the implementor of creating more tailored procedures to transform the device outputs and use them in other software, such as acquisition software. The EARS web services navigation module consumes this information to build the navigational history of the vessel.
EARS FRONT-END APPLICATION
The EARS 2 Front-end application is the main source of interaction of a user with the EARS infrastructure. It enables users to create and edit cruises and scientific programs, to modify the vocabulary of event-related terms (ie. actions and processes) and to log events that happen on board. It is a new development from the EARS 1 GUI, but has a similar functionality. In order to create events and to modify the program tree or the vessel tree, the application needs to be connected to the web services that operate on the local network of the vessel.
EARS2 still uses the same basic lay-out as EARS1 with on the left the trees and on the right the events