Difference between revisions of "Logs"

From PaparazziUAV
Jump to navigation Jump to search
m (according to plot (which is now a shell script) "plot has been renamed to logplotter", and then exec's $PAPARAZZI_SRC/sw/logalizer/logplotter)
 
(19 intermediate revisions by 9 users not shown)
Line 1: Line 1:
On each launch, the <tt>server</tt> creates a ''log'' of all the messages sent by the aircraft. This log can be used to analyse flight data and to replay the flight.
On each launch, the <tt>[[Server]]</tt> creates a ''log'' of all the messages sent by the aircraft. This log can be used to analyse flight data and to replay the flight.


==Format==
==Format==
Line 5: Line 5:
The log files are stored in the <tt>var/logs/</tt> folder (under <tt>PAPARAZZI_HOME</tt>). A log is split into two files:
The log files are stored in the <tt>var/logs/</tt> folder (under <tt>PAPARAZZI_HOME</tt>). A log is split into two files:
* A <tt>.log</tt> file, an XML file, which contains a copy of the whole configuration (airframes, flight plans, ...).
* A <tt>.log</tt> file, an XML file, which contains a copy of the whole configuration (airframes, flight plans, ...).
* A <tt>.data</tt> file, an ascii file, which contains the list of the received messages. Each message is time-stamped in seconds since the creation of the file and marked with the id of the sending aircraft.
* A <tt>.data</tt> file, an ascii file, which contains the list of the received messages. Each message is time-stamped in milliseconds since the creation of the file and marked with the ID of the sending aircraft.


The basename of these two files is the same and is build from the date and time at the creation.
The first part before the type of file of these two files is the same and is build from the date and time at the creation.


The lines of the <tt>.data</tt> file are formatted according to the message description listed in the <tt>conf/messages.xml</tt> file. For example, the following line:
The lines of the <tt>.data</tt> file are formatted according to the message description listed in the <tt>conf/messages.xml</tt> file. For example, the following line:
  3.300 6 GPS 3 36028551 481359212 899 18500 0 -8 960 31 0
  3.300 6 GPS 3 36028551 481359212 899 18500 0 -8 960 31 0
contains a <tt>GPS</tt> message received at time 3.3s, from aircraft 6. According the GPS message description:
contains a <tt>GPS</tt> message received at time 3.3s, from aircraft 6. According the GPS message description:
<source lang="xml" >
  <message name="GPS" ID="8">
  <message name="GPS" ID="8">
     <field name="mode"   type="uint8" unit="byte_mask"></field>
     <field name="mode"       type="uint8" unit="byte_mask"/>
     <field name="utm_east" type="int32" unit="cm"></field>
     <field name="utm_east"   type="int32" unit="cm"/>
     <field name="utm_north"  type="int32" unit="cm"></field>
     <field name="utm_north"  type="int32" unit="cm"/>
     <field name="course" type="int16" unit="decideg"></field>
     <field name="course"     type="int16" unit="decideg"/>
     <field name="alt"   type="int32" unit="cm"></field>
     <field name="alt"       type="int32" unit="cm"/>
     ...
     ...
  </message>
  </message>
</source>
UTM position is 360285.51m east, 4813592.12m north, course is 89.9° (east), altitude is 185m, ...
UTM position is 360285.51m east, 4813592.12m north, course is 89.9° (east), altitude is 185m, ...
Note that the appropriate <tt>messages.xml</tt> description, i.e. the one which has been used while the log was created, is itself stored in the associated <tt>.log</tt> file. It may differ from the current one installed in your <tt>conf/</tt> folder.
Note that the appropriate <tt>messages.xml</tt> description, i.e. the one which has been used while the log was created, is itself stored in the associated <tt>.log</tt> file. It may differ from the current one installed in your <tt>conf/</tt> folder.
Line 27: Line 30:
==Data Plotting==
==Data Plotting==


Data stored in log files can be plotted with the [[Plotter]] (<tt>sw/logalizer/plot</tt> or launchable from the [[Paparazzi Center]]). This tool can plot data from different logs in the same window and data from the same log in different windows. It also offers to export the track as a KML file for [http://earth.google.com Google Earth].
Data stored in log files can be plotted with the [[Plotter]] (<tt>sw/logalizer/logplotter</tt> or launchable from the [[Paparazzi Center]]). This tool can plot data from different logs in the same window and data from the same log in different windows. It also offers to export the track as a KML file for [http://earth.google.com Google Earth].


==Replay==
==Replay==
A flight can be replayed with the Log Flight Player (<tt>sw/logalizer/play</tt>, launchable from the [[Paparazzi Center]]). This agent then is a substitute for the Data Link agent and will send over the bus the messages which had been send by the aircraft while the log was recorded.
A flight can be replayed with the Log Flight Player (<tt>sw/logalizer/play</tt>) which can be started from the [[Paparazzi Center]] (from tools menu to start alone, or from the '''session''' selection box to start a complete replay session with GCS, server and player tool). This agent then is a substitute for the Data Link agent and will send over the bus the messages which had been send by the aircraft while the log was recorded.


Note: While replaying a log, it is a good idea to disable a new log creation from the server (<tt>-n</tt> option).  
Note: While replaying a log, it is a good idea to disable a new log creation from the server (<tt>-n</tt> option).  
Line 36: Line 39:
When doing a log replay it is very valuable to launch the messages window (in the tools menu) as well. This allows for the use of the the real time plotter and also gives the data from the aircraft to the user.
When doing a log replay it is very valuable to launch the messages window (in the tools menu) as well. This allows for the use of the the real time plotter and also gives the data from the aircraft to the user.


If the user wishes to speed up a replay gaia can be used (launch from paparazzi centers tools menu) to do this by changing the time scale.
If a<tt>.log</tt> file is added behind the replay command on the command line, this log will be loaded on startup immediately.
 
TIP: If one want to speed up a replay with 4x speedup of the time [[Simulation#Change_the_environment|Environment Simulator agent, Gaia]] can be used. Launch it from Paparazzi center tools menu "[[Simulation#Change_the_environment|Environment Simulator]]". Then changing the time scale with the slider or add a commandline option e.g. ''./gaia -t 4''
 


A <tt>.log</tt> file given in the command line will be loaded on startup.


===Output on a serial port===
===Output on a serial port===
Line 45: Line 50:
* <tt>-d</tt> Set the device (<tt>/dev/ttyUSB0</tt>)
* <tt>-d</tt> Set the device (<tt>/dev/ttyUSB0</tt>)


==Convert PprzLog into Matlab Workspace==
==Load PprzLog into Matlab Workspace==
Paparazzi log file could also be converted into matrices in matlab workspace for further data processing. An example matlab script is shown in [http://paparazzi.enac.fr/wiki/Image:PprzLog2Matlab_1_0.zip]. The example script needs the inputs of the log file name and directory. Just modify the line "fid = fopen('.\FlighLog20090321_MZN\09_03_19__15_03_14.data','r');". Then eight matrices are read into matlab workspace including GPS_SOL, GPS, PPRZ_MODE, DESIRED, ATTITUDE, NAVIGATION, NAVIGATION_REF, COMMANDS. The user could also modify the code easily to add more messages they want to know.
Paparazzi log file could also be converted into matrices in matlab workspace for further data processing. An example matlab script is shown in [http://paparazzi.enac.fr/wiki/Image:PprzLog2Matlab_1_0.zip PprzLog2Matlab_1_0.zip]. The example script needs the inputs of the log file name and directory. Just modify the line "fid = fopen('.\FlighLog20090321_MZN\09_03_19__15_03_14.data','r');". Eight matrices are read into matlab workspace including GPS_SOL, GPS, PPRZ_MODE, DESIRED, ATTITUDE, NAVIGATION, NAVIGATION_REF, COMMANDS. The user could also modify the code easily to add more messages they want to know.
 
[[Image:logviewer.png|thumb|right|The logviewer GUI]]
There is now another parser named <tt>log2struct</tt> available, see [http://lists.gnu.org/archive/html/paparazzi-devel/2009-08/msg00135.html this post on the mailing list].
It does not require code modification and can read any messages sent from any aircraft.
Usage examples can be found in the source code documentation.
 
A GUI built on this parser is available: [http://paparazzi.enac.fr/wiki_images/Logviewer.zip Logviewer.zip].
It is based on the GUI from <tt>sw/logalizer/matlab_log</tt> but features some improvements, e.g. in speed and reliability.
Standard plots like trajectory, attitude or speed are available for quick reviewing and Matlab's plotting tools can be accessed for plot cosmetics and printing.
 
==NMEA log output==
NMEA output is useful for [[Kinomap]], for example. Do not confuse this with the application ''ivy2nmea'' live NMEA stream out.
 
Two methods exist for generating an NMEA gps log from the paparazzi logs:
: 1- convert the .kml to .nmea by use of gpsbabel application (available on windows and linux)
: 2- convert the .data to .nmea by use of perl script in sw/in_progress/log_parser/log2nmea.pl (also works on windows and linux)
 
[[Category:Software]] [[Category:User_Documentation]]

Latest revision as of 07:35, 28 November 2017

On each launch, the Server creates a log of all the messages sent by the aircraft. This log can be used to analyse flight data and to replay the flight.

Format

The log files are stored in the var/logs/ folder (under PAPARAZZI_HOME). A log is split into two files:

  • A .log file, an XML file, which contains a copy of the whole configuration (airframes, flight plans, ...).
  • A .data file, an ascii file, which contains the list of the received messages. Each message is time-stamped in milliseconds since the creation of the file and marked with the ID of the sending aircraft.

The first part before the type of file of these two files is the same and is build from the date and time at the creation.

The lines of the .data file are formatted according to the message description listed in the conf/messages.xml file. For example, the following line:

3.300 6 GPS 3 36028551 481359212 899 18500 0 -8 960 31 0

contains a GPS message received at time 3.3s, from aircraft 6. According the GPS message description:

 <message name="GPS" ID="8">
     <field name="mode"       type="uint8" unit="byte_mask"/>
     <field name="utm_east"   type="int32" unit="cm"/>
     <field name="utm_north"  type="int32" unit="cm"/>
     <field name="course"     type="int16" unit="decideg"/>
     <field name="alt"        type="int32" unit="cm"/>
     ...
 </message>

UTM position is 360285.51m east, 4813592.12m north, course is 89.9° (east), altitude is 185m, ... Note that the appropriate messages.xml description, i.e. the one which has been used while the log was created, is itself stored in the associated .log file. It may differ from the current one installed in your conf/ folder.

Note: The .data files may be huge (about 4M per hour with a standard telemetry configuration). They can be efficiently compressed. The tools described in the next sections handle most of the compressed format (.zip, .gz, .bz2, ...). The bzip2 compression seems to perform better than others on these files.

Data Plotting

Data stored in log files can be plotted with the Plotter (sw/logalizer/logplotter or launchable from the Paparazzi Center). This tool can plot data from different logs in the same window and data from the same log in different windows. It also offers to export the track as a KML file for Google Earth.

Replay

A flight can be replayed with the Log Flight Player (sw/logalizer/play) which can be started from the Paparazzi Center (from tools menu to start alone, or from the session selection box to start a complete replay session with GCS, server and player tool). This agent then is a substitute for the Data Link agent and will send over the bus the messages which had been send by the aircraft while the log was recorded.

Note: While replaying a log, it is a good idea to disable a new log creation from the server (-n option).

When doing a log replay it is very valuable to launch the messages window (in the tools menu) as well. This allows for the use of the the real time plotter and also gives the data from the aircraft to the user.

If a.log file is added behind the replay command on the command line, this log will be loaded on startup immediately.

TIP: If one want to speed up a replay with 4x speedup of the time Environment Simulator agent, Gaia can be used. Launch it from Paparazzi center tools menu "Environment Simulator". Then changing the time scale with the slider or add a commandline option e.g. ./gaia -t 4


Output on a serial port

Launched with the -o option, the player will send to a serial port all the binary messages, as they have been received through the modem during the flight. Related options:

  • -s Set the baudrate (9600)
  • -d Set the device (/dev/ttyUSB0)

Load PprzLog into Matlab Workspace

Paparazzi log file could also be converted into matrices in matlab workspace for further data processing. An example matlab script is shown in PprzLog2Matlab_1_0.zip. The example script needs the inputs of the log file name and directory. Just modify the line "fid = fopen('.\FlighLog20090321_MZN\09_03_19__15_03_14.data','r');". Eight matrices are read into matlab workspace including GPS_SOL, GPS, PPRZ_MODE, DESIRED, ATTITUDE, NAVIGATION, NAVIGATION_REF, COMMANDS. The user could also modify the code easily to add more messages they want to know.

The logviewer GUI

There is now another parser named log2struct available, see this post on the mailing list. It does not require code modification and can read any messages sent from any aircraft. Usage examples can be found in the source code documentation.

A GUI built on this parser is available: Logviewer.zip. It is based on the GUI from sw/logalizer/matlab_log but features some improvements, e.g. in speed and reliability. Standard plots like trajectory, attitude or speed are available for quick reviewing and Matlab's plotting tools can be accessed for plot cosmetics and printing.

NMEA log output

NMEA output is useful for Kinomap, for example. Do not confuse this with the application ivy2nmea live NMEA stream out.

Two methods exist for generating an NMEA gps log from the paparazzi logs:

1- convert the .kml to .nmea by use of gpsbabel application (available on windows and linux)
2- convert the .data to .nmea by use of perl script in sw/in_progress/log_parser/log2nmea.pl (also works on windows and linux)