Difference between revisions of "Openlog"

From PaparazziUAV
Jump to navigation Jump to search
m (Cleanup of logger for PPRZ Wiki docs)
(6 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Openlog ==
<categorytree style="float:right; clear:right; margin-left:1ex; border: 1px solid gray; padding: 0.7ex;" mode=pages>Modules</categorytree>
In general [http://www.sparkfun.com/products/9530 OpenLog] is a Module from Sparkfun that contains an Atmega324 and a Micro-SD-Slot. It can handle FAT-formated files-systems up to 2GB. Once installed into the wiring (parallel to the dataradio) it records every message the UAS sends. That is very handy for using it as some kind of blackbox or for data-logging purposes if no Modem was involved. It is also nice for reconstructing what happened after a crash, since the radio-communication is gone, once the plane is near the ground. 2GB is enough to log a comlete season (several hundred flights).
 
There is also direct support for the Sparkfun [http://www.sparkfun.com/products/10216 Logomatic V2.6] board,
 
==Software==
 
Just use
 
conf/airframes/logomatic.xml
 
Currently autopilot data must be collected on UART0 for [[Data_Logger#Decoding|sd2log]] to work [20:40, 9 August 2012 (UTC)]
 
If you don't want to add a complete TWOG or full-blown LPC-Board to you setup you can also use the Sparkfun [http://www.sparkfun.com/products/9530 OpenLog] board.
Take a look at .
 
== UART Connection ==
The two serial inputs LPC_RXD0 (J7) and RXD1 (J2) are used to receive data. The received data is not forwarded to TX0/1 by software (the UART TX pins are deactivated). If the logger is put in between Tiny and the modem the connection should be done with a wire so that a logger failure does not cause any data link issue.
 
== I2C Connection ==
tbd
 
== SPI Connection ==
tbd
 
== Configuration ==
The logger is defined just like an aircraft. Take the
 
conf/airframes/logger_sd.xml
 
as example. You can enable one or both UARTs for logging and have to set the serial speed
 
<configure name="UART0_BAUD" value="B9600" />
<configure name="UART1_BAUD" value="B9600" />
 
The message type can be switched between the PPRZ and the XBEE format
 
<configure name="LOG_MSG_FMT" value="LOG_PPRZ"/>
''or''
<configure name="LOG_MSG_FMT" value="LOG_XBEE"/>
 
The SD card can either be connected to SPI0 or SPI1
 
<configure name="SPI_CHANNEL" value="0" />
''or''
<configure name="SPI_CHANNEL" value="1" />
 
Now create a new A/C with the logger airframe. Although they will not be used, you still need to supply dummy files for Flight Plan, Settings, Radio and Telemetry.
 
== Decoding ==
The telemetry is written in the Paparazzi .tlm [[Messages_Format|format]] that adds a timestamp to each message. The logger starts with 00000000.TLM in the main folder of the SD and increases the number with each log session. Already used numbers will not be overwritten. This data can be converted back to the Paparazzi .log and .data format using sd2log (make sure [[Installation#Environment_Variables|environment variables]] are set before running pprz programs from the commandline)
 
me@home:~/media/usbstick$ ~/paparazzi3/sw/logalizer/sd2log 00000002.TLM
Renaming produced file ...
09_08_13__20_55_03_SD.data file produced
09_08_13__20_55_03_SD.log file produced
09_08_13__20_55_03_SD.tlm file saved
 
It creates timestamps from the .tlm and changes the filename to the take-off time if a GPS message with correct time was available in the file or the current local PC time if no GPS was available. The .log file will be re-created either from the current configuration (if still in existence) or the MD5-labeled files that are stored in var/conf each time you build an aircraft. All resulting files are stored in var/logs with an _SD extension.
 
 
=About=
In general [http://www.sparkfun.com/products/9530 OpenLog] is a Module from Sparkfun that contains an Atmega328P and a Micro-SD-Slot. It can handle FAT16 and FAT32 microSD cards up to 16GB. Once installed into the wiring (parallel to the dataradio) it records every message the UAS sends. That is very handy for using it as some kind of blackbox or for data-logging purposes if no Modem was involved. It is also nice for reconstructing what happened after a crash, since the radio-communication is gone, once the plane is near the ground. 2GB is enough to log a comlete season (several hundred flights).
The general logging is done in two steps: First you record your log using the telemtry in transparent mode, with the openlog-module loaded. the normal paparazzi-protocal does not contain timestamps and the openlog-module adds timestamps to the datastream to have a reference for the time.
The general logging is done in two steps: First you record your log using the telemtry in transparent mode, with the openlog-module loaded. the normal paparazzi-protocal does not contain timestamps and the openlog-module adds timestamps to the datastream to have a reference for the time.
Second the recorded logs are extracted from the SD-Card to the normal logfiles. For doing so, there is a tool calles openlog2tlm to convert the recorded files to the TLM-format of the SD-logger. Next the created TLM-file is added to the logs by sd2log.  
Second the recorded logs are extracted from the SD-Card to the normal logfiles. For doing so, there is a tool calles openlog2tlm to convert the recorded files to the TLM-format of the SD-logger. Next the created TLM-file is added to the logs by sd2log.  
There is a helper-script that does this job.
There is a helper-script that does this job.
=== Hardware ===
=== Hardware ===


Line 9: Line 70:


The OpenLog is simply connected to the TX-Line of the AP. Once the AP is powered, it starts to record.
The OpenLog is simply connected to the TX-Line of the AP. Once the AP is powered, it starts to record.
In general there is one Problem with the stock-firmware of the OpenLog. If it gets an escape-character (CTRL+Z) it switches into command-mode and stops logging. But the telemetry stream is nearly arbitrary. So we have to deactivate this feature, it isn't needed for our application at all. Therefore we have to reprogram the OpenLog. You need an AVR-programmer like the i.e. [http://www.hobbyking.com/hobbycity/store/uh_viewItem.asp?idProduct=21321 usbasp]. You have to wire the ISP to the accoring pins of the OpenLog. You need the avrdude package installed.
First you need to checkout the modified OpenLog firmware on github:
git clone https://github.com/chni/OpenLog.git -b OpenLog_V1_noescape
then change to the directory where the hexfile is:
cd OpenLog/Code
The hexfile is contained by the directory, so you can flash it.
You should power the OpenLog with the Programmer, not by the LPC in the same time. The programming is done by running
avrdude -c usbasp -pm328p -U flash:w:openlog_noescape.hex -Ulfuse:w:0xff:m -Uhfuse:w:0xda:m -Uefuse:w:0x0:m
After running this, the OpenLog has a custom firmware making it ignore the escape-characters.


=== Software ===
=== Software ===
Line 51: Line 129:




[[Category:Software]] [[Category:User_Documentation]]
[[Category:Software]] [[Category:User_Documentation]] [[Category:Modules]]

Revision as of 09:21, 19 November 2015

There is also direct support for the Sparkfun Logomatic V2.6 board,

Software

Just use

conf/airframes/logomatic.xml

Currently autopilot data must be collected on UART0 for sd2log to work [20:40, 9 August 2012 (UTC)]

If you don't want to add a complete TWOG or full-blown LPC-Board to you setup you can also use the Sparkfun OpenLog board. Take a look at .

UART Connection

The two serial inputs LPC_RXD0 (J7) and RXD1 (J2) are used to receive data. The received data is not forwarded to TX0/1 by software (the UART TX pins are deactivated). If the logger is put in between Tiny and the modem the connection should be done with a wire so that a logger failure does not cause any data link issue.

I2C Connection

tbd

SPI Connection

tbd

Configuration

The logger is defined just like an aircraft. Take the

conf/airframes/logger_sd.xml

as example. You can enable one or both UARTs for logging and have to set the serial speed

<configure name="UART0_BAUD" value="B9600" />
<configure name="UART1_BAUD" value="B9600" />

The message type can be switched between the PPRZ and the XBEE format

<configure name="LOG_MSG_FMT" value="LOG_PPRZ"/>
or
<configure name="LOG_MSG_FMT" value="LOG_XBEE"/>

The SD card can either be connected to SPI0 or SPI1

<configure name="SPI_CHANNEL" value="0" />
or
<configure name="SPI_CHANNEL" value="1" />

Now create a new A/C with the logger airframe. Although they will not be used, you still need to supply dummy files for Flight Plan, Settings, Radio and Telemetry.

Decoding

The telemetry is written in the Paparazzi .tlm format that adds a timestamp to each message. The logger starts with 00000000.TLM in the main folder of the SD and increases the number with each log session. Already used numbers will not be overwritten. This data can be converted back to the Paparazzi .log and .data format using sd2log (make sure environment variables are set before running pprz programs from the commandline)

me@home:~/media/usbstick$ ~/paparazzi3/sw/logalizer/sd2log 00000002.TLM
Renaming produced file ...
09_08_13__20_55_03_SD.data file produced
09_08_13__20_55_03_SD.log file produced
09_08_13__20_55_03_SD.tlm file saved

It creates timestamps from the .tlm and changes the filename to the take-off time if a GPS message with correct time was available in the file or the current local PC time if no GPS was available. The .log file will be re-created either from the current configuration (if still in existence) or the MD5-labeled files that are stored in var/conf each time you build an aircraft. All resulting files are stored in var/logs with an _SD extension.


About

In general OpenLog is a Module from Sparkfun that contains an Atmega328P and a Micro-SD-Slot. It can handle FAT16 and FAT32 microSD cards up to 16GB. Once installed into the wiring (parallel to the dataradio) it records every message the UAS sends. That is very handy for using it as some kind of blackbox or for data-logging purposes if no Modem was involved. It is also nice for reconstructing what happened after a crash, since the radio-communication is gone, once the plane is near the ground. 2GB is enough to log a comlete season (several hundred flights). The general logging is done in two steps: First you record your log using the telemtry in transparent mode, with the openlog-module loaded. the normal paparazzi-protocal does not contain timestamps and the openlog-module adds timestamps to the datastream to have a reference for the time. Second the recorded logs are extracted from the SD-Card to the normal logfiles. For doing so, there is a tool calles openlog2tlm to convert the recorded files to the TLM-format of the SD-logger. Next the created TLM-file is added to the logs by sd2log. There is a helper-script that does this job.

Hardware

Openlog.jpg

The OpenLog is simply connected to the TX-Line of the AP. Once the AP is powered, it starts to record. In general there is one Problem with the stock-firmware of the OpenLog. If it gets an escape-character (CTRL+Z) it switches into command-mode and stops logging. But the telemetry stream is nearly arbitrary. So we have to deactivate this feature, it isn't needed for our application at all. Therefore we have to reprogram the OpenLog. You need an AVR-programmer like the i.e. usbasp. You have to wire the ISP to the accoring pins of the OpenLog. You need the avrdude package installed.

First you need to checkout the modified OpenLog firmware on github:

git clone https://github.com/chni/OpenLog.git -b OpenLog_V1_noescape

then change to the directory where the hexfile is:

cd OpenLog/Code

The hexfile is contained by the directory, so you can flash it.

You should power the OpenLog with the Programmer, not by the LPC in the same time. The programming is done by running

avrdude -c usbasp -pm328p -U flash:w:openlog_noescape.hex -Ulfuse:w:0xff:m -Uhfuse:w:0xda:m -Uefuse:w:0x0:m

After running this, the OpenLog has a custom firmware making it ignore the escape-characters.

Software

First of all the OpenLog needs to be configured. If it was powered one time without being configured, but with SD-Card plugged, it creates a file named CONFIG.txt or CONFIG.TXT (depends on the version of the firmware). You need to configure the Baudrate, it defaults to 9600-8-N1. In case you use 57600 Baud it should look like:

File: CONFIG.TXT
57600,26,3,0

Furthermore you need to add the OpenLog-module to your airframe.

File: conf/airframes/yourairframe.xml
[...]
  <modules>
    <load name="openlog.xml"/>
     [...]
  </modules>
[...]

Then compile, flash your AP and do your flight. Afterwards take the SD-Card from the logger and import the logs:

christoph@christoph:~/paparazzi/aktueller$ sw/logalizer/getopenlog LOG00003.TXT 
now converting LOG00003.TXT to LOG00003.TXT.tlm
The conversion from the logfile from the SD to TLM seems to be successful, 567 have been converted, 2 packages have been broken
Renaming produced file ...
12_06_17__01_59_45_SD.data file produced
Looking for 4ae62580cb19d37d2b30aa8d31cc20dc conf...
12_06_17__01_59_45_SD.log file produced
12_06_17__01_59_45_SD.tlm file saved

You can also do batch processing like:

christoph@christoph:~/paparazzi/aktueller$ sw/logalizer/getopenlog /media/disk-1/LOG*

In this case all logs will be imported. Afterwards you can find the logs as usual in var/logs/*_SD.log

If someting goes wrong during the import have a close look to the error messages, since openlog2tlm tries to identify the most common problems.