Difference between revisions of "User:Roirodriguez"

From PaparazziUAV
Jump to navigation Jump to search
Line 199: Line 199:
* I don't know if this affects (know nothing about arm7 actually), but there's a bug on gcc 3 arm7 cross-compilers described at [http://www.openhardware.net/Embedded_ARM/Toolchain/#gcc3bug http://www.openhardware.net/Embedded_ARM/Toolchain/#gcc3bug].
* I don't know if this affects (know nothing about arm7 actually), but there's a bug on gcc 3 arm7 cross-compilers described at [http://www.openhardware.net/Embedded_ARM/Toolchain/#gcc3bug http://www.openhardware.net/Embedded_ARM/Toolchain/#gcc3bug].
* Paparazzi-bin build process (making deb package) compiles some files with the cross compiler, and the switch -mapcs-32. That switch is deprecated in gcc version 4, so if we want to make a compiler that make the paparazzi-bin package compile, we need to use a gcc version 3 compiler.
* Paparazzi-bin build process (making deb package) compiles some files with the cross compiler, and the switch -mapcs-32. That switch is deprecated in gcc version 4, so if we want to make a compiler that make the paparazzi-bin package compile, we need to use a gcc version 3 compiler.
* For 64bit systems version 4 compilers for arm can be downloaded from the [http://www.gnuarm.org gnuarm] project. As told above, they don't support the -mapcs-32 switch. Version 3 compilers from gnuarm project doesn't have that switch enabled too.
* For 64bit systems version 4 compilers for arm can be downloaded from the [http://www.gnuarm.com gnuarm] project. As told above, they don't support the -mapcs-32 switch. Version 3 compilers from gnuarm project doesn't have that switch enabled too.


I'm sure there are tons of sites where you can download an arm7 cross-compiler which works well with that LPC2148 device. But my goal here is to build one that can after be used to build the paparazzi-bin deb package, so there's a need for that -mapcs-32 switch. Based on this, i decided to make the same versions as in the paparazzi debian repository, that's:
I'm sure there are tons of sites where you can download an arm7 cross-compiler which works well with that LPC2148 device. But my goal here is to build one that can after be used to build the paparazzi-bin deb package, so there's a need for that -mapcs-32 switch. Based on this, i decided to make the same versions as in the paparazzi debian repository, that's:

Revision as of 08:03, 5 August 2010

About me

  • Location: Santiago de Compostela (Spain).
  • Skills: 4 years working as a software developer over free software, and linux user and developer for the last 13 years. Now i'm back to the university to finish my physics studies.
  • Paparazzi status: Newbie. Just starting. I'm trying to get money to buy a plane and learn how to fly it (good start :-); after that i'll go into paparazzi. For now i'm just running some simulations and compiling it for my amd64 system.
  • Paparazzi goals:
    • Compile the sources for my ubuntu karmic amd64 platform. If possible generate deb packages to make installation on other amd64 systems easier.
    • Learn to fly a plane.
    • Learn to program AMD7 devices.
    • Study paparazzi source code, run lots of simulations... Until i've got a good overall idea of what's going on.
    • Build a paparazzi board, and install it into my plane.
    • Develope some ideas i've got, around paparazzi.

Ubuntu Karmic AMD64 Compilation log

Intro

The following explains my experience compiling all paparazzi-dev, paparazzi-arm7 and paparazzi-bin software for my amd64 machine, running an Ubuntu Karmic linux distribution.

It must be taken into account that i'm a complete newbie to paparazzi, arm7 and so. So my goal was primarly making it all compile, and patch where necessary for that, but testing on those patches was not done.

The final goal of this document is to clarify about which version of each piece of software has to be compiled, where to find that software, patches needed to make it compile on amd64... I'm making it public because i saw people tried to compile it for 64bit systems with no success, and because i need a few answers to some questions in order to make really tested and accurate deb packages.

I'm writting 'QUESTION:' at the begining of the lines that represent questions i've got, patches i'm not sure to be accurate, etc. This way everybody can find them easily...

You can find all the deb packages and modified source files (.tar.gz) at http://imasdtrade.x10hosting.com/paparazzi/.

Paparazzi-dev packages

About ivy-c, ivy-c-dev, ivy-ocaml and ivy-python packages. The rest of the software on which paparazzi-dev depends can be found in the ubuntu repositories for karmic. If you have a distribution different from ubuntu karmic which doesn't satisfy any other dependency and have instructions about compiling it, packaging it, etc, please feel free to add any reference on how here...

Ivy-python: The ivy-python package is architecture independant, so it can be downloaded from the ubuntu or debian paparazzi repository, at [1]. There you can find also the .tar.gz source package for it...

About Ivy versions

You can find the Ivy software bus homepage at http://www2.tls.cena.fr/products/ivy/. There you can find documentation, svn repository location, etc.

If you take a look to the files at http://paparazzi.enac.fr/ubuntu/dists/karmic/main/binary-i386/ you'll see two different version numbers for ivy-c packages, and 3 for ivy-ocaml packages:

If you take a look to the control files of each of those ivy-ocaml packages (you can do it downloading the package and then dpkg --info PACKAGE_DOWNLOADED from the command line. See the line starting with 'Depends:') you'll see all of them tell to be compatible with ivy-c versions >= 3.8.

However my experience with ivy-ocaml_1.1-7 (the only one for which source code is provided) is that it is not compatible with ivy-c 3.11.4. The civyloop.c source file in the ivy-ocaml_1.1-7 package makes use of the IvyMainLoop function (defined in the ivy-c package) in the following way:

value ivy_mainLoop(value unit)
{
  IvyMainLoop (NULL, NULL);
  return Val_unit;
}

Definitions for the IvyMainLoop function in the ivy-c package are as follows. For 3.8:

extern void IvyMainLoop(void(*BeforeSelect)(void),void(*AfterSelect)(void) );

And for 3.11:

extern void IvyMainLoop(void);

So 3.8 version of that function has 2 arguments and 3.11 has no arguments at all. So we've to use 3.8 version. We'll get an error about 'too much arguments for IvyMainLoop' if we try to compile ivy-ocaml_1.1-7 with ivy-c_3.11.4-1.

But there's no source for ivy-c_3.8.1-1 in the paparazzi repository, where did i get it from? From the ivy-c svn repository, the svn command is the following:

svn co http://svn.tls.cena.fr/svn/ivy/ivy-c/tags/debian_version_3_8_1-1 ivy-c_3.8.1

NOTE: Don't try to get ivy-ocaml too from the main ivy svn repository at http://svn.tls.cena.fr/svn/ivy/ivy-ocaml. It's main developer Pascal Brisset told me to use that ivy-ocaml_1.1-7.tar.gz file in the paparazzi repository instead. Svn repository for ivy-ocaml seems unused.

So we'll compile ivy-c_3.8.1 coming from the main ivy-c svn repository, and ivy-ocaml_1.1-7.tar.gz in the paparazzi repository (link above).

QUESTION: Where do i find sources (tar.gz) for ivy-ocaml_1.1 revisions 8 and 9??

QUESTION: Wouldn't it be better to change ivy-ocaml_1.1-7 control file to depend strictly on ivy-c 3.8 version?

Ivy-c (3.8.1)

As told above, source packages for this version of ivy-c can be downloaded with the following command (command line):

svn co http://svn.tls.cena.fr/svn/ivy/ivy-c/tags/debian_version_3_8_1-1 ivy-c_3.8.1
Compilation

To compile it you've just enter the src directory and type make, then make install as usual:

cd ivy-c_3.8.1/src
make
sudo make install

This will compile the sources and install header files, libraries and the ivyprobe program under /usr/local on your system.

Debian packages

The source we downloaded above is debianized. For 32bit systems you just need to:

cd ivy-c_3.8.1
dpkg-buildpackage -rfakeroot -uc -us

And done. You'll get your .deb packages in the directory containing ivy-c_3.8.1 (up one level).

For 64bit systems there's a problem: The Makefile at src/Makefile sets the relative path for the target libraries in the variable LIB. This is set to 'lib' or 'lib64', this last for 64bit systems. However, if you take a look at those debian/ivy-c.dirs and debian/ivy-c.install files there's no files being installed for usr/lib64 or usr/X11R6/lib64 (just usr/lib and usr/X11R6/lib libraries are installed in the debian package).

We'll modify debian/rules to override the LIB environment variable when calling make to equal 'lib' always (even if your system is a 64bit one, /usr/lib64 will be just a symbolic link to /usr/lib). Just change the line (debian/rules, remember):

cd src && $(MAKE) install DESTDIR=$(CURDIR)/debian/tmp PREFIX=/usr

By:

cd src && $(MAKE) install DESTDIR=$(CURDIR)/debian/tmp PREFIX=/usr LIB=/lib

And done, now you can run dpkg-buildpackage as usual.

You can find the sources for 3.8 already properly debianized for 64bit systems by me at http://imasdtrade.x10hosting.com/paparazzi/ivy-c_3.8.1-2.tar.gz. There you can find the .deb packages for ivy-c and ivy-dev (amd64) too. I've changed the debian/changelog file too...

Ivy-ocaml (1.1-7)

Compilation

That doesn't work on 64bit systems, if you just run 'make':

/usr/bin/ld: civyloop.o: relocation R_X86_64_32 against `timer_cb' can not be used when making a shared object; recompile with -fPIC

That's solved by just editing the Makefile. Change the line:

.c.o :
        $(CC) -Wall -c $(OCAMLINC) $(GLIBINC) $<

By:

.c.o :
       $(CC) -Wall -c -fPIC $(OCAMLINC) $(GLIBINC) $<

And done.

The following section (debian packaging) does the same but distributing the change as a dpatch to the sources instead editing the Makefile. This way we don't modify the sources, but just distribute a patch which can be integrated in debian/rules or used by its own (read dpatch manual page, man dpatch).

Debian packages

As told above, all we are going to do here is to distribute that change in the Makefile as a dpatch, and integrate the dpatch system into debian/rules. I'll not explain in depth how dpatch works, you can read http://www.debian.org/doc/maint-guide/ch-build.en.html for an introduction and then read the dpatch manual page (man dpatch).

Open debian/rules for editing and add, at the first line of the file (after the comments at the beginning), the following:

# Include dpatch stuff.
include /usr/share/dpatch/dpatch.make

That will include default dpatch rules for Makefiles, like 'patch', 'unpatch' and so.

Then edit the 'build' and 'clean' rules in debian/rules to depend on 'patch' and 'unpatch'. That means, replace the line:

build: build-stamp

By:

build: patch build-stamp

And the line:

clean:

By:

clean: unpatch

And done. This will make dpkg-buildpackage to apply the patches listed in debian/patches/00list before building, and deapply it on cleaning.

Now we've got to add our patch to debian/patches and list it in debian/patches/00list.amd64 (which lists patches being applied only for amd64 targets). For this run from the command line (you must be in the ivy-ocaml sources root dir!):

dpatch-edit-patch 01_ccfpic.dpatch

That will put you on a shell where you can edit all files you need. Just edit the Makefile as told above in Compiling and then type exit in that shell. After exiting a dpatch file will appear under debian/patches directory, named 01_ccfpic.dpatch.

Finally we've got to list that patch in a file called debian/patches/00list.amd64. Open that file for editing and add one single line saying:

01_ccfpic.dpatch

And now you can run dpkg-buildpackage to generate your deb packages. You can find mine (amd64), and modified sources (dpatch modifications described here, changelog file) in http://imasdtrade.x10hosting.com/paparazzi/.

NOTE: This solution is to avoid modifying the original sources. A proper patch for the original sources would be one that checks if the system is a 64bit system, and if so set FPIC env variable, the following code at the begining of the Makefile would be ok (taken from ivy-c Makefile):

PERHAPS64 := $(shell uname -m | perl -ne "print /64/ ? '64' : ;")
ifeq ($(PERHAPS64), "64")
        FPIC=
else
        FPIC=-fPIC
endif

And then let the rule to compile all .c and .o files be like the following:

.c.o :
        $(CC) -Wall -c $(FPIC) $(OCAMLINC) $(GLIBINC) $<

Paparazzi-arm7 packages

After going on please notice i didn't tested the cross-compiler below (toolchain). I need to test all of this, only built them just to make paparazzi-bin compile.

Toolchain (binutils, gcc, newlib)

About gcc versions

The following considerations must be taken into account:

  • I don't know if this affects (know nothing about arm7 actually), but there's a bug on gcc 3 arm7 cross-compilers described at http://www.openhardware.net/Embedded_ARM/Toolchain/#gcc3bug.
  • Paparazzi-bin build process (making deb package) compiles some files with the cross compiler, and the switch -mapcs-32. That switch is deprecated in gcc version 4, so if we want to make a compiler that make the paparazzi-bin package compile, we need to use a gcc version 3 compiler.
  • For 64bit systems version 4 compilers for arm can be downloaded from the gnuarm project. As told above, they don't support the -mapcs-32 switch. Version 3 compilers from gnuarm project doesn't have that switch enabled too.

I'm sure there are tons of sites where you can download an arm7 cross-compiler which works well with that LPC2148 device. But my goal here is to build one that can after be used to build the paparazzi-bin deb package, so there's a need for that -mapcs-32 switch. Based on this, i decided to make the same versions as in the paparazzi debian repository, that's:

  • GCC version 3.4.4.
  • Binutils version 2.16.1.
  • Newlib version 1.13.

QUESTION: Does that bug affect? QUESTION: Is that -mapcs-32 switch needed in the paparazzi build process (Makefiles under sw/airborne/arm7 directory)? I've readed that -mapcs-26 and -mapcs-32 are deprecated, and that apcs-32 is now the default. Because i do not have any idea about what this all means, it's possible that i'm wrong. Can anybody confirm or refute this?

Toolchain compilation

I've followed the general instructions found at http://www.madox.net/blog/2008/11/26/compiling-a-toolchain-for-arm7-under-ubuntu/, but modifing the t-arm-elf file. I reproduce all the steps below:

First of all you've got to download some dependencies:

apt-get install flex bison libgmp3-dev libmpfr-dev autoconf texinfo build-essential

IMPORTANT: In order to compile a gcc-3.4 cross-compiler we need a version 3 gcc native compiler in our system. Compilation will fail if you use a version 4 compiler. In ubuntu karmic there's no gcc-3 versions, so we need to download and install the following packages in our system (install it in the order specified, links below are for amd64):

Now we need to be sure our 3.4 compiler will be used to compile the cross-compiler instead the version 4 system default:

sudo rm /usr/bin/gcc
sudo ln -s /usr/bin/gcc-3.4 /usr/bin/gcc

We will undo this once finished.

QUESTION: Does anyone know how to specify the compiler to use instead of this dirty thing?

A: Change your CC environmental variable to the version you want, as in $ export CC=gcc-3.4 Before compiling a program, in the same terminal in which you are going to run configure/make, type: Code:

$ export CC=gcc32
$ export CXX=g++32

You have to install compat-gcc-32 and compat-gcc-32-g++ , also http://www.tellurian.com.au/whitepapers/multiplegcc.php may be ho help

Now we download gcc, binuitils and newlib, which can be found at:

Now we just follow the instructions given at http://www.madox.net/blog/2008/11/26/compiling-a-toolchain-for-arm7-under-ubuntu/ with two observations:

  • IMPORTANT: The t-arm-elf used there don't support the -mapcs-32 switch, change it by mine's at http://imasdtrade.x10hosting.com/paparazzi/t-arm-elf.
  • IMPORTANT: The configure script for gcc must be run with sudo (or as root), because it copies some of the newlib headers to /usr/local!!!!

If you download the versions above, and follow the instructions at that link being sure you follow my observations you are done.

Now, remember to undo that symlink:

sudo rm /usr/bin/gcc
sudo ln -s /usr/bin/gcc-4.4 /usr/bin/gcc # Change 4.4 by your most recent gcc compiler, see ls -l /usr/bin/gcc-*

QUESTION: Better build instructions? More accurate t-arm-ef file? Can anybody test the compiler (i'll do as soon as i can).

Debian packages

No debian packages for gcc, binutils or newlib were made, because as i told i'm not sure the way i compiled it all is correct. I don't know exactly how may i do to package such a complex package as a cross-compiler is...

QUESTION: Has Anybody got any link on packaging cross-compilers on debian (making deb packages)? QUESTION: Can anybody provide the debianized sources of that compiler in the paparazzi repository? That will tell me how to compile and package it all at once!

lpc21isp

The lpc21isp project is hosted in sourceforge. You can find the source packages at http://sourceforge.net/projects/lpc21isp/. I downloaded the latest code, was 1.64 at the time of this writting.

Version in the paparazzi repository seems to be 1.27 (i saw it inspecting the lpc21isp binary at the paparazzi repository with an hex editor). I'm going to compile 1.64, i guess it will be ok, but i didn't tested!

Compilation

Just:

make
sudo make install

Would work.

Debian packages

I've debianized the sources running dh_make (single binary package, after dh_make i removed unecessary files from the debian directory). Then i simply ran:

dpkg-buildpackage -r fakeroot -uc -us

You can get the debianized source package, as well as the deb packages from http://imasdtrade.x10hosting.com/paparazzi.

Paparazzi-bin package

The source code compiled below was obtained from the svn trunk:

svn co svn://svn.savannah.nongnu.org/paparazzi/paparazzi3/trunk paparazzi3

The revision obtained was r4525.

NOTE: The deb package generated has some problems (see below). The compilated non-packaged version works well. You must note too that some software in the deb package were compiled with the untested arm7 toolchain in the previous section, so it must not be used until that is properly tested.

Compilation

If you enter the sources directory and simply run make you will get the following error in an amd64 system:

cd sw/ground_segment/multimon; make PAPARAZZI_SRC=/home/roi/Codigo/paparazzi-dev/lenny_versions/paparazzi/temp PAPARAZZI_HOME=/home/roi/Codigo/paparazzi-dev/lenny_versions/paparazzi/temp
CC hdlc.c
hdlc.c:1: error: CPU you selected does not support x86-64 instruction set

This is because the code in sw/ground_segment/multimon/filter-i386.h which contains optimizations for some functions gets included and it doesn't work on amd64. filter.h says that filter-i386.h gets included if:

#ifdef ARCH_I386
#include "filter-i386.h"
#endif /* ARCH_I386 */

If we take a look to the Makefile (sw/ground_segment/multimon/Makefile):

CFLAGS          =-Wall -Wstrict-prototypes -I/usr/X11R6/include -I `ocamlc -where`
ifeq ($(DEBUG),y)
CFLAGS          +=-g -O -march=i486 -falign-loops=2 -falign-jumps=2 \
                 -falign-functions=2 -DARCH_I386
else
CFLAGS          +=-O3 -march=i486 -falign-loops=2 -falign-jumps=2 \
                 -falign-functions=2 -DARCH_I386
endif

We'll have to change it to not define ARCH_I386, not optimize for i486 and add -fPIC. So we need to leave the above lines like:

CFLAGS          =-Wall -fPIC -Wstrict-prototypes -I/usr/X11R6/include -I `ocamlc -where`
ifeq ($(DEBUG),y)
CFLAGS          +=-g -O -falign-loops=2 -falign-jumps=2 \
                 -falign-functions=2
else
CFLAGS          +=-O3 -falign-loops=2 -falign-jumps=2 \
                 -falign-functions=2
endif

Note that leaving the sources like this won't apply the optimizations if now you take the modified source and compile it in a 32bit system... Once my paparazzi-bin deb package (see below) works well i'll provide better patches for all the modifications explained here.

Now it compiles ok. I run ./paparazzi and test it with a simulation and everything seems fine.

Debian packages

The package generated here don't fully work (read at the end to see what fails), so don't use it but for testing purposes. The compiled sources compiled like above works well.

First of all, don't simply run dpkg-buildpackage, there's a Makefile rule instead! Just run:

make deb

With the revision i downloaded i get the following error:

install -o root -m 644 var/maps/trtqtttqtsrrtstq*.jpg var/maps/trtqtttqtsrrttsr*.jpg /home/roi/Codigo/paparazzi-dev/lenny_versions/paparazzi/temp/debian/paparazzi-bin/usr/share/paparazzi/data/maps
install: no se puede efectuar `stat' sobre «var/maps/trtqtttqtsrrtstq*.jpg»: No existe el fichero ó directorio
install: no se puede efectuar `stat' sobre «var/maps/trtqtttqtsrrttsr*.jpg»: No existe el fichero ó directorio
make[3]: *** [install_data] Error 1

It seems that there's a map trying to be installed that cannot be found in the sources. Just remove that jpg from the Makefile.install file, that's remove line 22 of Makefile.install:

$(INSTALLDATA) var/maps/trtqtttqtsrrtstq*.jpg var/maps/trtqtttqtsrrttsr*.jpg $(DESTDIR)/data/maps

QUESTION: Where am i supposed to get that map file from? Or does that line from Makefile.install need to be removed from the repository?

If now we run make deb again, the error is now:

dpkg-gencontrol: error: current host architecture 'amd64' does not appear in package's architecture list (i386)

That's because debian/control says it's a i386 only package. We've got to edit the control file, in my case i did:

cd debian
cp control.lenny control.karmic
cp changelog.lenny changelog.karmic

Then i edited debian/control.karmic and changed the line saying Architecture: i386 by Architecture: any. Now i run:

make DISTRO="karmic" deb

(See deb rule in the makefile to see how the DISTRO env variable is used).

Now it compiles properly. I'm going up one level and install that paparazzi-bin deb package but it fails:

cd ..
sudo dpkg --install paparazzi-bin_3.10-2_amd64.deb
Configurando paparazzi-bin (3.2-10) ...
File "_none_", line 1, characters 0-1:
Error: Error while linking kml.cmo:
Reference to undefined global `Server_globals'

So the postinst script is failing. It's located in debian/paparazzi-bin.postinst. Taking a look at that file one can see that all it does is to enter some directories and compile ocaml binaries. Even more, if one takes a look at those directories one can see that that binaries were compiled in the build process so:

QUESTION: Why aren't those binaries just being installed as usual? Is there any reason or simply there were problems trying to make them installed in the deb package?

I fixed it comparing the source dirs Makefiles with the compilation commands in the postinst file. I one compares between them for each directory in the postinst, it can be seen that there are some missing cma and/or cmo files in some ocamlc commands. Putting down here every modification i made to the debian/paparazzi-bin.postinst file would become very large, so you can simply download my modified sources here and diff it against the original sources.

Now it compiles, so i go up one directory, install it again and test:

cd ..
sudo dpkg --purge paparazzi-bin
sudo dpkg --install paparazzi-bin_3.10-2_amd64.deb

And everything goes ok. If i now run paparazzi from the command line it starts properly but says:

mv: cannot stat `/home/roi/paparazzi/data/maps/trtqtttqtsrrt*.jpg': No such file or directory

You can edit the Makefile.ac in the sources, eliminate the reference to that map file and reinstall and that disappears. Same QUESTION as above regarding to that map file: Where can i find that?

There's a problem that still persists. If i now run paparazzi (with my $HOME/paparazzi dir empty!) it fails to run a simulation, the error is the following:

BUILD /home/roi/paparazzi/var/MJ5/airframe.h
##################################################

 AIRFRAME MODEL: MJ5

##################################################
make[1]: *** No rule to make target `/home/roi/paparazzi/conf/modules/*.xml', needed by `/home/roi/paparazzi/var/MJ5/modules.h'.  Stop.
make[1]: Leaving directory `/usr/share/paparazzi'
make: *** [ac_h] Error 2
make: Leaving directory `/usr/share/paparazzi'

Inspecting my /home/roi/paparazzi directory and comparing to /usr/share/paparazzi i can see that not all files where copied when paparazzi was first run. The curious thing is that if i run ./paparazzi from the sources dir instead executing the installed one (/usr/bin) everything works ok (PAPARAZZI_SRC is taken as PAPARAZZI_HOME this way)...

QUESTION: Any ideas on why? I'm sure i can solve it by myself, but if anybody knows...