Context

Our team is porting a custom RTOS implementation (originally for an old Intel architecture) targetting the Zynq-7000 CPUs, therefor our main IDE/buildsystem of choice is Vitis. We are on the last version of Vitis Unified IDE (2024.2). It was decent until we got to the point of porting the packages, which on our system represent ELFs that have been specially compiled for the RTOS so that the loader can recognise and execute them. The problem is that we are unable to find how to link the GCC library (for the armv7-a+vfp) to our package, which is unfortunate because even a simple thing like a division needs a call to __aeabi_uidiv...

What we know

Vitis creates 2 binaries, the FSBL.elf and our_project.elf and performing an objdump or nm on both shows that the gcc and standart c functions are inside and are all resolved. Now to know how exactly they were linked, is extreamly complex since Vitis has it's build-system shared between custom pyhton files, custom configuration files, the Lopper and CMakeLists. I did try tracing the processes with ProcMon but it didn't give me any more clues, except pointing me to the directories where the pre-compiled libraries and the generated xil libraries seem to be.

Our package compilation

Since our packages include calls to functions/global variables of the OS we use the -r flag to make them relocatable, and when the package is getting loaded the OS loader will perform the necessary rellocation. Here is a sample makefile for the package compilation. The ld is not interesting to show and simply has several special sections for our package recognition and an entry definition.

CC = arm-none-eabi-gcc
LD = arm-none-eabi-ld
READELF = arm-none-eabi-readelf
OBJDUMP = arm-none-eabi-objdump
PACKAGE = package_another

# 
#
# no-common: Avoid multiple definition of global variables in each translation unit
# fPIC: Generate position-independent code
# ffunction-sections & -fdata-sections: Allows linker to optimize section removal 
# nthumb-interwork: Generate code that supports calling between ARM and Thumb instructions
# mpic-data-is-text-relative: Helps the linker to optimize PIC code
CFLAGS = -c \
         -mfpu=vfpv3 \
         -mfloat-abi=hard \
         -mcpu=cortex-a9 \
         -fno-common \
         -fPIC \
         -ffunction-sections \
         -fdata-sections \
         -mthumb-interwork \
         -mpic-data-is-text-relative \
         -O0 \
         -Iinclude 
# r: Generate relocatable output
# gc-sections: Remove unused sections
# build-id=none: Do not generate a build ID
LDFLAGS = -r \
          --gc-sections \
          --build-id=none \
          --no-undefined



all: $(PACKAGE).elf readelf objdump

$(PACKAGE).o: $(PACKAGE).c
    $(CC) $(CFLAGS) $< -o $@

$(PACKAGE).elf: $(PACKAGE).o $(PACKAGE).ld
    $(LD) -T $(PACKAGE).ld $< -o $@ $(LDFLAGS)

readelf:
    $(READELF) -a $(PACKAGE).elf > READELF

objdump:
    $(OBJDUMP) -d $(PACKAGE).elf > OBJDUMP

.PHONY: all clean readelf

clean:
    rm -f $(PACKAGE).o $(PACKAGE).elf b

What We have tried

Because of the -r flag the only way to partially link the libgcc and libc is with the --whole-archive. So since we are on v7 (armv7) and we want the hard FP this is the path we chose:

/mnt/c/Xilinx/Vitis/2024.2/gnu/aarch32/nt/gcc-arm-none-eabi/aarch32-xilinx-eabi/usr/lib/thumb/v7+fp/hard/arm-xilinxmllibv7fphard-eabi/13.3.0/libgcc.a

Therefor, adding this line to the LDFLAGS

          --whole-archive /mnt/c/Xilinx/Vitis/2024.2/gnu/aarch32/nt/gcc-arm-none-eabi/aarch32-xilinx-eabi/usr/lib/thumb/v7+fp/hard/arm-xilinxmllibv7fphard-eabi/13.3.0/libgcc.a --no-whole-archive \

But of course for one this is an overkill (way to many symbols and we will probably only ever need at most a dozen), and two there are still many undefined symbols so we need to link something else?

Question 1

What approach should we take to have a minimal version of the libgcc linked to our package? Guessing that it's not crucial for it to be the same one that the RTOS has (since it is barely used anyway even by the RTOS kernel)...

Question 2

Should we try to get away from the Vitis IDE? The only thing really holding us connected to it is the build system, and the lopper utility that allows to regenerate the device tree even if the XSA changes, and that also updates the BSP. Ease of compilation is still kind of in place, however going further it seems to be posing more problems than giving solutions. I know that it can be scripted using python, and we were successful in implementing a project rebuilding functionality (for github source control). However the scripting features are quite limited and not super well explained in the documentation.

Context

Our team is porting a custom RTOS implementation (originally for an old Intel architecture) targetting the Zynq-7000 CPUs, therefor our main IDE/buildsystem of choice is Vitis. We are on the last version of Vitis Unified IDE (2024.2). It was decent until we got to the point of porting the packages, which on our system represent ELFs that have been specially compiled for the RTOS so that the loader can recognise and execute them. The problem is that we are unable to find how to link the GCC library (for the armv7-a+vfp) to our package, which is unfortunate because even a simple thing like a division needs a call to __aeabi_uidiv...

What we know

Vitis creates 2 binaries, the FSBL.elf and our_project.elf and performing an objdump or nm on both shows that the gcc and standart c functions are inside and are all resolved. Now to know how exactly they were linked, is extreamly complex since Vitis has it's build-system shared between custom pyhton files, custom configuration files, the Lopper and CMakeLists. I did try tracing the processes with ProcMon but it didn't give me any more clues, except pointing me to the directories where the pre-compiled libraries and the generated xil libraries seem to be.

Our package compilation

Since our packages include calls to functions/global variables of the OS we use the -r flag to make them relocatable, and when the package is getting loaded the OS loader will perform the necessary rellocation. Here is a sample makefile for the package compilation. The ld is not interesting to show and simply has several special sections for our package recognition and an entry definition.

CC = arm-none-eabi-gcc
LD = arm-none-eabi-ld
READELF = arm-none-eabi-readelf
OBJDUMP = arm-none-eabi-objdump
PACKAGE = package_another

# https://developer.arm/documentation/101754/0623/armclang-Reference/armclang-Command-line-Options
#
# no-common: Avoid multiple definition of global variables in each translation unit
# fPIC: Generate position-independent code
# ffunction-sections & -fdata-sections: Allows linker to optimize section removal 
# nthumb-interwork: Generate code that supports calling between ARM and Thumb instructions
# mpic-data-is-text-relative: Helps the linker to optimize PIC code
CFLAGS = -c \
         -mfpu=vfpv3 \
         -mfloat-abi=hard \
         -mcpu=cortex-a9 \
         -fno-common \
         -fPIC \
         -ffunction-sections \
         -fdata-sections \
         -mthumb-interwork \
         -mpic-data-is-text-relative \
         -O0 \
         -Iinclude 
# r: Generate relocatable output
# gc-sections: Remove unused sections
# build-id=none: Do not generate a build ID
LDFLAGS = -r \
          --gc-sections \
          --build-id=none \
          --no-undefined



all: $(PACKAGE).elf readelf objdump

$(PACKAGE).o: $(PACKAGE).c
    $(CC) $(CFLAGS) $< -o $@

$(PACKAGE).elf: $(PACKAGE).o $(PACKAGE).ld
    $(LD) -T $(PACKAGE).ld $< -o $@ $(LDFLAGS)

readelf:
    $(READELF) -a $(PACKAGE).elf > READELF

objdump:
    $(OBJDUMP) -d $(PACKAGE).elf > OBJDUMP

.PHONY: all clean readelf

clean:
    rm -f $(PACKAGE).o $(PACKAGE).elf b

What We have tried

Because of the -r flag the only way to partially link the libgcc and libc is with the --whole-archive. So since we are on v7 (armv7) and we want the hard FP this is the path we chose:

/mnt/c/Xilinx/Vitis/2024.2/gnu/aarch32/nt/gcc-arm-none-eabi/aarch32-xilinx-eabi/usr/lib/thumb/v7+fp/hard/arm-xilinxmllibv7fphard-eabi/13.3.0/libgcc.a

Therefor, adding this line to the LDFLAGS

          --whole-archive /mnt/c/Xilinx/Vitis/2024.2/gnu/aarch32/nt/gcc-arm-none-eabi/aarch32-xilinx-eabi/usr/lib/thumb/v7+fp/hard/arm-xilinxmllibv7fphard-eabi/13.3.0/libgcc.a --no-whole-archive \

But of course for one this is an overkill (way to many symbols and we will probably only ever need at most a dozen), and two there are still many undefined symbols so we need to link something else?

Question 1

What approach should we take to have a minimal version of the libgcc linked to our package? Guessing that it's not crucial for it to be the same one that the RTOS has (since it is barely used anyway even by the RTOS kernel)...

Question 2

Should we try to get away from the Vitis IDE? The only thing really holding us connected to it is the build system, and the lopper utility that allows to regenerate the device tree even if the XSA changes, and that also updates the BSP. Ease of compilation is still kind of in place, however going further it seems to be posing more problems than giving solutions. I know that it can be scripted using python, and we were successful in implementing a project rebuilding functionality (for github source control). However the scripting features are quite limited and not super well explained in the documentation.

• c
• gcc
• ld
• vitis

• 2 Q2: FWIW, after paragraph 2 my head was spinning and I was thinking "vitis seems like a problem". For Q1: You don't need to do --whole-archive. See my answer: gcc ld: method to determine link order of static libraries If that's too much, here is the source: github/gcc-mirror/gcc/blob/master/libgcc/libgcc2.c You can include/exclude functions based on #define L_* cpp directives There are optimized asm versions as well. – Craig Estey Commented Mar 18 at 18:52
• BSP? Is the RTOS truly custom? Or, is it VxWorks? What about FreeRTOS or rtems? Or, linux? – Craig Estey Commented Mar 18 at 18:57
• You ask about linking the standard C library, but you seem actually to be talking about libgcc, which is something different (a compiler-specific support library). Please edit the question to clarify. – John Bollinger Commented Mar 19 at 4:13
• @JohnBollinger Yes indeed you are right! Sorry for th confusion – Vladouch Commented Mar 19 at 7:41
• @CraigEstey Yes... Sadly a necessity given the line of work, and tech debt (having a lot of client packages running on this OS)! I will have a look at your answer, thanks! – Vladouch Commented Mar 19 at 7:41

1 Answer 1

I found something that works for me pretty well! Basically just a way to find a local libgcc from the downloaded toolchain:

CC = arm-none-eabi-gcc
LD = arm-none-eabi-ld
READELF = arm-none-eabi-readelf
OBJDUMP = arm-none-eabi-objdump
PACKAGE = package_another

# Find the location of libgcc with the correct ABI
LIBGCC := $(shell $(CC) -mfpu=vfpv3 -mfloat-abi=hard -mcpu=cortex-a9 -print-libgcc-file-name)

# https://developer.arm/documentation/101754/0623/armclang-Reference/armclang-Command-line-Options
#
# no-common: Avoid multiple definition of global variables in each translation unit
# fPIC: Generate position-independent code
# ffunction-sections & -fdata-sections: Allows linker to optimize section removal 
# nthumb-interwork: Generate code that supports calling between ARM and Thumb instructions
# mpic-data-is-text-relative: Helps the linker to optimize PIC code
CFLAGS = -c \
         -mfpu=vfpv3 \
         -mfloat-abi=hard \
         -mcpu=cortex-a9 \
         -fno-common \
         -fPIC \
         -ffunction-sections \
         -fdata-sections \
         -mthumb-interwork \
         -mpic-data-is-text-relative \
         -O0 \
         -Iinclude 
# r: Generate relocatable output
# gc-sections: Remove unused sections
# build-id=none: Do not generate a build ID
LDFLAGS = -r \
          --gc-sections \
          --build-id=none \
          --no-undefined


all: $(PACKAGE).elf readelf objdump b

$(PACKAGE).o: $(PACKAGE).c
    $(CC) $(CFLAGS) $< -o $@

$(PACKAGE).elf: $(PACKAGE).o $(PACKAGE).ld
    $(LD) -T $(PACKAGE).ld $< -o $@ $(LDFLAGS) $(LIBGCC)

readelf:
    $(READELF) -a $(PACKAGE).elf > READELF

objdump:
    $(OBJDUMP) -d $(PACKAGE).elf > OBJDUMP

b: $(PACKAGE).elf
    python3 to_bin.py $< > $@

.PHONY: all clean readelf

clean:
    rm -f $(PACKAGE).o $(PACKAGE).elf b

Hope this can help anyone with a similar problem