Fixes for Problems with LLVM

Solutions to banach-space / llvm-tutor HelloWorld: Your First Pass

llvm opt gives error: unexpected type in metadata definition for clang

LLVM version 13.0 program opt gives with default version of Clang 3.4.2

error: unexpected type in metadata definition

Work around

Re-install LLVM using option -DLLVM_ENABLE_PROJECTS=clang
(takes depending on internet speed approx 3 hours)

make fails with llvm/IR/PassManager.h: No such file or directory

[  4%] Building CXX object CMakeFiles/MergeBB.dir/MergeBB.o
In file included from ____/tutor/lib/MergeBB.cpp:44:
_____/tutor/lib/../include/MergeBB.h:13:10: fatal error: llvm/IR/PassManager.h: No such file or directory
   13 | #include "llvm/IR/PassManager.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~

Work around

Recent versions of LLVM include llvm/IR/PassManager.h I.e. look in your release kit rather than in llvm-tutor

Hacky does not seem to easy to get llvm to point to LLVM release kit using environment variables. Possibly copy llvm/IR/PassManager.h and (all the other .h files you need) into your include directory (note directory structure) or even worse use soft directory link (you may also need llvm-c). 

cd ../include/
ln -s $LLVM_DIR/include/llvm
ln -s $LLVM_DIR/include/llvm-c

make fails with error: 'llvm::PassBuilder::OptimizationLevel' has not been declared

g++ -DOpcodeCounter_EXPORTS -I../include -fPIC -o CMakeFiles/OpcodeCounter.dir/OpcodeCounter.o -c OpcodeCounter.cpp
OpcodeCounter.cpp: In lambda function:
OpcodeCounter.cpp:124:37: error: ‘llvm::PassBuilder::OptimizationLevel’ has not been declared
  124 |                  llvm::PassBuilder::OptimizationLevel Level) {
      |                                     ^~~~~~~~~~~~~~~~~
OpcodeCounter.cpp:126:16: error: cannot convert ‘getOpcodeCounterPluginInfo()::::’ to ‘const std::function&, llvm::OptimizationLevel)>&’
  126 |               });
      |                ^
In file included from OpcodeCounter.cpp:35:
llvm/Passes/PassBuilder.h:448:76: note:   initializing argument 1 of ‘void llvm::PassBuilder::registerVectorizerStartEPCallback(const std::function&, llvm::OptimizationLevel)>&)’
  448 |       const std::function &C) {
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
139 ucacbbl@eden% g++ -DOpcodeCounter_EXPORTS -I../include -fPIC -o CMakeFiles/OpcodeCounter.dir/OpcodeCounter.o -c t.cpp

No Work around

Gave up. LLVM is broken.
Comment out PB.registerVectorizerStartEPCallback(..);

Remember to restart make to compile and link remaining files

With line 124 removed libOpcodeCounter.so fails with LLVM opt: 

$LLVM_DIR/bin/opt -load ../tutor/lib/libOpcodeCounter.so -legacy-opcode-counter -analyze input_for_cc.ll
Cannot specify -analyze under new pass manager, either specify '-enable-new-pm=0', or use the corresponding new pass manager pass, e.g. '-passes=print'. For a full list of passes, see the '--print-passes' flag.

stackoverflow.com says "The LLVM binaries are built without runtime type info"

LLVM opt fails missing undefined symbol: _ZTIN4llvm18format_object_baseE

$LLVM_DIR/bin/opt -load ../tutor/lib/libOpcodeCounter.so -legacy-opcode-counter -analyze input_for_cc.ll
Error opening '../tutor/lib/libOpcodeCounter.so': ../tutor/lib/libOpcodeCounter.so: undefined symbol: _ZTIN4llvm18format_object_baseE

LLVM opt fails missing undefined symbol: _ZTIN4llvm10ModulePassE

$LLVM_DIR/bin/opt -load ../tutor/lib/libInjectFuncCall.so -legacy-inject-func-call input_for_hello.bc -o instrumented.bin                                                                                                       Error opening '../tutor/lib/libInjectFuncCall.so': ../tutor/lib/libInjectFuncCall.so: undefined symbol: _ZTIN4llvm10ModulePassE
  -load request ignored.

Work around

Compile with -fno-rtti

https://stackoverflow.com/questions/17225956/developing-an-llvm-pass-with-cmake-out-of-llvm-source-directory actually suggests -Wall -fno-rtti but only need -fno-rtti Achieved by adding set(CMAKE_CXX_FLAGS "-Wall -fno-rtti") to CMakeLists.txt and running cmake followed by make again.

llvm opt gives unknown pass name 'legacy-inject-func-call'

https://llvm.org/docs/WritingAnLLVMPass.html says now (version 14 onwards?) LLVM opt assumes the new LLVM pass manager.

Work around

Force opt to use legacy pass manager with -enable-new-pm=0 For example:
$LLVM_DIR/bin/opt -enable-new-pm=0 -load ../tutor/lib/libInjectFuncCall.so -legacy-inject-func-call input_for_hello.bc -o instrumented.bin

llvm opt gives empty 'LegacyStaticCallCounter': output

Using https://github.com/banach-space/llvm-tutor#staticcallcounter gives unexpected output 
$LLVM_DIR/bin/opt -enable-new-pm=0 -load ../tutor/lib/libStaticCallCounter.so -legacy-static-cc -analyze input_for_cc.ll
Printing analysis 'LegacyStaticCallCounter':
=================================================
LLVM-TUTOR: static analysis results
=================================================
NAME                 #N DIRECT CALLS
-------------------------------------------------
-------------------------------------------------

Work around

Did you reuse input_for_cc.ll from an earlier example? Was it compiled with $LLVM_DIR/bin/clang -O1 ? Clang -O1 optimisation changes the function headers in LLVM IR intermediate representation. They are not recognised by StaticCallCounter

Fix recompile input_for_cc.ll without -O1 and re-run. 

$LLVM_DIR/bin/clang -S -emit-llvm -c ../tutor/inputs/input_for_cc.c -o input_for_cc.ll

$LLVM_DIR/bin/opt -enable-new-pm=0 -load ../tutor/lib/libStaticCallCounter.so -legacy-static-cc -analyze input_for_cc.ll
Printing analysis 'LegacyStaticCallCounter':
=================================================
LLVM-TUTOR: static analysis results
=================================================
NAME                 #N DIRECT CALLS
-------------------------------------------------
foo                  3         
bar                  2         
fez                  1         
-------------------------------------------------

$LLVM_DIR/bin/static Command not found.

LLVM tool static not included by default in LLVM 14 build

llvm opt gives error: expected top-level entity

$LLVM_DIR/bin/opt -enable-new-pm=0 -load ../tutor/lib/libMBAAdd.so -legacy-mba-add -mba-ratio=0.3 ../tutor/inputs/input_for_mba.c -o out_obfuscation.ll
/cs/sys/software2/llvm/llvm14//bin/opt: ../tutor/inputs/input_for_mba.c:1:1: error: expected top-level entity
//=============================================================================
^

Work around

Something weird with example taken from https://github.com/banach-space/llvm-tutor#mbaadd? Why is opt taking input from plain C code input_for_mba.c rather than IR intermediate code?
Perhaps command line should be 
$LLVM_DIR/bin/opt -enable-new-pm=0 -load ../tutor/lib/libMBAAdd.so -legacy-mba-add -mba-ratio=0.3 -S input_for_mba.ll -o out_obfuscation.ll

Find floating-point equality comparisons FindFCmpEq no output with New Pass Manager

The example FindFCmpEq which is supposed to find floating-point equality comparisons in LLVM IR does not give any output with the new pass manager. (Although opt gives an ok status code.) 
$LLVM_DIR/bin/opt --load-pass-plugin ../tutor/lib/libFindFCmpEq.so --passes="print" -disable-output input_for_fcmp_eq.ll
No output.

Find floating-point equality comparisons FindFCmpEq unexpected output with Legacy Pass Manager

The example FindFCmpEq which is supposed to find floating-point equality comparisons in LLVM IR gives output that is different from the example with the legacy pass manager.
It appears part of the answer lies in the IR produced by Clang not being as expected by the example.
Note we still have one IR instruction which is still unexplained. Ok (see below).

Note used of -enable-new-pm=0, which is needed for LLVM 14 onwards (see above). 

$LLVM_DIR/bin/opt -enable-new-pm=0 -load ../tutor/lib/libFindFCmpEq.so -find-fcmp-eq -analyze input_for_fcmp_eq.ll
Printing analysis 'Floating-point equality comparisons locator' for function 'sqrt_impl':
Floating-point equality comparisons in "sqrt_impl":
  %11 = fcmp oeq double %9, %10
Printing analysis 'Floating-point equality comparisons locator' for function 'sqrt':
Printing analysis 'Floating-point equality comparisons locator' for function 'main':
Floating-point equality comparisons in "main":
  %9 = fcmp oeq double %8, 1.000000e+00
  %13 = fcmp oeq double %11, %12
  %19 = fcmp oeq double %17, %18
opt's output is not as the example.

Notice %24 = fcmp olt double %22, %23 is not reported (due to being olt rather than a fcmp oeq).

LLVM Interpreter LLI run time errors

JIT session error: Symbols not found: [ _ZNKSt9basic_iosIcSt11char_traitsIcEEcvbEv ]
lli: Failed to materialize symbols: { (main, { ... } }

The missing symbol is a C++ mangled name. char_traits is part of the GNU C++ standard library. However it may not be in the standard C++ runtime library but instead in libstdc++_nonshared.a
It appears by default lli (LLVM version 14) does not find it and so reports this verbose error message.

Work around

Notice

  1. libstdc++_nonshared.a is an archive (.a), rather than a shared runtime library (.so), so use --extra-archive= (rather than --dlopen= for .so).
  2. libstdc++_nonshared.a is under the /lib/ directory of the gcc compiler installation.
  3. For missing C (rather than C++) symbols you may want libc_nonshared.a which might be in the directory /usr/lib64/
  4. lli --jit-kind=orc-lazy may help.
Example solution:
lli --extra-archive=/opt/rh/devtoolset-10/root/usr/lib/gcc/x86_64-redhat-linux/10/libstdc++_nonshared.a file_name.ll program arguments

W.B.Langdon Back 2 February 2022 (last update $Date: 2022/08/19 20:28:21 $)