UGENE Forum
http://ugene.net/forum/YaBB.pl
General Category >> Bugs and Issues >> Compilation fails on Smith-Waterman plugin
http://ugene.net/forum/YaBB.pl?num=1528031285

Message started by Louis Becquey on Jun 3rd, 2018 at 8:08pm

Title: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jun 3rd, 2018 at 8:08pm
Hello,
I am trying to build Ugene from source (because i want it to work with Cuda).

Something goes wrong with the Smith-Waterman plugin that makes the build fail, it looks like a path issue:

qmake -r

Code:
...
Reading /home/persalteas/ugene-1.30.0/src/plugins/smith_waterman/smith_waterman.pro
QDir::mkpath: Empty or null file name
WARNING: nvzz: Cannot access directory ''
...


And if i try to build it anyway, it works well for a while, and finally fails on the Smith-Waterman plugin again:

make

Code:
...

nvcc -m 32 -g -G -Xcompiler -Zi,-MD -Xlinker /DEBUG -c -I -Isrc -I_tmp -I../../include -I../../corelibs/U2View/_tmp -L -lcudart -o _tmp/obj/release/sw_cuda.o src/sw_cuda.cu
nvcc warning : Compiling in the 32-bit mode when the host compiler targets x86 or x86_64 is no longer supported on Linux
gcc-6: error: unrecognized command line option ‘-Zi’; did you mean ‘-Z’?
Makefile.Release:292: recipe for target '_tmp/obj/release/sw_cuda.o' failed


I tried to edit src/plugins/smith_waterman/smith_waterman.pri to force -m 64 (removed the warning about 32 bit mode) and to replace -Zi by -Z. The compilation went a bit further, but failed again:


Code:
make[1] : entering directory « /home/persalteas/ugene-1.30.0/src/plugins/smith_waterman »
/usr/lib/qt5/bin/qmake -o Makefile smith_waterman.pro
QDir::mkpath: Empty or null file name
WARNING: nvzz: Cannot access directory ''

and then linking fails.


Additional info:
- Issue observed on two different machines:
  • a Centos 7 workstation with 8 core Intel Xeon, GCC 5, Nvidia driver 390, CUDA 9.2,
  • an Ubuntu 18.04 with 16-core AMD Ryzen, GCC 6, Nvidia 390 driver, CUDA 9.1.

- Used the source of UGENE 1.30, downloaded from here : ugene.unipro.ru/downloads/ugene_get_latest_source.html


Any idea what to do ? Any help would be appreciated, thanks in advance !


Title: Re: Compilation fails on Smith-Waterman plugin
Post by Olga Golosova on Jun 4th, 2018 at 2:26pm
Hello, Louis!

Thanks for your request!

We were able to reproduce the issue. We're currently trying to solve it and will write you as soon as there are news on that.

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jun 6th, 2018 at 12:23am
Thank you, i am waiting to hear from you, then.  :)

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Olga Golosova on Jun 7th, 2018 at 5:44pm
Hi Louis!


Quote:
Thank you, i am waiting to hear from you, then.

Yeap, sorry for the delay! We were able to reproduce the issue, but we haven't found a solution yet. We'll spend some time more to investigate the issue.


Title: Re: Compilation fails on Smith-Waterman plugin
Post by Olga Golosova on Jun 20th, 2018 at 6:53pm
Hi again!

I'm writing to inform you that we're still working on the issue. We fixed the compilation issues, but we have some problems with testing the issue on a Linux machine. Hopefully, we'll solve them in the nearest future.

Sorry again for the delay!

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jun 29th, 2018 at 2:39pm
That is nice to read :) thank you.
I suppose that when everything will be tested and fine, you will propose a new source archive to download on the website ?

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Olga Golosova on Jul 2nd, 2018 at 2:16pm
The latest source version with the issue fix is available on GitHub.

Please inform us, if it works fine on your system also.

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jul 2nd, 2018 at 6:58pm
Hello Olga,

I just tried after cloning the github source.

I still have the path warning with qmake -r (as in my first post), and then the build  failed (but for a different reason):

Code:
g++ -Wl,-O1 -Wl,-rpath-link,/usr/lib/x86_64-linux-gnu -shared -Wl,-soname,libsmith_waterman.so.1 -o libsmith_waterman.so.1.0.0 _tmp/obj/release/sw_cuda.o _tmp/obj/release/PairAlignSequences.o _tmp/obj/release/SmithWatermanAlgorithm.o _tmp/obj/release/SmithWatermanAlgorithmSSE2.o _tmp/obj/release/SWAlgorithmPlugin.o _tmp/obj/release/SWAlgorithmTask.o _tmp/obj/release/SmithWatermanAlgorithmCUDA.o _tmp/obj/release/SmithWatermanAlgorithmOPENCL.o _tmp/obj/release/sw_cuda_cpp.o _tmp/obj/release/SWTaskFactory.o _tmp/obj/release/SmithWatermanTests.o _tmp/obj/release/SWWorker.o _tmp/obj/release/SWQuery.o _tmp/obj/release/PairwiseAlignmentSmithWatermanGUIExtension.o _tmp/obj/release/qrc_smith_waterman.o _tmp/obj/release/moc_SWAlgorithmPlugin.o _tmp/obj/release/moc_SWAlgorithmTask.o _tmp/obj/release/moc_SmithWatermanTests.o _tmp/obj/release/moc_SWWorker.o _tmp/obj/release/moc_SWQuery.o _tmp/obj/release/moc_PairwiseAlignmentSmithWatermanGUIExtension.o  -L../../_release -lU2Core -lU2Algorithm -lU2Formats -lU2Gui -lU2View -lU2Test -lU2Lang -lU2Designer -L -lcudart -lQt5WebKit -lQt5Svg -lQt5Widgets -lQt5Gui -lQt5Script -lQt5Network -lQt5Xml -lQt5Core -lpthread -lGL 
/usr/bin/x86_64-linux-gnu-ld : i386 architecture of input file  « _tmp/obj/release/sw_cuda.o » is incompatible with  i386:x86-64 output


I am on a 64 bits system, and i did not modified any -m32/-m64 this time.

(Edited)

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Igor Chebykin on Jul 3rd, 2018 at 3:35pm
Hi Louis!

We use Qt Creator on Linux, so you can try the steps below.
Your environment and hardware may be some different, but I think it’s not very important.

Anyway, our CUDA platform is the following:
1)      X64 desktop with ASUS motherboard, Intel processor, 16 Gb memory, 1 Tb HDD
2)      Tesla K20c CUDA card
3)      All video is processed by embedded Intel graphic card
4)      CUDA card is used only for accelerating of the numeric computations. More over the card hasn’t any video output (some X11 setting is needed to setup).
5)      The Linux version is Ubuntu 18.04 which was installed on clean HDD from desktop variant of the Ubuntu 18.04 iso distribution.
6)      CUDA version (nvcc –version)
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Wed_Apr_11_23:16:29_CDT_2018
Cuda compilation tools, release 9.2, V9.2.88
7)      Nvidia driver version (output of the nvidia-smi command)
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.26                 Driver Version: 396.26                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K20c          Off  | 00000000:01:00.0 Off |                    0 |
| 32%   43C    P0    54W / 225W |      0MiB /  4743MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+


As I mentioned, we use Qt Creator on Linux, and we used the following steps to set the CUDA environment:
1) In the main qt creator window view click the button "Projects"
2) Build settings windows is opened. You need just add three env vars for CUDA:
    CUDA_HOME=/usr/local/cuda
    CUDA_INC_PATH=/usr/local/cuda/include
    CUDA_LIB_PATH=/usr/local/cuda/lib64
3) Another solution is to set these vars as global, for example in ~/.bashrc (before build):
    export CUDA_HOME=/usr/local/cuda
    export CUDA_INC_PATH=/usr/local/cuda/include
    export CUDA_LIB_PATH=/usr/local/cuda/lib64

Both solution works for me
Please don't forget to add cuda to PATH and LD_LIBRARY_PATH global vars too.

Thanks,
--Igor.

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jul 3rd, 2018 at 6:42pm
Hi Igor,

I don't know why my variables do not work on Centos, but i do not have the issue today on Ubuntu.

I updated my previous post, the problem I have now is a conflict between 32 bit and 64 bit objects that cannot be linked together (if i understand correctly).
I have not spend enough time on it yet. If you have trivial advice about it, it would be appreciated, otherwise i may investigate it in the coming days.

I'll keep you updated.
Thanks for your work again ! :)

Louis

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Igor Chebykin on Jul 3rd, 2018 at 7:14pm
Hi Louis!

Can you try to add CONFIG+=x64 to qmake command? And rebuild Ugene.
As I know cuda needs x64-bit compiler option.
So, if you want to have cuda support you need x64 compilation :(

Thanks,
Igor.

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Igor Chebykin on Jul 4th, 2018 at 2:32pm
Hi Louis!

How it's going?

--Igor

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jul 5th, 2018 at 11:08pm
Hi Igor,

Thanks a lot, the compilation successfully finished !  :)

Unfortunately, Ugene fails to start:

Code:
ugene -ui


I get the "Ugene crash reporter" window with the following message:

Code:
Exception with code C++ exception - Segmentation fault: invalid permissions.

Stack dump: /tmp/ugene_crashes/7d16a762-d0f4-ec27-0ebe1457-23511074.dmp

Operation system: Linux

CPU Info:
  Vendor :AuthenticAMD
  logical cpus: 16
  cpu cores: 16
  hyper-threads: false

Memory Info: 16031Mb

UGENE version: 1.31.0-dev x64

UUID: None

ActiveWindow: None

Log:
AppMemory: 1051Mb;
None
Task tree:
None
Stack trace:
/usr/lib/ugene/libU2Private.so.1(_ZN2U229CrashHandlerPrivateUnixNotMac15storeStackTraceEv+0x1e5)[0x7f254e7be2b5]
/usr/lib/ugene/libU2Private.so.1(_ZN2U212CrashHandler15handleExceptionERK7QStringS3_+0x45)[0x7f254e7bc205]
/usr/lib/ugene/libU2Private.so.1(_ZN2U229CrashHandlerPrivateUnixNotMac16breakpadCallbackERKN15google_breakpad18MinidumpDescriptorEPvb+0x95)[0x7f254e7bddb5]
/usr/lib/ugene/libbreakpad.so.1(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3fe)[0x7f254772dd6e]
/usr/lib/ugene/libbreakpad.so.1(+0x9f94)[0x7f254772df94]
/usr/lib/ugene/libbreakpad.so.1(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xa4)[0x7f254772e0e4]
/lib/x86_64-linux-gnu/libc.so.6(+0x3ef20)[0x7f2548f8af20]
/usr/lib/x86_64-linux-gnu/libQt5Core.so.5(_ZN4QDir20fromNativeSeparatorsERK7QString+0x18)[0x7f2549a770f8]
/usr/lib/x86_64-linux-gnu/libQt5Core.so.5(_ZN9QFileInfoC1ERK7QString+0x43)[0x7f2549a8a163]
/usr/lib/x86_64-linux-gnu/libQt5Core.so.5(_ZN11QTranslator4loadERK7QStringS2_S2_S2_+0x64)[0x7f2549bab1f4]
/usr/lib/ugene/ugeneui(+0x4c15b)[0x55970f84215b]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f2548f6db97]
/usr/lib/ugene/ugeneui(+0x5c53a)[0x55970f85253a]


Any idea ?

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Igor Chebykin on Jul 9th, 2018 at 2:30pm
Hello Louise!

Let's try I reproduce the issue.
I need to know:
   - gcc/g++ versions
   - Qt version
   - CUDA version
   - additional options for qmake/make you have used
   - additional environment vars added before compilation
   - build command(s)

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Louis Becquey on Jul 10th, 2018 at 12:37am
Hi Igor,

I restarted all the installation from scratch to give you an exhaustive description of what i did and.... it worked !

I think i know what the issue was : Cuda is not compatible with GCC/G++ 7 on Ubuntu 18.04 (yet). I think i used the 7 without noticing in my previous compilation.

As i did the work of listing everything, let's post it for the case someone would have the same issue:

- Do a fresh install of Ubuntu 18.04 (minimal install is enough).
- Do your updates (sudo apt update, sudo apt upgrade)
- Install libraries and building packages: sudo apt install build-essential git htop zlib1g-dev qt5-default qttools5-dev-tools qtscript5-dev libqt5svg5-dev libqt5webkit5-dev
- Install nvidia driver : From control center > additional drivers > nvidia-driver-390
- Install Cuda : sudo apt install nvidia-cuda-toolkit
- Install GCC 6 and set it by default:

Code:
sudo apt install gcc-6 g++-6
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-6 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 20
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 20
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
sudo update-alternatives --set cc /usr/bin/gcc
sudo update-alternatives --set c++ /usr/bin/g++
sudo update-alternatives --config gcc
sudo update-alternatives --config g++

- Then, create a directory and cd to it (mkdir -p ~/Sources & cd ~/Sources)
- Clone Ugene : git clone https://github.com/ugeneunipro/ugene.git
- cd ugene
- nano src/ugene_globals.pri and edit the line "isEmpty(UGENE_CUDA_DETECTED) : UGENE_CUDA_DETECTED = 0", replace 0 by 1
- return to Sources/ugene
- qmake -r CONFIG+=x64 (a warning about paths still appears after reading Smith-Waterman .pro)
- make -j 16 release
- sudo make install
- ugene -ui


And everything works !!  ;D  This topic can be marked [Solved].
Thanks a lot !

Louis

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Igor Chebykin on Jul 10th, 2018 at 12:49am
Hi Louis!

It's great!
Of course, we need to to update readme.
Anyway, I will try gcc6/7 before, as I use gcc7 :)

Maybe some files was not recompiled after you have played with options? But after full clearing (re-installation) all is ok.
Or, maybe qmake has some bug inside...

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Olga Golosova on Jul 12th, 2018 at 11:02am
Hi Louis,

I also have a general question to you: do you any recommendations in case we would like to add more CUDA-optimized tools into UGENE?

Title: Re: Compilation fails on Smith-Waterman plugin
Post by Priscilla Hamer on Aug 28th, 2018 at 8:33pm
That is pleasant to peruse Smiley bless your heart.

I assume that when everything will be tried and fine, you will propose another source document to download on the site ?

UGENE Forum » Powered by YaBB 2.5 AE!
YaBB Forum Software © 2000-2010. All Rights Reserved.