I am having some issues installing Bio-DB-HTS (https://github.com/Ensembl/Bio-DB-HTS) requried to run a perl script from a cloned git repository.
System & Perl information
I am on Mac OSx High Sierra v.10.13.6 and using perl 5, version 18, subversion 2 (v5.18.2). I have added this information in my original question now.
Background information
When trying to perform a local installation, according to README instructions, i receive the error...
git clone https://github.com/Ensembl/Bio-DB-HTS.git
cd Bio-DB-HTS-2.10
perl INSTALL.pl
lzma.h library header not found in /usr/include
I tried to install the LZMA library but found that it actually was installed and that the lzma.h header file was simply missing from the path /usr/include. As LZMA is deprecated and replaced with XZ I installed the XZ library
brew install xz
After a quick search i found the lzma.h header in...
/usr/local/Cellar/xz/5.2.4/include/lzma.h
Now this is where I am not sure of how to proceed and if I have messed up something while trying to get around this. Since /usr/include has a directory with restricted access I added a line in the INSTALL.pl script to check existence for the file in both locations (which could probably disrupt something downstream in the analysis as I didn't change anything else except the if condition. However, when running the install this time i ran into a new problem.
perl INSTALL.pl
BioPerl does not seem to be installed. Please install it and try again.
On Debian/Ubuntu systems you can do this with the command:
apt-get install bioperl
On other systems use the CPAN shell:
perl -MCPAN -e 'install Bio::Perl'
And this is where my main issues reside. When trying to install Bio:Perl using cpan tests fail at various stage and I'm not sure which ones are the essential ones. The last lines from the output are
Result: FAIL
Failed 3/325 test programs. 16/19945 subtests failed.
CJFIELDS/BioPerl-1.007002.tar.gz
./Build test -- NOT OK
//hint// to see the cpan-testers results for installing this module,
try:
reports CJFIELDS/BioPerl-1.007002.tar.gz
Running Build install
make test had returned bad status, won't install without force
I reconfigured cpan to install dependancies automatically as mentioned here How do I tell CPAN to install all dependencies?.
perl -MCPAN -Mlocal::lib=~/perl5 -e 'my $c = "CPAN::HandleConfig"; $c->load(doit => 1, autoconfig => 1); $c->edit(prerequisites_policy => "follow"); $c->edit(build_requires_install_policy => "yes"); $c->commit'
And tried installing again forcing installation...
perl -f -MCPAN -e 'install Bio::Perl'
But I just get the same error
Result: FAIL
Failed 3/325 test programs. 16/19945 subtests failed.
CJFIELDS/BioPerl-1.007002.tar.gz
./Build test -- NOT OK
//hint// to see the cpan-testers results for installing this module,
try:
reports CJFIELDS/BioPerl-1.007002.tar.gz
Running Build install
make test had returned bad status, won't install without force
When I look at what modules have been installed using...
cpan -l
Bio::DB::HTS 2.11
Bio::DB::HTS::ReadIterator 2.11
Bio::DB::HTS::VCF 2.11
Bio::DB::HTS::Faidx 2.11
Bio::DB::HTS::PileupWrapper 2.11
Bio::DB::HTS::Alignment 2.11
Bio::DB::HTS::ConfigData undef
.
.
.
Bio::DB::HTS::VCF::Iterator 2.11
Bio::DB::HTS::VCF::Row 2.11
I can see that many of the ones I need from the Bio-DB-HTS package are there (assuming that they were one of the succesful installations when isntalling Bio::Perl) but now it prompts the error
Can't locate Bio/SeqFeature/Lite.pm in @INC
However, I never manage to actually get Bio:Seq nor Bio::Perl installed. I do have some perl knowledge but mostly work on python so I am feeling a bit lost of how to proceed.
Extra information
My cpan installs modules to
/usr/local/perl
And I have added the path to my environment variable
export PERL5LIB=/usr/local/perl
Edited information (failed tests & errors) as response to Shawn
@Shawn, Its a long list of errors, test failures and recommended installations. I I can post some examples of the first couple of errors. I am not really sure what to look neither.
Recommended installations
Checking prerequisites...
recommends:
* Algorithm::Munkres is not installed
* Array::Compare is not installed
* Bio::Phylo is not installed
* Convert::Binary::C is not installed
* GD is not installed
* Graph is not installed
* GraphViz is not installed
* HTML::TableExtract is not installed
* Inline::C (0.53) is installed, but we prefer to have 0.67
* PostScript::TextBlock is not installed
* SVG is not installed
* SVG::Graph is not installed
* Set::Scalar is not installed
* Sort::Naturally is not installed
* Spreadsheet::ParseExcel is not installed
* XML::DOM is not installed
* XML::DOM::XPath is not installed
* XML::Parser::PerlSAX is not installed
* XML::SAX::Writer is not installed
* XML::Twig is not installed
* YAML is not installed
Checking optional features...
EntrezGene............disabled
requires:
! Bio::ASN1::EntrezGene is not installed
MySQL Tests...........disabled
requires:
! DBD::mysql is not installed
Pg Tests..............disabled
requires:
! DBD::Pg is not installed
Here is the test summary report. I did not print the entire list of failed tests as it is extremely long. But What I cans see is that /LocalDB/SeqFeature_BDB.t is a part of the majority of the failed tests when looking at the verbose output.
Test Summary Report
-------------------
t/LocalDB/Fasta.t (Wstat: 1024 Tests: 109 Failed: 4)
Failed tests: 73, 91, 95, 101
Non-zero exit status: 4
t/LocalDB/Index/Index.t (Wstat: 20224 Tests: 36 Failed: 6)
Failed tests: 12-17
Non-zero exit status: 79
Parse errors: Bad plan. You planned 73 tests but ran 36.
t/LocalDB/Qual.t (Wstat: 1536 Tests: 56 Failed: 6)
Failed tests: 7-9, 49-50, 52
Non-zero exit status: 6
t/LocalDB/SeqFeature_BDB.t (Wstat: 0 Tests: 38 Failed: 4)
Failed tests: 17-19, 24
Parse errors: Bad plan. You planned 116 tests but ran 38.
t/Perl.t (Wstat: 512 Tests: 47 Failed: 16)
Failed tests: 28, 28, 28, 28-29, 29, 29, 29-30, 30, 30
30-31, 31, 31, 31
Non-zero exit status: 2
Parse errors: Tests out of sequence. Found (24) but expected (26)
Tests out of sequence. Found (25) but expected (27)
Tests out of sequence. Found (26) but expected (28)
Tests out of sequence. Found (26) but expected (29)
Tests out of sequence. Found (27) but expected (30)
Displayed the first 5 of 23 TAP syntax errors.
Re-run prove with the -p option to see them all.
t/RemoteDB/BioFetch.t (Wstat: 0 Tests: 83 Failed: 47)
Failed tests: 20-21, 21-22, 22-23, 23-24, 24-25, 25-26
26-27, 27-28, 28-29, 29-30, 30, 30, 30-31
31, 31, 31-32, 32, 32, 32-33, 33, 33, 33-34
34, 34, 34-35, 35, 35, 35-36, 36, 36, 36
Parse errors: Tests out of sequence. Found (4) but expected (6)
Tests out of sequence. Found (6) but expected (7)
Tests out of sequence. Found (7) but expected (8)
Tests out of sequence. Found (5) but expected (9)
Tests out of sequence. Found (6) but expected (10)
Displayed the first 5 of 79 TAP syntax errors.
Re-run prove with the -p option to see them all.
t/RemoteDB/GenBank.t (Wstat: 0 Tests: 658 Failed: 614)
Failed tests: 10-11, 11, 11-12, 12, 12-13, 13, 13-14
14, 14-15, 15, 15-16, 16, 16-17, 17, 17-18
18, 18-19, 19, 19, 19, 19, 19, 19-20, 20
20, 20, 20, 20, 20-21, 21, 21, 21, 21, 21
21-22, 22, 22, 22, 22, 22, 22-23, 23, 23
Thanks to everyone that helped out, I managed to figure it out in the end. I will explain the process here from beginning to end incase anyone else has the same issue.
The question I posted was to solve how to install Bio::Perl using CPAN as I was having issues with tests failing. Although I had some issues installing Bio-DB-HTS (which I solved before posting) and I will explain how I managed to install that as well in case anyone comes across the same issue.
It seems that Mac users tend to have an issue with the lzma.h header missing. In the case of installing Bio-DB-HTS. I had to modify a line in the Bio-DB-HTS/INSTALL.pl file that checked for the existence of the lzma.h file. See "Installing Bio-DB-HTS on Mac OSx" below for instructions.
Solving installation of Bio::Perl
Essentially I solved it by reinstalling/reconfiguring my CPAN installation. Although I believe the issue in the end was due to some environment variables that I had not set as I chose to manual sort out my CPAN directory structure, I recommend letting CPAN do this for you using the local:lib option as it will set or tell you how to set the environment variables at the end of the installation.
I had only set one of the environment variables below (PERL5LIB) which was probably the reason for my error. NOTE! that the paths you see below are specific for my system.
PATH="/Users/sjamal/perl5/bin${PATH:+:${PATH}}"; export PATH;
PERL5LIB="/Users/sjamal/perl5/lib/perl5${PERL5LIB:+:${PERL5LIB}}"; export PERL5LIB;
PERL_LOCAL_LIB_ROOT="/Users/sjamal/perl5${PERL_LOCAL_LIB_ROOT:+:${PERL_LOCAL_LIB_ROOT}}"; export PERL_LOCAL_LIB_ROOT;
PERL_MB_OPT="--install_base \"/Users/sjamal/perl5\""; export PERL_MB_OPT;
PERL_MM_OPT="INSTALL_BASE=/Users/sjamal/perl5"; export PERL_MM_OPT;
If you have already it configured like me but what to start from a clean slate you will need to remove the CPAN folder created on the user that you installed cpan on.
/Users/<USERNAME>/.cpan
rm -rf /Users/<USERNAME>/.cpan
Now you should be able to run the cpan command as done at the first instance and get prompted with lots of questions of how you want to set up the installation and this is where you will be able to chose 'local:lib' (if you are able to access sudo you can choose the 'sudo' option as well). I installed cpanm based on multiple recommendations and also since it prompts less question apparently and then installed Bio::Perl. Although I should mention that the installation did failed a test and refused to install so I had to run the force command for it to build Bio::Perl.
cpan -i App:cpanminus
cpanm --force Bio::Perl
You should now have Bio::Perl installed.
Installing Bio-DB-HTS on Mac OSx
The lzma library is deprecated on Mac OSx but has been replaced with the XZ library so if you are missing the file as well you can install XZ using brew. If you don't have brew installed you can find how to install it here https://brew.sh/
brew install xz
You will now have a lzma.h header file in the location where XZ was installed using brew, in my case /usr/local/Cellar/xz/5.2.4/include/lzma.h.
git clone https://github.com/Ensembl/Bio-DB-HTS.git
cd Bio-DB-HTS-2.10
vim Bio-DB-HTS/INSTALL.pl
So, I changed the line using vim...
-e '/usr/include/lzma.h' or die <<END;
to
-e '/usr/include/lzma.h' **|| '/usr/local/Cellar/xz/5.2.4/include/lzma.h'** or die <<END;
NOTE! You have to change '/usr/local/Cellar/xz/5.2.4/include/lzma.h' path to where you have the lzma.h in XZ package just installed using brew. However, I want to clarify that this does not solve anything other than telling the script that the file does exist. The Install script won't be able to make use of the file (if that is what it needs to do) as nothing else has been changed in the installation script.
You should now be able to install Bio-DB-HTS by simply running the perl script as below
cd Bio-DB-HTS-2.10
perl INSTALL.pl
Hope that helps!
Sabri