Editor’s Note 07/26/2017: This post provides an overview of why the gfortran error is occurring along with solutions to fix the macOS R versions:
Intro
Over the past weekend, I’ve been working on translating R code into Rcpp, a plugin C++ for R, to increase the speed of Markov Chain Monte Carlo (MCMC). There have been quite a few things I’ve learned. From the top: 1.) RStudio is fantastic since not only does it handle R, but it also works splendidly with Rcpp! 2.) Be prepared to write your own operations (i.e. matrix multiplication, solve, etc.) or use RcppArmadillo. 3.) Buzz Lightyear voice, “Compiler errors … everywhere!” 4.) Benchmark, benchmark, and then benchmark. I had such a positive experience with Rcpp, I most likely will be talking about it more in the future. The same cannot be said for initial experience I had with RcppArmadillo, a critical extension package of Rcpp that all Statisticians will want to use.
The Problem
The initial experience I had with RcppArmadillo was problematic since it didn’t work out of the box even with the slew of development tools (XCode) I have installed on macOS. Therefore, I would like to draw attention to a wee bit of a hiccup users can potential experience on macOS starting at OS X Mavericks (OS X 10.9) and going until macOS 10.12 Sierra. Chiefly, the show-stopping errors of “-lgfortran” and “-lquadmath.”
Examples of the error texts in R 3.0.0 - 3.3.0 are:
“-lgfortran” Error
ld: warning: directory not found for option
'-L/usr/local/lib/gcc/i686-apple-darwin8/4.2.3/x86_64'
ld: warning: directory not found for option '-L/usr/local/lib/x86_64'
ld: warning: directory not found for option
'-L/usr/local/lib/gcc/i686-apple-darwin8/4.2.3'
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see
invocation)“-lquadmath” Error
ld: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'
ld: library not found for -lquadmath
clang: error: linker command failed with exit code 1 (use -v to see invocation) make: ***
[sourceCpp_93074.so] Error 1 clang++
-I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include"
-I"/Library/Frameworks/R.framework/Versions/3.1/Resources/library/RcppArmadillo/include"
-fPIC -Wall -mtune=core2 -g -O2 -c rmvrnorm_arma.cpp -o rmvrnorm_arma.o clang++ -dynamiclib -Wl,-headerpad_max_install_names
-undefined dynamic_lookup -single_module -multiply_defined suppress -L/usr/local/lib -o sourceCpp_93074.so rmvrnorm_arma.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2 -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundationExamples of the error text in R 3.4.0 would be:
ld: warning: directory not found for option
'-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: warning: directory not found for option '-L/usr/local/gfortran/lib'
ld: warning: directory not found for option
'-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see
invocation)The good news is that both errors originate from the same problem. Mainly, R for macOS was compiled using either gfortran-4.8 (R 3.0.0 - R 3.3.0) or gfortran-6.1 (>= R 3.4.0). What does this mean?
For starters, the official cran mirror of R tools for OS X and R tools for OS X on r.research.att.com that lists the gfortran binary are out of date. However, obtaining the correct gfortran binary is slightly different depending on your version of R. The reason for this is R 3.4.0 began using OpenMP whereas prior versions omitted it.
In particular, for those users on R 3.0.0 - R 3.3., the official gfortran binaries break the compile chain because the binaries listed were not used to compile R. As a result, using gfortran will lead you to switch from receiving a “-lgfortran” error to a “-lquadmath” error since a few files are missing from the expected locations.
In constrast, for R 3.4.0 and later versions of R, you must obtain the official gfortran binary.
The Solution
Please note, the solution has changed to be dependent on the R version being run.
R 3.0.0 - R 3.3.0
Go to the optional libraries, frameworks and applications for Mac OS X on r.research.att.com and download gfortran-4.8.2-darwin13.tar.bz2. Extract the package in /, which is root. The files should be unpacked into /usr/local/ (see the appendix for file structure).
Alternatively, open Terminal and type:
curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
sudo tar fvxz gfortran-4.8.2-darwin13.tar.bz2 -C /Removing R’s gfortran-4.8.2 binary
With the transition to R 3.4.0 requiring a different set of binaries, you may wish to remove the above binary to free up space. The best way to go about it is by running a bash script that first deletes files found in the archive and then attempts to remove the directories containing the files (assuming they are empty). The reason for this approach is this binary was designed at a time when code was integrated tightly with macOS routines.
# Download installer into working directory
curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
# Remove _files_ associated with the binary
for file in $(tar tfz gfortran-4.8.2-darwin13.tar.bz2); do
sudo rm -f /$file;
done
# Remove empty _folders_ associated with the binary
for file in $(tar tfz gfortran-4.8.2-darwin13.tar.bz2); do
sudo rmdir -p /$file;
done
# Delete the installer
rm -rf gfortran-4.8.2-darwin13.tar.bz2>= R 3.4.*
Depending on whether you are on OS X El Capitan (10.11) or macOS Sierra (10.12) there are two installation paths to take regarding obtaining official gfortran binary from http://gcc.gnu.org/wiki/GFortranBinaries#MacOS.
OS X El Capitan (10.11) or gfortran 6.1.0
By default, the R core team builds the macOS R application on OS X El Capitan. Thus, the paths for compilation are configured to use the official gfortran 6.1.0 binary for OS X El Capitan (10.11) from http://gcc.gnu.org/wiki/GFortranBinaries#MacOS. As a result, all one needs to do is run the downloaded installer and rejoice!
Note: This advice holds even if you are not on El Capitan (10.11) but are on the macOS Sierra (10.12) or later macOS versions on the R 3.4.* line.
macOS Sierra (10.12) or gfortran 6.3.0
The following provides a procedure to use the macOS Sierra gfortran routine with R for those on macOS Sierra. Please note, if you installed the gfortran 6.1.0 binary, then do NOT use the directions below for installing gfortran 6.3.0.
First, download the official gfortran 6.3.0 binary for macOS Sierra (10.12) from http://gcc.gnu.org/wiki/GFortranBinaries#MacOS and then install it.
Next, we need to set a path in a file called ~/.R/Makevars, which handles compilation, so that it recognize the gfortran 6.3.0 binary. So, we’ll create a ~/.R/Makevars file – if one does not exist – and then add a special implicit variable called FLIBS that controls which Fortran library is used. This is detailed in Section 6.3.2: macOS of the R Installation and Administration
Open Terminal and type:
mkdir ~/.R
cat << EOF >> ~/.R/Makevars
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin16/6.3.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
EOFLater
Check back when new versions of R are released.
Fin
Happy Rcpp and RcppArmadillo ’in on OS X Mavericks through macOS!
Appendix
To view the file structure of the gfortran binaries from http://r.research.att.com/libs/ we can extract it from the archive using:
## Assumes homebrew is installed
# brew install tree
curl -O ~/Downloads/gfortran-4.8.2-darwin13.tar.bz2 http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
tar fvxz gfortran-4.8.2-darwin13.tar.bz2
tree usrExtracted directory structure:
usr
└── local
├── bin
│ ├── gfortran -> gfortran-4.8
│ ├── gfortran-4.8 -> x86_64-apple-darwin13.0.0-gfortran
│ └── x86_64-apple-darwin13.0.0-gfortran
├── lib
│ ├── gcc
│ │ └── x86_64-apple-darwin13.0.0
│ │ └── 4.8.2
│ │ ├── crt3.o
│ │ ├── crtfastmath.o
│ │ ├── crtprec32.o
│ │ ├── crtprec64.o
│ │ ├── crtprec80.o
│ │ ├── crttme.o
│ │ ├── crttms.o
│ │ ├── finclude
│ │ │ ├── omp_lib.f90
│ │ │ ├── omp_lib.h
│ │ │ ├── omp_lib.mod
│ │ │ └── omp_lib_kinds.mod
│ │ ├── i386
│ │ │ ├── crt3.o
│ │ │ ├── crtfastmath.o
│ │ │ ├── crtprec32.o
│ │ │ ├── crtprec64.o
│ │ │ ├── crtprec80.o
│ │ │ ├── crttme.o
│ │ │ ├── crttms.o
│ │ │ ├── libcaf_single.a
│ │ │ ├── libcaf_single.la
│ │ │ ├── libgcc.a
│ │ │ ├── libgcc_eh.a
│ │ │ ├── libgcov.a
│ │ │ ├── libgfortranbegin.a
│ │ │ └── libgfortranbegin.la
│ │ ├── include
│ │ │ ├── adxintrin.h
│ │ │ ├── ammintrin.h
│ │ │ ├── avx2intrin.h
│ │ │ ├── avxintrin.h
│ │ │ ├── bmi2intrin.h
│ │ │ ├── bmiintrin.h
│ │ │ ├── bmmintrin.h
│ │ │ ├── cpuid.h
│ │ │ ├── cross-stdarg.h
│ │ │ ├── emmintrin.h
│ │ │ ├── f16cintrin.h
│ │ │ ├── float.h
│ │ │ ├── fma4intrin.h
│ │ │ ├── fmaintrin.h
│ │ │ ├── fxsrintrin.h
│ │ │ ├── ia32intrin.h
│ │ │ ├── immintrin.h
│ │ │ ├── iso646.h
│ │ │ ├── lwpintrin.h
│ │ │ ├── lzcntintrin.h
│ │ │ ├── mm3dnow.h
│ │ │ ├── mm_malloc.h
│ │ │ ├── mmintrin.h
│ │ │ ├── nmmintrin.h
│ │ │ ├── omp.h
│ │ │ ├── pmmintrin.h
│ │ │ ├── popcntintrin.h
│ │ │ ├── prfchwintrin.h
│ │ │ ├── quadmath.h
│ │ │ ├── quadmath_weak.h
│ │ │ ├── rdseedintrin.h
│ │ │ ├── rtmintrin.h
│ │ │ ├── smmintrin.h
│ │ │ ├── ssp
│ │ │ │ ├── ssp.h
│ │ │ │ ├── stdio.h
│ │ │ │ ├── string.h
│ │ │ │ └── unistd.h
│ │ │ ├── stdalign.h
│ │ │ ├── stdarg.h
│ │ │ ├── stdbool.h
│ │ │ ├── stddef.h
│ │ │ ├── stdfix.h
│ │ │ ├── stdint-gcc.h
│ │ │ ├── stdint.h
│ │ │ ├── stdnoreturn.h
│ │ │ ├── tbmintrin.h
│ │ │ ├── tgmath.h
│ │ │ ├── tmmintrin.h
│ │ │ ├── unwind.h
│ │ │ ├── varargs.h
│ │ │ ├── wmmintrin.h
│ │ │ ├── x86intrin.h
│ │ │ ├── xmmintrin.h
│ │ │ ├── xopintrin.h
│ │ │ ├── xsaveintrin.h
│ │ │ ├── xsaveoptintrin.h
│ │ │ └── xtestintrin.h
│ │ ├── include-fixed
│ │ │ ├── README
│ │ │ ├── limits.h
│ │ │ ├── math.h
│ │ │ ├── stdint.h
│ │ │ └── syslimits.h
│ │ ├── install-tools
│ │ │ ├── fixinc_list
│ │ │ ├── gsyslimits.h
│ │ │ ├── include
│ │ │ │ ├── README
│ │ │ │ └── limits.h
│ │ │ ├── macro_list
│ │ │ └── mkheaders.conf
│ │ ├── libcaf_single.a
│ │ ├── libcaf_single.la
│ │ ├── libgcc.a
│ │ ├── libgcc_eh.a
│ │ ├── libgcov.a
│ │ ├── libgfortranbegin.a
│ │ ├── libgfortranbegin.la
│ │ └── plugin
│ │ ├── gtype.state
│ │ └── include
│ │ ├── ada
│ │ │ └── gcc-interface
│ │ │ └── ada-tree.def
│ │ ├── alias.h
│ │ ├── all-tree.def
│ │ ├── alloc-pool.h
│ │ ├── ansidecl.h
│ │ ├── auto-host.h
│ │ ├── b-header-vars
│ │ ├── basic-block.h
│ │ ├── bitmap.h
│ │ ├── builtins.def
│ │ ├── bversion.h
│ │ ├── c-family
│ │ │ ├── c-common.def
│ │ │ ├── c-common.h
│ │ │ ├── c-objc.h
│ │ │ ├── c-pragma.h
│ │ │ └── c-pretty-print.h
│ │ ├── c-tree.h
│ │ ├── cfg-flags.def
│ │ ├── cfghooks.h
│ │ ├── cfgloop.h
│ │ ├── cgraph.h
│ │ ├── cif-code.def
│ │ ├── config
│ │ │ ├── darwin-protos.h
│ │ │ ├── darwin-sections.def
│ │ │ ├── darwin.h
│ │ │ ├── darwin10.h
│ │ │ ├── darwin9.h
│ │ │ ├── i386
│ │ │ │ ├── biarch64.h
│ │ │ │ ├── darwin.h
│ │ │ │ ├── darwin64.h
│ │ │ │ ├── i386-opts.h
│ │ │ │ ├── i386-protos.h
│ │ │ │ └── i386.h
│ │ │ ├── initfini-array.h
│ │ │ └── vxworks-dummy.h
│ │ ├── config.h
│ │ ├── configargs.h
│ │ ├── coretypes.h
│ │ ├── cp
│ │ │ ├── cp-tree.def
│ │ │ ├── cp-tree.h
│ │ │ ├── cxx-pretty-print.h
│ │ │ └── name-lookup.h
│ │ ├── cppdefault.h
│ │ ├── cpplib.h
│ │ ├── debug.h
│ │ ├── defaults.h
│ │ ├── diagnostic-core.h
│ │ ├── diagnostic.def
│ │ ├── diagnostic.h
│ │ ├── double-int.h
│ │ ├── dumpfile.h
│ │ ├── emit-rtl.h
│ │ ├── except.h
│ │ ├── filenames.h
│ │ ├── fixed-value.h
│ │ ├── flag-types.h
│ │ ├── flags.h
│ │ ├── function.h
│ │ ├── gcc-plugin.h
│ │ ├── genrtl.h
│ │ ├── ggc.h
│ │ ├── gimple-pretty-print.h
│ │ ├── gimple.def
│ │ ├── gimple.h
│ │ ├── gsstruct.def
│ │ ├── gtm-builtins.def
│ │ ├── gtype-desc.h
│ │ ├── hard-reg-set.h
│ │ ├── hashtab.h
│ │ ├── highlev-plugin-common.h
│ │ ├── hwint.h
│ │ ├── incpath.h
│ │ ├── input.h
│ │ ├── insn-constants.h
│ │ ├── insn-flags.h
│ │ ├── insn-modes.h
│ │ ├── insn-notes.def
│ │ ├── internal-fn.def
│ │ ├── internal-fn.h
│ │ ├── intl.h
│ │ ├── ipa-prop.h
│ │ ├── ipa-ref-inline.h
│ │ ├── ipa-ref.h
│ │ ├── ipa-reference.h
│ │ ├── ipa-utils.h
│ │ ├── is-a.h
│ │ ├── java
│ │ │ └── java-tree.def
│ │ ├── langhooks.h
│ │ ├── libiberty.h
│ │ ├── line-map.h
│ │ ├── machmode.h
│ │ ├── md5.h
│ │ ├── mode-classes.def
│ │ ├── objc
│ │ │ └── objc-tree.def
│ │ ├── obstack.h
│ │ ├── omp-builtins.def
│ │ ├── options.h
│ │ ├── opts.h
│ │ ├── output.h
│ │ ├── params.def
│ │ ├── params.h
│ │ ├── plugin-api.h
│ │ ├── plugin-version.h
│ │ ├── plugin.def
│ │ ├── plugin.h
│ │ ├── pointer-set.h
│ │ ├── predict.def
│ │ ├── predict.h
│ │ ├── prefix.h
│ │ ├── pretty-print.h
│ │ ├── real.h
│ │ ├── realmpfr.h
│ │ ├── reg-notes.def
│ │ ├── rtl.def
│ │ ├── rtl.h
│ │ ├── safe-ctype.h
│ │ ├── sanitizer.def
│ │ ├── sbitmap.h
│ │ ├── splay-tree.h
│ │ ├── statistics.h
│ │ ├── symtab.h
│ │ ├── sync-builtins.def
│ │ ├── system.h
│ │ ├── target-hooks-macros.h
│ │ ├── target.def
│ │ ├── target.h
│ │ ├── timevar.def
│ │ ├── timevar.h
│ │ ├── tm-preds.h
│ │ ├── tm.h
│ │ ├── tm_p.h
│ │ ├── toplev.h
│ │ ├── tree-check.h
│ │ ├── tree-dump.h
│ │ ├── tree-flow-inline.h
│ │ ├── tree-flow.h
│ │ ├── tree-inline.h
│ │ ├── tree-iterator.h
│ │ ├── tree-pass.h
│ │ ├── tree-pretty-print.h
│ │ ├── tree-ssa-alias.h
│ │ ├── tree-ssa-operands.h
│ │ ├── tree-ssa-sccvn.h
│ │ ├── tree.def
│ │ ├── tree.h
│ │ ├── treestruct.def
│ │ ├── vec.h
│ │ └── version.h
│ ├── libgcc_ext.10.4.dylib
│ ├── libgcc_ext.10.5.dylib
│ ├── libgcc_s.1.dylib -> libgcc_s_x86_64.1.dylib
│ ├── libgcc_s_x86_64.1.dylib
│ ├── libgfortran.3.dylib
│ ├── libgfortran.a
│ ├── libgfortran.dylib -> libgfortran.3.dylib
│ ├── libgfortran.la
│ ├── libgfortran.spec
│ ├── libgomp.1.dylib
│ ├── libgomp.a
│ ├── libgomp.dylib -> libgomp.1.dylib
│ ├── libgomp.la
│ ├── libgomp.spec
│ ├── libquadmath.0.dylib
│ ├── libquadmath.a
│ ├── libquadmath.dylib -> libquadmath.0.dylib
│ └── libquadmath.la
├── libexec
│ └── gcc
│ └── x86_64-apple-darwin13.0.0
│ └── 4.8.2
│ ├── cc1
│ ├── collect2
│ ├── f951
│ ├── lto-wrapper
│ └── plugin
│ └── gengtype
└── share
└── man
├── man1
│ └── gfortran.1
└── man7
├── fsf-funding.7
├── gfdl.7
└── gpl.7