R Compiler Tools for Rcpp on macOS before R 3.6.0

programming
cpp
Author

TheCoatlessProfessor

Published

May 27, 2018

Editor’s Note: This is an updated post that covers how to install the macOS toolchain for versions of R starting at 3.5.z.

For other R versions, please see:

Intro

The objective behind this post is to provide users with information on how to setup the macOS toolchain for compiling used in the 3.5.* series of R. This has been a bit problematic for many R users since OS X Mavericks, which resulted in gfortran binaries being dropped from the R installer. More curiously, the additional demand to have access to a compiler vs. downloading a binary from CRAN became apparent slightly after Rcpp’s 0.10.0 version, when attributes where added that removed the necessity to use R’s SEXP objects.

Removing Old Installation Files

If you previously used the clang4 installer, please consider deleting the old components that were installed.

Instructions for uninstallation can be found here:

Uninstalling the R development toolchain on macOS

Installation Instructions

There are two ways to go about setting up the R toolchain for compiled code on macOS:

  1. Bash script that downloads and installs the elements
  2. Individually installing each element yourself.

Pick one and follow it to completion.

There are three components to the R 3.5.x toolchain based on Section: C.3 macOS of R Installation and Administration. These components are:

  1. Xcode Command Line Tools (“Xcode CLI”)
  2. clang6
  3. gfortran6.1

Historical Information

The toolchain has changed overtime with respect to the compiler and gfortran requirements. In particular, in prior version of R the compiler was:

  • R 3.4.z: clang4
  • R 3.0.0 - 3.3.3: the Xcode CLI clang binary

The gfortran binary has shifted from a custom version of gfortran compiled by the CRAN macOS maintainers to the official binaries by the GNU project.

Bash Script

The bash script acts as a way to automatically acquire all of the official components of the R toolchain on macOS.

To use this bash script please:

  1. Open the Terminal from /Applications/Utilities/.
  2. Copy and paste the following bash script:
########### Xcode CLI

# Headless install of Xcode CLI
# Based on a script by Timothy Sutton, MIT licensed 2013 - 2014
# The code used is given at:
# https://github.com/timsutton/osx-vm-templates/blob/ce8df8a7468faa7c5312444ece1b977c1b2f77a4/scripts/xcode-cli-tools.sh#L8-L14

# Check if the Xcode CLI tool directory exists.
# See technical note: https://developer.apple.com/library/content/technotes/tn2339/_index.html#//apple_ref/doc/uid/DTS40014588-CH1-WHAT_IS_THE_COMMAND_LINE_TOOLS_PACKAGE_
# Note: This is not a rigorous check... So, if a user has deleted contents
# inside the folder but left the folder intact, then this will _not_ trigger
# an installation
if [ ! -d "/Library/Developer/CommandLineTools" ]; then

    # Create a temporary file for the header
    touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress

    # Figure out the correct Xcode CLI for the given mac OS
    PROD=$(sudo softwareupdate -l |
        grep "\*.*Command Line" |
        tail -n 1 | awk -F"*" '{print $2}' |
        sed -e 's/^ *//' |
        tr -d '\n')

    # Install Xcode CLI      
    sudo softwareupdate -i "$PROD" --verbose;

    rm -rf /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
else
    echo "Xcode CLI is installed..."    
fi

########### clang6

# Download and Install the clang6 binary 
# Download ~440mb -> 2 gb installed
curl -O https://cran.r-project.org/bin/macosx/tools/clang-6.0.0.pkg
sudo installer -pkg clang-6.0.0.pkg -target /
rm -rf clang-6.0.0.pkg

# Create an R environment file if it doesn't exist to store a modified path
# VARIABLE
if [ ! -e "~/.Renviron" ] ; then
   touch ~/.Renviron
fi

# Add the clang6 binary path to R's local paths
echo 'PATH="/usr/local/clang6/bin:${PATH}"' >> ~/.Renviron

########### gfortran

# Download and install the gfortran used in R 3.5.0
curl -O https://cloud.r-project.org/bin/macosx/tools/gfortran-6.1.pkg
sudo installer -pkg gfortran-6.1.pkg -target /
rm -rf gfortran-6.1.pkg

# Establish a symlink of gfortran into /usr/local/bin
sudo ln -s /usr/local/gfortran/bin/gfortran /usr/local/bin/gfortran

Manual Install Guide

This guide provides a step-by-step breakdown of the actions the bash script takes. As a result, this guide will use both installers and script commands ia Terminal.app found in /Applications/Utilities/. Terminal is macOS’s equivalent to Linux’s shell and Window’s command line. From Terminal, we will install only the XCode Command Line Tools. These provide the system headers used to build the official CRAN binary for R.

XCode Command Line Tools

  1. Open the Terminal from /Applications/Utilities/
  2. Type the following into Terminal
xcode-select --install
  1. Press “Install”
  2. Verify installation by typing into terminal:
gcc --version

Installing the clang6 R binary

The clang6 binary can be obtained at:

https://cran.r-project.org/bin/macosx/tools/clang-6.0.0.pkg

from the macOS Tools Page

  1. Download and run the installer.

  2. Open the Terminal from /Applications/Utilities/

  3. Type:

touch ~/.Renviron
echo 'PATH="/usr/local/clang6/bin:${PATH}"' >> ~/.Renviron

Install gfortran6.1 binary

The gfortran6.1 binary can be obtained at:

https://cran.r-project.org/bin/macosx/tools/gfortran-6.1.pkg

from the macOS Tools Page

Download and run the installer.

Quick check

To verify that everything is working appropriately, let’s do a quick C++ program using Rcpp and Armadillo.

First, let’s install Rcpp and RcppArmadillo within R.

install.packages(c('Rcpp', 'RcppArmadillo'))

Create a new file, name the follow: helloworld.cpp

By adding the .cpp extension, the file is viewed as being C++ code.

Within the file write:

#include <RcppArmadillo.h>   

// [[Rcpp::depends(RcppArmadillo)]]

// [[Rcpp::export]]
void hello_world() {
  Rcpp::Rcout << "Hello World!" << std::endl;  
}

// After compile, this function will be immediately called using
// the below snippet and results will be sent to the R console.

/*** R
hello_world() 
*/

Compile the function using:

Rcpp::sourceCpp('path/to/file/helloworld.cpp')

where 'path/to/file/' is the location containing helloworld.cpp

If everything is installed appropriately, then you should see the following in the console:

> hello_world()

Hello World!

In addition, you should have a new function within the global environment scope called “hello_world”. You can call this function like a normal R function via:

hello_world()

Common Errors

The following are debugged errors that you may run into.

Why can’t R find the clang6 compiler even though I installed it and setup the ~/.Renviron?

There are three ways to encounter this issue:

First, the issue will appear as a warning when you first install RcppArmadillo from source. e.g.

install.packages("RcppArmadillo", type = "source")

The configure script will say:

checking for macOS... found
checking for macOS Apple compiler... found
configure: WARNING: OpenMP unavailable and turned off.

Secondly, if you try to compile a script locally using the pre-built CRAN binary for RcppArmadillo, then you will run into:

clang: warning: argument unused during compilation: '-fopenmp'
fatal error: 'omp.h' file not found

Third way – and the least common – is to have missing build commands. This error is likely due to an improperly specified PATH variable.

sh: make: command not found
sh: rm: command not found
Warning message:
In system(cmd) : error in running command

The fixes for each of these problems can be broken down related to:

  1. Launching R from terminal
    • Do you have any custom bindings, e.g. alias R ="R --flag1 --flag2"?
    • Are there any flags with the R call like --vanilla or --no-environ?
      • For --vanilla, please switch to --no-save --no-restore --no-site-file --no-init-file.
      • Otherwise, please remove --no-environ from the list.
  2. Poorly formatted ~/.Renviron
    • Are there multiple PATH variables or just one?
      • If so, delete and/or merge together the multiple copies of PATH.
    • Did you type the path correctly in ~/.Renviron?
      • Verify PATH="/usr/local/clang6/bin:${PATH}"
  3. Location of Installation
    • Did you install the clang-6.0.0.pkg?
      • Check if a folder is present: ls /usr/local/clang6

Why doesn’t the shared object load?

This is a tricky question to answer. Feel free to leave a comment below if the next solution doesn’t work for you.

If you are receiving the error message of Symbol not found: ___addtf3, then there is an old installation of R’s custom gfortran binary sitting in /usr/local.

The full error message is given as:

Error: 
 package or namespace load failed for ‘RcppArmadillo’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object
 '/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo/libs/RcppArmadillo.so':
dlopen(/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo/libs/RcppArmadillo.so, 6):
Symbol not found: ___addtf3
Referenced from: /usr/local/lib/libquadmath.0.dylib
Expected in: /usr/local/lib/libgcc_s_x86_64.1.dylib
in /usr/local/lib/libquadmath.0.dylib

In short, we need to remove the CRAN-built gfortran-4.8.2 binary that was in use from R 3.0.0 - R 3.3.3. To do so, we’ll download the binary and iterate through each file to remove them. Please run in Terminal the following shell script.

# Download installer into working directory
curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2

# Remove _files_ associated with the binary
for file in $(tar tfz gfortran-4.8.2-darwin13.tar.bz2); do
sudo rm -f /$file; 
done

# Remove empty _folders_ associated with the binary
for file in $(tar tfz gfortran-4.8.2-darwin13.tar.bz2); do 
sudo rmdir -p /$file; 
done

# Delete the installer
rm -rf gfortran-4.8.2-darwin13.tar.bz2

Then, reinstall the gfortran-6.1.pkg binary.