This post is a version of the rcpp-and-mirai README from the R&D Rcpp collection of compiled-code examples. The repository holds the full, runnable source.
This small example R package demonstrates how to house a compiled C++ compute kernel inside a package and distribute it in parallel across a pool of mirai daemons, so that each worker reaches the compiled function directly without copying source or recompiling. It is one of the R&D Rcpp prototype packages, a collection of minimal examples for integrating compiled code with R. The walkthrough below covers the kernel, the driver that splits the work, and the few details that matter when running compiled code on daemons.
The RcppMirai R package provides an example of housing a C++ function in a package and distributing it in parallel across a pool of mirai daemons. It mirrors the rcpp-and-doparallel example, swapping the doParallel and foreach backend for mirai.
Usage
To install the package, you must first have a compiler on your system that is compatible with R. For help on obtaining a compiler consult either macOS or Windows guides.
With a compiler in hand, one can then install the package from GitHub by:
# install.packages("remotes")
remotes::install_github("coatless-rd-rcpp/rcpp-and-mirai")
library("RcppMirai")Implementation Details
Within this project, a compute kernel is written in C++ with Rcpp and an R function distributes chunks of the work across mirai daemons. By packaging the kernel, each daemon reaches the compiled function directly through a namespace-qualified call, with no copying of source or recompilation on the workers. The example task is a bootstrap of the mean: the kernel draws bootstrap resamples and returns the mean of each, and the R driver splits the total number of resamples across the available daemons.
.
├── DESCRIPTION # Package metadata
├── NAMESPACE # Function and dependency registration
├── R # R functions
│ ├── RcppExports.R # Autogenerated R to C++ bindings by Rcpp
│ ├── RcppMirai-package.R # Package documentation
│ └── parallel_bootstrap_mean.R # mirai driver that distributes the kernel
├── README.md
├── RcppMirai.Rproj
├── man # Package documentation
│ ├── RcppMirai-package.Rd
│ ├── bootstrap_mean.Rd
│ └── parallel_bootstrap_mean.Rd
└── src # Compiled code
├── RcppExports.cpp # Autogenerated R bindings
└── bootstrap_mean.cpp # Compute kernel: bootstrap the mean in C++The C++ Kernel
The kernel lives in src/bootstrap_mean.cpp. It draws reps resamples of x with replacement and returns the mean of each, producing the bootstrap distribution. It uses R’s random number generator through R::unif_rand(), which matters for the parallel case described below.
#include <Rcpp.h>
// [[Rcpp::export]]
Rcpp::NumericVector bootstrap_mean(Rcpp::NumericVector x, int reps) {
int n = x.size();
Rcpp::NumericVector out(reps);
for (int b = 0; b < reps; b++) {
double sum = 0.0;
for (int i = 0; i < n; i++) {
int idx = (int)(R::unif_rand() * n); // draw an index in [0, n)
sum += x[idx];
}
out[b] = sum / n;
}
return out;
}The R Function
parallel_bootstrap_mean() is the driver. In keeping with the design of mirai, it does not set up daemons itself: it reads the number of connected daemons from mirai::status() and, when there are none, runs the kernel once in the current session. When a daemon pool is present, it splits the requested number of resamples into one chunk per daemon and distributes them with mirai::mirai_map(). The work is sent through the package worker bootstrap_chunk() (not an inline function), and the pieces are combined back into a single bootstrap distribution with the [.flat] collector.
parallel_bootstrap_mean = function(x, reps = 10000L, chunks = NULL) {
reps = as.integer(reps)
# mirai leaves daemon set up to the caller. With no connected daemons,
# evaluate the compiled kernel once in the current session.
n_daemons = mirai::status()$connections
if (n_daemons < 1L) {
return(bootstrap_mean(x, reps))
}
# One task per daemon unless the caller asks for a different split.
if (is.null(chunks)) {
chunks = n_daemons
}
chunks = max(1L, min(as.integer(chunks), reps))
# Split `reps` as evenly as possible into `chunks` pieces.
sizes = diff(round(seq(0L, reps, length.out = chunks + 1L)))
sizes = sizes[sizes > 0L]
# Distribute the pieces across the daemons. `bootstrap_chunk` is a package
# function, so only a reference to it travels to each daemon, which then
# loads the package to reach the compiled kernel.
mirai::mirai_map(
sizes,
bootstrap_chunk,
.args = list(x = x)
)[.flat]
}
# Internal worker run on each daemon. Kept at the top level (its environment is
# the package namespace) so that mirai serialises a reference to it instead of a
# copy of the caller's frame.
bootstrap_chunk = function(reps, x) {
RcppMirai::bootstrap_mean(x, reps)
}To run in parallel, the user starts a pool of daemons first, then resets it when finished:
library("RcppMirai")
mirai::daemons(4) # start 4 local daemons
boot = parallel_bootstrap_mean(rnorm(1000), reps = 1e5)
mirai::daemons(0) # reset when finished
quantile(boot, c(0.025, 0.5, 0.975))Using Compiled Code on Daemons
A daemon starts with no user packages attached, so the work sent to it must reach the compiled code on its own. A namespace-qualified call such as RcppMirai::bootstrap_mean() is all that is needed to use a compiled function from an installed package on a daemon, since :: loads the package and its compiled code on first use. There is nothing special to do for Rcpp code beyond installing the package, a point confirmed in the mirai discussion on package development and the mirai package vignette.
The function handed to mirai_map() is the package worker bootstrap_chunk(), defined at the top level of the package rather than inline. Because its environment is the package namespace, mirai transmits only a reference to it. A function written inline inside parallel_bootstrap_mean() would instead capture that call’s environment and copy it, including x, to every daemon, which is both wasteful and contrary to the explicit-dependency design of mirai.
Daemon set up is left to the caller. mirai does not rely on global options or environment variables, so letting the user call mirai::daemons() keeps the function composable with any other mirai work in the same session. Each daemon also draws from its own independent random number stream, which keeps resamples drawn in parallel statistically sound; passing a seed to mirai::daemons() makes those streams reproducible for a fixed number of chunks.
Two further notes apply to compiled code in particular. If the kernel were itself threaded, for example with OpenMP, every daemon would launch its own threads, so the total thread count becomes the number of daemons multiplied by the threads per daemon; keep the kernel single-threaded, or cap the threads, to avoid oversubscribing the machine. The exported examples also run in the current session by default and only start daemons inside an if (interactive()) guard, so package checks do not spawn extra processes.
DESCRIPTION
Surfacing the C++ code with Rcpp requires Rcpp under both LinkingTo (for the headers used at compile time) and Imports (so it is available at run time). The driver calls mirai through namespaced calls, so mirai is listed under Imports as well.
LinkingTo:
Rcpp
Imports:
Rcpp (>= 1.0.12),
mirai (>= 2.0.0)Source
The full package source is available on GitHub, and the rendered package site can be found at rd-rcpp.thecoatlessprofessor.com/rcpp-and-mirai.