Publishing to Connect using Bitbucket Pipelines

Overview of Deploying to Connect

Connect hosts a variety of data artifacts with different development life cycles. Whenever you want to publish one of these data artifacts to Connect, there are three paths you can follow:

This page focuses on the third option, programmatic deployment using Bitbucket Pipelines as a continuous integration and deployment pipeline. Continuous integration (CI) is the practice of automating the integration of code changes. That automation can entail running different tests or other pre-deployment activities. Continuous deployment (CD) is the practice of automating the deployment of code changes to a test or production environment. Many popular code hosting providers and independent software companies offer CI and CD services. These pipelines allow you to build for specific operating systems/environments, integrate tests and publish to Connect from private repositories without a service account.

The following section examines the deployment of a Shiny application to Connect using Bitbucket Pipelines.

Bitbucket Pipelines Example

Demo Bitbucket Repository: https://bitbucket.org/lisa-sandbox-code/shiny-app-demo-cicd-bitbucket/src/main/

The first thing to consider is how to manage the R packages as dependencies within the CI/CD service. One solution is to do a one-by-one installation of every package the Shiny app uses, however, this gets cumbersome as the app grows bigger. To simplify package management of the environment, it is recommended to use the renv package. This package helps reproducibility because, for all packages used by your project, it saves both the package version information and the repository location(s) where you installed the packages from so that it can restore the environment on other computers. You use renv in this deployment process to maintain consistency between the development and build environments.

Without renv:

install.packages(c(“shiny”, “jsonlite”, “ggplot2”, “stringr”))

With renv:

renv::snapshot() #Development Environment
renv::restore() # CI

To learn more about renv, visit the package documentation website.

Below you can see how the files are organized in the Demo Bitbucket repository.

.
├── .Rbuildignore
├── .Rprofile
├── .gitignore
├── README.md
├── app.R
├── manifest.json
├── bitbucket-pipelines.yml
├── deploy-to-connect.sh
├── install-pandoc.sh
├── set-rspm.sh
├── renv.lock
└── shiny-app-demo-cicd-bitbucket.Rproj

1 directory, 12 files

The Shiny application is app.R.

The main file of interest is bitbucket-pipelines.yml. This file defines the CI/CD pipeline and is set up to run on any push to the main branch. First, it sets up the proper environment, including restoring the renv environment. Second, it publishes the Shiny application to Connect using the Connect API.

image: rstudio/r-base:4.1.2-focal

pipelines:
  default:
    - step:
        name: 'Deploy to Connect'
        deployment: production
        script:
          - sudo apt-get update && sudo apt-get install -y libcurl4-openssl-dev libssl-dev curl jq
          - /bin/bash install-pandoc.sh
          - /bin/bash set-rspm.sh
          - sudo rm .Rprofile
          - sudo R -e 'install.packages("renv")'
          - sudo R -e 'renv::consent(TRUE)'
          - sudo R -e 'renv::restore()'
          - sudo R -e 'install.packages("rsconnect")'
          - sudo R -e 'rsconnect::writeManifest()'
          - /bin/bash deploy-to-connect.sh

The rest of this article reviews the file’s contents so you understand in detail what is happening.

The beginning of bitbucket-pipelines.yml specifies the image for the pipeline, which allows for the specification of what OS should be used on the virtual machine and the R version. We can use the rstudio/r-base:4.1.2-focal image to provide a container with R installed since this CI/CD pipeline will be running R code and deploying a Shiny app. Any Docker Hub image may be used and for a full list of available images with versions of R pre-installed visit the rstudio/r-base repository.

image: rstudio/r-base:4.1.2-focal

To successfully deploy to Connect this pipeline will need several environment variables. CONNECT_SERVER, CONNECT_API_KEY, UNIQ_NAME, and CONTENT_FILES should be set up at the pipeline level by going to Repository Settings on the left side of the screen, scrolling down and clicking on Repository variables under Pipelines, and using the Name and Value input fields to add your variables. Variables added can be secured, meaning that the variable will be encrypted and will be masked from the logs.

  • CONNECT_SERVER: The URL for your Connect server address. URL should end with /
  • CONNECT_API_KEY: API key for programmatically accessing and publishing content on Connect
  • UNIQ_NAME: The name of the application
  • CONTENT_FILES: The files that will be deployed, for example app.R or app.R config.csv (note that multiple files should be space separated)

The screenshot below illustrates where to go in the Bitbucket settings.

Bitbucket setup showing Step 1 click on repository variables under pipelines; Step 2 add variable; Step 3 optionally make a secured variable to encrypt while stored and mask from logs

Bitbucket Pipeline Variables Setup

The rest of the bitbucket-pipelines.yml file determines a series of steps to be performed, the last of which deploys the Shiny application to the Connect server.

The below lines install system dependencies and set up Package Manager as the default R repository for faster package installation from binaries. If your organization does not have Package Manager, it is recommended to use Posit’s Public Package Manager for access to binaries.

pipelines:
  default:
    # The following deployment steps will be executed for each pipeline run. To configure your steps and conditionally deploy see https://support.atlassian.com/bitbucket-cloud/docs/configure-bitbucket-pipelinesyml/
    - step:
        name: 'Deploy to Connect'
        deployment: production
        script:
          - sudo apt-get update && sudo apt-get install -y libcurl4-openssl-dev libssl-dev curl jq
          - /bin/bash install-pandoc.sh
          - /bin/bash set-rspm.sh

The next steps restore the environment. First, the .RProfile is removed to avoid conflicts with renv. Then we install renv and restore the renv environment. Using renv is recommended rather than manually installing packages, as mentioned at the beginning of this article.

          - sudo rm .Rprofile
          - sudo R -e 'install.packages("renv")'
          - sudo R -e 'renv::consent(TRUE)'
          - sudo R -e 'renv::restore()'

After the proper environment is set up we create the manifest.json file required by Connect for deploying content programmatically.

          - sudo R -e 'install.packages("rsconnect")'
          - sudo R -e 'rsconnect::writeManifest()'

The last step is to run the deploy-to-connect.sh script, which interfaces with the Connect API and deploys the Shiny application. This script utilizes both the pipeline defined environment variables and the locally defined variables to determine the server location, API key, content files, and unique name.

          - /bin/bash deploy-to-connect.sh
Back to top