Splitting a code base into packages keeps compilation fast, and avoids unwanted dependencies between layers. You may use a package per layer (persistence, network, etc.), and even a package per feature if the project is big.

The highest iteration speed when developing visual work is to be able to launch a feature in isolation, relying only in test objects. xcodegen can help to generate small projects for feature testing.

An interesting talk on the subject is Architecting Giant Apps for Scalability and Build Speed. It talks about package granularity and dependency injection related to project sizes.

I recently split a pet project and found two challenges:

  • Synchronizing changes. Increasing package versions and picking up changes from the app.
  • Preventing regressions. Running a CI to spot regressions and maintain semantic versioning.

I’m using local SPM references and GitHub Actions for this.

Local SPM references

If packages are only used in that project keep them in a monorepo. Otherwise, you need a repository per package and versioning changes. I went with the latter.

My first idea was to slice user stories so I could work in one package at the time. However this assumes perfect knowledge of the problem. In practice, you may need to try to solve a problem in order to fully understand it, which results in back and forth changes between packages.

Then I thought of setting every package reference to local, touch several packages, then submit one update per package when the feature is complete. Switching every reference to local is not a feature of SPM, so I’m using a Makefile for this.

How to switch a package reference to local

In Package.swift change a dependency as follows: 
.package(path: "../Dependency"),
#.package(url: "git@github.com:janodevorg/Dependency.git", from: "1.0.0")

For Xcodegen a local dependency would be:

packages:
  Dependency:
    path: ../Dependency

For Xcode click Local while adding the dependency, then choose a folder.

How to switch Package.swift between local and remote

Triplicate Package.swift for each dependency: 
Package.swift
Package.local.swift
Package.remote.swift
Let’s say your dependencies are in the same folder, and they are called Dep1 Dep2 Dep3 Dep4. Place a Makefile on the same folder. The layout should look like this: 
.
├── Dep1
├── Dep2
├── Dep3
├── Dep4
└── Makefile

The contents of the Makefile are:

#!/bin/bash -e -o pipefail

FOLDERS = Dep1 Dep2 Dep3 Dep4

.PHONY: local remote

local:
	$(foreach folder, $(FOLDERS), test -s ./$(folder)/Package.local.swift && { cat ./$(folder)/Package.local.swift > ./$(folder)/Package.swift; };)

remote:
	$(foreach folder, $(FOLDERS), test -s ./$(folder)/Package.remote.swift && { cat ./$(folder)/Package.remote.swift > ./$(folder)/Package.swift; };)

Duplication may lead to errors so I wonder if this can be done with Sourcery keeping the remote repository as a comment. I didn’t look into it so duplication became a great solution by virtue of being the only working solution I have right now.

Once changes were submitted I needed to update to these changes from the main project. I thought of pointing to a branch, but sometimes SPM refuses to update to the last commit in a branch (SR-15262) unless I remove Xcode and SPM caches. Tags however, do work so I’m increasing tag numbers according to semantic versioning.

How to remove Derived Data and the SPM cache

rm -rf ~/Library/Caches/org.swift.swiftpm/
rm -rf ~/Library/Developer/Xcode/DerivedData/*

Self-hosted GitHub Actions

If you touch several places at once you need integration testing, lots of it. I tried several solutions and found these concerns:

  • Most CIs are complex. I want to clone + swift test triggered by PRs. This requires a ridiculous number of steps.
  • CIs are expensive and slow. I don’t want to pay upwards of $40 for a virtual machine that runs 5x slower than my Mac.
  • CIs on macOS hardware are specially expensive. Currently GitHub Actions (GA) is 10x more expensive when ran in macOS.

The best option for me was GitHub Actions with self-hosted runners.

What is a runner?

A runner is a local virtual machine that recreate the conditions of the CI using software installed locally. Some CI providers let you host runners in your machine.

Advantages of using a runner:

  • It’s free.
  • It runs at the speed of a local machine (yours).
  • It uses the software you provide. Want the newest Xcode? install it, use it.

There are three kinds of runners: repository, organization, enterprise. The repository runner is limited to one repo, while the others manage several. It’s almost as if GitHub wanted me to create an organization isn’t it? Organizations with public repos are free. For advanced features (like private repos) pricing starts at 40€ user/year.

Installation

GitHub instructions to host a runner start in the article Adding self-hosted runners. In summary, run the following:

mkdir actions-runner && cd actions-runner
curl -o actions-runner-osx-x64-2.288.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.288.1/actions-runner-osx-x64-2.288.1.tar.gz
echo "08d6dd274a1a497d052f5dae740a058d28660cfdcd8864d53857bd31a8521cdf  actions-runner-osx-x64-2.288.1.tar.gz" | shasum -a 256 -c
tar xzf ./actions-runner-osx-x64-2.288.1.tar.gz
rm actions-runner-osx-x64-2.288.1.tar.gz

# ./config.sh

I commented ./config.sh at the end because it asks you things I’m going to discuss below:

  • What is the URL of your repository?
  • What is your runner register token?

Runner token for an organization

To create a runner registration token for an organization:

  • Go to profile > Settings > Developer settings and create a personal token that has the admin:org scope (leave other scopes unchecked). Note that this has to be a personal token because organizations don’t have permissions associated with them.
  • Use the personal admin token to request an organization token:
    • curl \
-u janodev:PERSONAL_ADMIN_TOKEN \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/orgs/janodevorg/actions/runners/registration-token
  • Take the token you got from the previous call and replace it in the call below where it says TOKEN. Accept the defaults for every question you get. Note that included in the defaults are the tags self-hosted, X64, macOS. We’ll use these tags later. 
    ./config.sh --url https://github.com/janodevorg --token TOKEN
  • To verify that the runner works correctly you need a personal token with workflow scope, but you can use for this the personal token with admin:org scope you created before. This is going to display a number of Pass checks, or an error report. 
    ./run.sh --check --url https://github.com/janodevorg/Report --pat PERSONAL_ADMIN_TOKEN
  • Start the runner. It will start listening for jobs. 
    ./run.sh

If you prefere to run the agent as a service see Configuring the self-hosted runner application as a service. This is what you need if you intend to have a Mac mini as the CI server for your team. It saves money, time, and lets the dev team own the CI as god intended. FYI: one runner means one concurrent build, which prevents your server from choking and it’s still faster than a virtual machine.

Runner token for a single repo

Included for completion, here is the case where you intend to use one runner per repo.

To create a runner registration token for a single repository:

  • Go to profile > Settings > Developer settings and create a personal token that has the repo scope (leave other scopes unchecked).
  • Run the curl below replacing MY_TOKEN with your personal token.
curl \
    -u janodev:MY_TOKEN
    -X POST \
    -H "Accept: application/vnd.github.v3+json" \
    https://api.github.com/repos/janodev/Dependency/actions/runners/registration-token

Run a workflow

Create a file .github/workflows/swift.yml in your repository with this content. Mind these lines:

  • 2: It runs when you push to the repo.
  • 6: It requires a runner configured with the tags [self-hosted, X64, macOS].
  • 15: You need to create a SSH_KEY secret containing your private key. ⚠️ Be mindful that anyone able to edit this action will be able to run arbitrary operations with your private key.
  • 17: You need to create a KNOWN_HOSTS secret containing the string github.com

Start the runner (./run.sh) and push to your repo. You can track the runner activity from the tab Actions of the GitHub repo. If you want a badge for your repo click the ellipsis besides the button Re-run all jobs.

Troubleshooting

If you see a non descriptive error, go to the folder where you installed your runner, and open the logs generated in the _diag child folder.

If your runner is not picking up jobs:

  • If your repo is public check that the group of your runner is set to execute on public repos. You can do so in Runner groups. This is a secure measure so up to you. If the machine runs PRs from unknown users is a good idea to keep it in its own network.
  • Check that the element “runs-on” on your workflow file contains the tags your runner is configured for. These tags default to [self-hosted, X64, macOS] on macOS.