Lighthouse CI Client fails in AWS CodePipeline / CodeBuild

You did setup a build pipeline for your JAM stack and now you would like to check the lighthouse score on each deployment as part of the build process. After adding the lighthouse client to the buildspec it fails:

1$ /codebuild/output/src463792406/src/node_modules/.bin/lhci autorun
2 ✅ .lighthouseci/ directory writable
3 ✅ Configuration file found
4 ✅ Chrome installation found
5 ⚠️ Ancestor hash not determinable
6 ERROR: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`
7 ✅ LHCI server reachable
8 ✅ LHCI server API-compatible
9 ✅ LHCI server token valid
10 ⚠️ LHCI server non-unique build for this hash
11 ERROR: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`

The Problem

The reason for this error is not a missing git client, it is the missing .git directory.

CodePipeline downloads the source as a zip from the source provider rather than doing a Git clone, which means the .git folder won’t be retained and git commands like the one you’re running won’t work.

Have a look at the requirements for lighthouse

The documentation gives us some clues in the Build Context Section and a list of environment variables to provide the values:

When uploading to the Lighthouse CI server, the CLI will attempt to automatically infer the build context such as git hash, author, message, ancestor, etc. In most cases, there's nothing you need to change about this, but if you're running without a git repo, in a Jenkins environment, a CI provider we haven't documented yet, or are just running into errors, you can control the build context yourself.

The following environment variables override the inferred build context settings.

NameExample Format
LHCI_BUILD_CONTEXT__GIT_REMOTEgit@github.com:GoogleChrome/lighthouse-ci.git
LHCI_BUILD_CONTEXT__GITHUB_REPO_SLUGGoogleChrome/lighthouse-ci
LHCI_BUILD_CONTEXT__CURRENT_HASHe7f1b0fa3aebb6ef95e44c0d0b820433ffdd2e63
LHCI_BUILD_CONTEXT__ANCESTOR_HASH78bafdcaf40e204c0d0b81b90bb76b9aa5834e11
LHCI_BUILD_CONTEXT__COMMIT_TIME2019-11-29T16:43:39-05:00
LHCI_BUILD_CONTEXT__CURRENT_BRANCHdev_branch_1234
LHCI_BUILD_CONTEXT__COMMIT_MESSAGEDaily run of Lighthouse
LHCI_BUILD_CONTEXT__AUTHORPatrick Hulce <patrick.hulce@example.com>
LHCI_BUILD_CONTEXT__AVATAR_URLhttps://example.com/patrickhulce.jpg
LHCI_BUILD_CONTEXT__EXTERNAL_BUILD_URLhttps://my-jenkins.example.com/jobs/123

Can we get this information from the CodeBuild enviroment?

The AWS CloudBuild documentation will list this variables:

AWS CodeBuild provides several variables that you can use in your build environment:

NameDescription
AWS_DEFAULT_REGIONThe AWS Region where the build is running (for example, us-east-1). This environment variable is used primarily by the AWS CLI.
AWS_REGIONThe AWS Region where the build is running (for example, us-east-1). This environment variable is used primarily by the AWS SDKs.
CODEBUILD_BUILD_ARNThe Amazon Resource Name (ARN) of the build (for example, arn:aws:codebuild:region-ID:account-ID:build/codebuild-demo-project:b1e6661e-e4f2-4156-9ab9-82a19EXAMPLE).
CODEBUILD_BUILD_IDThe CodeBuild ID of the build (for example, codebuild-demo-project:b1e6661e-e4f2-4156-9ab9-82a19EXAMPLE).
CODEBUILD_BUILD_IMAGEThe CodeBuild build image identifier (for example, aws/codebuild/standard:2.0).
CODEBUILD_BUILD_NUMBERThe current build number for the project.
CODEBUILD_BUILD_SUCCEEDINGWhether the current build is succeeding. Set to 0 if the build is failing, or 1 if the build is succeeding.
CODEBUILD_INITIATORThe entity that started the build. If CodePipeline started the build, this is the pipeline's name (for example, codepipeline/my-demo-pipeline). If an IAM user started the build, this is the user's name (for example, MyUserName). If the Jenkins plugin for CodeBuild started the build, this is the string CodeBuild-Jenkins-Plugin.
CODEBUILD_KMS_KEY_IDThe identifier of the AWS KMS key that CodeBuild is using to encrypt the build output artifact (for example, arn:aws:kms:region-ID:account-ID:key/key-ID or alias/key-alias).
CODEBUILD_LOG_PATHThe log stream name in CloudWatch Logs for the build.
CODEBUILD_RESOLVED_SOURCE_VERSIONAn identifier for the version of a build's source code.
CODEBUILD_SOURCE_REPO_URLThe URL to the input artifact or source code repository. For Amazon S3, this is s3:// followed by the bucket name and path to the input artifact. For CodeCommit and GitHub, this is the repository's clone URL. If a build originates from CodePipeline, then this might be empty.
CODEBUILD_SOURCE_VERSIONFor CodeCommit, it is the commit ID or branch name associated with the version of the source code to be built. For GitHub, GitHub Enterprise, and Bitbucket it is the commit ID, branch name, or tag name associated with the version of the source code to be built.
CODEBUILD_SRC_DIRThe directory path that CodeBuild uses for the build (for example, /tmp/src123456789/src).
CODEBUILD_START_TIMEThe start time of the build specified as a Unix timestamp in milliseconds.
CODEBUILD_WEBHOOK_ACTOR_ACCOUNT_IDThe account ID of the user that triggered the webhook event.
CODEBUILD_WEBHOOK_BASE_REFThe base reference name of the webhook event that triggers the current build. For a pull request, this is the branch reference.
CODEBUILD_WEBHOOK_EVENTThe webhook event that triggers the current build.
CODEBUILD_WEBHOOK_PREV_COMMITThe ID of the most recent commit before the webhook push event that triggers the current build.
CODEBUILD_WEBHOOK_HEAD_REFThe head reference name of the webhook event that triggers the current build. It can be a branch reference or a tag reference.
CODEBUILD_WEBHOOK_TRIGGERShows the webhook event that triggered the build. This variable is available only for builds triggered by a webhook.

We do have the current commit hash but we do not have the commit message and there is no way to get the ancestor hash.

Fixing the Problem

After searching on google and failing with some other solutions I found this great post from Timothy Jones: How to access git metadata in CodeBuild when using CodePipeline/CodeCommit

He did a very good research on the topic and provides a solution which allows to integrate lighthouse without a need for changes when lighthouse or any other software needs access to git metadata.

1) Allow CodeBuild to clone your repository

This is only needed, if you use a CodeCommit repository. For github, gitlab, Bitbucket this step can be skipped.

Give your CodeBuild execution role the permission to clone the repository:

1{
2 "Effect": "Allow",
3 "Action": [
4 "codecommit:GitPull"
5 ],
6 "Resource": [
7 "arn:aws:codecommit:*:*:<YOUR_REPOSITORY_NAME>"
8 ]
9},

2) Add git credential helper to CodeBuild

This is only needed, if you use a CodeCommit repository. For github, gitlab, Bitbucket this step can be skipped.

Add this to the start of your builspec.yml:

1version: 0.2
2
3env:
4 git-credential-helper: yes

This will allow git to access the repository with the existing AWS credentials in the build container.

3) Clone repository

Our requirements are:

  • clone the repository to a temporal directory
  • switch current branch and commit based om the values in the AWS build environment variables
  • move the .git directory into the source directory

Happily for us Timothy Jones already create a script which does this for us. You can download this script here or with this code:

1curl https://raw.githubusercontent.com/TimothyJones/codepipeline-git-metadata-example/master/scripts/codebuild-git-wrapper.sh \
2-o codebuild-git-wrapper.sh

4) Add repository url and branch to enviroment

CodeBuild Console / Add Enviroment Variables
[CodeBuild Console / Add Enviroment Variables]

We need the following variables in our CodeBuild:

  • REPO_URL - vaild url for git clone
  • BRANCH - branch wich was used to run the build

There are three methods depending on the way you did setup your CodePipeline to add environment variables:

  • set the enviroment variables in the CodePipeline Console
  • set the enviroment variables in the CodeBuild Console
  • use CloudFormation to set the environment variables in CodePipeline / CodeBuild

Final buildspec for Lighthouse CI

This simple example, which is very typical for node based applications e.g. gatsby. The lhci autorun combines collecting, asserting and uploading to a Lighhouse CI server and requires a lighthouserc.js file in your root to define your environment and tasks.

builspec.yml:

1version: 0.2
2env:
3 git-credential-helper: yes
4phases:
5 install:
6 commands:
7 - npm install -g @lhci/cli
8 pre_build:
9 commands:
10 - npm install --pure-lockfile
11 build:
12 commands:
13 - scripts/codebuild-git-wrapper.sh $REPO_URL $REPO_BRANCH
14 - git log -2
15 - echo Build started on `date`
16 - npm build
17 - lhci autorun

The resulted output of your deployment log should look like this now:

1$ /codebuild/output/src401949355/src/node_modules/.bin/lhci autorun
22157 ✅ .lighthouseci/ directory writable
32158 ✅ Configuration file found
42159 ✅ Chrome installation found
52160 ✅ Ancestor hash determinable
62161 ✅ LHCI server reachable
72162 ✅ LHCI server API-compatible
82163 ✅ LHCI server token valid
92164 ✅ LHCI server unique build for this hash
102165 Healthcheck passed!
112166
122167 Started a web server on port 45377...
132168 Running Lighthouse 3 time(s) on http://localhost:45377/index.html
142169 Run #1...done.
152170 Run #2...done.
162171 Run #3...done.
172172 Running Lighthouse 3 time(s) on http://localhost:45377/about/index.html
182173 Run #1...done.
192174 Run #2...done.
202175 Run #3...done.
212176 Running Lighthouse 3 time(s) on http://localhost:45377/en/blog/lighthouse-ci-fails-in-aws-codepipeline-codebuild/index.html
222177 Run #1...done.
232178 Run #2...done.
242179 Run #3...done.
252180 Done running Lighthouse!
262181
272182 Saving CI project website-test (15a1683b-3072-46f7-a72f-c6f9da1d2a3e)
282183 Saving CI build (dc92ed97-c7f8-4171-b74d-1af9dbf77b81)
292184 Saved LHR to http://lighhouse-ci-server.example:9001 (d2cc722b-9a89-4497-82e7-70fd4acaca77)
302185 Saved LHR to http://lighhouse-ci-server.example:9001 (2fd47ecd-d03e-4a47-b3ac-24f9301dcd80)
312186 Saved LHR to http://lighhouse-ci-server.example:9001 (f1bfb641-2e40-44a7-8773-a29b7c7b1cec)
322187 Saved LHR to http://lighhouse-ci-server.example:9001 (23692d1a-cb13-4347-837e-71288e1d7141)
332188 Saved LHR to http://lighhouse-ci-server.example:9001 (732eb3b0-2dd6-4237-bfd4-0a917627c24e)
342189 Saved LHR to http://lighhouse-ci-server.example:9001 (c64e1d7c-ff9c-42dc-8467-027c37ecb2be)
352190 Saved LHR to http://lighhouse-ci-server.example:9001 (062a732b-0130-4172-9192-87708cc47be9)
362191 Saved LHR to http://lighhouse-ci-server.example:9001 (274aa877-f69f-47c7-823a-127c7d21b89a)
372192 Saved LHR to http://lighhouse-ci-server.example:9001 (8e8b09c4-8969-4300-9cc6-12cba17bb27d)
382193 Done saving build results to Lighthouse CI
392194 View build diff at http://nagios.dienst.at:9001/app/projects/website2020/compare/dc92ed95-c6f8-4671-b74d-1af9dbf77b81
402195 No GitHub token set, skipping GitHub status check.
412196
422197 Done running autorun.
432198 Done in 123.33s.