Lighthouse CI Client fails in AWS CodePipeline / CodeBuild

You did setup a build pipeline for your JAM stack and now you would like to check the lighthouse score on each deployment as part of the build process. After adding the lighthouse client to the buildspec it fails:

$ /codebuild/output/src463792406/src/node_modules/.bin/lhci autorun
	✅  .lighthouseci/ directory writable
	✅  Configuration file found
	✅  Chrome installation found
	⚠️   Ancestor hash not determinable
	    ERROR: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`
	✅  LHCI server reachable
	✅  LHCI server API-compatible
	✅  LHCI server token valid
	⚠️   LHCI server non-unique build for this hash
	    ERROR: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`

The Problem

The reason for this error is not a missing git client, it is the missing .git directory.

CodePipeline downloads the source as a zip from the source provider rather than doing a Git clone, which means the .git folder won’t be retained and git commands like the one you’re running won’t work.

Have a look at the requirements for lighthouse

The documentation gives us some clues in the Build Context Section and a list of environment variables to provide the values:

When uploading to the Lighthouse CI server, the CLI will attempt to automatically infer the build context such as git hash, author, message, ancestor, etc. In most cases, there’s nothing you need to change about this, but if you’re running without a git repo, in a Jenkins environment, a CI provider we haven’t documented yet, or are just running into errors, you can control the build context yourself.

The following environment variables override the inferred build context settings.

Name	Example Format
`LHCI_BUILD_CONTEXT__GIT_REMOTE`	`git@github.com:GoogleChrome/lighthouse-ci.git`
`LHCI_BUILD_CONTEXT__GITHUB_REPO_SLUG`	`GoogleChrome/lighthouse-ci`
`LHCI_BUILD_CONTEXT__CURRENT_HASH`	`e7f1b0fa3aebb6ef95e44c0d0b820433ffdd2e63`
`LHCI_BUILD_CONTEXT__ANCESTOR_HASH`	`78bafdcaf40e204c0d0b81b90bb76b9aa5834e11`
`LHCI_BUILD_CONTEXT__COMMIT_TIME`	`2019-11-29T16:43:39-05:00`
`LHCI_BUILD_CONTEXT__CURRENT_BRANCH`	`dev_branch_1234`
`LHCI_BUILD_CONTEXT__COMMIT_MESSAGE`	`Daily run of Lighthouse`
`LHCI_BUILD_CONTEXT__AUTHOR`	`Patrick Hulce <patrick.hulce@example.com>`
`LHCI_BUILD_CONTEXT__AVATAR_URL`	https://example.com/patrickhulce.jpg
`LHCI_BUILD_CONTEXT__EXTERNAL_BUILD_URL`	https://my-jenkins.example.com/jobs/123

Can we get this information from the CodeBuild enviroment?

The AWS CloudBuild documentation will list this variables:

AWS CodeBuild provides several variables that you can use in your build environment:

Name	Description
AWS_DEFAULT_REGION	The AWS Region where the build is running (for example, us-east-1). This environment variable is used primarily by the AWS CLI.
AWS_REGION	The AWS Region where the build is running (for example, us-east-1). This environment variable is used primarily by the AWS SDKs.
CODEBUILD_BUILD_ARN	The Amazon Resource Name (ARN) of the build (for example, arn:aws:codebuild:region-ID:account-ID:build/codebuild-demo-project:b1e6661e-e4f2-4156-9ab9-82a19EXAMPLE).
CODEBUILD_BUILD_ID	The CodeBuild ID of the build (for example, codebuild-demo-project:b1e6661e-e4f2-4156-9ab9-82a19EXAMPLE).
CODEBUILD_BUILD_IMAGE	The CodeBuild build image identifier (for example, aws/codebuild/standard:2.0).
CODEBUILD_BUILD_NUMBER	The current build number for the project.
CODEBUILD_BUILD_SUCCEEDING	Whether the current build is succeeding. Set to 0 if the build is failing, or 1 if the build is succeeding.
CODEBUILD_INITIATOR	The entity that started the build. If CodePipeline started the build, this is the pipeline’s name (for example, codepipeline/my-demo-pipeline). If an IAM user started the build, this is the user’s name (for example, MyUserName). If the Jenkins plugin for CodeBuild started the build, this is the string CodeBuild-Jenkins-Plugin.
CODEBUILD_KMS_KEY_ID	The identifier of the AWS KMS key that CodeBuild is using to encrypt the build output artifact (for example, arn:aws:kms:region-ID:account-ID:key/key-ID or alias/key-alias).
CODEBUILD_LOG_PATH	The log stream name in CloudWatch Logs for the build.
CODEBUILD_RESOLVED_SOURCE_VERSION	An identifier for the version of a build’s source code.
CODEBUILD_SOURCE_REPO_URL	The URL to the input artifact or source code repository. For Amazon S3, this is s3:// followed by the bucket name and path to the input artifact. For CodeCommit and GitHub, this is the repository’s clone URL. If a build originates from CodePipeline, then this might be empty.
CODEBUILD_SOURCE_VERSION	For CodeCommit, it is the commit ID or branch name associated with the version of the source code to be built. For GitHub, GitHub Enterprise, and Bitbucket it is the commit ID, branch name, or tag name associated with the version of the source code to be built.
CODEBUILD_SRC_DIR	The directory path that CodeBuild uses for the build (for example, /tmp/src123456789/src).
CODEBUILD_START_TIME	The start time of the build specified as a Unix timestamp in milliseconds.
CODEBUILD_WEBHOOK_ACTOR_ACCOUNT_ID	The account ID of the user that triggered the webhook event.
CODEBUILD_WEBHOOK_BASE_REF	The base reference name of the webhook event that triggers the current build. For a pull request, this is the branch reference.
CODEBUILD_WEBHOOK_EVENT	The webhook event that triggers the current build.
CODEBUILD_WEBHOOK_PREV_COMMIT	The ID of the most recent commit before the webhook push event that triggers the current build.
CODEBUILD_WEBHOOK_HEAD_REF	The head reference name of the webhook event that triggers the current build. It can be a branch reference or a tag reference.
CODEBUILD_WEBHOOK_TRIGGER	Shows the webhook event that triggered the build. This variable is available only for builds triggered by a webhook.

We do have the current commit hash but we do not have the commit message and there is no way to get the ancestor hash.

Fixing the Problem

After searching on google and failing with some other solutions I found this great post from Timothy Jones: How to access git metadata in CodeBuild when using CodePipeline/CodeCommit

He did a very good research on the topic and provides a solution which allows to integrate lighthouse without a need for changes when lighthouse or any other software needs access to git metadata.

1) Allow CodeBuild to clone your repository

This is only needed, if you use a CodeCommit repository. For github, gitlab, Bitbucket this step can be skipped.

Give your CodeBuild execution role the permission to clone the repository:

{
    "Effect": "Allow",
    "Action": [
        "codecommit:GitPull"
    ],
    "Resource": [
        "arn:aws:codecommit:*:*:<YOUR_REPOSITORY_NAME>"
    ]
},

2) Add git credential helper to CodeBuild

This is only needed, if you use a CodeCommit repository. For github, gitlab, Bitbucket this step can be skipped.

Add this to the start of your builspec.yml:

version: 0.2

env:
  git-credential-helper: yes

This will allow git to access the repository with the existing AWS credentials in the build container.

3) Clone repository

Our requirements are:

clone the repository to a temporal directory
switch current branch and commit based om the values in the AWS build environment variables
move the .git directory into the source directory

Happily for us Timothy Jones already create a script which does this for us. You can download this script here or with this code:

curl https://raw.githubusercontent.com/TimothyJones/codepipeline-git-metadata-example/master/scripts/codebuild-git-wrapper.sh \
-o codebuild-git-wrapper.sh

4) Add repository url and branch to enviroment

CodeBuild Console / Add Enviroment Variables

We need the following variables in our CodeBuild:

REPO_URL - vaild url for git clone
BRANCH - branch wich was used to run the build

There are three methods depending on the way you did setup your CodePipeline to add environment variables:

set the enviroment variables in the CodePipeline Console
set the enviroment variables in the CodeBuild Console
use CloudFormation to set the environment variables in CodePipeline / CodeBuild

Final buildspec for Lighthouse CI

This simple example, which is very typical for node based applications e.g. gatsby. The lhci autorun combines collecting, asserting and uploading to a Lighhouse CI server and requires a lighthouserc.js file in your root to define your environment and tasks.

builspec.yml:

version: 0.2
env:
  git-credential-helper: yes
phases:
  install:
    commands:
      - npm install -g @lhci/cli
  pre_build:
    commands:
      - npm install --pure-lockfile
  build:
    commands:
      - scripts/codebuild-git-wrapper.sh $REPO_URL $REPO_BRANCH
      - git log -2
      - echo Build started on `date`
      - npm build
      - lhci autorun

The resulted output of your deployment log should look like this now:

$ /codebuild/output/src401949355/src/node_modules/.bin/lhci autorun
2157	✅  .lighthouseci/ directory writable
2158	✅  Configuration file found
2159	✅  Chrome installation found
2160	✅  Ancestor hash determinable
2161	✅  LHCI server reachable
2162	✅  LHCI server API-compatible
2163	✅  LHCI server token valid
2164	✅  LHCI server unique build for this hash
2165	Healthcheck passed!
2166
2167	Started a web server on port 45377...
2168	Running Lighthouse 3 time(s) on http://localhost:45377/index.html
2169	Run #1...done.
2170	Run #2...done.
2171	Run #3...done.
2172	Running Lighthouse 3 time(s) on http://localhost:45377/about/index.html
2173	Run #1...done.
2174	Run #2...done.
2175	Run #3...done.
2176	Running Lighthouse 3 time(s) on http://localhost:45377/en/blog/lighthouse-ci-fails-in-aws-codepipeline-codebuild/index.html
2177	Run #1...done.
2178	Run #2...done.
2179	Run #3...done.
2180	Done running Lighthouse!
2181
2182	Saving CI project website-test (15a1683b-3072-46f7-a72f-c6f9da1d2a3e)
2183	Saving CI build (dc92ed97-c7f8-4171-b74d-1af9dbf77b81)
2184	Saved LHR to http://lighhouse-ci-server.example:9001 (d2cc722b-9a89-4497-82e7-70fd4acaca77)
2185	Saved LHR to http://lighhouse-ci-server.example:9001 (2fd47ecd-d03e-4a47-b3ac-24f9301dcd80)
2186	Saved LHR to http://lighhouse-ci-server.example:9001 (f1bfb641-2e40-44a7-8773-a29b7c7b1cec)
2187	Saved LHR to http://lighhouse-ci-server.example:9001 (23692d1a-cb13-4347-837e-71288e1d7141)
2188	Saved LHR to http://lighhouse-ci-server.example:9001 (732eb3b0-2dd6-4237-bfd4-0a917627c24e)
2189	Saved LHR to http://lighhouse-ci-server.example:9001 (c64e1d7c-ff9c-42dc-8467-027c37ecb2be)
2190	Saved LHR to http://lighhouse-ci-server.example:9001 (062a732b-0130-4172-9192-87708cc47be9)
2191	Saved LHR to http://lighhouse-ci-server.example:9001 (274aa877-f69f-47c7-823a-127c7d21b89a)
2192	Saved LHR to http://lighhouse-ci-server.example:9001 (8e8b09c4-8969-4300-9cc6-12cba17bb27d)
2193	Done saving build results to Lighthouse CI
2194	View build diff at http://nagios.dienst.at:9001/app/projects/website2020/compare/dc92ed95-c6f8-4671-b74d-1af9dbf77b81
2195	No GitHub token set, skipping GitHub status check.
2196
2197	Done running autorun.
2198	Done in 123.33s.