Lighthouse CI Client fails in AWS CodePipeline / CodeBuild
You did setup a build pipeline for your JAM stack and now you would like to check the lighthouse score on each deployment as part of the build process. After adding the lighthouse client to the buildspec it fails:
1$ /codebuild/output/src463792406/src/node_modules/.bin/lhci autorun2 ✅ .lighthouseci/ directory writable3 ✅ Configuration file found4 ✅ Chrome installation found5 ⚠️ Ancestor hash not determinable6 ERROR: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`7 ✅ LHCI server reachable8 ✅ LHCI server API-compatible9 ✅ LHCI server token valid10 ⚠️ LHCI server non-unique build for this hash11 ERROR: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`
The Problem
The reason for this error is not a missing git client, it is the missing .git directory.
CodePipeline downloads the source as a zip from the source provider rather than doing a Git clone, which means the .git folder won’t be retained and git commands like the one you’re running won’t work.
Have a look at the requirements for lighthouse
The documentation gives us some clues in the Build Context Section and a list of environment variables to provide the values:
When uploading to the Lighthouse CI server, the CLI will attempt to automatically infer the build context such as git hash, author, message, ancestor, etc. In most cases, there's nothing you need to change about this, but if you're running without a git repo, in a Jenkins environment, a CI provider we haven't documented yet, or are just running into errors, you can control the build context yourself.
The following environment variables override the inferred build context settings.
| Name | Example Format |
|---|---|
LHCI_BUILD_CONTEXT__GIT_REMOTE | git@github.com:GoogleChrome/lighthouse-ci.git |
LHCI_BUILD_CONTEXT__GITHUB_REPO_SLUG | GoogleChrome/lighthouse-ci |
LHCI_BUILD_CONTEXT__CURRENT_HASH | e7f1b0fa3aebb6ef95e44c0d0b820433ffdd2e63 |
LHCI_BUILD_CONTEXT__ANCESTOR_HASH | 78bafdcaf40e204c0d0b81b90bb76b9aa5834e11 |
LHCI_BUILD_CONTEXT__COMMIT_TIME | 2019-11-29T16:43:39-05:00 |
LHCI_BUILD_CONTEXT__CURRENT_BRANCH | dev_branch_1234 |
LHCI_BUILD_CONTEXT__COMMIT_MESSAGE | Daily run of Lighthouse |
LHCI_BUILD_CONTEXT__AUTHOR | Patrick Hulce <patrick.hulce@example.com> |
LHCI_BUILD_CONTEXT__AVATAR_URL | https://example.com/patrickhulce.jpg |
LHCI_BUILD_CONTEXT__EXTERNAL_BUILD_URL | https://my-jenkins.example.com/jobs/123 |
Can we get this information from the CodeBuild enviroment?
The AWS CloudBuild documentation will list this variables:
AWS CodeBuild provides several variables that you can use in your build environment:
| Name | Description |
|---|---|
| AWS_DEFAULT_REGION | The AWS Region where the build is running (for example, us-east-1). This environment variable is used primarily by the AWS CLI. |
| AWS_REGION | The AWS Region where the build is running (for example, us-east-1). This environment variable is used primarily by the AWS SDKs. |
| CODEBUILD_BUILD_ARN | The Amazon Resource Name (ARN) of the build (for example, arn:aws:codebuild:region-ID:account-ID:build/codebuild-demo-project:b1e6661e-e4f2-4156-9ab9-82a19EXAMPLE). |
| CODEBUILD_BUILD_ID | The CodeBuild ID of the build (for example, codebuild-demo-project:b1e6661e-e4f2-4156-9ab9-82a19EXAMPLE). |
| CODEBUILD_BUILD_IMAGE | The CodeBuild build image identifier (for example, aws/codebuild/standard:2.0). |
| CODEBUILD_BUILD_NUMBER | The current build number for the project. |
| CODEBUILD_BUILD_SUCCEEDING | Whether the current build is succeeding. Set to 0 if the build is failing, or 1 if the build is succeeding. |
| CODEBUILD_INITIATOR | The entity that started the build. If CodePipeline started the build, this is the pipeline's name (for example, codepipeline/my-demo-pipeline). If an IAM user started the build, this is the user's name (for example, MyUserName). If the Jenkins plugin for CodeBuild started the build, this is the string CodeBuild-Jenkins-Plugin. |
| CODEBUILD_KMS_KEY_ID | The identifier of the AWS KMS key that CodeBuild is using to encrypt the build output artifact (for example, arn:aws:kms:region-ID:account-ID:key/key-ID or alias/key-alias). |
| CODEBUILD_LOG_PATH | The log stream name in CloudWatch Logs for the build. |
| CODEBUILD_RESOLVED_SOURCE_VERSION | An identifier for the version of a build's source code. |
| CODEBUILD_SOURCE_REPO_URL | The URL to the input artifact or source code repository. For Amazon S3, this is s3:// followed by the bucket name and path to the input artifact. For CodeCommit and GitHub, this is the repository's clone URL. If a build originates from CodePipeline, then this might be empty. |
| CODEBUILD_SOURCE_VERSION | For CodeCommit, it is the commit ID or branch name associated with the version of the source code to be built. For GitHub, GitHub Enterprise, and Bitbucket it is the commit ID, branch name, or tag name associated with the version of the source code to be built. |
| CODEBUILD_SRC_DIR | The directory path that CodeBuild uses for the build (for example, /tmp/src123456789/src). |
| CODEBUILD_START_TIME | The start time of the build specified as a Unix timestamp in milliseconds. |
| CODEBUILD_WEBHOOK_ACTOR_ACCOUNT_ID | The account ID of the user that triggered the webhook event. |
| CODEBUILD_WEBHOOK_BASE_REF | The base reference name of the webhook event that triggers the current build. For a pull request, this is the branch reference. |
| CODEBUILD_WEBHOOK_EVENT | The webhook event that triggers the current build. |
| CODEBUILD_WEBHOOK_PREV_COMMIT | The ID of the most recent commit before the webhook push event that triggers the current build. |
| CODEBUILD_WEBHOOK_HEAD_REF | The head reference name of the webhook event that triggers the current build. It can be a branch reference or a tag reference. |
| CODEBUILD_WEBHOOK_TRIGGER | Shows the webhook event that triggered the build. This variable is available only for builds triggered by a webhook. |
We do have the current commit hash but we do not have the commit message and there is no way to get the ancestor hash.
Fixing the Problem
After searching on google and failing with some other solutions I found this great post from Timothy Jones: How to access git metadata in CodeBuild when using CodePipeline/CodeCommit
He did a very good research on the topic and provides a solution which allows to integrate lighthouse without a need for changes when lighthouse or any other software needs access to git metadata.
1) Allow CodeBuild to clone your repository
This is only needed, if you use a CodeCommit repository. For github, gitlab, Bitbucket this step can be skipped.
Give your CodeBuild execution role the permission to clone the repository:
1{2 "Effect": "Allow",3 "Action": [4 "codecommit:GitPull"5 ],6 "Resource": [7 "arn:aws:codecommit:*:*:<YOUR_REPOSITORY_NAME>"8 ]9},
2) Add git credential helper to CodeBuild
This is only needed, if you use a CodeCommit repository. For github, gitlab, Bitbucket this step can be skipped.
Add this to the start of your builspec.yml:
1version: 0.223env:4 git-credential-helper: yes
This will allow git to access the repository with the existing AWS credentials in the build container.
3) Clone repository
Our requirements are:
- clone the repository to a temporal directory
- switch current branch and commit based om the values in the AWS build environment variables
- move the
.gitdirectory into the source directory
Happily for us Timothy Jones already create a script which does this for us. You can download this script here or with this code:
1curl https://raw.githubusercontent.com/TimothyJones/codepipeline-git-metadata-example/master/scripts/codebuild-git-wrapper.sh \2-o codebuild-git-wrapper.sh
4) Add repository url and branch to enviroment
We need the following variables in our CodeBuild:
- REPO_URL - vaild url for git clone
- BRANCH - branch wich was used to run the build
There are three methods depending on the way you did setup your CodePipeline to add environment variables:
- set the enviroment variables in the CodePipeline Console
- set the enviroment variables in the CodeBuild Console
- use CloudFormation to set the environment variables in CodePipeline / CodeBuild
Final buildspec for Lighthouse CI
This simple example, which is very typical for node based applications e.g. gatsby.
The lhci autorun combines collecting, asserting and uploading to a Lighhouse CI server and requires a lighthouserc.js
file in your root to define your environment and tasks.
builspec.yml:
1version: 0.22env:3 git-credential-helper: yes4phases:5 install:6 commands:7 - npm install -g @lhci/cli8 pre_build:9 commands:10 - npm install --pure-lockfile11 build:12 commands:13 - scripts/codebuild-git-wrapper.sh $REPO_URL $REPO_BRANCH14 - git log -215 - echo Build started on `date`16 - npm build17 - lhci autorun
The resulted output of your deployment log should look like this now:
1$ /codebuild/output/src401949355/src/node_modules/.bin/lhci autorun22157 ✅ .lighthouseci/ directory writable32158 ✅ Configuration file found42159 ✅ Chrome installation found52160 ✅ Ancestor hash determinable62161 ✅ LHCI server reachable72162 ✅ LHCI server API-compatible82163 ✅ LHCI server token valid92164 ✅ LHCI server unique build for this hash102165 Healthcheck passed!112166122167 Started a web server on port 45377...132168 Running Lighthouse 3 time(s) on http://localhost:45377/index.html142169 Run #1...done.152170 Run #2...done.162171 Run #3...done.172172 Running Lighthouse 3 time(s) on http://localhost:45377/about/index.html182173 Run #1...done.192174 Run #2...done.202175 Run #3...done.212176 Running Lighthouse 3 time(s) on http://localhost:45377/en/blog/lighthouse-ci-fails-in-aws-codepipeline-codebuild/index.html222177 Run #1...done.232178 Run #2...done.242179 Run #3...done.252180 Done running Lighthouse!262181272182 Saving CI project website-test (15a1683b-3072-46f7-a72f-c6f9da1d2a3e)282183 Saving CI build (dc92ed97-c7f8-4171-b74d-1af9dbf77b81)292184 Saved LHR to http://lighhouse-ci-server.example:9001 (d2cc722b-9a89-4497-82e7-70fd4acaca77)302185 Saved LHR to http://lighhouse-ci-server.example:9001 (2fd47ecd-d03e-4a47-b3ac-24f9301dcd80)312186 Saved LHR to http://lighhouse-ci-server.example:9001 (f1bfb641-2e40-44a7-8773-a29b7c7b1cec)322187 Saved LHR to http://lighhouse-ci-server.example:9001 (23692d1a-cb13-4347-837e-71288e1d7141)332188 Saved LHR to http://lighhouse-ci-server.example:9001 (732eb3b0-2dd6-4237-bfd4-0a917627c24e)342189 Saved LHR to http://lighhouse-ci-server.example:9001 (c64e1d7c-ff9c-42dc-8467-027c37ecb2be)352190 Saved LHR to http://lighhouse-ci-server.example:9001 (062a732b-0130-4172-9192-87708cc47be9)362191 Saved LHR to http://lighhouse-ci-server.example:9001 (274aa877-f69f-47c7-823a-127c7d21b89a)372192 Saved LHR to http://lighhouse-ci-server.example:9001 (8e8b09c4-8969-4300-9cc6-12cba17bb27d)382193 Done saving build results to Lighthouse CI392194 View build diff at http://nagios.dienst.at:9001/app/projects/website2020/compare/dc92ed95-c6f8-4671-b74d-1af9dbf77b81402195 No GitHub token set, skipping GitHub status check.412196422197 Done running autorun.432198 Done in 123.33s.