# How I broke Gatsby JS conditional page build and learned to debug the tool chain - URL: https://www.heissenberger.at/en/blog/gatsby-conditional-build-broken-debugging/ - Language: en - Published: 2020-05-29 - Tags: AWS, CodeBuild, Gatsby JS, FixIt > Learn about Gatsby internal architecture for GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES which does not work as expected in an CI environment. I built my AWS CodeBuild Pipeline with a new feature called "[Conditional Page Builds](https://www.gatsbyjs.org/docs/conditional-page-builds/)" and it did not worked as expected in the build environment. Starting the build process again with no change in the source code and with identical copies of the `.cache` and `public` folder generated this output: ``` ... info One or more of your plugins have changed since the last time you ran Gatsby. As a precaution, we're deleting your site's cache to ensure there's no stale data. success initialize cache - 0.011s ... success Building production JavaScript and CSS bundles - 30.860s success Rewriting compilation hashes - 0.005s ... success run queries - 49.184s - 44/44 0.89/s success Building static HTML for pages - 17.453s - 40/40 2.29/s success Delete previous page data - 0.000s success Generating image thumbnails - 166.557s - 316/316 1.90/s ... ``` a second call to `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true gatsby build --log-pages` immediately following the first one shows the expected output: ```bash ... success onPreInit - 0.020s success initialize cache - 0.012s ... success Building production JavaScript and CSS bundles - 4.623s success run queries - 4.804s - 4/4 0.83/s success Building static HTML for pages - 0.732s - 0/0 0.00/s success Delete previous page data - 0.002s ... ``` > This proves the fact that the conditional page build is working and no page contains any data which is different on each creation. ## Comparing the two outputs I found these problems: 1. Cache invalidation by changed plugins `info One or more of your plugins have changed since the last time you ran Gatsby. As a precaution, we're deleting your site's cache to ensure there's no stale data.` 2. Rewriting compilation hashes 3. Regenerating Images `success Generating image thumbnails` To open an issue on github I had to create a simple setup simulating the problems with the environment. [Issue 1 and 2 (posted on github) where my fault](https://github.com/gatsbyjs/gatsby/issues/24568) by creating a temporal file with a name starting with `gatsby-*` which conflicted with a part of gatsby creating a hash on the content of all files starting with this name. ## creating a simpler setup 1. create a gatsby site based on the default starter template 2. upgrade gatsby and the tool chain to the latest stable versions ```bash npx gatsby new gatsby-starter cd gatsby-starter yarn upgrade --latest ``` on the first run of `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true yarn build --log-pages` the result was as expected - there was no `.cache` and no `public` folder. The build process creates the compilation hashes [24], the static HTML [26] and the image thumbnails [28]. ```bash $ gatsby build --log-pages success open and validate gatsby-configs - 0.034s success load plugins - 1.153s success onPreInit - 0.020s success initialize cache - 0.012s success copy gatsby files - 0.122s success onPreBootstrap - 0.018s success createSchemaCustomization - 0.008s success source and transform nodes - 0.140s success building schema - 0.535s success createPages - 0.002s success createPagesStatefully - 0.082s success onPreExtractQueries - 0.002s success update schema - 0.051s success extract queries from components - 0.340s success write out requires - 0.007s success write out redirect data - 0.002s success Build manifest and related icons - 0.211s success onPostBootstrap - 0.220s ⠀ info bootstrap finished - 8.052s ⠀ success Building production JavaScript and CSS bundles - 6.059s success Rewriting compilation hashes - 0.061s success run queries - 6.757s - 7/7 1.04/s success Building static HTML for pages - 0.904s - 4/4 4.43/s success Delete previous page data - 0.003s success Generating image thumbnails - 7.766s - 6/6 0.77/s success onPostBuild - 0.018s info Done building in 15.908345021 sec info Built pages: Updated page: /404/ Updated page: / Updated page: /page-2/ Updated page: /404.html ✨ Done in 16.24s. ``` running the same build command, a second time gives this output: ```bash $ gatsby build --log-pages success open and validate gatsby-configs - 0.033s success load plugins - 0.827s success onPreInit - 0.018s success initialize cache - 0.007s success copy gatsby files - 0.109s success onPreBootstrap - 0.009s success createSchemaCustomization - 0.006s success source and transform nodes - 0.067s success building schema - 0.375s success createPages - 0.002s success createPagesStatefully - 0.139s success onPreExtractQueries - 0.004s success update schema - 0.046s success extract queries from components - 0.286s success write out requires - 0.016s success write out redirect data - 0.004s success Build manifest and related icons - 0.223s success onPostBootstrap - 0.249s ⠀ info bootstrap finished - 6.451s ⠀ success Building production JavaScript and CSS bundles - 4.623s success run queries - 4.804s - 4/4 0.83/s success Building static HTML for pages - 0.732s - 0/0 0.00/s success Delete previous page data - 0.002s success onPostBuild - 0.002s info Done building in 12.096156853 sec ✨ Done in 12.45s. ``` as expected the following steps have been skipped: - Rewriting compilation hashes - Building static HTML 0/0 - Generating Images ## simulating the build process of the build Pipeline 1. create temporal directory 2. clone the git repository to a temporal directory 3. restore the build cache 4. run the build command 5. save the build cache a simple bash script [`codepipeline.sh`](https://github.com/aheissenberger/gatsby-cache-problem/blob/master/scripts/codepipeline.sh) would look like this (don't use this for production as many edge cases are missing!): ```bash #!/bin/bash set -x BUILD_CACHE=/tmp/buildcache.tgz TMPDIR=$(mktemp -d) git clone https://github.com/aheissenberger/gatsby-cache-problem.git $TMPDIR if [ -f "$BUILD_CACHE" ]; then tar -xf "$BUILD_CACHE" --directory $TMPDIR fi pushd $TMPDIR yarn install GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true yarn build --log-pages || exit 1 if [ -f "$BUILD_CACHE" ]; then rm "$BUILD_CACHE" fi tar czf "$BUILD_CACHE" .cache/ public/ popd ``` Run this script twice and you will see that on the second run the image generation still happens: ``` ... success Building static HTML for pages - 1.621s - 0/0 0.00/s success Delete previous page data - 0.004s success Generating image thumbnails - 14.491s - 6/6 0.41/s <<==LOOK HERE success onPostBuild - 0.012s info Done building in 21.23928199 sec ✨ Done in 21.56s. + '[' -f /tmp/buildcache.tgz ']' + rm /tmp/buildcache.tgz + tar czf /tmp/buildcache.tgz .cache/ public/ + popd ``` I documented my environment and [posted an issue on github](https://github.com/gatsbyjs/gatsby/issues/24571) after not finding any other issue with a similar problem. > Since I am not very patient, I decided to look at the problem myself. ## How to debug Gatsby JS in VS Code The first thing I do when testing out a new framework is setting up the Javascript debugger in VS Code. Gatsby provides a very [good documentation](https://www.gatsbyjs.org/docs/debugging-the-build-process/#vs-code-debugger-manual-config) which makes this a breeze. Add this to your existing VS Code `launch.json` to debug **conditional page building**: ```json { "name": "Gatsby build Conditional", "type": "node", "request": "launch", "protocol": "inspector", "program": "${workspaceRoot}/node_modules/gatsby/dist/bin/gatsby", "args": ["build", "--write-to-file", "--log-pages"], "env": { "GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES": "true", "CI": "true" }, "skipFiles": [], "stopOnEntry": false, "runtimeArgs": ["--nolazy"], "sourceMaps": false } ``` ## activate searching in the `node_modules` folder open the VS Code configuration `.vscode/settings.json` and add this: ```json { "search.exclude": { "**/node_modules": false }, "search.useIgnoreFiles": false } ``` ## Dig into the problem The goal is to find the place in the code where the logic is which compares values from the cache with the existing environment and decides to rerun the image generation. 1. let's search for `Generating image thumbnails` in the `node_modules` folder the result will list `CHANGELOG.md` and `node_modules/gatsby-plugin-sharp/utils.js:38` looking into the second file will show us the code which prints the success message 2. set a break point in this line [38] 3. remove the `.cache` and `public` folder: `rm -fr .cache/ ./public/` 4. start the VS Code Debugger and choose `Gatsby build Conditional` from the pulldown 5. the debugger stops at line 38 ::figure[VS Code debugging / utils.js:38]{src="./debug-01.png" alt="VS Code debugging / utils.js:38" width="100%"} 6. look at the `CALL STACK` in the left column and click on the second line - this is the script which called this function ::figure[VS Code debugging / gatsby-node.js:91]{src="./debug-02.png" alt="VS Code debugging / gatsby-node.js:91" width="100%"} 7. we see that this code part of the `gatsby-plugin-sharp` is activated by the event `CREATE_JOB_V2` 8. if this would have been a normal function call, simple looking up the CALL STACK would be easy, but this is an event which could have been raised anywhere in the code based 9. cancel the debugger and search again for `CREATE_JOB_V2` inside the `node_modules` folder the search result contains 6 files where `node_modules/gatsby/dist/redux/actions/public.js` looks most promising ::figure[VS Code search result / public.js:1198]{src="./search_CREATE_JOB_V2.png" alt="VS Code search result / public.js:1198" width="100%"} 10. looking at the 8 lines of this function: ```javascript actions.createJobV2 = (job, plugin) => (dispatch, getState) => { const currentState = getState() const internalJob = createInternalJob(job, plugin) const jobContentDigest = internalJob.contentDigest // Check if we already ran this job before, if yes we return the result // We have an inflight (in progress) queue inside the jobs manager to make sure // we don't waste resources twice during the process if (currentState.jobsV2 && currentState.jobsV2.complete.has(jobContentDigest)) { return Promise.resolve(currentState.jobsV2.complete.get(jobContentDigest).result) } // removed... } ``` - line 2 gets the state from the redux store which was populated from the `.cache` folder - line 3 creates a new Job object - line 4 gets the content digest hash from the new job - line 8 does a lookup in the redux store if the job has been done before 11. set a breakpoint on `const internalJob = createInternalJob(job, plugin);` 12. remove the `.cache` and `public` folder: `rm -fr .cache/ ./public/` 13. run debugger - choose `Gatsby build Conditional` from the pulldown 14. when the debugger pauses we can inspect the job attributes ::figure[VS Code debug / public.js:1182]{src="./debug-inspect-variables.png" alt="VS Code debug / public.js:1182" width="100%"} the left column shows us the following attributes with absolute paths `/User/xxxxxxxx/...`: _job.inputPaths, job.outputDir, plugin.pluginFilepath, plugin.resolve_ 15. step into the function `createInternalJob` to check on the creation of the diggest ::figure[VS Code debug / jobs-manager.js]{src="./debug-jobs-manager.png" alt="VS Code debug / jobs-manager.js" width="100%"} 16. scrolldown until you see the place where the `contentDigest` is created: ```javascript internalJob.contentDigest = createContentDigest({ name: job.name, inputPaths: internalJob.inputPaths.map(inputPath => inputPath.contentDigest), outputDir: internalJob.outputDir, args: internalJob.args, plugin: internalJob.plugin, }) ``` ### We found the Problem > The hash created by Gatsby Job Queue Manager will always break when compared to values in the cache which > have been created in a source folder with a different absolute path as long as the Job and the Plugin object > contain absolute paths. I also [create an issue on github](https://github.com/gatsbyjs/gatsby/issues/24571) where you can follow the next steps. ## Quick Fix Always copy the source folder to the same path which has been used when the `.cache` folder was created. ## Long term Solutions to the Problem - convert the object to a JSON string and replace the path to the root folder with a stable string or hash (still only a quick fix) - do not allow to store any absolute path in any Gatsby data structures: `Job` objects, `plugin` objects I hope I could give some people a first impression on how to track down a problem by yourselves and provide maintainers of a project a better documentation to fix architectural problems. [Write a response on Medium](https://medium.com/p/edfd4cfa3ae/responses/show)