Shopify Theme Check in CI: Fail on What Matters

Run shopify theme check in CI with an explicit --fail-level, a committed .theme-check.yml that extends theme-check:recommended and switches on the asset size checks that preset leaves off, and, for an older theme with hundreds of existing offences, a baseline file so the build fails on new errors instead of historical ones.

Each of those pieces exists because the default leaves a gap, and the gaps are specific.

Theme Check is Shopify’s linter for the Liquid and JSON in a theme and in theme app extensions. The current version is the TypeScript rewrite that ships inside Shopify CLI (the old Ruby gem is what theme-check-action@v1 ran), and the same engine powers the Shopify Liquid VS Code extension. So the warnings a developer sees in the editor are the ones CI will enforce, as long as both read the same config file.

What does shopify theme check actually fail on?

Every check has a severity: error, warning or info, which the config also accepts as 0, 1 and 2. The command exits with code 1 when any offence is at or above the --fail-level, and the default fail level is error.

The flag accepts six values. The command reference lists the flag without spelling out what each value does, but the CLI source shows them collapsing onto three severities:

  • error fails on errors only.
  • warning and suggestion both fail on warnings and errors.
  • info and style both fail on anything.
  • crash never fails on offences at all. The process still exits non-zero if Theme Check itself throws.

That last one looks useless until you want a job that always produces a report and lets a later step decide. We will use it below.

Which checks count as errors is set by the preset. In theme-check:recommended, ParserBlockingScript, ImgWidthAndHeight, LiquidHTMLSyntaxError, UnknownFilter, MissingAsset and TranslationKeyExists are errors; RemoteAsset, UnusedAssign, OrphanedSnippet, DeprecatedFilter and UndefinedObject are warnings. That last one is listed as an error in the checks reference, but the generated preset ships it at severity 1, so check the preset rather than the table when you decide what will break a build.

If a .theme-check.yml at the theme root has no extends key, Theme Check extends theme-check:recommended for you. That preset is generated from each check’s own recommended flag, and a handful are off. Diff it against theme-check:all in the theme-tools repo and the theme-relevant ones missing are AssetSizeJavaScript and AssetSizeCSS. (The other four are app-block checks for theme app extensions.)

Those two are the performance checks. AssetSizeJavaScript flags a script whose compressed size exceeds thresholdInBytes, default 10000. AssetSizeCSS does the same for stylesheets, default 100000. On a theme that has picked up a few apps and a slider library, these are the checks with something to say, and by default they say nothing.

A config worth committing:

# .theme-check.yml
extends: theme-check:recommended

ignore:
  - node_modules/**

AssetSizeJavaScript:
  enabled: true
  severity: error
  threshold_in_bytes: 10000

AssetSizeCSS:
  enabled: true
  severity: warning
  threshold_in_bytes: 100000

RemoteAsset:
  severity: error

Snake case and camel case both work for check options; the loader converts threshold_in_bytes to thresholdInBytes. Promoting RemoteAsset to an error is a judgement call. It fires on scripts and stylesheets served from third-party hosts, and on a Shopify storefront those are usually either an app that should be a theme app extension or a CDN copy of a library that belongs in assets/.

Run shopify theme check --print to see the resolved config after extends are merged, and --list to see which checks are enabled. Do both once locally before trusting CI.

How do I run Theme Check in GitHub Actions?

For a theme that is already clean, Shopify’s own theme-check-action is the shortest route. Version 2 runs shopify theme check through the CLI (its version input takes an @shopify/theme release, 3.50 or later), and, given token: ${{ github.token }}, writes annotations onto the pull request. Its base input limits annotations to files that differ from that ref, and flags passes anything else through, for example --fail-level warning.

Pin the CLI version with the action’s version input. The recommended preset is generated from the checks themselves, so a CLI upgrade can add a check or raise a severity. Pinned, that change arrives in a pull request that bumps the version, where someone is looking for it.

Can I add Theme Check to an old theme without fixing everything first?

Point shopify theme check at a five-year-old theme and you can get hundreds of offences. The first CI run is red, and the quick fix, --fail-level crash on every run, switches the gate off entirely.

The alternative is a ratchet. Commit a count of today’s errors per check, fail the build when any count goes up, and lower the numbers as people fix things. --output json gives you what you need: an array with one entry per file, each holding an offenses array whose items carry check, severity (as the strings error, warning or info), message and zero-based start_row and end_row.

// scripts/theme-check-ratchet.mjs
import { existsSync, readFileSync, writeFileSync } from 'node:fs';

const [reportPath, baselinePath = '.theme-check-baseline.json'] = process.argv.slice(2);
const report = JSON.parse(readFileSync(reportPath, 'utf8'));

const counts = {};
for (const file of report) {
  for (const offense of file.offenses) {
    if (offense.severity !== 'error') continue;
    counts[offense.check] = (counts[offense.check] ?? 0) + 1;
  }
}

if (process.env.UPDATE_BASELINE === '1') {
  writeFileSync(baselinePath, JSON.stringify(counts, null, 2) + '\n');
  console.log(`Wrote ${baselinePath}`);
  process.exit(0);
}

if (!existsSync(baselinePath)) {
  console.error(`No baseline at ${baselinePath}. Run with UPDATE_BASELINE=1 and commit it.`);
  process.exit(1);
}

const baseline = JSON.parse(readFileSync(baselinePath, 'utf8'));
let failed = false;

for (const check of new Set([...Object.keys(counts), ...Object.keys(baseline)])) {
  const now = counts[check] ?? 0;
  const allowed = baseline[check] ?? 0;
  if (now > allowed) {
    console.error(`${check}: ${now} errors, baseline allows ${allowed}`);
    failed = true;
  } else if (now < allowed) {
    console.log(`${check}: down to ${now} from ${allowed}. Lower the baseline.`);
  }
}

process.exit(failed ? 1 : 0);

The workflow runs the check with --fail-level crash so it always writes the report, then hands the decision to the script:

# .github/workflows/theme-check.yml
name: Theme Check
on: [pull_request]

jobs:
  theme-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm install -g @shopify/[email protected]
      - run: shopify theme check --output json --fail-level crash > theme-check.json
      - run: node scripts/theme-check-ratchet.mjs theme-check.json

Counting per check rather than per file is deliberate. Moving a broken snippet from one file to another should not fail the build, and adding a new ParserBlockingScript anywhere should.

When should I disable a check in the file instead?

Sometimes the offence is correct and you are shipping it anyway: a payment provider’s script that genuinely has to block, or a chat widget over the size limit that the client insists on. Theme Check reads Liquid comments for this, and the configuration docs list three forms: {% # theme-check-disable %} for everything that follows, {% # theme-check-disable CheckName %} for one check, and {% # theme-check-disable-next-line %} for a single line.

{% # theme-check-disable-next-line ParserBlockingScript %}
<script src="{{ 'payment-sdk.js' | asset_url }}"></script>

Name the check every time. A bare disable hides whatever the next CLI release decides to catch on that line. Syntax errors cannot be disabled this way, because they are raised while parsing, before comments are read.

shopify theme check --auto-correct fixes the offences that have an automatic fix. Run it locally and review the diff; do not wire it into CI, where it would edit files nobody reads.

What we would set up

On a new theme, or one built from Dawn, commit the config above, add the action with --fail-level error and a pinned version, and turn on annotations. On an inherited theme, start with the ratchet, keep --fail-level crash only for the report step, and put a “lower the baseline” line in every pull request that fixes something. We would not promote warnings to failures on day one on either kind of theme; UnusedAssign and OrphanedSnippet are useful to read and poor reasons to block a merge.

For related theme structure, our note on theme blocks versus section blocks covers the block model that checks such as UniqueStaticBlockId and ValidBlockTarget police, and the theme app extension post is where RemoteAsset offences from apps should end up.

Setting up this gate, and fixing what the asset size checks find, is part of our Shopify development work on existing themes.

Need this built properly?

Whoooop Ltd has spent 15+ years building and maintaining web applications in TypeScript, React, Node.js and serverless — the same ground this post covers.

Get in touch