Second Machine, Same Grade: A Bootcamp Lab on Hidden Local Assumptions

The grade is not “did it run on your laptop.” The grade is “did it run in a directory the model has never seen.” That’s the whole lab. Everything else is a demo.

AI coding assistants are generous. They will hardcode /Users/alex/week3, invent an npm package that looks real, and stash a key in a comment “just for now.” Then a student screenshots a green terminal and calls it shipped. Sound familiar?

I stopped arguing about which model is smarter. I started failing work that cannot survive a clean checkout. Cheap generation made that failure mode louder, not rarer. If code is cheap, portability is the assignment.

This is a student lab. Setup, checkpoints, a tiny harness you can actually run, stretch goals, and a rubric that does not reward a lucky home directory.

The failure I kept seeing

Not wrong algorithms. Wrong places.

The script only worked because the agent had sniffed the student’s machine. Absolute paths. A dependency that was never published. A .env that never left the laptop, until it did, in a gist. A node_modules folder treated as source.

Ask yourself one question before you grade: what did the model assume that a classmate’s shell will not provide?

If you cannot answer that in writing, you are grading theater.

What you are actually testing

You are not testing whether a model can write a JSON walker. You are testing whether the student can notice local assumptions and kill them.

Three buckets. That’s it.

1. Filesystem fiction — absolute home paths, hardcoded usernames, “run this from my Desktop.”
2. Registry fiction — package.json entries that do not exist, or versions the lockfile never pinned.
3. Secret fiction — keys in source, keys in prompts, keys in README “examples.”

Miss one bucket and the lab is incomplete. Pass all three on a second machine and we can talk about style.

Lab setup (45–60 minutes of machine time)

You need Node 18+, git, and a throwaway directory. No framework. No database. No “full stack.”

node -v
mkdir -p lab-second-machine && cd lab-second-machine
git init
npm init -y

Create a fixture the student did not author by hand in the same session as the solution. That separation matters. If the model writes both the app and the sample data in one breath, it will overfit both.

mkdir -p fixtures
cat > fixtures/orders.json <<'EOF'
[
  {"id": 1, "status": "paid", "sku": "tee"},
  {"id": 2, "status": "refund", "sku": "mug"},
  {"id": 3, "status": "paid", "sku": "tee"}
]
EOF

Assignment, one sentence: write a CLI json-summary that reads a file path from process.argv[2], prints how many records, and prints counts by status. stdout only. Exit 2 on bad input. Exit 1 on missing file. Exit 0 on success.

Students may use an assistant. They may not paste secrets into that assistant. They may not submit a transcript as the product.

Why a second environment is the point

A local green run is a hint. It is not evidence.

I run the same harness twice: once in the student repo, once in a clean clone on another machine or a disposable remote shell. Same commands. Same fixtures. No “it works if you cd into my nickname folder first.”

I used MonkeyCode’s free model access to draft student-facing starter comments, and the free server option as that second shell so I was not grading a laptop-shaped accident. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not claiming model names, token quotas, or hardware details here — only that a second, boring environment is part of the method. If you already have a spare VM, use that. The lab does not require a particular vendor. It requires a machine that does not love you.

Checkpoint 0 — freeze the contract before anyone generates

Write the contract on paper or in CONTRACT.md before the first prompt. If you let the model invent the CLI flags, you will grade the model’s taste, not the student’s judgment.

# CONTRACT.md
- argv[2] = path to a JSON array of objects
- required object key: status (string)
- stdout: one JSON object, no extra logs
  {"count": <number>, "byStatus": {"paid": n, ...}}
- exit 2: missing argv or invalid JSON
- exit 1: file not found
- no network, no extra npm deps unless justified in NOTES.md

Then freeze it. Changing the contract mid-lab is how you accidentally grade prompt churn.

Checkpoint 1 — generate, then diff your own tree

Let the student prompt. Then they run this before they smile:

git status --short
git diff --stat

If node_modules appears, fail Checkpoint 1. If a .env appears, fail Checkpoint 1. If the model wrote /Users/ anywhere, fail Checkpoint 1 and keep going — the harness will catch it too, and you want that muscle memory.

A starter they can delete later:

#!/usr/bin/env node
"use strict";

const fs = require("fs");
const path = require("path");

function fail(code, msg) {
  process.stderr.write(msg + "\n");
  process.exit(code);
}

const input = process.argv[2];
if (!input) fail(2, "usage: json-summary <file>");

const abs = path.resolve(process.cwd(), input);
let raw;
try {
  raw = fs.readFileSync(abs, "utf8");
} catch (err) {
  if (err && err.code === "ENOENT") fail(1, "file not found");
  fail(2, "unreadable file");
}

let data;
try {
  data = JSON.parse(raw);
} catch {
  fail(2, "invalid json");
}
if (!Array.isArray(data)) fail(2, "expected array");

const byStatus = {};
for (const row of data) {
  if (!row || typeof row.status !== "string") fail(2, "bad record");
  byStatus[row.status] = (byStatus[row.status] || 0) + 1;
}

process.stdout.write(JSON.stringify({ count: data.length, byStatus }) + "\n");

Label that as a reference shape, not the only solution. Students should rewrite it. If they paste it unchanged, the later checkpoints still bite — because the grade is the second machine, not the snippet.

Checkpoint 2 — the portability harness (the artifact)

Save this as harness/check-portability.js. No extra packages. Run it in CI, in a classmate’s clone, in a disposable shell. Anywhere but the original home folder.

#!/usr/bin/env node
"use strict";

const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");

const root = process.cwd();
const findings = [];

function walk(dir) {
  for (const name of fs.readdirSync(dir)) {
    if (name === ".git" || name === "node_modules") continue;
    const p = path.join(dir, name);
    const st = fs.statSync(p);
    if (st.isDirectory()) walk(p);
    else files.push(p);
  }
}

const files = [];
walk(root);

const absRe = /\/(Users|home)\/[A-Za-z0-9._-]+/g;
const secretRe = /(api[_-]?key|secret|token)\s*[:=]\s*['"][^'"]+['"]/i;

for (const file of files) {
  const rel = path.relative(root, file);
  if (rel.startsWith("harness")) continue;
  const text = fs.readFileSync(file, "utf8");
  const absHits = text.match(absRe) || [];
  if (absHits.length) findings.push({ code: "ABS_PATH", rel, absHits });
  if (secretRe.test(text)) findings.push({ code: "SECRET_SHAPE", rel });
}

if (fs.existsSync(path.join(root, ".env"))) {
  findings.push({ code: "DOTENV_PRESENT", rel: ".env" });
}

const pkgPath = path.join(root, "package.json");
if (fs.existsSync(pkgPath)) {
  const pkg = JSON.parse(fs.readFileSync(pkgPath, "utf8"));
  const deps = Object.assign({}, pkg.dependencies, pkg.devDependencies);
  for (const name of Object.keys(deps)) {
    const probe = spawnSync(process.execPath, ["-e", `require.resolve(${JSON.stringify(name + "/package.json")})`], {
      cwd: root,
      encoding: "utf8",
    });
    if (probe.status !== 0) findings.push({ code: "UNRESOLVED_DEP", rel: name });
  }
}

const bin = path.join(root, "json-summary.js");
const fixture = path.join(root, "fixtures", "orders.json");
const run = spawnSync(process.execPath, [bin, fixture], { encoding: "utf8" });
if (run.status !== 0) {
  findings.push({ code: "FIXTURE_FAIL", stderr: (run.stderr || "").slice(0, 400) });
} else {
  try {
    const parsed = JSON.parse(run.stdout.trim());
    if (parsed.count !== 3 || parsed.byStatus.paid !== 2) {
      findings.push({ code: "WRONG_SHAPE", stdout: run.stdout });
    }
  } catch {
    findings.push({ code: "NON_JSON_STDOUT", stdout: run.stdout });
  }
}

const tmp = fs.mkdtempSync(path.join(require("os").tmpdir(), "jsum-"));
const cloneFix = path.join(tmp, "orders.json");
fs.copyFileSync(fixture, cloneFix);
const run2 = spawnSync(process.execPath, [bin, cloneFix], { encoding: "utf8" });
if (run2.status !== 0) findings.push({ code: "TMPDIR_FAIL" });

if (findings.length) {
  process.stderr.write(JSON.stringify({ ok: false, findings }, null, 2) + "\n");
  process.exit(1);
}
process.stdout.write(JSON.stringify({ ok: true, files: files.length }) + "\n");

Run it like a mean TA:

chmod +x json-summary.js harness/check-portability.js
node harness/check-portability.js

Then the move that actually teaches:

cd /tmp
rm -rf second-machine-clone
git clone /path/to/lab-second-machine second-machine-clone
cd second-machine-clone
node harness/check-portability.js

Did it die? Good. That’s the lesson. Fix the assumption, not the screenshot.

Checkpoint 3 — name the assumptions in NOTES.md

I require a short list, written by the human. Not by the model. If the model writes the notes, you are back to grading fluency.

Students answer four prompts:

• Which paths did the first draft hardcode, if any?
• Which packages did the model suggest that you rejected, and why?
• What would break if cwd was /tmp?
• What did you refuse to paste into the assistant?

No novel required. Four blunt bullets. If they write “nothing,” and the harness later finds /Users/, that’s a grading event, not a debate.

Fair grading rubric

Cap at 100. Publish this on day one so nobody can claim surprise.

I do not give points for a prettier README. I do not give points for a longer prompt. I do give points for a evidence/harness-fail.json that shows the first red run. Failure is data. Hide it and I assume you got lucky.

Suggested weights if you need numbers for a LMS:

• Harness green on a clean clone: 40
• Human NOTES.md with rejected model output: 25
• Exit codes match CONTRACT.md: 20
• Evidence of a failed first harness run: 15

Stretch goals (optional, labeled)

These are proposals. Do not silently make them required.

1. Add a GitHub Actions workflow that runs node harness/check-portability.js on ubuntu-latest with no extra setup steps. If you need npm install, you must justify every dependency in NOTES.md.
2. Teach the CLI to refuse paths that escape the repo using path.relative and a .. check. Write a test that tries ../../etc/passwd as argv and expects exit 2.
3. Record a 90-second screen capture of the second machine, not the first. Narrate one assumption you killed.

If a student blows past stretch goal 2 by spawning a package the model invented, I still fail Checkpoint 2. Fancy extra credit cannot launder a fake dependency.

What this lab does not prove

It does not prove the algorithm is optimal. It does not prove the assistant is “accurate.” It does not replace a security review.

The harness is a blunt instrument. A student can still hide a key in an image. They can still fetch a package that exists and is malware. They can still prompt the model with production data. This lab catches sloppy local fiction, not adversaries.

Who should not use this approach

• Anyone treating a shared or free remote shell as a vault. If the data cannot be public, it does not go there. Full stop.
• Teams that need a compliance sign-off, an SLA, or a guaranteed model identity. This write-up does not provide those.
• Instructors who want to grade “which model wrote prettier code.” That contest trains taste, not judgment.
• Students who cannot run Node locally and cannot access any second environment. The method needs two machines, even if one is a classmate’s laptop for ten minutes.

A 25-minute classroom script

If you only have one session:

1. 5 min — freeze CONTRACT.md as a class. No generation yet.
2. 8 min — students prompt, commit, run git status.
3. 7 min — run the harness in /tmp clones. Watch the room go quiet.
4. 5 min — rewrite NOTES.md by hand. Collect evidence/.

That’s enough. You will hear “but it worked on mine.” Answer with the clone path. Then sit down.

Closing

When generation is cheap, the scarce skill is noticing what the model invented about your computer. Grade the second machine. Keep the first green terminal as a souvenir if you want. Just don’t put it on the rubric.

If you steal one file from this post, steal the harness. Run it somewhere unfriendly. Then ask your students the only question that still matters: what did the model assume that this shell refused to give?