Compare commits

..

3 Commits

Author SHA1 Message Date
jaybe 5dce80e56d ci: verify 단계 폴더 보장 및 요약 로그 출력 추가\n\n- artifacts/export 폴더 보장 스텝 추가\n- 무결성 검증 요약(ok/total/missing/...) 한 줄 출력
Auto PR / open-pr (push) Successful in 17s
Auto Label PR / add-automerge-label (pull_request) Successful in 6s
CI / test (pull_request) Successful in 1m14s
2025-11-16 20:58:22 +09:00
jaybe e32d59ca44 ci: 무결성 검증 통합(권장 플로우)
Auto PR / open-pr (push) Successful in 16s
Auto Label PR / add-automerge-label (pull_request) Successful in 6s
CI / test (pull_request) Successful in 1m13s
- scripts/make-verify-json.js: 산출물 폴더 → extras/assets JSON 생성
- scripts/verify.js: JSON 입력으로 무결성 검증(종료코드 반영)
- package.json: verify 스크립트 추가
- .gitea/workflows/ci.yml: ./artifacts/export 존재 시 검증 스텝 실행
2025-11-16 20:52:18 +09:00
jaybe 86155c9fd3 feat: verifyIntegrity CLI 래퍼 추가 및 문서화 (TDD)
Auto PR / open-pr (push) Successful in 16s
Auto Label PR / add-automerge-label (pull_request) Successful in 5s
CI / test (pull_request) Successful in 1m11s
- verify.cli.ts/verify.cli.test.ts 추가
- README 사용법/종료코드/CI 가이드 문서화
- 전체 테스트 그린 유지
2025-11-16 20:24:30 +09:00
10 changed files with 446 additions and 7 deletions
+27
View File
@@ -43,6 +43,33 @@ jobs:
if-no-files-found: ignore
retention-days: 3
- name: Ensure artifacts export folder
run: |
set -euo pipefail
mkdir -p ./artifacts/export
- name: Verify bundle integrity (optional)
run: |
set -euo pipefail
if [ -f ./artifacts/export/assets.integrity.index.json ]; then
echo "Found integrity index; generating verify inputs"
npm run verify:make-json
echo "Running verify CLI"
npm run verify:run | tee /tmp/verify_output.json
echo "--- verify summary (for quick view) ---"
node -e '
const fs = require("fs");
try {
const txt = fs.readFileSync("/tmp/verify_output.json", "utf-8");
const obj = JSON.parse(txt);
const s = obj.summary || {};
console.log(`ok=${obj.ok} total=${s.total||0} missing=${s.missing||0} integMismatch=${s.integrityMismatches||0} sizeMismatch=${s.sizeMismatches||0} bundleHashMismatch=${s.bundleHashMismatch||false}`);
} catch (e) { console.log("(verify summary parse skipped)"); }
'
else
echo "No integrity index at ./artifacts/export/assets.integrity.index.json; skipping integrity verification"
fi
- name: Auto-merge PR on green
if: ${{ github.event_name == 'pull_request' }}
env:
+19 -6
View File
@@ -361,7 +361,7 @@
- [ci] auto pipeline 최종 재검증 최종 ping (2025-11-14 22:26:59)
## Export ZIP 무결성 강화(Option2) 진행 현황
### Export ZIP 무결성 강화(Option2) 진행 현황
### 목표
- Export ZIP 무결성 강화: 자산 무결성 지표와 번들 검증 체계 구축
@@ -382,6 +382,14 @@
- Export Options(폰트 프리커넥트/프리로드, 이미지 로딩/디코딩, robots 메타) UI/문서화
- 브랜치 푸쉬 완료: `feat/export-bundle-integrity`
- 이미지 최적화 옵션(1차): `fetchpriority`/`srcset`/`sizes` 추가 및 테스트 그린
- 이미지 최적화 옵션(2차): `imageAutoSizes`(srcset만 있을 때 기본 sizes 자동 적용)
- 이미지 최적화 옵션(3차): `imageSizesPreset`(mobile-first/desktop-fixed) + 우선순위(imageSizes > preset > auto > none)
- 브랜치/PR: `feat/image-optimization` 푸쉬 완료
- verifyIntegrity CLI 래퍼(TDD) 추가: `lib/exporter/verify.cli.ts` + 테스트
- 폴더 기반 직렬화 유틸(TDD) 추가: `lib/exporter/verify.serialize.ts` + 테스트
- CI 통합(권장 플로우): `.gitea/workflows/ci.yml``./artifacts/export` 존재 시 검증 스텝 자동 실행
- 스크립트: `scripts/make-verify-json.js`, `scripts/verify.js` + `npm run verify:make-json`/`verify:run`
- 브랜치/PR: `feat/verify-integrity-cli` 푸쉬 완료
### 하드닝
- ZIP 경로 가드: `../`, 절대경로(`/`), 역슬래시(`\`) 차단 (테스트 포함)
@@ -397,11 +405,16 @@
- Node/브라우저 호환 btoa/atob 제거, 안전한 변환 로직 적용
### 다음 작업
1) PR 생성(템플릿/라벨): `feat/export-bundle-integrity``main`
2) CI 무결성 검증 스텝 추가: `verifyIntegrity`로 빌드 산출물 검사(요약 리포트 로깅)
3) PR 라벨 자동화 및 조건부 머지(Gitea Actions/웹훅)
4) README에 사용 가이드 보완(verifyIntegrity 활용 예시)
5) 이미지 최적화 2차: `sizes` 기본 정책 프리셋 및 변환 파이프라인(AVIF/WebP) 설계 초안(TDD 스텁)
1) PR 생성/머지 순서 관리: `feat/image-optimization`, `feat/verify-integrity-cli` → main
2) CI 검증 리포트 형식 보강: 요약 필드(total/missing/...)를 워크플로 로그에 하이라이트
3) PR 라벨 자동화 및 조건부 머지(Gitea Actions/웹훅) 유지
4) 변환 파이프라인(AVIF/WebP) 설계 초안(TDD 스텁)
5) README/개발자 가이드에 `./artifacts/export` 폴더 정책 명시(로컬/CI 공통)
### 산출물 경로 정책(CI)
- 기본 폴더: `./artifacts/export/` (ZIP 해제 또는 직접 생성)
- 검증 입력: `npm run verify:make-json``./artifacts/extras.json`/`./artifacts/assets.json`
- 검증 실행: `npm run verify:run` (종료코드 0/1/2)
### CI/CD 계획(초안)
- PR 오픈 시: 라벨 자동 지정, 테스트/빌드, 무결성 검증 스텝 실행
+20
View File
@@ -80,6 +80,26 @@ The Builder can export a production-ready ZIP that contains:
### Asset paths and grouping
- AssetManager writes files to `assets/` with stable hash-based filenames.
### verifyIntegrity CLI
- Purpose: Validate exported bundle assets against `assets.integrity.index.json`.
- Input format: two JSON files mapping file path → base64 bytes
- Extras JSON: must contain `"assets.integrity.index.json"` key with base64 of the index file
- Assets JSON: keys are asset paths, values are base64 of file bytes
- Usage:
```bash
node scripts/verify.js --extras extras.json --assets assets.json
```
- Output: prints a JSON with fields of `VerifyResult` (`ok`, `errors`, `summary`)
- Exit codes:
- 0: ok
- 1: verification failed or invalid inputs
- 2: usage error (missing args)
- CI: add a step to run the CLI after build/export; fail the job if exit code != 0.
- Path strategy can be configured:
- `flat` (default): `assets/<hash>.<ext>`
- `grouped`: `assets/images/...`, `assets/fonts/...`, `assets/other/...`
+54
View File
@@ -0,0 +1,54 @@
import { describe, it, expect } from 'vitest'
import { verifyCli } from '@/lib/exporter/verify.cli'
// helpers to build base64 from string
function b64(s: string) {
if (typeof Buffer !== 'undefined') return Buffer.from(s, 'utf-8').toString('base64')
throw new Error('Buffer not available')
}
describe('verify.cli', () => {
const EMPTY_SHA256_B64 = '47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU='
it('returns 0 and logs summary when ok', async () => {
// empty entries integrity index; bundleHash = sha256("") in base64
const extras = {
'assets.integrity.index.json': b64(JSON.stringify({ entries: [], bundleHash: 'sha256-' + EMPTY_SHA256_B64 }))
}
const assets = {}
const extrasJson = JSON.stringify(extras)
const assetsJson = JSON.stringify(assets)
const paths: Record<string, string> = {
'/tmp/extras.ok.json': extrasJson,
'/tmp/assets.ok.json': assetsJson,
}
const logs: string[] = []
const readText = async (p: string) => {
const v = paths[p]
if (v == null) throw new Error('no such file: ' + p)
return v
}
const code = await verifyCli(['--extras', '/tmp/extras.ok.json', '--assets', '/tmp/assets.ok.json'], readText, (s: string) => logs.push(s))
expect(code).toBe(0)
expect(logs.join('\n')).toMatch(/"ok": true/)
expect(logs.join('\n')).toMatch(/"total": 0/)
})
it('returns 1 and logs errors when invalid JSON', async () => {
const paths: Record<string, string> = {
'/tmp/extras.bad.json': JSON.stringify({ 'assets.integrity.index.json': 'not-base64' }),
'/tmp/assets.empty.json': JSON.stringify({}),
}
const logs: string[] = []
const readText = async (p: string) => {
const v = paths[p]
if (v == null) throw new Error('no such file: ' + p)
return v
}
const code = await verifyCli(['--extras', '/tmp/extras.bad.json', '--assets', '/tmp/assets.empty.json'], readText, (s: string) => logs.push(s))
expect(code).toBe(1)
expect(logs.join('\n')).toMatch(/"ok": false/)
expect(logs.join('\n')).toMatch(/Invalid assets\.integrity\.index\.json JSON|Missing assets\.integrity\.index\.json/)
})
})
+68
View File
@@ -0,0 +1,68 @@
import { verifyIntegrity } from '@/lib/exporter/verify'
export type ReadText = (path: string) => Promise<string>
export type Logger = (line: string) => void
// base64 -> Uint8Array
function fromBase64(b64: string): Uint8Array {
if (typeof Buffer !== 'undefined') return new Uint8Array(Buffer.from(b64, 'base64'))
throw new Error('Buffer not available')
}
// parse args like: [--extras, path, --assets, path]
function parseArgs(argv: string[]): { extras?: string; assets?: string } {
const out: { extras?: string; assets?: string } = {}
for (let i = 0; i < argv.length; i++) {
const a = argv[i]
if (a === '--extras') out.extras = argv[++i]
else if (a === '--assets') out.assets = argv[++i]
}
return out
}
export async function verifyCli(argv: string[], readText: ReadText, log: Logger): Promise<number> {
try {
const { extras: extrasPath, assets: assetsPath } = parseArgs(argv)
if (!extrasPath || !assetsPath) {
log('Usage: verify --extras <extras.json> --assets <assets.json>')
return 2
}
const extrasJson = await readText(extrasPath)
const assetsJson = await readText(assetsPath)
let extrasMap: Record<string, string>
let assetsMap: Record<string, string>
try {
extrasMap = JSON.parse(extrasJson)
assetsMap = JSON.parse(assetsJson)
} catch (e) {
log('Invalid JSON input files')
return 1
}
// decode base64 into Uint8Array maps
const extrasBytes: Record<string, Uint8Array> = {}
for (const [k, v] of Object.entries(extrasMap)) {
try {
extrasBytes[k] = fromBase64(v)
} catch {
// mark invalid; let verifyIntegrity handle missing/invalid as possible
log(`Failed to decode base64 for extras entry: ${k}`)
return 1
}
}
const assetsBytes: Record<string, Uint8Array> = {}
for (const [k, v] of Object.entries(assetsMap)) {
try {
assetsBytes[k] = fromBase64(v)
} catch {
log(`Failed to decode base64 for asset entry: ${k}`)
return 1
}
}
const result = verifyIntegrity(extrasBytes, assetsBytes)
log(JSON.stringify(result, null, 2))
return result.ok ? 0 : 1
} catch (e) {
log(String(e instanceof Error ? e.message : e))
return 1
}
}
+48
View File
@@ -0,0 +1,48 @@
import { describe, it, expect } from 'vitest'
import { generateVerifyInputs } from '@/lib/exporter/verify.serialize'
// helper to b64
function toB64(u8: Uint8Array) {
if (typeof Buffer !== 'undefined') return Buffer.from(u8).toString('base64')
throw new Error('Buffer not available')
}
describe('verify.serialize (from folder maps)', () => {
it('collects assets.integrity.index.json into extras and assets/* into assets map', async () => {
// virtual FS
const files: Record<string, Uint8Array> = {
'/artifacts/export/assets.integrity.index.json': new TextEncoder().encode(JSON.stringify({ entries: [], bundleHash: 'sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=' })),
'/artifacts/export/index.html': new TextEncoder().encode('<!doctype html>'),
'/artifacts/export/assets/img/a.png': new Uint8Array([1,2,3]),
'/artifacts/export/assets/fonts/f.woff2': new Uint8Array([4,5,6,7]),
}
const listDir = async (dir: string): Promise<string[]> => {
// return child paths (recursive listing simulated by filtering)
return Object.keys(files).filter((p) => p.startsWith(dir)).map((p) => p)
}
const readFile = async (p: string) => {
const v = files[p]
if (!v) throw new Error('not found: ' + p)
return v
}
const { extras, assets } = await generateVerifyInputs('/artifacts/export', listDir, readFile)
// extras must have only the integrity index
expect(Object.keys(extras)).toEqual(['assets.integrity.index.json'])
expect(extras['assets.integrity.index.json']).toBe(toB64(files['/artifacts/export/assets.integrity.index.json']))
// assets must include assets/* files only
expect(Object.keys(assets).sort()).toEqual(['assets/fonts/f.woff2','assets/img/a.png'])
expect(assets['assets/img/a.png']).toBe(toB64(files['/artifacts/export/assets/img/a.png']))
})
it('ignores non-asset files and missing integrity index', async () => {
const files: Record<string, Uint8Array> = {
'/artifacts/export/index.html': new TextEncoder().encode('<!doctype html>'),
'/artifacts/export/assets/img/a.png': new Uint8Array([9,9]),
}
const listDir = async (dir: string) => Object.keys(files).filter((p) => p.startsWith(dir))
const readFile = async (p: string) => files[p]!
const { extras, assets } = await generateVerifyInputs('/artifacts/export', listDir, readFile)
expect(Object.keys(extras)).toEqual([])
expect(Object.keys(assets)).toEqual(['assets/img/a.png'])
})
})
+47
View File
@@ -0,0 +1,47 @@
export type ListDir = (dir: string) => Promise<string[]>
export type ReadFile = (path: string) => Promise<Uint8Array>
function toBase64(u8: Uint8Array): string {
if (typeof Buffer !== 'undefined') return Buffer.from(u8).toString('base64')
throw new Error('Buffer not available')
}
// Generate inputs for verify CLI from a folder tree.
// - root: export folder (contains assets.integrity.index.json and assets/*)
// - listDir: returns list of file paths under root (can be recursive listing)
// - readFile: reads a file into bytes
export async function generateVerifyInputs(
root: string,
listDir: ListDir,
readFile: ReadFile
): Promise<{ extras: Record<string, string>; assets: Record<string, string> }> {
const all = await listDir(root)
const norm = (p: string) => p.replace(/\\/g, '/')
const rootNorm = norm(root).replace(/\/$/, '')
const rel = (p: string) => {
const pn = norm(p)
return pn.startsWith(rootNorm + '/') ? pn.slice(rootNorm.length + 1) : pn
}
const extras: Record<string, string> = {}
const assets: Record<string, string> = {}
// assets.integrity.index.json at root
const integrityPath = all.find((p) => rel(p) === 'assets.integrity.index.json')
if (integrityPath) {
const bytes = await readFile(integrityPath)
extras['assets.integrity.index.json'] = toBase64(bytes)
}
// collect assets/**/*
for (const abs of all) {
const r = rel(abs)
if (!r.startsWith('assets/')) continue
// skip directories: best-effort by checking trailing slash
if (r.endsWith('/')) continue
const bytes = await readFile(abs)
assets[r] = toBase64(bytes)
}
return { extras, assets }
}
+3 -1
View File
@@ -10,7 +10,9 @@
"test": "vitest run",
"test:watch": "vitest -w",
"coverage": "vitest run --coverage",
"test:e2e": "playwright test"
"test:e2e": "playwright test",
"verify:make-json": "node scripts/make-verify-json.js --root ./artifacts/export --extras ./artifacts/extras.json --assets ./artifacts/assets.json",
"verify:run": "node scripts/verify.js --extras ./artifacts/extras.json --assets ./artifacts/assets.json"
},
"dependencies": {
"@dnd-kit/core": "^6.3.1",
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env node
/*
Generate verify inputs from an export folder
Usage: node scripts/make-verify-json.js --root <export_folder> --extras <out_extras.json> --assets <out_assets.json>
*/
const fs = require('fs');
const fsp = require('fs/promises');
const path = require('path');
function parseArgs(argv) {
const out = { root: '', extras: '', assets: '' };
for (let i = 2; i < argv.length; i++) {
const a = argv[i];
if (a === '--root') out.root = argv[++i];
else if (a === '--extras') out.extras = argv[++i];
else if (a === '--assets') out.assets = argv[++i];
}
return out;
}
function toB64(buf) { return Buffer.from(buf).toString('base64'); }
async function readFileB64(p) { const b = await fsp.readFile(p); return toB64(b); }
async function walk(dir) {
const out = [];
const entries = await fsp.readdir(dir, { withFileTypes: true });
for (const ent of entries) {
const abs = path.join(dir, ent.name);
if (ent.isDirectory()) {
const sub = await walk(abs);
out.push(...sub);
} else if (ent.isFile()) {
out.push(abs);
}
}
return out;
}
(async () => {
try {
const { root, extras, assets } = parseArgs(process.argv);
if (!root || !extras || !assets) {
console.log('Usage: node scripts/make-verify-json.js --root <export_folder> --extras <out_extras.json> --assets <out_assets.json>');
process.exit(2);
}
const rootAbs = path.resolve(process.cwd(), root);
const files = await walk(rootAbs);
const norm = (p) => p.split(path.sep).join('/');
const rootNorm = norm(rootAbs).replace(/\/$/, '');
const rel = (p) => {
const pn = norm(p);
return pn.startsWith(rootNorm + '/') ? pn.slice(rootNorm.length + 1) : pn;
};
const extrasMap = {};
const assetsMap = {};
// integrity index
const idx = files.find((p) => rel(p) === 'assets.integrity.index.json');
if (idx) {
extrasMap['assets.integrity.index.json'] = await readFileB64(idx);
}
// assets/**/*
for (const abs of files) {
const r = rel(abs);
if (!r.startsWith('assets/')) continue;
const b64 = await readFileB64(abs);
assetsMap[r] = b64;
}
await fsp.mkdir(path.dirname(path.resolve(process.cwd(), extras)), { recursive: true });
await fsp.mkdir(path.dirname(path.resolve(process.cwd(), assets)), { recursive: true });
await fsp.writeFile(extras, JSON.stringify(extrasMap));
await fsp.writeFile(assets, JSON.stringify(assetsMap));
console.log(`Wrote ${extras} and ${assets}`);
} catch (e) {
console.error(e && e.stack || String(e));
process.exit(1);
}
})();
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env node
/*
Verify bundle integrity using extras/assets JSON maps.
Usage: node scripts/verify.js --extras <extras.json> --assets <assets.json>
*/
const fs = require('fs/promises');
const crypto = require('crypto');
function parseArgs(argv) {
const out = { extras: '', assets: '' };
for (let i = 2; i < argv.length; i++) {
const a = argv[i];
if (a === '--extras') out.extras = argv[++i];
else if (a === '--assets') out.assets = argv[++i];
}
return out;
}
function fromB64(b64) { return Buffer.from(b64, 'base64'); }
function sha256b64(buf) { return crypto.createHash('sha256').update(buf).digest('base64'); }
(async () => {
try {
const { extras, assets } = parseArgs(process.argv);
if (!extras || !assets) {
console.log('Usage: node scripts/verify.js --extras <extras.json> --assets <assets.json>');
process.exit(2);
}
const extrasJson = await fs.readFile(extras, 'utf-8');
const assetsJson = await fs.readFile(assets, 'utf-8');
const extrasMap = JSON.parse(extrasJson);
const assetsMap = JSON.parse(assetsJson);
const result = { ok: true, errors: [], summary: { total: 0, missing: 0, integrityMismatches: 0, sizeMismatches: 0, bundleHashMismatch: false } };
if (!Object.prototype.hasOwnProperty.call(extrasMap, 'assets.integrity.index.json')) {
result.ok = false; result.errors.push('Missing assets.integrity.index.json in extras');
console.log(JSON.stringify(result, null, 2)); process.exit(1);
}
let parsed;
try {
parsed = JSON.parse(fromB64(extrasMap['assets.integrity.index.json']).toString('utf-8'));
} catch {
result.ok = false; result.errors.push('Invalid assets.integrity.index.json JSON');
console.log(JSON.stringify(result, null, 2)); process.exit(1);
}
const recomputedLines = [];
let missing = 0, integrityMismatches = 0, sizeMismatches = 0;
for (const e of parsed.entries) {
const b64 = assetsMap[e.path];
if (!b64) { result.errors.push(`Missing asset for path in integrity index: ${e.path}`); missing++; continue; }
const buf = fromB64(b64);
if (typeof e.size === 'number' && e.size !== buf.length) {
result.errors.push(`Size mismatch for ${e.path}: expected ${e.size}, got ${buf.length}`); sizeMismatches++; }
const integ = 'sha256-' + sha256b64(buf);
if (integ !== e.integrity) { result.errors.push(`Integrity mismatch for ${e.path}`); integrityMismatches++; }
recomputedLines.push(e.path); recomputedLines.push(integ);
}
const bundle = 'sha256-' + sha256b64(Buffer.from(recomputedLines.join('\n'), 'utf-8'));
const bundleHashMismatch = bundle !== parsed.bundleHash;
if (bundleHashMismatch) result.errors.push('BundleHash mismatch');
result.summary.total = parsed.entries.length;
result.summary.missing = missing;
result.summary.integrityMismatches = integrityMismatches;
result.summary.sizeMismatches = sizeMismatches;
result.summary.bundleHashMismatch = bundleHashMismatch;
result.ok = result.errors.length === 0;
console.log(JSON.stringify(result, null, 2));
process.exit(result.ok ? 0 : 1);
} catch (e) {
console.error(e && e.stack || String(e));
process.exit(1);
}
})();