diff --git a/docs/next-steps.md b/docs/next-steps.md index 9060f57..f9984a8 100644 --- a/docs/next-steps.md +++ b/docs/next-steps.md @@ -19,3 +19,23 @@ Focused follow-up work for `@knighted/develop`. - Detect transient CDN/module loading failures and surface a clear recovery action in-app. - Add a user-triggered retry path (for example, Reload page / Force reload) when runtime bootstrap imports fail. - Consider an optional automatic one-time retry before showing recovery controls, while avoiding infinite reload loops. + +5. **Type reference parsing hardening (TS preprocessor-first)** + - Transition declaration/reference discovery in in-browser type diagnostics to a TypeScript preprocessor-first flow (`ts.preProcessFile`) instead of regex-driven parsing. + - Scope this to the lazy React type environment loader first, then evaluate whether the same parser path should be reused for all type package graph walking. + - Keep current lazy-loading behavior intact: no React type graph fetch until the user switches to React render mode and triggers Typecheck. + - Preserve CDN provider fallback behavior and existing diagnostics UX while changing parser internals. + - Add a strict fallback contract: + - Primary: `preProcessFile` outputs (`importedFiles`, `referencedFiles`, `typeReferenceDirectives`). + - Secondary fallback only when unavailable: current lightweight parsing logic. + - Never treat commented example code as imports/references. + - Add guardrails around known failure classes discovered during development: + - Relative declaration references like `global.d.ts` must resolve as file paths, not package names. + - Extensionless declaration references (for example `./user-context`) must attempt `.d.ts` candidates first. + - Avoid noisy parallel fetch fan-out for bad candidates; use ordered fallback to reduce 404/CORS console noise. + - Add focused test coverage (unit or Playwright) that proves: + - React-mode typecheck does not trigger fake fetches from commented examples in declaration files. + - React-mode typecheck resolves `react` and `react-dom/client` without module-not-found diagnostics. + - DOM mode still avoids React type graph hydration. + - Suggested implementation prompt: + - "Refactor `src/modules/type-diagnostics.js` to make TypeScript preprocessor parsing (`preProcessFile`) the source of truth for declaration graph discovery in the lazy React type loader. Keep current CDN fallback and lazy hydration semantics. Ensure references from comments are ignored, `*.d.ts`/relative path handling is correct, and candidate fetch ordering minimizes noisy failed requests. Add regression coverage for `global.d.ts` and commented `./user-context` examples. Validate with `npm run lint`, `npm run build:esm`, and targeted React/typecheck Playwright runs." diff --git a/docs/type-checking.md b/docs/type-checking.md new file mode 100644 index 0000000..5d6bc53 --- /dev/null +++ b/docs/type-checking.md @@ -0,0 +1,190 @@ +# Type Checking In The Browser + +This document explains how `@knighted/develop` performs TypeScript diagnostics directly in the browser, including the general flow for all render modes and the React-mode-specific type graph loading path. + +## Goals + +- Provide on-demand TypeScript diagnostics without a local build step. +- Keep render/preview UX responsive while type checks run. +- Support a generic baseline in DOM mode. +- Support a realistic React typing environment in React mode. +- Preserve CDN fallback behavior so diagnostics can still run when one provider fails. + +## High-Level Architecture + +Browser type checking is implemented by combining three pieces: + +1. TypeScript compiler runtime loaded from CDN. +2. Virtual filesystem assembled in memory from source + declaration files. +3. Custom TypeScript host and module resolution bridge that reads from the virtual filesystem. + +At runtime this is managed by `createTypeDiagnosticsController` in `src/modules/type-diagnostics.js` and wired from `src/app.js`. + +## Generic Typecheck Flow (All Modes) + +When the user clicks Typecheck: + +1. Ensure TypeScript compiler runtime is loaded from CDN. +2. Build (or reuse) TypeScript standard library declarations (`lib.*.d.ts`). +3. Read current editor source (`component.tsx`). +4. Build a virtual file map for the current run. +5. Create a TypeScript `Program` with a custom host. +6. Collect diagnostics and display formatted output in the diagnostics UI. + +### Compiler Loading + +- TypeScript runtime is loaded using `importFromCdnWithFallback`. +- The selected provider is remembered for downstream declaration URL generation. + +### Standard Library Hydration + +- `getTypeScriptLibUrls(...)` provides provider-prioritized URLs for TS lib declarations. +- Triple-slash `reference lib` and `reference path` directives are followed recursively. +- Loaded files are cached in memory so repeated checks do not re-fetch. + +### Program Options (Generic) + +- `jsx: Preserve` +- `target: ES2022` +- `module: ESNext` +- `moduleResolution: Bundler` (fallback NodeNext/NodeJs) +- `strict: true` +- `noEmit: true` +- `skipLibCheck: true` +- `types: []` to disable implicit ambient type package scanning in this virtual environment + +The explicit `types: []` avoids TypeScript attempting implicit `@types/*` discovery that does not map to a real disk `node_modules` in browser. + +## DOM Mode Behavior + +DOM mode uses a lightweight ambient JSX definition that is injected into the virtual filesystem as a synthetic declaration file. + +This keeps the baseline path minimal and avoids loading React type packages when they are not needed. + +## React Mode Behavior: Lazy CDN Type Hydration + +React mode enables an additional lazy type graph loader: + +- Trigger condition: render mode is `react` and Typecheck is run. +- Root packages: `@types/react` and `@types/react-dom`. +- Transitive dependencies are discovered and loaded on demand. +- Everything is cached after first load. + +### CDN Type Package URL Strategy + +`getTypePackageFileUrls(...)` generates candidate URLs for type package files with a fallback order that favors raw package CDNs before esm-hosted variants. + +Current priority for type package files: + +1. jsDelivr +2. unpkg +3. active TypeScript provider (if present) +4. esm.sh + +This ordering reduces issues from transformed declaration content. + +### Declaration Graph Discovery + +For each loaded declaration file: + +1. Parse references with `ts.preProcessFile` when available. +2. Fallback to a minimal regex parser only if preprocessor is unavailable. +3. Follow imports/references/type directives recursively. + +Guardrails: + +- Relative declaration references are treated as paths. +- Extensionless references try `.d.ts` candidates first. +- Absolute URL specifiers are ignored. +- Commented example imports are not treated as real dependencies. + +### Candidate File Resolution + +When a declaration path is ambiguous, candidates are tried in ordered fallback: + +1. `.d.ts` +2. script-extension-normalized `.d.ts` +3. `/index.d.ts` +4. raw `` + +This reduces noisy failed requests and improves compatibility with DefinitelyTyped layouts. + +## Virtual Filesystem Design + +The virtual filesystem is a `Map` where keys are normalized virtual paths. + +Typical entries include: + +- `component.tsx` +- `lib.esnext.full.d.ts` and referenced TS lib files +- `knighted-jsx-runtime.d.ts` (DOM mode only) +- `node_modules/@types/react/...` +- `node_modules/@types/react-dom/...` +- transitive type deps like `node_modules/csstype/...` + +The loader maintains: + +- loaded file content cache +- package manifest cache +- package entrypoint cache +- in-flight promise dedupe for concurrent requests + +## TypeScript Host + Resolver Bridge + +A custom host is supplied to TypeScript `createProgram(...)` and reads from the virtual map: + +- `fileExists` +- `readFile` +- `directoryExists` +- `getDirectories` +- `getSourceFile` +- `resolveModuleNames` + +Resolver strategy: + +1. Ask TypeScript `resolveModuleName(...)` first. +2. If unresolved and React type graph is active, resolve via virtual `node_modules` candidates. + +This allows TypeScript diagnostics to behave like a project-backed environment while operating purely in browser memory. + +## Diagnostics UI Integration + +- Typecheck state is surfaced via loading/neutral/ok/error states. +- Results are formatted with line/column when available. +- Existing render status is preserved and adjusted when type errors are present. +- Re-check scheduling is supported when unresolved type errors already exist. + +## Known Constraints + +- This is intentionally diagnostics-only (`noEmit`). +- Type package compatibility still depends on CDN availability. +- Browser security and CDN headers may surface noisy network failures on provider fallback paths. +- Complex package resolution edge cases may still require targeted guardrails. + +## Why This Approach + +Compared to a server-side typecheck service, this approach keeps feedback local to the browser session and aligns with the CDN-first architecture of `@knighted/develop`. + +Compared to a purely regex-driven declaration walker, TypeScript preprocessor parsing gives a more robust dependency graph with fewer false positives. + +## Validation And Regression Coverage + +Recent changes are protected with Playwright coverage that checks: + +- React-mode Typecheck succeeds. +- Expected `@types/react` loading occurs. +- Malformed type fetch URL patterns do not occur. + +Recommended local validation when changing this system: + +```bash +npm run lint +npm run build:esm +npm run test:e2e -- --grep "react mode typecheck" +``` + +## Future Improvements + +- Add explicit lazy-loading assertions (no `@types/*` requests before first React-mode Typecheck). +- Expand diagnostics UI with jump-to-line navigation and richer context. +- Consider optional user-configurable extra type roots after baseline stability is proven. diff --git a/playwright/app.spec.ts b/playwright/app.spec.ts index 700dc3d..339c95e 100644 --- a/playwright/app.spec.ts +++ b/playwright/app.spec.ts @@ -334,6 +334,72 @@ test('transpiles TypeScript annotations in component source', async ({ page }) = await expect(page.locator('#preview-host button')).toContainText('typed') }) +test('react mode typecheck loads types without malformed URL fetches', async ({ + page, +}) => { + await waitForInitialRender(page) + + await ensurePanelToolsVisible(page, 'component') + + const typeRequestUrls: string[] = [] + page.on('request', request => { + const url = request.url() + if (url.includes('@types/')) { + typeRequestUrls.push(url) + } + }) + + await page.locator('#render-mode').selectOption('react') + await page.getByRole('button', { name: 'Typecheck' }).click() + + await page.locator('#diagnostics-toggle').click() + await expect(page.locator('#diagnostics-component')).toContainText( + 'No TypeScript errors found.', + ) + + const diagnosticsText = await page.locator('#diagnostics-component').innerText() + expect(diagnosticsText).not.toContain("Cannot find type definition file for 'react'") + expect(diagnosticsText).not.toContain( + "Cannot find type definition file for 'react-dom'", + ) + + expect(typeRequestUrls.some(url => url.includes('@types/react'))).toBeTruthy() + + const malformedTypeRequestPatterns = [ + '/@types/global.d.ts/package.json', + '/user-context', + '/https:/', + ] + + for (const pattern of malformedTypeRequestPatterns) { + expect(typeRequestUrls.some(url => url.includes(pattern))).toBeFalsy() + } +}) + +test('react mode executes default React import without TDZ runtime failure', async ({ + page, +}) => { + await waitForInitialRender(page) + + await ensurePanelToolsVisible(page, 'component') + + await page.getByLabel('ShadowRoot (open)').uncheck() + await page.locator('#render-mode').selectOption('react') + await setComponentEditorSource( + page, + [ + "import React from 'react'", + 'const App = () => ', + ].join('\n'), + ) + + await expect(page.locator('#status')).toHaveText('Rendered') + await expect(page.locator('#preview-host button')).toContainText( + 'react default import works', + ) + await expect(page.locator('#preview-host pre')).toHaveCount(0) +}) + test('clearing component source reports clear action without error status', async ({ page, }) => { diff --git a/src/app.js b/src/app.js index 181713c..9e93209 100644 --- a/src/app.js +++ b/src/app.js @@ -1,5 +1,6 @@ import { cdnImports, + getTypePackageFileUrls, getTypeScriptLibUrls, importFromCdnWithFallback, } from './modules/cdn.js' @@ -391,7 +392,9 @@ const typeDiagnostics = createTypeDiagnosticsController({ cdnImports, importFromCdnWithFallback, getTypeScriptLibUrls, + getTypePackageFileUrls, getJsxSource: () => getJsxSource(), + getRenderMode: () => renderMode.value, setTypecheckButtonLoading, setTypeDiagnosticsDetails, setStatus, diff --git a/src/bootstrap.js b/src/bootstrap.js index f7c6032..0f7e3ae 100644 --- a/src/bootstrap.js +++ b/src/bootstrap.js @@ -9,7 +9,7 @@ import { getPrimaryCdnImportUrls } from './modules/cdn.js' const preloadImportKeys = [ 'cssBrowser', 'jsxDom', - 'jsxTranspile', + 'jsxTransform', 'jsxReact', 'react', 'reactDomClient', diff --git a/src/modules/cdn.js b/src/modules/cdn.js index 83939bd..efe56ae 100644 --- a/src/modules/cdn.js +++ b/src/modules/cdn.js @@ -54,10 +54,10 @@ export const cdnImportSpecs = { esm: '@knighted/jsx', jspmGa: 'npm:@knighted/jsx', }, - jsxTranspile: { - importMap: '@knighted/jsx/transpile', - esm: '@knighted/jsx/transpile', - jspmGa: 'npm:@knighted/jsx/transpile', + jsxTransform: { + importMap: '@knighted/jsx/transform', + esm: '@knighted/jsx/transform', + jspmGa: 'npm:@knighted/jsx/transform', }, jsxReact: { importMap: '@knighted/jsx/react', @@ -234,6 +234,16 @@ const typeScriptLibBaseByProvider = { jsdelivr: `https://cdn.jsdelivr.net/npm/typescript@${typeScriptVersion}/lib`, } +const typePackageVersionByName = { + '@types/react': '19.2.2', + '@types/react-dom': '19.2.1', + '@types/prop-types': '15.7.15', + '@types/scheduler': '0.26.0', + csstype: '3.1.3', +} + +const getTypePackageVersion = packageName => typePackageVersionByName[packageName] + /* * Keep a reliable fallback order for .d.ts files when the active module provider * does not host TypeScript lib declarations consistently (e.g. import maps/jspmGa). @@ -257,6 +267,47 @@ const getTypeScriptLibProviderPriority = typeScriptProvider => { return [...new Set(ordered)] } +const typePackageBaseByProvider = { + esm: packageName => { + const version = getTypePackageVersion(packageName) + const versionSegment = version ? `@${version}` : '' + return `https://esm.sh/${packageName}${versionSegment}` + }, + unpkg: packageName => { + const version = getTypePackageVersion(packageName) + const versionSegment = version ? `@${version}` : '' + return `https://unpkg.com/${packageName}${versionSegment}` + }, + jsdelivr: packageName => { + const version = getTypePackageVersion(packageName) + const versionSegment = version ? `@${version}` : '' + return `https://cdn.jsdelivr.net/npm/${packageName}${versionSegment}` + }, +} + +export const getTypePackageFileUrls = ( + packageName, + fileName, + { typeScriptProvider } = {}, +) => { + const normalizedFileName = + typeof fileName === 'string' && fileName.length > 0 ? fileName : 'package.json' + const typePackageProviderPriority = [ + 'jsdelivr', + 'unpkg', + ...(typeof typeScriptProvider === 'string' ? [typeScriptProvider] : []), + 'esm', + ] + const providerOrderedBases = [...new Set(typePackageProviderPriority)] + .map(provider => { + const createBase = typePackageBaseByProvider[provider] + return typeof createBase === 'function' ? createBase(packageName) : null + }) + .filter(Boolean) + + return providerOrderedBases.map(baseUrl => `${baseUrl}/${normalizedFileName}`) +} + export const getTypeScriptLibUrls = (fileName, { typeScriptProvider } = {}) => { const providerOrderedBases = getTypeScriptLibProviderPriority(typeScriptProvider) .map(provider => typeScriptLibBaseByProvider[provider]) diff --git a/src/modules/defaults.js b/src/modules/defaults.js index 1becee3..0d187f0 100644 --- a/src/modules/defaults.js +++ b/src/modules/defaults.js @@ -26,10 +26,13 @@ export const defaultJsx = [ ].join('\n') export const defaultReactJsx = [ + "import { useState } from 'react'", + "import type { MouseEvent } from 'react'", + '', 'type CounterButtonProps = {', ' label: string', ' active: boolean', - ' onClick: (event: MouseEvent) => void', + ' onClick: (event: MouseEvent) => void', '}', '', 'const CounterButton = ({ label, active, onClick }: CounterButtonProps) => (', @@ -44,9 +47,8 @@ export const defaultReactJsx = [ ')', '', 'const App = () => {', - ' const { useState } = React', ' const [count, setCount] = useState(0)', - ' const handleClick = (_event: MouseEvent) => {', + ' const handleClick = (_event: MouseEvent) => {', ' setCount(current => current + 1)', ' }', '', diff --git a/src/modules/render-runtime.js b/src/modules/render-runtime.js index a234d41..d666ed4 100644 --- a/src/modules/render-runtime.js +++ b/src/modules/render-runtime.js @@ -43,10 +43,10 @@ export const createRenderRuntimeController = ({ if (coreRuntime) return coreRuntime try { - const [cssBrowser, jsxDom, jsxTranspile] = await Promise.all([ + const [cssBrowser, jsxDom, jsxTransform] = await Promise.all([ importFromCdnWithFallback(cdnImports.cssBrowser), importFromCdnWithFallback(cdnImports.jsxDom), - importFromCdnWithFallback(cdnImports.jsxTranspile), + importFromCdnWithFallback(cdnImports.jsxTransform), ]) if (typeof cssBrowser.module.cssFromSource !== 'function') { @@ -57,16 +57,16 @@ export const createRenderRuntimeController = ({ throw new Error(`jsx export was not found from ${jsxDom.url}`) } - if (typeof jsxTranspile.module.transpileJsxSource !== 'function') { + if (typeof jsxTransform.module.transformJsxSource !== 'function') { throw new Error( - `transpileJsxSource export was not found from ${jsxTranspile.url}`, + `transformJsxSource export was not found from ${jsxTransform.url}`, ) } coreRuntime = { cssFromSource: cssBrowser.module.cssFromSource, jsx: jsxDom.module.jsx, - transpileJsxSource: jsxTranspile.module.transpileJsxSource, + transformJsxSource: jsxTransform.module.transformJsxSource, } return coreRuntime @@ -261,7 +261,133 @@ export const createRenderRuntimeController = ({ return React.cloneElement(value, nextProps) } - const shouldAttemptTranspileFallback = error => error instanceof SyntaxError + const shouldAttemptTranspileFallback = error => { + if (error instanceof SyntaxError) { + return true + } + + if (!(error instanceof Error)) { + return false + } + + return /Unexpected token|Cannot use import statement|Unexpected identifier/.test( + error.message, + ) + } + + const isImportRange = range => + Array.isArray(range) && + range.length === 2 && + Number.isInteger(range[0]) && + Number.isInteger(range[1]) + + const stripImportDeclarations = (code, imports) => { + const ranges = imports + .map(entry => entry?.range) + .filter(isImportRange) + .slice() + .sort((first, second) => second[0] - first[0]) + + let output = code + + for (const [start, end] of ranges) { + if (start < 0 || end < start || end > output.length) { + continue + } + + output = `${output.slice(0, start)}${output.slice(end)}` + } + + return output + } + + const buildRuntimeImportPlan = imports => { + const preamble = [] + const unsupportedSources = new Set() + let requiresReactRuntime = false + let hasReactRuntimeAlias = false + + const ensureReactRuntimeAlias = () => { + if (hasReactRuntimeAlias) { + return '__knightedReactRuntime' + } + + hasReactRuntimeAlias = true + preamble.push('const __knightedReactRuntime = React') + return '__knightedReactRuntime' + } + + for (const entry of imports) { + if (!entry || entry.importKind !== 'value') { + continue + } + + if (entry.source !== 'react') { + unsupportedSources.add(entry.source) + continue + } + + requiresReactRuntime = true + + for (const binding of entry.bindings ?? []) { + if (!binding || binding.isTypeOnly) { + continue + } + + if (binding.kind === 'default' || binding.kind === 'namespace') { + if (binding.local === 'React') { + continue + } + + preamble.push(`const ${binding.local} = React`) + continue + } + + if (binding.kind === 'named') { + if (binding.imported === 'default') { + if (binding.local === 'React') { + continue + } + + preamble.push(`const ${binding.local} = React`) + } else { + if (binding.local === 'React') { + const reactRuntimeAlias = ensureReactRuntimeAlias() + preamble.push(`const React = ${reactRuntimeAlias}.${binding.imported}`) + } else { + preamble.push(`const ${binding.local} = React.${binding.imported}`) + } + } + } + } + } + + return { + preamble, + requiresReactRuntime, + unsupportedSources: [...unsupportedSources], + } + } + + const formatTransformDiagnosticsError = diagnostics => { + const firstDiagnostic = diagnostics?.[0] + + if (!firstDiagnostic) { + return '[jsx] Failed to transform source.' + } + + const lines = [`[jsx] ${firstDiagnostic.message}`] + + if (firstDiagnostic.codeframe) { + lines.push(firstDiagnostic.codeframe) + } + + if (firstDiagnostic.helpMessage) { + lines.push(firstDiagnostic.helpMessage) + } + + return lines.join('\n') + } const createUserModuleFactory = source => new Function( @@ -464,7 +590,8 @@ export const createRenderRuntimeController = ({ } const evaluateUserModule = async (helpers = {}) => { - const { jsx, transpileJsxSource } = await ensureCoreRuntime() + const { jsx, transformJsxSource } = await ensureCoreRuntime() + let runtimeHelpers = helpers const userCode = getJsxSource() .replace(/^\s*export\s+default\s+function\b/gm, '__defaultExport = function') .replace(/^\s*export\s+default\s+class\b/gm, '__defaultExport = class') @@ -482,34 +609,97 @@ export const createRenderRuntimeController = ({ const transpileMode = helpers.React && helpers.reactJsx ? 'react' : 'dom' const transpileOptionsByMode = { dom: { - sourceType: 'script', + sourceType: 'module', createElement: 'jsx.createElement', fragment: 'jsx.Fragment', typescript: 'strip', }, react: { - sourceType: 'script', + sourceType: 'module', createElement: 'React.createElement', fragment: 'React.Fragment', typescript: 'strip', }, } - const transpiledUserCode = transpileJsxSource( + const transformedResult = transformJsxSource( userCode, transpileOptionsByMode[transpileMode], - ).code - const moduleFactory = createUserModuleFactory(transpiledUserCode) + ) - if (helpers.React && helpers.reactJsx) { - return moduleFactory(helpers.jsx ?? jsx, helpers.reactJsx, helpers.React) + if (transformedResult.diagnostics.length > 0) { + throw new Error(formatTransformDiagnosticsError(transformedResult.diagnostics), { + cause: error, + }) + } + + const importAnalysisResult = transformJsxSource(transformedResult.code, { + sourceType: 'module', + typescript: 'preserve', + }) + + if (importAnalysisResult.diagnostics.length > 0) { + throw new Error( + formatTransformDiagnosticsError(importAnalysisResult.diagnostics), + { + cause: error, + }, + ) + } + + const runtimeImportPlan = buildRuntimeImportPlan(importAnalysisResult.imports) + + if (runtimeImportPlan.unsupportedSources.length > 0) { + throw new Error( + `Unsupported runtime imports in playground execution: ${runtimeImportPlan.unsupportedSources + .map(specifier => `'${specifier}'`) + .join(', ')}.`, + { + cause: error, + }, + ) + } + + if (runtimeImportPlan.requiresReactRuntime && !runtimeHelpers.React) { + const { React, reactJsx } = await ensureReactRuntime() + runtimeHelpers = { + ...runtimeHelpers, + React, + reactJsx: runtimeHelpers.reactJsx ?? reactJsx, + } + } + + const runtimeCode = stripImportDeclarations( + transformedResult.code, + importAnalysisResult.imports, + ) + const executableUserCode = runtimeImportPlan.preamble.length + ? `${runtimeImportPlan.preamble.join('\n')}\n${runtimeCode}` + : runtimeCode + + const moduleFactory = createUserModuleFactory(executableUserCode) + + if (runtimeHelpers.React && runtimeHelpers.reactJsx) { + return moduleFactory( + runtimeHelpers.jsx ?? jsx, + runtimeHelpers.reactJsx, + runtimeHelpers.React, + ) } if (transpileMode === 'dom') { - return moduleFactory(helpers.jsx ?? jsx, helpers.reactJsx, helpers.React) + return moduleFactory( + runtimeHelpers.jsx ?? jsx, + runtimeHelpers.reactJsx, + runtimeHelpers.React, + ) } const { React, reactJsx } = await ensureReactRuntime() - return moduleFactory(helpers.jsx ?? jsx, helpers.reactJsx ?? reactJsx, React) + return moduleFactory( + runtimeHelpers.jsx ?? jsx, + runtimeHelpers.reactJsx ?? reactJsx, + React, + ) } } diff --git a/src/modules/type-diagnostics.js b/src/modules/type-diagnostics.js index 46f038f..e51ff2c 100644 --- a/src/modules/type-diagnostics.js +++ b/src/modules/type-diagnostics.js @@ -1,10 +1,224 @@ const ignoredTypeDiagnosticCodes = new Set([2318, 6053]) +const reactTypeRootPackages = ['@types/react', '@types/react-dom'] +const typeImportPattern = + /(?:import|export)\s+(?:type\s+)?(?:[^'"\n]*?\s+from\s+)?['"]([^'"\n]+)['"]|import\(['"]([^'"\n]+)['"]\)/g +const typeReferencePathPattern = /\/\/\/\s*/g +const typeReferenceTypesPattern = /\/\/\/\s*/g + +const isTypeDeclarationPathReference = reference => { + if (typeof reference !== 'string') { + return false + } + + return reference.endsWith('.d.ts') || reference.endsWith('.ts') +} + +const isAbsoluteUrlReference = reference => { + if (typeof reference !== 'string') { + return false + } + + return /^(https?:)?\/\//.test(reference) +} + +const domJsxTypes = + 'declare namespace React {\n' + + ' type Key = string | number\n' + + ' interface Attributes { key?: Key | null }\n' + + '}\n' + + 'declare namespace JSX {\n' + + ' type Element = unknown\n' + + ' interface ElementChildrenAttribute { children: unknown }\n' + + ' interface IntrinsicAttributes extends React.Attributes {}\n' + + ' interface IntrinsicElements { [elemName: string]: Record }\n' + + '}\n' + +const normalizeVirtualFileName = fileName => + typeof fileName === 'string' && fileName.startsWith('/') ? fileName.slice(1) : fileName + +const normalizeRelativePath = path => { + const normalized = String(path ?? '') + .replace(/\\/g, '/') + .replace(/^\.\//, '') + const parts = normalized.split('/') + const resolved = [] + + for (const part of parts) { + if (!part || part === '.') { + continue + } + if (part === '..') { + resolved.pop() + continue + } + resolved.push(part) + } + + return resolved.join('/') +} + +const dirname = path => { + const normalized = normalizeRelativePath(path) + const lastSlashIndex = normalized.lastIndexOf('/') + return lastSlashIndex === -1 ? '' : normalized.slice(0, lastSlashIndex) +} + +const joinPath = (...segments) => + normalizeRelativePath(segments.filter(Boolean).join('/')) + +const toDtsPathCandidates = path => { + const normalized = normalizeRelativePath(path) + if (!normalized) { + return [] + } + + if (normalized.endsWith('.d.ts')) { + return [normalized] + } + + const withDtsFromScriptExt = normalized.replace(/\.(c|m)?[jt]sx?$/, '.d.ts') + + return [ + `${normalized}.d.ts`, + withDtsFromScriptExt, + `${normalized}/index.d.ts`, + normalized, + ].filter((candidate, index, all) => candidate && all.indexOf(candidate) === index) +} + +const splitBareSpecifier = specifier => { + if (typeof specifier !== 'string' || specifier.length === 0) { + return null + } + + if (specifier.startsWith('@')) { + const [scope, name, ...rest] = specifier.split('/') + if (!scope || !name) { + return null + } + return { + packageName: `${scope}/${name}`, + subpath: rest.join('/'), + } + } + + const [name, ...rest] = specifier.split('/') + return { + packageName: name, + subpath: rest.join('/'), + } +} + +const toTypePackageName = runtimePackageName => { + if (runtimePackageName === 'csstype') { + return 'csstype' + } + if (runtimePackageName === 'prop-types') { + return '@types/prop-types' + } + if (runtimePackageName === 'scheduler') { + return '@types/scheduler' + } + if (runtimePackageName.startsWith('@types/')) { + return runtimePackageName + } + if (runtimePackageName.startsWith('@')) { + return `@types/${runtimePackageName.slice(1).replace('/', '__')}` + } + return `@types/${runtimePackageName}` +} + +const parseTypeReferencesWithRegexFallback = sourceText => { + const references = new Set() + const sourceWithoutComments = sourceText + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/\/\/.*$/gm, '') + + for (const match of sourceWithoutComments.matchAll(typeImportPattern)) { + const specifier = (match[1] ?? match[2] ?? '').trim() + if (specifier) { + references.add(specifier) + } + } + + for (const match of sourceText.matchAll(typeReferencePathPattern)) { + const path = match[1]?.trim() + if (path) { + references.add(path) + } + } + + for (const match of sourceText.matchAll(typeReferenceTypesPattern)) { + const packageName = match[1]?.trim() + if (packageName) { + references.add(packageName) + } + } + + return [...references] +} + +const parseTypeReferences = (compiler, sourceText) => { + if (typeof compiler.preProcessFile === 'function') { + const references = new Set() + const preProcessed = compiler.preProcessFile(sourceText, true, true) + + for (const importedFile of preProcessed.importedFiles ?? []) { + const fileName = importedFile.fileName?.trim() + if (fileName) { + references.add(fileName) + } + } + + for (const referencedFile of preProcessed.referencedFiles ?? []) { + const fileName = referencedFile.fileName?.trim() + if (fileName) { + references.add(fileName) + } + } + + for (const typeDirective of preProcessed.typeReferenceDirectives ?? []) { + const fileName = typeDirective.fileName?.trim() + if (fileName) { + references.add(fileName) + } + } + + return [...references] + } + + return parseTypeReferencesWithRegexFallback(sourceText) +} + +const parseTypeScriptLibReferences = sourceText => { + const references = new Set() + const libReferencePattern = /\/\/\/\s*/g + const pathReferencePattern = /\/\/\/\s*/g + + for (const match of sourceText.matchAll(libReferencePattern)) { + const libName = match[1]?.trim() + if (libName) { + references.add(`lib.${libName}.d.ts`) + } + } + + for (const match of sourceText.matchAll(pathReferencePattern)) { + const pathName = match[1]?.trim() + if (pathName) { + references.add(pathName.replace(/^\.\//, '')) + } + } + + return [...references] +} export const createTypeDiagnosticsController = ({ cdnImports, importFromCdnWithFallback, getTypeScriptLibUrls, + getTypePackageFileUrls, getJsxSource, + getRenderMode = () => 'dom', defaultTypeScriptLibFileName = 'lib.esnext.full.d.ts', setTypecheckButtonLoading, setTypeDiagnosticsDetails, @@ -20,6 +234,9 @@ export const createTypeDiagnosticsController = ({ let typeScriptCompiler = null let typeScriptCompilerProvider = null let typeScriptLibFiles = null + let reactTypeFiles = null + let reactTypePackageEntries = null + let reactTypeLoadPromise = null let lastTypeErrorCount = 0 let hasUnresolvedTypeErrors = false let scheduledTypeRecheck = null @@ -60,6 +277,35 @@ export const createTypeDiagnosticsController = ({ return `L${position.line + 1}:${position.character + 1} TS${diagnostic.code}: ${message}` } + const fetchTextFromUrls = async (urls, errorPrefix) => { + const attempts = urls.map(async url => { + const response = await fetch(url) + if (!response.ok) { + throw new Error(`HTTP ${response.status} from ${url}`) + } + + return response.text() + }) + + try { + return await Promise.any(attempts) + } catch (error) { + let message = error instanceof Error ? error.message : String(error) + + if (error instanceof AggregateError) { + const reasons = Array.from(error.errors ?? []) + .slice(0, 3) + .map(reason => (reason instanceof Error ? reason.message : String(reason))) + const reasonSummary = reasons.length ? ` Causes: ${reasons.join(' | ')}` : '' + message = `Tried URLs: ${urls.join(', ')}.${reasonSummary}` + } + + throw new Error(`${errorPrefix}: ${message}`, { + cause: error, + }) + } + } + const ensureTypeScriptCompiler = async () => { if (typeScriptCompiler) { return typeScriptCompiler @@ -93,65 +339,11 @@ export const createTypeDiagnosticsController = ({ return ignoredTypeDiagnosticCodes.has(diagnostic.code) } - const normalizeVirtualFileName = fileName => - typeof fileName === 'string' && fileName.startsWith('/') - ? fileName.slice(1) - : fileName - const fetchTypeScriptLibText = async fileName => { const urls = getTypeScriptLibUrls(fileName, { typeScriptProvider: typeScriptCompilerProvider, }) - - const attempts = urls.map(async url => { - const response = await fetch(url) - if (!response.ok) { - throw new Error(`HTTP ${response.status} from ${url}`) - } - - return response.text() - }) - - try { - return await Promise.any(attempts) - } catch (error) { - let message = error instanceof Error ? error.message : String(error) - - if (error instanceof AggregateError) { - const reasons = Array.from(error.errors ?? []) - .slice(0, 3) - .map(reason => (reason instanceof Error ? reason.message : String(reason))) - const reasonSummary = reasons.length ? ` Causes: ${reasons.join(' | ')}` : '' - - message = `Tried URLs: ${urls.join(', ')}.${reasonSummary}` - } - - throw new Error(`Unable to fetch TypeScript lib file ${fileName}: ${message}`, { - cause: error, - }) - } - } - - const parseTypeScriptLibReferences = sourceText => { - const references = new Set() - const libReferencePattern = /\/\/\/\s*/g - const pathReferencePattern = /\/\/\/\s*/g - - for (const match of sourceText.matchAll(libReferencePattern)) { - const libName = match[1]?.trim() - if (libName) { - references.add(`lib.${libName}.d.ts`) - } - } - - for (const match of sourceText.matchAll(pathReferencePattern)) { - const pathName = match[1]?.trim() - if (pathName) { - references.add(pathName.replace(/^\.\//, '')) - } - } - - return [...references] + return fetchTextFromUrls(urls, `Unable to fetch TypeScript lib file ${fileName}`) } const hydrateTypeScriptLibFiles = async (pendingFileNames, loaded) => { @@ -186,40 +378,361 @@ export const createTypeDiagnosticsController = ({ return typeScriptLibFiles } + const getTypePackageManifestUrls = packageName => { + return getTypePackageFileUrls(packageName, 'package.json', { + typeScriptProvider: typeScriptCompilerProvider, + }) + } + + const getTypePackageFileUrlsWithProvider = (packageName, fileName) => { + return getTypePackageFileUrls(packageName, fileName, { + typeScriptProvider: typeScriptCompilerProvider, + }) + } + + const fetchTypePackageDeclaration = async (packageName, requestedFileName) => { + const fileNameCandidates = toDtsPathCandidates(requestedFileName) + + const tryCandidateAt = async (index, firstError = null) => { + if (index >= fileNameCandidates.length) { + throw new Error( + `Unable to fetch type declaration ${packageName}/${requestedFileName}. Tried candidates: ${fileNameCandidates.join(', ')}.${firstError ? ` ${firstError}` : ''}`, + ) + } + + const candidateFileName = fileNameCandidates[index] + + try { + const sourceText = await fetchTextFromUrls( + getTypePackageFileUrlsWithProvider(packageName, candidateFileName), + `Unable to fetch type declaration ${packageName}/${candidateFileName}`, + ) + + return { + fileName: candidateFileName, + sourceText, + } + } catch (error) { + const message = error instanceof Error ? error.message : String(error) + return tryCandidateAt(index + 1, firstError ?? message) + } + } + + return tryCandidateAt(0) + } + + const ensureReactTypeFiles = async compiler => { + if (reactTypeFiles && reactTypePackageEntries) { + return { + files: reactTypeFiles, + packageEntries: reactTypePackageEntries, + } + } + + if (reactTypeLoadPromise) { + return reactTypeLoadPromise + } + + reactTypeLoadPromise = (async () => { + const files = new Map() + const packageEntryByName = new Map() + const packageManifestByName = new Map() + const pending = [] + const visited = new Set() + + const getVirtualTypeFileName = (packageName, packageFileName) => { + return joinPath('node_modules', packageName, packageFileName) + } + + const enqueueTypeFile = (packageName, fileName) => { + for (const candidate of toDtsPathCandidates(fileName)) { + const key = `${packageName}:${candidate}` + if (visited.has(key)) { + continue + } + visited.add(key) + pending.push({ packageName, fileName: candidate }) + } + } + + const ensureTypePackageManifest = async packageName => { + if (packageManifestByName.has(packageName)) { + return packageManifestByName.get(packageName) + } + + const manifestText = await fetchTextFromUrls( + getTypePackageManifestUrls(packageName), + `Unable to fetch type package manifest ${packageName}`, + ) + const manifest = JSON.parse(manifestText) + packageManifestByName.set(packageName, manifest) + + const entry = + typeof manifest.types === 'string' + ? manifest.types + : typeof manifest.typings === 'string' + ? manifest.typings + : 'index.d.ts' + + packageEntryByName.set(packageName, normalizeRelativePath(entry)) + enqueueTypeFile(packageName, entry) + + const dependencies = { + ...(manifest.dependencies ?? {}), + ...(manifest.peerDependencies ?? {}), + } + + await Promise.all( + Object.keys(dependencies).map(dependencyName => { + const dependencyPackageName = toTypePackageName(dependencyName) + return ensureTypePackageManifest(dependencyPackageName) + }), + ) + + return manifest + } + + await Promise.all( + reactTypeRootPackages.map(packageName => ensureTypePackageManifest(packageName)), + ) + + const drainPendingTypeFiles = async () => { + const next = pending.shift() + if (!next) { + return + } + + const { packageName, fileName } = next + const primaryVirtualFileName = getVirtualTypeFileName(packageName, fileName) + + if (files.has(primaryVirtualFileName)) { + await drainPendingTypeFiles() + return + } + + const fetched = await fetchTypePackageDeclaration(packageName, fileName) + const resolvedVirtualFileName = getVirtualTypeFileName( + packageName, + fetched.fileName, + ) + + files.set(primaryVirtualFileName, fetched.sourceText) + if (resolvedVirtualFileName !== primaryVirtualFileName) { + files.set(resolvedVirtualFileName, fetched.sourceText) + } + + const references = parseTypeReferences(compiler, fetched.sourceText) + const pendingPackageManifestLoads = [] + + for (const reference of references) { + if (!reference || reference.startsWith('node:')) { + continue + } + + if (isAbsoluteUrlReference(reference)) { + continue + } + + if (reference.startsWith('.') || isTypeDeclarationPathReference(reference)) { + const relativeBase = dirname(fileName) + enqueueTypeFile(packageName, joinPath(relativeBase, reference)) + continue + } + + const parsedReference = splitBareSpecifier(reference) + if (!parsedReference) { + continue + } + + const targetPackageName = toTypePackageName(parsedReference.packageName) + pendingPackageManifestLoads.push( + ensureTypePackageManifest(targetPackageName).then(() => { + const targetSubpath = normalizeRelativePath(parsedReference.subpath) + if (targetSubpath) { + enqueueTypeFile(targetPackageName, targetSubpath) + return + } + + const targetEntry = packageEntryByName.get(targetPackageName) + if (targetEntry) { + enqueueTypeFile(targetPackageName, targetEntry) + } + }), + ) + } + + if (pendingPackageManifestLoads.length > 0) { + await Promise.all(pendingPackageManifestLoads) + } + + await drainPendingTypeFiles() + } + + await drainPendingTypeFiles() + + reactTypeFiles = files + reactTypePackageEntries = packageEntryByName + reactTypeLoadPromise = null + + return { + files, + packageEntries: packageEntryByName, + } + })() + + try { + return await reactTypeLoadPromise + } catch (error) { + reactTypeLoadPromise = null + throw error + } + } + + const toVirtualTypeFileCandidates = ( + packageEntries, + runtimeSpecifier, + containingFile, + ) => { + if (runtimeSpecifier.startsWith('.')) { + const containingDirectory = dirname(containingFile) + return toDtsPathCandidates(joinPath(containingDirectory, runtimeSpecifier)) + } + + const parsedSpecifier = splitBareSpecifier(runtimeSpecifier) + if (!parsedSpecifier) { + return [] + } + + const packageName = toTypePackageName(parsedSpecifier.packageName) + const subpath = normalizeRelativePath(parsedSpecifier.subpath) + if (!subpath) { + const packageEntry = packageEntries.get(packageName) + if (!packageEntry) { + return [] + } + return [joinPath('node_modules', packageName, packageEntry)] + } + + return toDtsPathCandidates(subpath).map(candidate => + joinPath('node_modules', packageName, candidate), + ) + } + const collectTypeDiagnostics = async (compiler, sourceText) => { const sourceFileName = 'component.tsx' const jsxTypesFileName = 'knighted-jsx-runtime.d.ts' + const renderMode = getRenderMode() + const isReactMode = renderMode === 'react' const libFiles = await ensureTypeScriptLibFiles() - const jsxTypes = - 'declare namespace React {\n' + - ' type Key = string | number\n' + - ' interface Attributes { key?: Key | null }\n' + - '}\n' + - 'declare namespace JSX {\n' + - ' type Element = unknown\n' + - ' interface ElementChildrenAttribute { children: unknown }\n' + - ' interface IntrinsicAttributes extends React.Attributes {}\n' + - ' interface IntrinsicElements { [elemName: string]: Record }\n' + - '}\n' - - const files = new Map([ - [sourceFileName, sourceText], - [jsxTypesFileName, jsxTypes], - ...libFiles.entries(), - ]) + + let reactTypes = null + if (isReactMode) { + reactTypes = await ensureReactTypeFiles(compiler) + } + + const files = new Map([[sourceFileName, sourceText], ...libFiles.entries()]) + + if (!isReactMode) { + files.set(jsxTypesFileName, domJsxTypes) + } + + if (reactTypes) { + for (const [fileName, text] of reactTypes.files.entries()) { + files.set(fileName, text) + } + } const options = { jsx: compiler.JsxEmit?.Preserve, target: compiler.ScriptTarget?.ES2022, module: compiler.ModuleKind?.ESNext, + moduleResolution: + compiler.ModuleResolutionKind?.Bundler ?? + compiler.ModuleResolutionKind?.NodeNext ?? + compiler.ModuleResolutionKind?.NodeJs, + types: [], strict: true, noEmit: true, skipLibCheck: true, } - const host = { + const listVirtualDirectories = targetDirectory => { + const normalizedDirectory = normalizeRelativePath(targetDirectory) + const prefix = normalizedDirectory ? `${normalizedDirectory}/` : '' + const nextDirectories = new Set() + + for (const fileName of files.keys()) { + const normalizedFileName = normalizeRelativePath(fileName) + if (!normalizedFileName.startsWith(prefix)) { + continue + } + + const remainder = normalizedFileName.slice(prefix.length) + const nextSegment = remainder.split('/')[0] + if (nextSegment && remainder.includes('/')) { + nextDirectories.add(nextSegment) + } + } + + return [...nextDirectories] + } + + const moduleResolutionHost = { fileExists: fileName => files.has(normalizeVirtualFileName(fileName)), readFile: fileName => files.get(normalizeVirtualFileName(fileName)), + directoryExists: directoryName => { + const normalized = normalizeRelativePath(normalizeVirtualFileName(directoryName)) + return listVirtualDirectories(normalized).length > 0 + }, + getDirectories: directoryName => { + const normalized = normalizeRelativePath(normalizeVirtualFileName(directoryName)) + return listVirtualDirectories(normalized) + }, + realpath: fileName => normalizeVirtualFileName(fileName), + getCurrentDirectory: () => '/', + } + + const resolveModuleNames = (moduleNames, containingFile) => { + return moduleNames.map(moduleName => { + const resolved = compiler.resolveModuleName( + moduleName, + containingFile, + options, + moduleResolutionHost, + ) + + if (resolved.resolvedModule) { + return resolved.resolvedModule + } + + if (!reactTypes) { + return undefined + } + + const candidates = toVirtualTypeFileCandidates( + reactTypes.packageEntries, + moduleName, + containingFile, + ) + + const matched = candidates.find(candidate => files.has(candidate)) + if (!matched) { + return undefined + } + + return { + resolvedFileName: matched, + extension: compiler.Extension?.Dts ?? '.d.ts', + isExternalLibraryImport: matched.startsWith('node_modules/'), + } + }) + } + + const host = { + fileExists: moduleResolutionHost.fileExists, + readFile: moduleResolutionHost.readFile, + directoryExists: moduleResolutionHost.directoryExists, + getDirectories: moduleResolutionHost.getDirectories, getSourceFile: (fileName, languageVersion) => { const normalizedFileName = normalizeVirtualFileName(fileName) const text = files.get(normalizedFileName) @@ -244,14 +757,28 @@ export const createTypeDiagnosticsController = ({ getDefaultLibFileName: () => defaultTypeScriptLibFileName, writeFile: () => {}, getCurrentDirectory: () => '/', - getDirectories: () => [], getCanonicalFileName: fileName => normalizeVirtualFileName(fileName), useCaseSensitiveFileNames: () => true, getNewLine: () => '\n', + resolveModuleNames, + } + + const rootNames = [sourceFileName] + if (!isReactMode) { + rootNames.push(jsxTypesFileName) + } + if (reactTypes) { + for (const packageName of reactTypeRootPackages) { + const packageEntry = reactTypes.packageEntries.get(packageName) + if (!packageEntry) { + continue + } + rootNames.push(joinPath('node_modules', packageName, packageEntry)) + } } const program = compiler.createProgram({ - rootNames: [sourceFileName, jsxTypesFileName], + rootNames, options, host, })