Skip to content

Add Tzafon Northstar computer use template#134

Open
mttias wants to merge 2 commits intokernel:mainfrom
mttias:tzafon-computer-use
Open

Add Tzafon Northstar computer use template#134
mttias wants to merge 2 commits intokernel:mainfrom
mttias:tzafon-computer-use

Conversation

@mttias
Copy link

@mttias mttias commented Mar 13, 2026

Adds TypeScript and Python templates for Tzafon's Northstar CUA Fast model using Kernel Computer Controls API. Tested end-to-end on Kernel.


Note

Low Risk
Low risk: changes are additive (new template directories and template registry entries) with minimal impact on existing logic beyond listing/sorting supported templates.

Overview
Adds a new tzafon-computer-use template option to pkg/create/templates.go, including display metadata, ordering priority among computer-use templates, and deploy/invoke samples for both TypeScript and Python.

Introduces new TypeScript and Python template projects under pkg/templates/*/tzafon-computer-use that run a Tzafon Lightcone responses-based computer-use loop wired to Kernel Computer Controls, with optional replay recording, env/key scaffolding, and minimal project configs (Python pyproject.toml; TypeScript package.json/pnpm-lock.yaml/tsconfig.json).

Written by Cursor Bugbot for commit 1aeabe5. This will update automatically on new commits. Configure here.

@mttias mttias force-pushed the tzafon-computer-use branch from e9427c2 to f599a30 Compare March 13, 2026 21:10
Add new CLI templates for Tzafon's Northstar CUA Fast model, enabling
users to scaffold browser automation projects using Kernel's infrastructure.

New templates:
- TypeScript: `kernel create --template ts-tzafon-cua`
- Python: `kernel create --template python-tzafon-cua`

Both templates include:
- Agentic sampling loop using Tzafon's Lightcone SDK responses API
- Computer tool mapping Northstar actions (click, type, scroll, drag,
  key, navigate, wait) to Kernel's Computer Controls API
- Session management with optional replay recording
- 1280x800 default viewport

Files changed:
- pkg/templates/typescript/tzafon-computer-use/ - TypeScript template
- pkg/templates/python/tzafon-computer-use/ - Python template
- pkg/create/templates.go - Template registration
@mttias mttias force-pushed the tzafon-computer-use branch from f599a30 to 00b5338 Compare March 13, 2026 21:11
@dprevoznik dprevoznik requested review from Sayan- and dprevoznik March 17, 2026 04:39

case 'drag':
await this.kernel.browsers.computer.dragMouse(this.sessionId, {
path: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dp-cursor-review-analysis

Drag actions from the Tzafon model are incomplete. The model emits {"type":"drag","x":470,"y":344} with only a start point — no end_x/end_y or x2/y2. This causes dragMouse to receive NaN for the destination, so the drag silently does nothing. This is a Tzafon model-side limitation; the template code handles drag correctly when both start and end coordinates are provided.

self.kernel.browsers.playwright.execute(
self.session_id, code=f"await page.goto({json.dumps(action.url)})",
)
elif t == "drag":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dp-cursor-review-analysis

Same drag issue as TypeScript. The Tzafon model emits drag actions with only a start point ({"type":"drag","x":470,"y":344}) and no end coordinates. The getattr fallbacks resolve to 0, so every drag ends up dragging to (0, 0). This is a Tzafon model-side limitation.

@mttias mttias force-pushed the tzafon-computer-use branch from 7ae85f2 to 1aeabe5 Compare March 17, 2026 08:54
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Copy link
Contributor

@Sayan- Sayan- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of small q's. overall looks good!

requires-python = ">=3.11"
dependencies = [
"kernel>=0.35.0",
"tzafon",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any version pinning required for tzafon?

elif t == "wait":
await asyncio.sleep(2)
else:
raise ToolError(f"Unknown action type: {t}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the model ever send right_click as its own action type? the README lists it but there's no case for it here -- only click with button="right" on line 80.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants