Sylab: A unified backbone for robotics, ML, and agentic work.
A multi-machine lab platform that gives every capability the same composable vocabulary: model inference, robotics simulation, media generation, autonomous agents, hardware control. Designed, written, and operated single-handedly, in active expansion.
The premise is architectural: the durable edge in this field is not any single model or tool, but the orchestration layer that makes them composable. sylab gives every capability the same five primitives: skill, cell, timeline, monitor, resource. They recurse freely. One vocabulary at every scope, observable end-to-end, version-controlled as plain directories.
Drop in a new domain by writing one skill contract: the canvas, monitors, vault, lifecycle, and audit log all come for free. The architecture is the product, and individual tools get swapped over time.
Five primitives · recursive composition · one vocabulary at every scope
Living Canvas · seven-cell character-design pipeline, all stages observableThe home cluster: orchestrator (Strix Halo + 3090 eGPU) and compute node (DGX Spark), 10 GbE direct-link · mobile-workstation footprint · <650 W with all GPUs crunching · quiet, low heat
Pipelines are visible: cells dropped on a canvas, ports wired between them, every cell publishing live observables: image strips, log tails, embedded mesh viewers, agent transcripts. Today the system runs deterministic and agentic skills across research, media, robotics, code-assistant agents, and benchmarking, driven by execution backends spanning local LLM endpoints, ComfyUI, Isaac Sim, Blender Cycles, AniGen, Playwright-driven coding agents, and arbitrary HTTP services.
Cluster topology · orchestrator dispatches over a 10 GbE bus to a compute node
The same vocabulary covers robotics & ML infrastructure: a simulator-agnostic primitive runs Isaac Sim showcases on the GPU node and MuJoCo on the CPU host through a single contract, so Spot, H1, custom platforms, and classical control benchmarks all share the same lifecycle. Future Drake, or Genesis backends slot in by registering one async function. Long-lived services have audited lifecycle endpoints, and cluster topology is declared, not imperative.
robotics.sim · H1 humanoid · MuJoCo / Isaac Sim · single contract
media.rig · full studio monitor · AniGen auto-rig from generated mesh
It also drives a full media & 3D pipeline: concept image → background isolate → multi-view character sheet → PBR mesh (Hunyuan3D) → AniGen rig → Blender Cycles render-QA, one canvas with one vocabulary, fully observable per stage. Each step is a skill cell with a typed contract, and intermediate artifacts are content-addressed in a local vault and linkable across runs. The same primitive set powers robotics-asset authoring, character pipelines, and product-style renders.
Layered on top are workflows: a multi-step deep-research agent with sub-agent delegation, threading query expansion, broad search, dedup-extract, synthesis, a human-gated checkpoint, and vault compile into one composition. Coding agents run inside containers with persistent workspaces and mid-run chat steering. Every action is an event, and every artifact is content-addressed in the same vault.
composite.deep-research · six cells, prose previews on every face
media.asset · 6-DOF arm · multi-view PBR mesh QA
media.concept · 6-DOF arm asset, fresh from Flux Kontext + Hunyuan3D
A substrate, not a single tool. The same canvas, the same vocabulary, hosts work across four directions in active development:
Model benchmarking & training: throughput, quality, vision-QA, coding, long-context, and article batteries, reinforcement-learning trajectory wrappers, memory layers.
The whole thing is built solo, every primitive, every runner, every monitor view. Active alpha-client engagements, with ongoing expansion across robotics policy training, in-browser interactive demos, and on-device deployment.
case 02 · 2019 · franka × aescape
Aescape: robotic massage, taught by hand.
A control system that lets a human expert teach a massage to a robot: record the technique, modify it, and replay it on a real person with the right forces in the right places, every time.
Aescape, a Franka partner now operating its own commercial robotic-massage product, wanted to capture the technique of an expert massage professional and give it back through a robot. Not a recorded path, not an animation. The actual technique: motion, depth, pressure, rhythm, reproducible across bodies. Lead engineer on the control system, built end-to-end during the engagement.
LfD control loop · taught trajectory in · compliant + bounded execution out
At the core sits a learning-from-demonstration pipeline tuned for compliant, force-regulated contact across a human body. The trajectory representation keeps motion and force jointly editable, so speed, depth, region, and repetitions are all addressable post-capture. Replay runs under compliant control with task-frame safety envelopes. Force-regulated robot contact directly on a person is a distillation of pHRI work: safety, compliance, intent transfer, repeatability.
Control-system prototyping was carried out on-site at Franka's R&D offices in Munich. Integration work then continued in New York with the Aescape team, plugging the controller into the wider station software (perception, orchestration, and the surrounding production stack). Feedback from the Aescape engineers, "control system is buttery smooth", reflected how cleanly the compliant motion carried through the full integrated system. Aescape went on to build a commercial product around this category.
case 03 · 2018 · 2020 · franka emika
Dual-arm teaching & compliant control.
A bimanual control system that extends Franka's single-arm kinesthetic teaching into a richer family of two-arm tasks, with the user's task and shared payload at the coordinated mode.
Bimanual research platform · Franka-built hardware in partner-university research source: arxiv.org/html/2604.08726v1
Franka's single arm was widely known for kinesthetic teaching that felt direct and physical: pull the arm, record, replay. The dual-arm work began from that intuition and asked what the natural extension to two arms ought to feel like, so an operator could teach a richer set of bimanual tasks with the same directness, rather than puppeting two arms in parallel.
The architecture supports a family of teaching and replay modes spanning the spectrum from fully independent per-arm operation to tightly coordinated bimanual behaviour, with selectable coupling at both position and torque level. The user picks the right mix for the task at hand, and the same teaching idiom carries across.
Technical direction was selected after a wide review of the literature, with a strong physical foundation: compliance and motion compose predictably under contact and switching, and the system stays inside the platform's capability envelope rather than degrading at the edge. Limits, reach, and self-interaction are first-class in the model, not afterthoughts.
Validation ran on three tracks: analytical work on the underlying control laws, simulation, and a test harness designed to exercise teaching and automatic-run scenarios at scale. Repeatable, observable behaviour across many task configurations was the engineering target throughout.
I led the dual-arm control system development, prototyping the early version alone before the work grew across a team as the foundations stabilised. Variants of the system found their way, in one form or another, into a series of bimanual Franka-built research platforms delivered to partners over the years (a small selection lives in case 04).
case 04 · 2014 · 2020 · franka emika
Inside the Panda.
The compliant arm at Franka Emika (presented as Franka Emika at the Hannover Messe 2016 launch, branded the Panda shortly after, and now the FR3 at Franka Robotics). Work across compliant control, pHRI strategies, simulation and testing, and the Service Robotics R&D program that branched off from the platform.
Hannover Messe 2016 · two-arm DJ act on the Franka Emika booth, prepared as the show's main attractor footage: youtube.com/watch?v=iQH0NLUd4nU
On the control side, the platform was an exercise in compliant manipulation: a force-sensitive, torque-controlled arm whose defining feature was direct, physical kinesthetic teaching. The work spanned the control architecture itself and the pHRI strategies and algorithms layered on top, working out what makes a force-sensitive arm trustworthy around people and what makes teaching feel direct rather than mediated.
Simulation models and test harnesses that let new control variants be exercised quickly across many scenarios, and validation work to confirm the system held up at the limits of what the hardware could do.
In 2016 I led the Service Robotics R&D program inside Franka Emika. The goal was to take the platform's compliant lineage and point it at care-assistance use cases: handing, supporting, guiding, and simple object interactions on prototype humanoids built around the same arms, controlled under whole-body objectives that fold base, arms, and torso under one controller. The work seeded several research and partner threads that ran for years afterwards, including the partnership with Aescape. MIRMI / TUM's Garmi geriatric-care prototype (alongside) is one of the Franka-built bimanual research platforms used in academic work over those years.
Garmi · MIRMI / TUM · care-robot prototype on Franka-derived bimanual arms source: die-neue-sammlung.de
Automated covid-19 swab-testing booth · Panda-arm prototype, rapid-built in the early weeks of the 2020 pandemic footage: youtube.com/watch?v=Mreb3Xv-Eo0
In the very first weeks of the 2020 pandemic, the Franka team rapid-prototyped an automated covid-19 swab-testing booth on Panda arms: a feasibility demonstration that compliant arms could safely perform a clinical-grade nasopharyngeal swab through a glass partition, with the patient on one side and the robot on the other. A dedicated team picked the concept up afterwards and developed it into a full solution to a real public-health problem at scale.
Before leaving Franka Emika in 2020, I rejoined the control team for a parting stretch, passing on what I'd picked up across years of R&D and helping where I could on the next round of the platform's evolution. The arm continues today as the FR3 at Franka Robotics.
