IDL Walkthrough

An inference service, from interface to deployment.

A streaming inference backend whose contract and calling model can stay stable locally, over TLS, or inside an SGX enclave, while deployment and security composition change around it.

The scenario

You are building a C++ inference backend and you need callers in other processes, other machines, or a browser UI to be able to start an inference run, receive tokens as they are generated, and cancel if needed.

Without a framework you write serialisation on both sides, manage a callback channel, design a wire protocol for streaming output, handle object lifetimes across the connection, and repeat that work for every transport you need. Canopy replaces that with a single IDL file and generated glue on both sides.

  • Token streaming arrives via a callback interface — the server calls back into the client over the same connection, fire-and-forget.
  • A session object is created on the server and returned as an rpc::shared_ptr — the session stays alive exactly as long as the caller holds it.
  • The service factory is the top-level entry point — callers connect to it and ask it to create sessions.
  • The same interface supports blocking and co_await builds through Canopy's bi-modal API.

Step 1 — write the IDL

The IDL describes the callable surface: interfaces, methods, argument direction, and one-way calls. It says nothing about transport, serialisation, or execution model.

// inference/inference.idl // Copyright notice here #import "rpc/rpc_types.idl" namespace inference { // The namespace is part of the generated type name. // Adding v2 later is additive — v1 stays callable. namespace v1 { // Sampling parameters. rpc::optional means the caller may omit the field; // the default value (= …) fills it in. [status=production] struct generation_options { [description="Maximum tokens to generate"] rpc::optional<uint32_t> max_tokens = 512; [description="Sampling temperature 0.0–2.0"] rpc::optional<float> temperature = 0.7; [description="Halt on any matching sequence"] rpc::optional<std::vector<std::string>> stop_sequences; }; [status=production] enum class finish_reason : uint64_t { complete = 0, max_tokens = 1, cancelled = 2, error = 3 }; // Callback: the server delivers tokens into this interface on the client side. // [post] = fire-and-forget — no reply is awaited and no round-trip is added // per token. on_finish is a normal call so the client gets confirmation. [status=production] interface i_token_sink { [post] rpc::result on_token(const std::string& token); rpc::result on_finish(finish_reason reason, uint32_t tokens_generated); }; // A live inference session. Returned as rpc::shared_ptr by the factory. // The server keeps the session alive for exactly as long as the caller holds // its shared_ptr — distributed reference counting, no manual cleanup. [status=production] interface i_session { rpc::result generate( const std::string& prompt, const generation_options& options, // optimistic_ptr: a callable non-owning reference to the client's // token sink. Does not prevent the sink from being released. const rpc::optimistic_ptr<i_token_sink>& sink); rpc::result cancel(); }; // The top-level service factory. Callers connect to this, list available // models, and request a session for the one they want. [status=production] interface i_inference_service { rpc::result list_models([out] std::vector<std::string>& model_ids); rpc::result create_session( const std::string& model_id, [out] rpc::shared_ptr<i_session>& session); }; } }

IDL features used here

  • namespace v1 — the recommended explicit, stable versioned name in every generated type
  • rpc::optional<T> — omissible field with an explicit default
  • [post] — one-way fire-and-forget; no reply channel opened per token
  • rpc::optimistic_ptr<T> — callable non-owning reference (breaks the callback cycle)
  • rpc::shared_ptr<T> — returned session whose lifetime is controlled by the caller
  • [out] — output parameter direction annotation
  • [description=…] — description embedded in the generated JSON schema

What you are not deciding here

  • No transport — TCP, IPC, local, DLL, or SGX is chosen at construction time
  • No serialisation format — YAS binary, JSON, or Protocol Buffers is selected by build and connection configuration
  • No client binding baked into the contract — C++ and supported generated JavaScript/Protocol Buffers paths come from the same IDL; broader runtimes are planned
  • No execution model — the same source compiles blocking or co_await
  • No separate version field — the namespace name is the version; v1 and v2 can coexist
  • No hand-written wire protocol — framing, encoding, dispatch, and generated contract identities are handled by Canopy

The IDL is the type system

The IDL file is not just documentation — it is the single canonical definition from which every downstream artefact is generated. Change a type in the IDL and the C++ proxies, protobuf descriptors, JSON schemas, and JavaScript stubs all update in the next build. Nothing can drift out of sync because nothing is written twice.

// One IDL file. Many generated targets.
inference.idl
single source of truth
├──
C++ proxy + stub — yas_binary, yas_json, protocol_buffers variants
├──
Language-runtime roadmap — Rust rewrite and broader C ABI-based bindings
├──
JavaScript WebSocket client — reduced-trust browser layer, same contract
├──
JSON schema (config profile) — VS Code IntelliSense, validation, hover docs
└──
JSON schema (MCP profile) — minimal tool schema for AI agent use

Version management

An explicit namespace v1 is the recommended versioning mechanism. The qualified name inference::v1::i_session is the version — there is no separate version field to keep in sync with the interface shape, and a later v2 can coexist without silently redirecting existing callers.

At build time, Canopy writes a fingerprint for every interface marked [status=production] into a check_sums/production/ file. If the interface definition changes, the fingerprint changes, and a CI check can reject the build — enforcing that a production contract is never silently modified.

To evolve a production interface, you rename it: bump v1 to v2. The old name stays callable by any code compiled against it; the new name starts a fresh contract. Both can coexist in the same IDL file and the same running service.

Serialisation format as a deployment detail

Each CanopyGenerate target (yas_binary, yas_json, protocol_buffers) is a different transformation of the same IDL types. The caller holds an rpc::shared_ptr<i_session> — it does not know or specify which wire format is in use.

Format is chosen through the build and the service or transport configuration used for newly created proxies. A C++-to-C++ path might use YAS binary for throughput; the generated browser client uses Protocol Buffers for cross-language compatibility, and future runtime bindings can use the same schema. The interface code does not change.

The same IDL attributes that annotate types for C++ also annotate the generated JSON schemas: [description=…] becomes hover documentation in VS Code and method descriptions in MCP tool definitions. Annotations written once appear everywhere they are needed.

A struct field added in the IDL appears in the C++ proxy, the protobuf descriptor, the JSON schema, and the JavaScript client after the next build — consistently and without touching any of those targets by hand. A field removed from the IDL causes a compile error in any code that still references it, in any language that has a generated binding.

Step 2 — what Canopy generates

One CanopyGenerate CMake call produces C++ for each requested serialisation format. The generated headers expose the same virtual interface the IDL described — no hand-written serialisation, dispatch, or transport code on either side.

# CMakeLists.txt CanopyGenerate( inference inference/inference.idl ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_BINARY_DIR}/generated "" yas_binary # high-performance C++ to C++ path yas_json # human-readable debug and schema-oriented path protocol_buffers # browser and cross-language interoperability path include_paths ${CMAKE_CURRENT_SOURCE_DIR}/. install_dir ${GENERATED_INSTALL_DIR})

After generation the caller-side header exposes the pure virtual interface exactly as written in the IDL. The caller holds an rpc::shared_ptr<i_inference_service> and calls it like a local object:

// Caller — the generated interface looks like a local C++ object. // RPC_TASK / CO_AWAIT compile for Canopy's blocking and coroutine builds. RPC_TASK(rpc::result) run_inference() { rpc::shared_ptr<inference::v1::i_inference_service> svc = connect(); // see Step 4 std::vector<std::string> models; if (auto err = CO_AWAIT svc->list_models(models); err != rpc::error::OK()) CO_RETURN err; rpc::shared_ptr<inference::v1::i_session> session; if (auto err = CO_AWAIT svc->create_session(models[0], session); err != rpc::error::OK()) CO_RETURN err; // Register a callback sink. The server will call on_token() on this // object from the other side of the connection. auto sink = rpc::make_shared<my_token_sink>(); inference::v1::generation_options opts; opts.max_tokens = 256; opts.temperature = 0.8f; // When session goes out of scope, the remote object is released automatically. CO_RETURN CO_AWAIT session->generate( "Explain RAII in one paragraph.", opts, sink); }
The caller does not know or care whether the service is in the same process, on another machine, or inside an SGX enclave. That decision is made where the transport is constructed — not here.

Step 3 — write the implementation

The server-side implementation inherits from rpc::base and overrides the interface methods. There is no serialisation, no transport plumbing, and no callback channel to set up — that is all in generated code.

// server/my_session.h #include "generated/inference/inference.h" class my_session : public rpc::base<my_session, inference::v1::i_session> { my_model_state model_; public: RPC_TASK(rpc::result) generate( const std::string& prompt, const inference::v1::generation_options& opts, const rpc::optimistic_ptr<inference::v1::i_token_sink>& sink) override { auto max_tokens = opts.max_tokens.value_or(512); model_.begin(prompt, opts.temperature.value_or(0.7f)); while (model_.has_next() && max_tokens-- > 0) { CO_AWAIT sink->on_token(model_.next_token()); // [post]: no remote reply wait } CO_RETURN CO_AWAIT sink->on_finish( inference::v1::finish_reason::complete, model_.tokens_generated()); } RPC_TASK(rpc::result) cancel() override { model_.cancel(); CO_RETURN rpc::error::OK(); } }; class my_inference_service : public rpc::base<my_inference_service, inference::v1::i_inference_service> { public: RPC_TASK(rpc::result) list_models(std::vector<std::string>& model_ids) override { model_ids = { "llama3-8b", "mistral-7b" }; CO_RETURN rpc::error::OK(); } RPC_TASK(rpc::result) create_session( const std::string& /*model_id*/, rpc::shared_ptr<inference::v1::i_session>& session) override { // The runtime registers the distributed-capable shared_ptr when it is // returned across the boundary. The caller's copy can keep the session alive. session = rpc::make_shared<my_session>(); CO_RETURN rpc::error::OK(); } };
The [post] annotation on on_token means no application response is awaited for each token. That removes a per-token request/response round trip; throughput still depends on serialisation, transport capacity, scheduling, and backpressure.

Step 4 — choose a deployment

The interface and enclave-compatible business logic can stay the same between these deployments. Transport construction, platform packaging, and security policy are supplied separately.

Local (same process)

In-process, plugin, or DLL boundary with low-overhead local and binary paths. Useful during development and for component isolation.

TCP + TLS

Network deployment between processes or machines. The stream transformer wraps each accepted TCP connection in TLS before handing it to the transport — application code unchanged.

SGX enclave

Inference inside a trusted-execution environment. An SGX transport crosses the enclave boundary; when route security and DCAP policy are configured, evidence is verified and bound to the protected session before sensitive RPC is admitted.

// Schematic deployment composition — the application interface is unchanged. // Local or DLL: // generated interface -> local / dynamic-library transport -> implementation // Network: // generated interface -> streaming transport -> TCP -> OpenSSL TLS // Confidential SGX route: // generated interface -> protected RPC -> route security -> SGX transport // route policy: end-to-end encryption + credential authentication + mutual DCAP // // The connection factory materialises the selected transport and stream layers. // SGX packaging and enclave-owned attestation policy are deployment concerns; // the business interface does not claim that transport alone supplies attestation.

The client-side connection factory materialises the matching profile and returns an rpc::shared_ptr<i_inference_service>. From that point the typed application calls use the same contract across all three deployments.

Step 5 — schemas and agent discovery

The same IDL that drives generated C++ also drives generated JSON schemas. Two schema profiles serve different consumers from a single source:

Config profile

Full authoring schema: descriptions from [description=…], default values, string|integer enums, additionalProperties: false, and cross-file $ref with $id.

Used by VS Code and other JSON-schema-aware editors to provide completion and validation when writing Canopy configuration files.

MCP profile

Minimal tool schema: everything inlined, no $id, string-only enums, no defaults. Sized and shaped for LLM tool-use layers.

An AI agent receives this schema and can build a valid JSON call for any method in the interface — with no hand-written schema maintenance.

At runtime, a caller can ask a Canopy service to describe itself. The service returns method names, parameter schemas, and interface metadata — all derived from the generated code, with no hand-written MCP configuration needed.

// ── What the generator produces (build-time, no runtime cost) ──────────────── // Config profile schema for generation_options (fragment): { "$schema": "http://json-schema.org/draft-07/schema#", "title": "generation_options", "properties": { "max_tokens": { "type": "integer", "default": 512, "description": "Maximum tokens to generate" }, "temperature": { "type": "number", "default": 0.7, "description": "Sampling temperature 0.0–2.0" } }, "additionalProperties": false } // MCP profile for i_session.generate (fragment): // — no defaults (fewer tokens to the LLM), string-only enums, self-contained { "name": "i_session.generate", "inputSchema": { "type": "object", "properties": { "prompt": { "type": "string", "description": "The prompt text" }, "options": { "type": "object", "properties": { "max_tokens": { "type": "integer", "description": "Maximum tokens to generate" }, "temperature": { "type": "number", "description": "Sampling temperature 0.0–2.0" } } } }, "required": ["prompt"] } }

Canopy's built-in get_schema RPC retrieves generated interface descriptors across local, streaming, SGX, and protected routes. The C++ runtime also provides schema-driven dynamic call and post helpers, which are used by the MCP integration. Generated browser clients remain the supported typed JavaScript path; a completely generic no-stub browser facade is future work.

// C++ runtime discovery over an existing local or remote object. std::vector<rpc::rpc_types::v1::interface_descriptor> descriptors; auto result = CO_AWAIT rpc::casting_interface::get_schema( *session, descriptors, rpc::rpc_types::v1::encoding::yas_json, rpc::rpc_types::v1::schema_flavor::mcp); // The descriptors carry generated interface and method schemas. // rpc::casting_interface::call/post provide the corresponding dynamic C++ path.

What was written vs what was generated

inference.idlThe complete service contract: three interfaces, one options struct, one enum, and their attributes.
CMakeLists.txt changeOne CanopyGenerate call specifying the IDL file and desired serialisation formats.
my_session.h / .cppThe actual inference logic: generate, cancel, and the session factory — no transport or serialisation code.
Deployment compositionChoose local, TCP+TLS, or SGX transport wiring and add the route-security policy required by that deployment.
Generated (not written)Proxy and stub code for each serialisation format; JSON schemas for editor IntelliSense and AI tooling; version fingerprints; JavaScript WebSocket client.

Try the calculator demo first

The WebSocket calculator uses the same IDL pattern — simpler interface, same transport and generation story. Run it to see the generated call path end to end before exploring a fuller service contract.

Open live demos IDL reference GitHub