Activity

A focused day of releases and documentation alignment across the VGI ecosystem.

On vgi-lint-check, a release bump to 0.81.0 shipped support for protocol 1.3.0 and VGI905 preconditions, marking progress on protocol compliance.

The TypeScript RPC library (vgi-rpc-typescript) received two rapid releases addressing HTTP handling. Version 0.19.1 fixed an issue where Cloudflare was double-encoding compressed responses, followed by 0.19.2 which guarded the dispatch debug flag behind globalThis to ensure blended and table-in-out functions work correctly on Cloudflare Workers. These fixes resolve interoperability issues with the CDN platform.

Units support in vgi-units was bumped to 0.1.5, aligning with VGI 0.29 / vgi-rpc 0.21 for the 1.3.0 protocol release.

The haybarn community extensions were updated to track the latest vgi main branch, pulling in changes that publish worker-declared global functions into system.main, retire the describe.json contract, and improve handling of required options on attach.

A subtle bug fix in adbc_scanner corrected wrong results when scanning ANALYZE'd tables — the kind of edge case that only surfaces under specific query optimization conditions.

Documentation received overdue attention on vgi-rpc-python. The wire protocol spec had drifted significantly across nine releases, missing three request-metadata keys, lagging on HTTP headers and endpoints, and containing actively misleading claims. PR #39 brings the spec current with the actual implementation. This also includes private work across 4 commits on 2 internal repos.

A major coordination effort across the VGI protocol ecosystem today, with 42 commits spanning eight repositories. The overarching theme: migrating from static describe.json endpoints to a dynamic catalog system that clients read over the protocol itself.

vgi-java reached version 0.23.0, implementing the new landing page behavior that reads the catalog over the protocol rather than relying on describe.json. Rusty also dropped the describe.json producer and added a required option to attach configurations.

vgi-rpc-rust bumped to 0.21.0 with parallel changes: the HTTP server now serves the VGI client bundle directly and no longer exposes describe.json routes. Internal dependency pins were updated to maintain consistency across the ecosystem.

vgi-rust advanced to 0.29.0, with similar landing page refactoring. Notably, the protocol now advertises every unary method's real parameter schema, and wrapped parameter schemas are advertised directly instead of being hidden behind describe.json. The required option was added to attach options, and formatting was standardized with cargo fmt.

cupola received several polish improvements: fixed lossless Arrow type handling and eliminated a terminal-readiness race condition. The landing page now strips trailing slashes from the client base URL and reads the catalog over the protocol. Perspective was upgraded to v5.1.0 with a rebased fork, npm-layout vendoring, and virtual-server coverage.

vgi-rpc-typescript shipped version 0.19.0 and a preceding 0.18.1. The landing surface gained test coverage, the client bundle is now served by HTTP routes (describe.json removed), and the base URL handling was standardized. The package is now installable under npm with a browser-compatible webpack build.

vgi-rpc-java released 0.21.0, echoing the Rust changes: HTTP routes serve the client bundle and describe.json endpoints are gone.

vgi-go completed the rollout at 0.19.0. The protocol layer now declares wrapped parameter schemas on request-wrapped methods, the catalog advertises these schemas, and required was added to attach options. Like its peers, the landing page now serves the JavaScript client instead of producing describe.json.

Beyond the public repositories, Rusty also made 13 commits across four private repositories, indicating additional work on related infrastructure or applications.

Daily Activity Summary for August 9, 2026

Rusty executed a large-scale synchronization cycle across the DuckDB community extensions ecosystem. The haybarn-community-extensions-sync repository recorded the coordination of 41 dispatches, syncing upstream changes at commit 6b41a3c8.

The bulk of the work manifested across haybarn-community-extensions, where he synchronized 40+ extension repositories with their upstream sources. Notable updates included:

• New extensions onboarded: stac, quackiso, python_udf, motorsport_telemetry, ml, luajit, gdrive, and erpl_tunnel were all brought into the sync system for the first time
• Version bumps: zarr advanced from v0.1.2 → v0.1.3, pst from v0.2.0 → v0.2.0, anofox_statistics pinned to v0.8.1, and python_udf set to v0.2.1
• Branch/commit synchronizations: Extensions like zim, urlpattern, snowflake, raster, quackscale, quackformers, quackapi, plinking_duck, pdf, pdal, osmium, mlpack, ldbc_data_gen, laterite_ags4, gpudb, gaggle, fit, eurostat, duckgql, duckdb_opendalfs, and adbc_scanner all received upstream commit updates
• Stable tracking: duck_lineage remained at its current pin despite the sync cycle

The systematic nature of these updates—marked with [skip ci] flags—reflects an automated or highly organized synchronization workflow managing the health and currency of the broader DuckDB community extension ecosystem. Additionally, Rusty made 3 commits on a private repository, though the specifics remain undisclosed.

Rusty focused on several interconnected projects, with particular attention to cloud data handling and worker infrastructure.

Work on vgi-cbor advanced the CBOR serialization layer with two key improvements. The glob source expansion completed the port of cloud.rs, bringing cloud data handling capabilities into the serialization pipeline. Additionally, he refactored MessagePack timestamp handling to properly carry instant values using the reserved timestamp extension type, preserving nanosecond precision — a critical detail for temporal correctness in data interchange.

haybarn-wasm received test infrastructure refinement: the OPFS (Origin Private File System) test suite was scoped to its own describe block, improving test organization and isolation for the browser-based storage implementation.

The bulk of effort landed in cupola, where Rusty shipped two releases while addressing accessibility and error handling. Version 0.4.105 bundled significant improvements including SQL quoting fixes, an auth classifier, consolidated Arrow handling, prompt facts, and a bridge split refactor—all while ensuring the e2e test suite runs green. Version 0.4.106 followed quickly with a branded error page for outdated or missing worker versions, making the user experience clearer when deployments drift. The accessibility work fixed 7 failing e2e tests, restoring full test compliance across the suite.

SQL quoting and Arrow migration work in cupola

Rusty focused on fixing several interconnected issues in cupola. Two commits addressed SQL quoting, catalog field name handling, and completing an Apache Arrow migration. The work involved escaping SQL literals properly, tightening authentication classification logic, and finishing the transition to Arrow-based data handling. These fixes improve both data safety and the robustness of catalog operations.

HTTP error handling across the ecosystem

Work extended to related projects addressing error surface and reporting. In haybarn-community-extensions, a dependency bump pulled in upstream fixes from the main branch, including improvements to how HTTP worker errors are surfaced after the stream header has been written. Over in vgi-rpc-python, he implemented a complementary fix to flag producer errors on their first turn, ensuring errors are caught and reported at the right point in the HTTP request lifecycle.

Private repository work

Additional commits were made across 2 private repositories (4 commits total), continuing parallel development efforts.

Rusty focused on community engagement and support work throughout the day, fielding eleven issue and pull request comments across various projects. Rather than pushing new features, he concentrated on reviewing, providing feedback, and helping others navigate their work—a solid day of stewardship and collaboration.

Behind the scenes, private repository activity shows nine commits distributed across two private codebases, suggesting ongoing maintenance or feature development on projects not yet public.

A heavy day across the VGI ecosystem, focusing on payload handling and protocol updates. Rusty addressed a critical issue: payloads larger than 2 GiB were being corrupted or rejected across multiple language implementations. This required careful fixes to transport layer bounds checking and honest ceiling reporting instead of wrapping errors.

VGI RPC implementations saw coordinated updates to fix the large-payload issue. In vgi-rpc-rust, the fix involved bounding every transport write and removing the illegitimate 256 MiB cap on request bodies—released as v0.20.0. The Go and TypeScript implementations followed suit: vgi-rpc-go and vgi-rpc-typescript received similar transport bounds fixes, with CI adjustments to run large-payload test suites in isolation (preventing the access log from interfering). vgi-rpc-java shipped v0.20.2 with proper error semantics for bodies that exceed 2 GiB, and vgi-rpc-python received multiple releases (0.40.0–0.42.0) addressing httpx → httpx2 migration, clamped reads, and required huge-payload conformance testing.

The Rust ecosystem advanced significantly. vgi-rust released three versions (0.26.0–0.28.0) adopting VGI protocol 1.3.0 with global functions and streaming COPY FROM via CopyFromFunction::read_stream. vgi-cbor matured from 0.2.1 to 0.4.0 with CBOR/MessagePack COPY format support, exact DECIMAL and timestamp handling, s3:// and http(s):// path support via object store, streaming COPY FROM instead of buffering, and improved container path diagnostics. vgi-go adopted VGI protocol 1.3.0 and the latest vgi-rpc-go v0.20.0.

haybarn-wasm saw a release bump to 1.5.5-rc2 with coordinated dependency upgrades (css-loader 7, webpack-cli 7, actions/setup-node v7, wrangler pinning) and CI improvements to decouple shell deployment from npm publishing. The wasm base image tag now derives from the workspace version for consistency. cupola followed with a haybarn-wasm 1.5.5-rc1 dependency bump.

Rusty also managed 11 community reviews and replies on dependabot PRs across haybarn-wasm and vgi-rpc-python, shepherding dependency updates through the pipeline. Private repo activity added 9 commits across 4 repos, continuing parallel work.

A high-velocity day across the vgi-rpc language ports, driving conformance to the 0.36 protocol specification. Rusty systematized a major cross-language update involving token architecture changes, standardized error handling, and access logging infrastructure.

vgi-rpc-python saw the heaviest activity with 19 commits shipping features and refinements. He released versions 0.34.0 through 0.37.0, tracking the specification milestones: token introspection endpoints (off by default), egress accounting and claim redaction in the access log, async log emission, Cross-Origin-Resource-Policy standardization, and conformance test automation. Notable fixes include capturing introspection logs correctly, emitting request_data as a real IPC stream to speed up the request path, and relaxing the conformance validator to accept zero-parameter calls. A security fix bumped the cryptography dependency floor past GHSA-g6cj-pr64-35w5.

vgi-rpc-rust, vgi-rpc-go, vgi-rpc-typescript, and vgi-rpc-java each received parallel updates implementing the 0.36 spec surface. He opened PR #33 on vgi-rpc-typescript, PR #1 on vgi-rpc-java, PR #1 on vgi-rpc-rust, and PR #3 on vgi-rpc-go—each bringing the respective language port to feature parity with the Python reference implementation. The ports now include:

• Call/cursor token split — separating call tokens from state tokens with cursor version bumps (v4→v6 in Rust, v5→v6 in Java, v5 in TypeScript, bounded call-state caches)
• Token payload compression inside sealed envelopes with codec tagging
• Standardized 401 responses with reason codes, cache directives, and JSON envelopes
• Introspection endpoints and access log verification integrated into CI pipelines at DEBUG level across all ports

Rust encountered a build issue fixed by ensuring clean compilation without the HTTP feature. Java required a fix to serialize request_data before the stream drains.

On the open-source front, Rusty provided feedback on apache/arrow-rs PR #10075, discussing the use of Metadata struct instead of HashMap<String, String> for ordered, cheap-to-clone metadata handling.

A productive day spanning multiple repositories, with heavy focus on performance optimization and release work in vgi-rpc-python, alongside UI fixes and community engagement.

cupola received a bug fix to preserve query results when switching between editor tabs, improving the user experience for multi-tab workflows.

vgi-go saw protocol buffer regeneration to align with the 1.3.0 release of vgi-python, which introduced global function support.

vgi-rpc-python absorbed the bulk of today's effort with a substantial performance optimization sprint. Key improvements include:

• HTTP layer: Fixed waitress server defaults for the workload, resolved dispatch function resolution to happen once rather than per-request, and implemented retry logic for requests lost to stale keep-alive connections
• State management: Split stream state into separate call and cursor tokens, encoded flat stream state without Arrow IPC overhead, and reused deserialized stream-pinned state across turns to avoid repeated rebuilds
• Batch handling: Built one-row batches per column and shared a single IpcWriteOptions instance
• Cryptography: Switched from PyNaCl to PyCryptodome for state token sealing and added per-platform AEAD backend selection
• Caching: Cached empty batches and shared IPC read options; reverted some over-aggressive serialization caches that proved counterproductive
• Testing: Fixed drain-timer stubs to coexist with pytest-timeout on Windows, and ran the full suite under xdist to expose and fix concurrency issues

Two releases were cut: version 0.32.0 introduced a msgpack state codec replacing the positional codec, while 0.31.0 featured the call/cursor token split, compact state codec, and per-platform AEAD selection.

Two feature PRs were opened: PR #37 replaces the circular seal brand asset throughout the project with a new shield mark design (updating the logo, landing page, error pages, and other assets), and PR #36 standardizes the 401 response format with machine-readable error signals and a proxy-configuration note, replacing ad-hoc HTML pages with structured responses that won't break Arrow clients.

Rusty also engaged with upstream projects: replying on PR #10075 in apache/arrow-rs regarding metadata struct optimization, and on PR #22564 in duckdb/duckdb for the new secret type parameters system view.

Private repository activity included 16 commits across 4 repos.