Activity

A high-volume day focused on open-source stewardship, documentation standardization, and security fixes across multiple projects.

Community Extensions & Documentation

Rusty made substantial contributions to the DuckDB ecosystem by standardizing documentation links across duckdb/community-extensions. This involved updating links for 24 extensions—including airport, bitfilters, cronjob, crypto, datasketches, and many others—to ensure consistent documentation formatting. Beyond documentation, he also took on maintainer roles for five community extensions: webmacro, cronjob, quickjs, redis, and tsid.

URI Credential Security Fixes

A significant security issue was addressed across two related repositories. In vgi-rpc-rust, commits fixed a critical bug where JWKS fetch errors were echoing URL credentials and query parameters in error messages. This fix propagated to grainlift, where he released version 0.6.1 with URI credential redaction. The fix ensures that sensitive authentication information (user:password@ patterns) is stripped from URIs before logging or error reporting, with documentation clarifying that OAuth messages were the source of the leak, not the reqwest HTTP client itself.

Cupola Report Storage & UI Improvements

cupola saw a flurry of activity across multiple releases (0.4.225–0.4.230). Key improvements included:

• Integration with HTTP report stores, unifying report storage browsing and transfers
• Ability to browse and compare worker report revisions
• UI fixes aligning report controls and editor saves with the worker contract
• Resolution of a Pivot catalog table discovery bug (quoted Perspective table IDs)
• Improved HTTPS localhost connection error messaging
• Test enhancements including e2e test isolation and regression testing for protocol 0.5 session schemas

Filter Library Documentation

On FastFilter/xor_singleheader, documentation was added for DuckDB bindings for XOR and binary fuse filters, supporting PR #79.

Routine Maintenance

Other updates included releasing grainlift 0.6.1 with the credential redaction fix and vgi-rpc 0.31.4 dependency bump, CI toolchain repinning on grainlift, and housekeeping across davidgasquez/awesome-duckdb to update Query.Farm extension links.

A prolific day across the Query.Farm ecosystem and upstream community projects. Work focused on bug fixes, extension maintenance, and broad ecosystem documentation.

RPC and observability work. vgi-rpc-go received multiple stability fixes: resolved a conflict where OpenTelemetry and Sentry instrumentation hooks were replacing each other, fixed an issue where only one dispatch hook could be held at a time, and corrected trace correlation loss in access logging. Two patch releases shipped as v0.34.0 and v0.34.1.

Extension updates and releases. quickjs was bumped to DuckDB v1.5.6 and QuickJS-NG v0.17.0, with the extension version incremented to 2026100801. redis merged PR #10 adding TTL support (EXPIRE, TTL, EXPIREAT functions) and updated its DuckDB submodule to the v1.5-variegata branch. cupola released v0.4.223 with multi-catalog inventory and AI prompt caching fixes, with hardening applied across those systems and DDL navigation. airport fixed an issue where Flight exchange lifetime was materializing results unnecessarily. radio fixed VARCHAR message handling—payloads were being escaped, and all messages were sent as binary frames instead of text where appropriate. vgi-csharp corrected run_tests.sh to execute the actual test tree rather than a compiled-in unittest version.

Community extension submissions. Rusty opened 16 PRs submitting Query.Farm extensions to upstream projects' adoption and integration lists: Crypto to BLAKE3, Hashfuncs to xxHash and RapidHash, Tributary to Kafka integrations and librdkafka bindings, Bitfilters to XOR and binary fuse filters, Marisa to marisa-trie bindings, DataSketches adapter to the DataSketches website, Radio to IXWebSocket users, MiniJinja integration to MiniJinja use cases, and Query.Farm projects to Apache Arrow's Powered By page. Additionally, he submitted comprehensive updates to awesome-duckdb covering six existing extension entries and filling in missing projects from the Query.Farm catalog.

Community extension registry updates. Opened 15 PRs against duckdb/community-extensions to sync the latest commits for: tsid, stochastic, shellfs, redis, radio, quickjs, openprompt, minijinja, lindel, json_schema, inflector, httpserver, hashfuncs, fuzzycomplete, cronjob, and geosilo.

Awesome list submissions. Submitted documentation improvements to multiple community curated lists: added DuckDB projects to awesome-typesafe-jev, awesome-adbc, awesome-kafka, awesome-duckdb-spatial, and awesome-iroh. Also opened an issue against json-schema-org/website proposing Query.Farm's JSON Schema for DuckDB in the ecosystem.

Website and documentation. query-farm-astro published an article on Iroh browser SQL with diagrams and metadata, connected product pages and technical guides to published articles, updated VARIANT and Arrow support tracking, and documented new TTL functions in Redis extension documentation. datasketches-website and quickjs-ng received commits documenting their respective DuckDB adapters.

Community support. Replied to issues on a5 regarding overlapping polygon detection and on quickjs about DuckDB community extension details. Also provided community stewardship across upstream projects with targeted, well-documented submissions.

Private activity. 16 commits across 2 private repositories.

Rusty delivered a major coordinated release across the VGI ecosystem today, addressing security, stability, and protocol improvements across multiple language implementations.

VGI RPC Layer

The vgi-rpc-cpp library received several critical fixes bundled into releases 0.8.1 through 0.9.0. A bearer authenticator that refused anonymous callers now supports optional bearer authentication via HttpConfig::bearer_optional. The access log was oversharing request payloads and stream state tokens—this has been corrected. Additionally, serve_unix workers lacked an idle timeout, causing launched processes to hang indefinitely; this is now fixed. The reflection client tests were reformatted to match CI's clang-format 22.1 standards.

Similar fixes landed in vgi-rpc-rust (releases 0.31.1–0.31.3) and vgi-rpc-go (releases 0.33.1–0.33.2), where request payloads were inadvertently reaching access logs and unary records still carried a transitional payload_omitted marker. The vgi-rpc-typescript library received corresponding updates for request data logging at DEBUG level.

VGI Core Implementations

vgi-java saw substantial protocol advancement across releases 0.39.0–0.41.1. Port attach tickets now allow a runner holding only a grant to reattach a user's catalog using a sealed ticket. Opaque values are no longer unsealed with a fallback—they either validate or reject, and secret options no longer appear in the attach ID. The VgiService is now generated from the reference specification rather than hand-written, and the vgi.v2 registry reflects this change with the reference hash reported by reflection. AttachTickets now seal using vgirpc's XChaCha20Poly1305 instead of a local copy. HTTP lane error handling was improved so worker errors are treated as skips, and the full vgi.v2 surface is now hosted.

In vgi-cpp, attach tickets introduce a critical security feature: a user's ATTACH is sealed so only a runner holding their grant can replay it. Opaque attach data is now sealed and bound to the caller, eliminating plaintext secret options. The integration harnesses were fixed to fail on every HTTP-related worker error rather than silently skipping them, and the vgi.v2 hash is pinned to the reference to prevent surface drift. Launched workers now honour the --idle-timeout flag on --unix sockets. The vgi.v2 registry is generated from the reference specification, and optional bearer support surfaces the principal. Releases span v0.8.0 through v0.10.1.

vgi-rust (releases 0.41.0–0.43.0) mirrors these advances: attach tickets seal user ATTACHes for runner replay, opaque data is sealed on HTTP, and integration lanes no longer hide failures as skips. The vgi.v2 surface is now fully reference-generated with a hash matching vgi-python 0.43.0. HTTP test infrastructure was cleaned up—run_http_tests.sh was leaking all five servers on every run. CI now runs unit tests in the staged directory where HTTP workers live.

Grainlift and Integration

grainlift-go now advertises per-connection statistics support via Grainlift protocol 0.5. The grainlift-cloudflare deployment was updated to protocol 0.5 and declares statistics as unsupported; PR #1 merging durable-objects-and-permissions support was integrated.

Auxiliary Work

cupola received a fix to open the default schema and simplify empty sidebar sections. Across the ecosystem, integration test scripts in vgi-java and elsewhere were cleaned up to prevent HTTP worker leaks during CI runs. Overall, Rusty coordinated security hardening (sealed attach tickets, secret option protection), stability improvements (idle timeouts, error visibility), and specification alignment (vgi.v2 generated from reference) across 22 repositories.

A particularly active day across the VGI ecosystem, with 91 commits spanning 18 repositories and focused work on grant-based authentication, catalog capabilities, and cross-language RPC implementations.

Grant Authentication Across the Stack

The dominant theme was rolling out bearer credential support for minted grants across all VGI RPC language bindings. vgi-rpc-cpp (v0.7.0), vgi-rpc-rust (v0.30.0), vgi-rpc-go (v0.32.0), vgi-rpc-typescript (v0.27.0), vgi-rpc-java (v0.29.0), and vgi-rpc-csharp (v0.14.0) all received updates to accept sealed grants from issue_grant and resolved tokens as HTTP bearer credentials. This represents a significant authentication flow improvement across the distributed worker infrastructure.

Core Language Workers

vgi-rust received dual releases (v0.39.0 and v0.40.0) adding catalog_contents fixture catalogs, DDL-capable in-memory catalogs, and client-side load_catalog support. The worker now accepts sealed grants via HTTP bearer authentication configured through --grant-key or VGI_RPC_GRANT_KEYS. A flaky test suite issue was addressed where tampered-grant tests intermittently passed with real grants.

vgi-cpp (v0.7.0) underwent substantial work: ported five missing fixtures to published-extension lanes, reverted and re-applied catalog_contents fixture handling, served six catalog_contents fixture catalogs, and delivered DDL-capable memory catalogs. A critical scalar bug was fixed where result() moved an array before reading its length on x86-64. Build parallelism was capped at 2 to prevent GCC runners from exhausting memory. The worker now properly authenticates incoming grant-based calls.

vgi-java shipped two releases (v0.37.0 and v0.38.0) with catalog_contents fixture support for code-defined and in-memory DDL catalogs, plus grant-based bearer authentication.

vgi-csharp (v0.14.0 and v0.15.0) added catalog_contents fixture catalogs, DDL-capable in-memory catalogs, and full support for sealed-grant and resolve_token authentication over HTTP.

Specialized Workers & Extensions

vgi-polars (v0.9.0–0.9.1) tackled several improvements: migrated to PyPI-hosted vgi-python with dependency pinning to v0.41.0, added Filter Encoding v2 support for pushed-down filters, fixed string/binary is_in needle handling, resolved hanging scans that ended before their streams, and implemented proper cleanup of abandoned table_function streams. Polars 2.0 is now the build target with 1.x still supported.

vgi-lint-check fixed buffering-function argument attachment and improved simulation output to show result columns.

vgi-rpc-go (v0.32.0) extended bearer credential support to HTTP-based sealed grants and resolve_token flows.

Frontend & User-Facing Tools

cupola saw steady improvements across three releases (v0.4.219–0.4.221): bulk catalog content loading with legacy fallback, sidebar context menus for reports and notebooks, file attachment support across AI assistants, HTTP request abort handling on cancellation, Iroh node startup deferred to first use for performance, upgrade notification dismissal, and local document sidebar organization with restored report creation.

haybarn-wasm bumped to v1.5.5-rc8 to support request cancellation in flight.

Open Source & Private Work

Rusty also contributed to the duckdb/duckdb-wasm repository as part of broader community engagement. Additionally, 22 commits across 4 private repositories indicate ongoing internal development.

A prolific day across the Query Farm ecosystem, with 91 commits spanning 17 repositories and significant feature development in both backend services and the Cupola UI.

VGI Language Bindings & Core Updates

Rusty pushed releases and maintenance updates across the VGI stack. The vgi-rust crate received version 0.37.1 with documentation improvements, including fixes to broken intra-doc links in vgi-client and removal of secrets guidance from attach-option docs. Similar maintenance appeared in vgi-java with CI adjustments to handle upstream attach_secrets skips. The vgi-excel binding was prepared for Cupola 0.5.1, pinning VGI releases and adding sign-in recovery support.

Haybarn WASM & HTTP Layer

Work on haybarn-wasm focused on request lifecycle management. He upgraded the engine dependency and implemented abort support for HTTP requests in flight, ensuring httpfs requests properly carry the query's interrupt flag. This honors request cancellation in retry loops, improving responsiveness when queries are stopped mid-execution.

Cupola: Major Multi-Release Sprint

The bulk of today's effort landed in cupola, with a coordinated sequence of 8 releases (0.4.208–0.4.216) delivering substantial UI and workspace features. Early releases tackled inspector and editor polish: sidebar Inspector with call snippets and signature help (0.4.209), per-tab revision history and a redesigned split Run button (0.4.214), distinct summary and documentation display (0.4.210), Inspector sections with view SQL (0.4.211), and toolbar regrouping plus a History diff stack (0.4.212). The inspector also now stringifies LIST columns in Perspective snapshots (0.4.208).

Following these incremental improvements, he merged multi-catalog phases 3A, 3B, and 3C. Phase 3A introduced the workspace manager editor for managing multiple catalogs. Phase 3B added workspace files, DuckDB script export/import, and JSON Schema support. Phase 3C brought alias rename dialogs, report requires with Rebind/Attach, command palette, and a tokenizer for catalog alias references. Supporting these features are structured attach options with a typed options form and consent screen, plus a portable workspace format with aliases and #ws= codec for catalog-qualified routing.

Final releases in the sequence (0.4.215–0.4.216) delivered native SQL notebooks with charts and AI editing, query cancellation consistency across all surfaces, and fixes to arrowLosslessConversion and R2 cache policy handling. Dependencies were updated to vgi 0.37.1 and test workers to vgi-python 0.38.

Ecosystem & Private Work

Additional activity touched vgi-rpc-typescript, vgi-csharp, vgi-go, vgi-rpc-python, vgi-rpc-csharp, vgi-github, adbc_scanner, query-farm-astro, and community extensions. He also contributed 13 commits across 2 private repositories, and engaged in 8 community support interactions—likely reviewing or advising on DuckDB extensions and open-source projects.

Rusty released version 0.4.207 of cupola with a notable feature addition: a resizable value panel and formatted JSON display in Lines mode. The work involved both the feature implementation and expanded test coverage for results handling, as captured in the release commit and preceding feature branch work.

Over on arrow-rs, he opened PR #11374 to address issue #9837, introducing a zero-copy pull-based BufferStreamReader for IPC streams. The motivation is clear: when an entire IPC stream is already loaded in memory—whether from an HTTP body, memory-mapped file, shared-memory segment, or bytes::Bytes from object storage—the existing StreamReader<R: Read> API forces unnecessary copying. This new approach eliminates that overhead by leveraging the data already in hand.

Rusty also weighed in on the DuckDB community with feedback on PR #22715, offering support for the Arrow export enhancement targeting VARIANT type support. His involvement across these three projects reflects ongoing work on columnar data handling, zero-copy optimizations, and interoperability between query execution and Arrow serialization.

Rusty shipped two releases of cupola focused on improving how query results are displayed. The first release (0.4.205) added support for reading full values in query results through a new value panel and Lines layout. He then refined the presentation in 0.4.206 by restyling the Lines results layout to display results as row cards with a key/value table structure, improving readability and visual hierarchy.

Work on vgi-github centered on enhancing the GitHub data integration extension with better user experience and observability. The README was rewritten to focus on user-facing documentation, with internal details moved to a separate DEVELOPMENT.md file. On the technical side, Rusty added a token_source ATTACH option that allows users to supply their own GitHub login credentials rather than relying on a default token. Servers can now opt into this feature via the VGI_GITHUB_ALLOW_TOKEN_SOURCE environment variable. He also implemented logging improvements to report rate-limit waits, retries, and token budget information to the DuckDB log, giving users better visibility into API usage and performance.

A major release day across the Grainlift ecosystem, with significant work on object storage support, request handling, and production readiness across multiple language implementations and supporting libraries.

Core Grainlift Framework

The grainlift repository reached version 0.4.3, introducing object storage for large HTTP requests and results. The driver now accepts bound batches as large as the request itself, enabling more efficient data transfer. Work included SDK object storage implementation for the development Service, shared conformance validation for request limits and object storage contracts, and strict type checking via mypy and pydoclint. CI improvements added haybarn-cli installation for hello-world tests and refined candidate selection, while documentation clarified that connection commands are CALL table functions.

Language Implementations

The Go implementation (grainlift-go) received updates to handle large HTTP requests and results through S3-compatible storage, with result batches now able to fill responses entirely. The TypeScript SDK (grainlift-typescript) and Python driver (grainlift-python) both released versions 0.3.0–0.3.1, implementing the same object storage and request limit improvements. The grainlift-hello-world-python example was bumped to 0.2.0 with proper connection cleanup via CALL adbc_disconnect, while grainlift-hello-world-go and grainlift-hello-world-typescript were updated to use the latest SDK versions. Documentation across examples now highlights object storage flags and DuckDB's attached catalog patterns.

RPC and Transport Layer

vgi-rpc-go and vgi-rpc-typescript received refinements to handle response size constraints more robustly. The Go implementation now declares body size in zstd frames to prevent decode failures under small response limits, while the TypeScript version (0.25.8) fixed type-checking for consumers without the optional iroh-http-node peer dependency.

Turso Integration

The new grainlift-turso driver reached production readiness with support for Turso Cloud's database engine and concurrent transactions. Work included adding deadlines and cancellation semantics, a comprehensive release workflow for binaries and container images, and production host configuration. Documentation was significantly expanded with a user-focused README, quick-start guide from release downloads, SQL examples using DuckDB's attached catalog, and a changelog. CI improvements ensure unique table names across concurrent jobs.

Query Engine and WebAssembly

haybarn-wasm (1.5.5-rc7) and cupola (0.4.204) benefited from query cancellation improvements. The WebAssembly engine now properly interrupts running queries through connection flags and mirrors HTTPFSParams fields for consistency. Cupola's editor now cancels running queries via the Stop button and utilizes every engine thread for cancellable queries. He also added a feature to remove earlier revisions from report history.

Community Extensions and Ecosystem

haybarn-community-extensions was updated to track latest versions: grainlift driver 0.4.3, adbc_scanner from main (with telemetry thread and decimal fixes), and vgi from main (with cancel dispatcher thread safety improvements). The duckdb-grainlift extension bumped grainlift to 0.4.3, while query-farm-astro received related updates.

Supporting Work

The Cloudflare example and hello-world projects were synchronized with the latest dependency versions. Across all repositories, work emphasized production readiness, proper resource management, and comprehensive documentation. A private repository also received 2 commits.

Overall, this represents a coordinated push to ship object storage support, production hardening, and improved concurrent request handling across the entire Grainlift stack.

Daily Activity Summary — October 1, 2026

RPC and Authentication Infrastructure

Rusty made significant progress on OAuth and device-flow authentication across the RPC ecosystem. In vgi-rpc-rust, he advertised a separate OAuth device-flow client and fixed the resource metadata to properly expose OAuth client fields. The TypeScript SDK (vgi-rpc-typescript) gained support for OAuth PKCE on Node ESM and Workers, while also exporting the Arrow facade as a first-class module (./arrow).

The vgi-rpc-website received a substantial redesign and content refresh, including new transport icons, updated SDK capability summaries, documentation of Iroh transports, improved social previews, and a general restyle aligned with the Query.Farm brand. Product logos and technical concepts now link to their respective documentation.

Grainlift Driver and Gateway

The grainlift driver saw major feature work around OAuth: interactive sign-in flows within the driver, device-flow client support on the gateway, and OAuth discovery and token refresh capabilities. A new release (0.4.1) captured these authentication improvements. Separately, Rusty fixed session recovery to recognize lost sessions regardless of wording and implemented fail-fast timeouts.

The duckdb-grainlift extension received extensive updates to support this new authentication layer. Notable changes include preparing iroh:// endpoints for ATTACH operations, reporting catalog type and version 0.4.0, naming attached databases instead of connection handles, removing rowid from remote tables, adding OAuth refresh-token ATTACH options, and fixing type casting for loosely typed scan results. Autocommit writes now work correctly on non-transactional remotes. Documentation for OAuth ATTACH options was also added.

Data Layer and Database Attachments

Work in cupola focused on integrating Grainlift databases into the query interface, including support for Iroh-based connections. Multiple releases (0.4.197 through 0.4.203) delivered incremental improvements: the old block-grid report builder was removed, Evidence sandboxed components now load and render properly, follow-up messages were added to both the AI chat and report agent workflows, and catalog refresh now correctly clears the grainlift extension's schema cache. Users can now open Grainlift gateways using ?service=grainlift+https://…&target=… parameters.

The adbc_scanner fix ensures that remote tables without rowid properly read a real column for count(*) operations.

TypeScript Runtime and Community Extensions

grainlift-typescript can now run on Cloudflare Workers with flechette support, expanding the reach of the toolkit to serverless environments.

In haybarn-community-extensions, Rusty bumped the grainlift extension repeatedly to track the latest driver improvements: OAuth sign-in, timeout handling, session recovery, loosely typed remote support, D1 write operations, database naming, and iroh endpoint preparation. This ensures community-facing DuckDB installations stay current with the core driver development.

Query.Farm Website Updates

The query-farm-astro site received a small but important fix to the churches map in the VGI project to accommodate Overture's updated places taxonomy, along with a responsive layout adjustment for the hero logo at phone widths.

Publishing and Package Naming

Rusty opened PR #29 on grainlift to resolve a PyPI publishing conflict. The server wheel was named grainlift, which conflicts with the Python worker toolkit (grainlift-python, version 0.2.1) and causes PyPI to reject uploads since only grainlift-python is a trusted publisher for that package name. The solution is to publish the server wheel as grainlift-adbc-gateway instead.