diff --git a/README.md b/README.md index 5a8c94e..38473fb 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ This `Go` branch keeps the outside interface compatible with the original projec - same default TSV / optional JSON / NDJSON batch outputs - same `--jdbc-url` flag for connection strings -The implementation is now a **single self-contained Go binary**. It does **not** require Java, JDBC drivers, `mysql`, `psql`, `sqlite3`, or any other runtime/client software to be installed. SQLite support is pure Go, and MySQL/PostgreSQL use native Go drivers. The only external commands involved are macOS built-ins for optional launchd install and optional Keychain lookup. +The implementation is now a **single self-contained Go binary**. It does **not** require Java, JDBC drivers, `mysql`, `psql`, `sqlite3`, or any other runtime/client software to be installed. SQLite support is pure Go, MySQL/PostgreSQL use native Go drivers, and DuckDB is statically linked (the one CGO dependency). The only external commands involved are macOS built-ins for optional launchd install and optional Keychain lookup. ## Supported URLs @@ -20,6 +20,42 @@ The implementation is now a **single self-contained Go binary**. It does **not** - PostgreSQL: `jdbc:postgresql://localhost:5432/app` - SQLite file: `jdbc:sqlite:/tmp/app.db` - SQLite memory: `jdbc:sqlite::memory:` +- DuckDB file: `jdbc:duckdb:/tmp/analytics.duckdb` +- DuckDB memory: `jdbc:duckdb:` (or `jdbc:duckdb::memory:`) +- DuckDB composite: `jdbc:duckdb:?attach=alias1,alias2` — see below + +## Cross-database joins (DuckDB composite) + +A duckdb alias can ATTACH other **already-open aliases** by name, giving one +SQL session that joins across servers (e.g. MySQL × PostgreSQL): + +```bash +database-cli open --alias shop --jdbc-url jdbc:mysql://db1:3306/shop --user shop --password-stdin +database-cli open --alias orders --jdbc-url jdbc:postgresql://db2:5432/orders --user app --password-stdin + +database-cli open --alias duck --jdbc-url 'jdbc:duckdb:?attach=shop,orders' +database-cli query --alias duck ' + select o.id, c.name + from orders.public.orders o + join shop.shop.customers c on c.id = o.customer_id' +``` + +Notes: + +- Credentials are reused from the source aliases **inside the daemon** via + temporary DuckDB secrets — in-memory only, never on disk, never in SQL you + type. +- A source opened `--read-only` is attached `READ_ONLY`; a composite opened + `--read-only` attaches all its sources `READ_ONLY`. +- Tables are addressed as `..` (MySQL: `..
`). +- The duckdb mysql/postgres/sqlite extensions are installed on first use + (downloaded from extensions.duckdb.org into `~/.duckdb/extensions`, needs + network once). +- DuckDB opens its own connections to the same host/port as the source alias, + so SSH-tunneled sources work — keep the tunnel (and the source alias) open + while using the composite. +- In-memory sqlite sources cannot be attached (their data is private to the + source connection). ## Install @@ -27,7 +63,7 @@ The implementation is now a **single self-contained Go binary**. It does **not** ./scripts/install.sh ``` -That script builds the binary with `CGO_ENABLED=0`, installs it to `~/.local/share/database-cli/database-cli`, writes a launchd agent, and drops a wrapper at `/opt/homebrew/bin/database-cli`. +That script builds the binary (CGO enabled for the statically linked duckdb driver), installs it to `~/.local/share/database-cli/database-cli`, writes a launchd agent, and drops a wrapper at `/opt/homebrew/bin/database-cli`. You only need Go to build from source. Once built, the resulting `database-cli` binary is standalone and can be copied to another compatible macOS machine without installing Java or database client libraries. @@ -48,6 +84,21 @@ database-cli rollback --alias A database-cli batch [--alias A] ``` +## Read-only aliases + +`open --read-only` blocks writes in two layers: + +- The database session rejects writes and DDL on every pooled connection: + MySQL/MariaDB `SET SESSION TRANSACTION READ ONLY`, PostgreSQL + `default_transaction_read_only=on`, SQLite `mode=ro`, file-backed DuckDB + `access_mode=read_only`. DuckDB cannot open an in-memory database read-only, + so an in-memory composite attaches its sources `READ_ONLY` instead. +- The daemon refuses `exec`, `begin`, `commit` and `rollback`, and `query` + accepts only statements starting with `SELECT`, `WITH`, `SHOW`, + `DESCRIBE`/`DESC` or `EXPLAIN`. + +`list` shows the flag per alias: `[{"alias":"shop","readOnly":true}]`. + ## Examples ```bash diff --git a/docs/spec.md b/docs/spec.md index 38f1d40..7c6f4f4 100644 --- a/docs/spec.md +++ b/docs/spec.md @@ -55,7 +55,7 @@ SQLite is pure Go so the binary stays self-contained. - `query` / `exec` accept the original positional SQL form, while `--sql` is still accepted as a compatibility fallback. - `jdbc:sqlite::memory:` is pinned to a single SQLite connection so memory databases behave the same across commands. - `schema` and `describe` use dialect-specific metadata queries. -- `read-only` is enforced in the daemon for `exec`. +- `read-only` is enforced by the database session (writes and DDL rejected on every pooled connection) and in the daemon: `exec`/`begin`/`commit`/`rollback` are refused and `query` accepts only read statements. ### Packaging @@ -68,6 +68,6 @@ SQLite is pure Go so the binary stays self-contained. 1. Open MySQL, PostgreSQL, and SQLite aliases using the existing flag contract. 2. Run `ping`, then run queries in TSV and JSON mode. 3. Run `begin` / `commit` / `rollback` against an alias. -4. Reject `exec` on a read-only alias. +4. Reject `exec`, and writes or DDL through `query`, on a read-only alias. 5. Confirm batch query ops honor their JSON flag. 6. Confirm the built binary runs without Java or external database client software. diff --git a/go.mod b/go.mod index a6191e1..72e0cbc 100644 --- a/go.mod +++ b/go.mod @@ -10,17 +10,39 @@ require ( require ( filippo.io/edwards25519 v1.2.0 // indirect + github.com/apache/arrow-go/v18 v18.4.1 // indirect + github.com/duckdb/duckdb-go-bindings v0.1.21 // indirect + github.com/duckdb/duckdb-go-bindings/darwin-amd64 v0.1.21 // indirect + github.com/duckdb/duckdb-go-bindings/darwin-arm64 v0.1.21 // indirect + github.com/duckdb/duckdb-go-bindings/linux-amd64 v0.1.21 // indirect + github.com/duckdb/duckdb-go-bindings/linux-arm64 v0.1.21 // indirect + github.com/duckdb/duckdb-go-bindings/windows-amd64 v0.1.21 // indirect github.com/dustin/go-humanize v1.0.1 // indirect + github.com/go-viper/mapstructure/v2 v2.4.0 // indirect + github.com/goccy/go-json v0.10.5 // indirect + github.com/google/flatbuffers v25.2.10+incompatible // indirect github.com/google/uuid v1.6.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect + github.com/klauspost/compress v1.18.0 // indirect + github.com/klauspost/cpuid/v2 v2.3.0 // indirect + github.com/marcboeker/go-duckdb/arrowmapping v0.0.21 // indirect + github.com/marcboeker/go-duckdb/mapping v0.0.21 // indirect + github.com/marcboeker/go-duckdb/v2 v2.4.3 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/ncruces/go-strftime v1.0.0 // indirect + github.com/pierrec/lz4/v4 v4.1.22 // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + github.com/zeebo/xxh3 v1.0.2 // indirect + golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 // indirect + golang.org/x/mod v0.33.0 // indirect golang.org/x/sync v0.20.0 // indirect golang.org/x/sys v0.42.0 // indirect + golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 // indirect golang.org/x/text v0.29.0 // indirect + golang.org/x/tools v0.42.0 // indirect + golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect modernc.org/libc v1.72.3 // indirect modernc.org/mathutil v1.7.1 // indirect modernc.org/memory v1.11.0 // indirect diff --git a/go.sum b/go.sum index f4aebaf..6c9cf5c 100644 --- a/go.sum +++ b/go.sum @@ -1,12 +1,33 @@ filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo= filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc= +github.com/apache/arrow-go/v18 v18.4.1 h1:q/jVkBWCJOB9reDgaIZIdruLQUb1kbkvOnOFezVH1C4= +github.com/apache/arrow-go/v18 v18.4.1/go.mod h1:tLyFubsAl17bvFdUAy24bsSvA/6ww95Iqi67fTpGu3E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= +github.com/duckdb/duckdb-go-bindings v0.1.21 h1:bOb/MXNT4PN5JBZ7wpNg6hrj9+cuDjWDa4ee9UdbVyI= +github.com/duckdb/duckdb-go-bindings v0.1.21/go.mod h1:pBnfviMzANT/9hi4bg+zW4ykRZZPCXlVuvBWEcZofkc= +github.com/duckdb/duckdb-go-bindings/darwin-amd64 v0.1.21 h1:Sjjhf2F/zCjPF53c2VXOSKk0PzieMriSoyr5wfvr9d8= +github.com/duckdb/duckdb-go-bindings/darwin-amd64 v0.1.21/go.mod h1:Ezo7IbAfB8NP7CqPIN8XEHKUg5xdRRQhcPPlCXImXYA= +github.com/duckdb/duckdb-go-bindings/darwin-arm64 v0.1.21 h1:IUk0FFUB6dpWLhlN9hY1mmdPX7Hkn3QpyrAmn8pmS8g= +github.com/duckdb/duckdb-go-bindings/darwin-arm64 v0.1.21/go.mod h1:eS7m/mLnPQgVF4za1+xTyorKRBuK0/BA44Oy6DgrGXI= +github.com/duckdb/duckdb-go-bindings/linux-amd64 v0.1.21 h1:Qpc7ZE3n6Nwz30KTvaAwI6nGkXjXmMxBTdFpC8zDEYI= +github.com/duckdb/duckdb-go-bindings/linux-amd64 v0.1.21/go.mod h1:1GOuk1PixiESxLaCGFhag+oFi7aP+9W8byymRAvunBk= +github.com/duckdb/duckdb-go-bindings/linux-arm64 v0.1.21 h1:eX2DhobAZOgjXkh8lPnKAyrxj8gXd2nm+K71f6KV/mo= +github.com/duckdb/duckdb-go-bindings/linux-arm64 v0.1.21/go.mod h1:o7crKMpT2eOIi5/FY6HPqaXcvieeLSqdXXaXbruGX7w= +github.com/duckdb/duckdb-go-bindings/windows-amd64 v0.1.21 h1:hhziFnGV7mpA+v5J5G2JnYQ+UWCCP3NQ+OTvxFX10D8= +github.com/duckdb/duckdb-go-bindings/windows-amd64 v0.1.21/go.mod h1:IlOhJdVKUJCAPj3QsDszUo8DVdvp1nBFp4TUJVdw99s= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw= github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk= +github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs= +github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= +github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4= +github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= +github.com/google/flatbuffers v25.2.10+incompatible h1:F3vclr7C3HpB1k9mxCGRMXq6FdUalZ6H/pNX4FP1v0Q= +github.com/google/flatbuffers v25.2.10+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8= github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs= github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= @@ -21,12 +42,25 @@ github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw= github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= +github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= +github.com/marcboeker/go-duckdb/arrowmapping v0.0.21 h1:geHnVjlsAJGczSWEqYigy/7ARuD+eBtjd0kLN80SPJQ= +github.com/marcboeker/go-duckdb/arrowmapping v0.0.21/go.mod h1:flFTc9MSqQCh2Xm62RYvG3Kyj29h7OtsTb6zUx1CdK8= +github.com/marcboeker/go-duckdb/mapping v0.0.21 h1:6woNXZn8EfYdc9Vbv0qR6acnt0TM1s1eFqnrJZVrqEs= +github.com/marcboeker/go-duckdb/mapping v0.0.21/go.mod h1:q3smhpLyv2yfgkQd7gGHMd+H/Z905y+WYIUjrl29vT4= +github.com/marcboeker/go-duckdb/v2 v2.4.3 h1:bHUkphPsAp2Bh/VFEdiprGpUekxBNZiWWtK+Bv/ljRk= +github.com/marcboeker/go-duckdb/v2 v2.4.3/go.mod h1:taim9Hktg2igHdNBmg5vgTfHAlV26z3gBI0QXQOcuyI= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU= +github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= @@ -34,6 +68,10 @@ github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UV github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0= +github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA= +golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 h1:R84qjqJb5nVJMxqWYb3np9L5ZsaDtB+a39EqjV0JSUM= +golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0/go.mod h1:S9Xr4PYopiDyqSyp5NjCrhFrqg6A5zA2E/iPHPhqnS8= golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8= golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w= golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= @@ -41,10 +79,14 @@ golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 h1:bTLqdHv7xrGlFbvf5/TXNxy/iUwwdkjhqQTJDjW7aj0= +golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4/go.mod h1:g5NllXBEermZrmR51cJDQxmJUHUOfRAaNyWBM+R+548= golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk= golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4= golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k= golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0= +golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da h1:noIWHXmPHxILtqtCOPIhSt0ABwskkZKjD3bXGnZGpNY= +golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= diff --git a/internal/cli/cli.go b/internal/cli/cli.go index fdfa863..e26cc31 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -347,10 +347,13 @@ Usage: database-cli ping ` case "list": - return `list List all open connection aliases. + return `list List all open connection aliases and whether each is read-only. Usage: database-cli list + +Output: + JSON array of {alias, readOnly} objects. ` case "open": return `open Open a new JDBC connection and register it under an alias. @@ -364,7 +367,9 @@ Flags: --user Database username --password-stdin Read password from stdin --password-keychain Look up password from macOS Keychain (service/account) - --read-only Block write operations (exec, begin, commit, rollback) on this alias + --read-only Make the alias read-only: the database session rejects writes and DDL, + exec/begin/commit/rollback are refused, and query accepts only + SELECT, WITH, SHOW, DESCRIBE/DESC and EXPLAIN ` case "close": return `close Close an open connection. @@ -388,6 +393,8 @@ Flags: Example: database-cli query --alias mydb 'SELECT * FROM users LIMIT 10' database-cli query --alias mydb --json 'SELECT id, name FROM users' + +Note: on a read-only alias only SELECT, WITH, SHOW, DESCRIBE/DESC and EXPLAIN are accepted. ` case "exec": return `exec Run a write SQL statement (INSERT, UPDATE, DELETE, DDL). @@ -476,7 +483,8 @@ Each stdin line is a JSON object with an "op" field: Output is one NDJSON result line per input line. -Note: blocked if the alias was opened with --read-only. +Note: on a read-only alias, exec/begin/commit/rollback ops are rejected and query ops +accept only SELECT, WITH, SHOW, DESCRIBE/DESC and EXPLAIN. ` default: return fmt.Sprintf("No help available for %q.\n", cmd) diff --git a/internal/daemon/duckdb.go b/internal/daemon/duckdb.go new file mode 100644 index 0000000..f9766f0 --- /dev/null +++ b/internal/daemon/duckdb.go @@ -0,0 +1,68 @@ +package daemon + +import ( + "fmt" + "regexp" + "strings" +) + +var attachNamePattern = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) + +// buildAttachStatements produces the SQL a duckdb composite alias runs at +// open time to make a source alias's database queryable. Credentials go +// into a TEMPORARY duckdb secret — in-memory only, scoped to the duckdb +// instance, never written to disk or echoed back. +func buildAttachStatements(name string, source connectionConfig, readOnly bool) ([]string, error) { + if !attachNamePattern.MatchString(name) { + return nil, fmt.Errorf("alias %q cannot be used as an ATTACH name (letters, digits, underscore only)", name) + } + + switch source.Dialect { + case dialectMySQL, dialectPostgres: + if source.Remote == nil { + return nil, fmt.Errorf("alias %q has no stored connection parameters", name) + } + secretType := "mysql" + if source.Dialect == dialectPostgres { + secretType = "postgres" + } + options := "TYPE " + secretType + ", SECRET secret_" + name + if readOnly { + options += ", READ_ONLY" + } + return []string{ + "INSTALL " + secretType, + "LOAD " + secretType, + fmt.Sprintf( + "CREATE TEMPORARY SECRET secret_%s (TYPE %s, HOST %s, PORT %s, DATABASE %s, USER %s, PASSWORD %s)", + name, + secretType, + duckdbStringLiteral(source.Remote.Host), + source.Remote.Port, + duckdbStringLiteral(source.Remote.Database), + duckdbStringLiteral(source.Remote.User), + duckdbStringLiteral(source.Remote.Password), + ), + fmt.Sprintf("ATTACH '' AS %s (%s)", name, options), + }, nil + case dialectSQLite: + if source.SingleConnection { + return nil, fmt.Errorf("alias %q is an in-memory sqlite database and cannot be attached", name) + } + options := "TYPE sqlite" + if readOnly { + options += ", READ_ONLY" + } + return []string{ + "INSTALL sqlite", + "LOAD sqlite", + fmt.Sprintf("ATTACH %s AS %s (%s)", duckdbStringLiteral(source.DSN), name, options), + }, nil + default: + return nil, fmt.Errorf("alias %q has dialect %s, which cannot be attached to duckdb", name, source.Dialect) + } +} + +func duckdbStringLiteral(value string) string { + return "'" + strings.ReplaceAll(value, "'", "''") + "'" +} diff --git a/internal/daemon/duckdb_test.go b/internal/daemon/duckdb_test.go new file mode 100644 index 0000000..89baebf --- /dev/null +++ b/internal/daemon/duckdb_test.go @@ -0,0 +1,195 @@ +package daemon + +import ( + "encoding/json" + "strings" + "testing" + + "github.com/scriptease/database-cli/internal/protocol" +) + +func TestBuildAttachStatementsMySQL(t *testing.T) { + source := connectionConfig{ + Dialect: dialectMySQL, + Remote: &remoteConfig{Host: "db.example.com", Port: "3306", Database: "shop", User: "alice", Password: "it's"}, + } + + statements, err := buildAttachStatements("shop", source, true) + if err != nil { + t.Fatalf("buildAttachStatements() error = %v", err) + } + if len(statements) != 4 { + t.Fatalf("statements = %d, want 4", len(statements)) + } + secret := statements[2] + if !strings.Contains(secret, "TYPE mysql") || !strings.Contains(secret, "HOST 'db.example.com'") || !strings.Contains(secret, "PASSWORD 'it''s'") { + t.Fatalf("secret statement = %q, want mysql secret with escaped password", secret) + } + attach := statements[3] + if attach != "ATTACH '' AS shop (TYPE mysql, SECRET secret_shop, READ_ONLY)" { + t.Fatalf("attach statement = %q", attach) + } +} + +func TestBuildAttachStatementsRejectsBadName(t *testing.T) { + source := connectionConfig{ + Dialect: dialectPostgres, + Remote: &remoteConfig{Host: "h", Port: "5432", Database: "d", User: "u", Password: "p"}, + } + if _, err := buildAttachStatements("bad-name", source, false); err == nil { + t.Fatal("expected error for hyphenated attach name") + } +} + +func TestBuildAttachStatementsRejectsMemorySQLite(t *testing.T) { + source := connectionConfig{Dialect: dialectSQLite, DSN: ":memory:", SingleConnection: true} + if _, err := buildAttachStatements("mem", source, false); err == nil { + t.Fatal("expected error for in-memory sqlite source") + } +} + +func TestStoreDuckDBLifecycle(t *testing.T) { + store := newStore() + t.Cleanup(store.CloseAll) + + if err := store.Open(protocol.OpenRequest{ + Alias: "duck", + JDBCURL: "jdbc:duckdb:", + }); err != nil { + t.Fatalf("Open() error = %v", err) + } + + if _, err := store.Exec("duck", "create table answers(id integer, label text)"); err != nil { + t.Fatalf("Exec(create table) error = %v", err) + } + if _, err := store.Exec("duck", "insert into answers values (42, 'life')"); err != nil { + t.Fatalf("Exec(insert) error = %v", err) + } + + tsvBody, _, err := store.Query("duck", "select id, label from answers", false) + if err != nil { + t.Fatalf("Query(tsv) error = %v", err) + } + if string(tsvBody) != "id\tlabel\n42\tlife\n" { + t.Fatalf("Query(tsv) = %q", string(tsvBody)) + } + + schemaBody, err := store.Schema("duck") + if err != nil { + t.Fatalf("Schema() error = %v", err) + } + var tables []string + if err := json.Unmarshal(schemaBody, &tables); err != nil { + t.Fatalf("json.Unmarshal(schema) error = %v", err) + } + if len(tables) != 1 || !strings.HasSuffix(tables[0], ".answers") { + t.Fatalf("schema tables = %#v, want [...answers]", tables) + } + + describeBody, err := store.Describe("duck", "answers") + if err != nil { + t.Fatalf("Describe() error = %v", err) + } + var columns []describeColumn + if err := json.Unmarshal(describeBody, &columns); err != nil { + t.Fatalf("json.Unmarshal(describe) error = %v", err) + } + if len(columns) != 2 || columns[0].Name != "id" || columns[1].Name != "label" { + t.Fatalf("describe columns = %#v", columns) + } + + if err := store.Close("duck"); err != nil { + t.Fatalf("Close() error = %v", err) + } +} + +// TestStoreDuckDBAttachSQLite exercises the full composite flow with a +// sqlite file standing in for a remote database. Requires the duckdb +// sqlite extension; skips when it cannot be installed (e.g. offline). +func TestStoreDuckDBAttachSQLite(t *testing.T) { + store := newStore() + t.Cleanup(store.CloseAll) + + dbPath := t.TempDir() + "/source.db" + if err := store.Open(protocol.OpenRequest{Alias: "src", JDBCURL: "jdbc:sqlite:" + dbPath}); err != nil { + t.Fatalf("Open(sqlite) error = %v", err) + } + if _, err := store.Exec("src", "create table users(id integer primary key, name text)"); err != nil { + t.Fatalf("Exec(create table) error = %v", err) + } + if _, err := store.Exec("src", "insert into users values (1, 'Ada')"); err != nil { + t.Fatalf("Exec(insert) error = %v", err) + } + + if err := store.Open(protocol.OpenRequest{ + Alias: "duck", + JDBCURL: "jdbc:duckdb:?attach=src", + }); err != nil { + if strings.Contains(err.Error(), "attach src") { + t.Skipf("duckdb sqlite extension unavailable: %v", err) + } + t.Fatalf("Open(duckdb attach) error = %v", err) + } + + tsvBody, _, err := store.Query("duck", "select name from src.users where id = 1", false) + if err != nil { + t.Fatalf("Query(attached) error = %v", err) + } + if string(tsvBody) != "name\nAda\n" { + t.Fatalf("Query(attached) = %q, want Ada", string(tsvBody)) + } +} + +func TestStoreDuckDBReadOnlyFile(t *testing.T) { + store := newStore() + t.Cleanup(store.CloseAll) + + jdbcURL := "jdbc:duckdb:" + t.TempDir() + "/ro.duckdb" + if err := store.Open(protocol.OpenRequest{Alias: "rw", JDBCURL: jdbcURL}); err != nil { + t.Fatalf("Open(rw) error = %v", err) + } + if _, err := store.Exec("rw", "create table answers(id integer, label text)"); err != nil { + t.Fatalf("Exec(create table) error = %v", err) + } + if err := store.Close("rw"); err != nil { + t.Fatalf("Close(rw) error = %v", err) + } + + if err := store.Open(protocol.OpenRequest{Alias: "ro", JDBCURL: jdbcURL, ReadOnly: true}); err != nil { + t.Fatalf("Open(ro) error = %v", err) + } + requireReadOnlyAlias(t, store, "ro", "select label from answers", + "create or replace view answers_view as select * from answers", + "update answers set label = 'changed'", + ) + if _, err := store.Describe("ro", "answers"); err != nil { + t.Fatalf("Describe(ro) error = %v", err) + } +} + +// TestStoreDuckDBReadOnlyAttachesSourcesReadOnly covers an in-memory +// composite, which duckdb cannot open read-only: its sources must be +// attached READ_ONLY even when they were opened writable. +func TestStoreDuckDBReadOnlyAttachesSourcesReadOnly(t *testing.T) { + store := newStore() + t.Cleanup(store.CloseAll) + + dbPath := t.TempDir() + "/source.db" + if err := store.Open(protocol.OpenRequest{Alias: "src", JDBCURL: "jdbc:sqlite:" + dbPath}); err != nil { + t.Fatalf("Open(sqlite) error = %v", err) + } + if _, err := store.Exec("src", "create table users(id integer primary key, name text)"); err != nil { + t.Fatalf("Exec(create table) error = %v", err) + } + + if err := store.Open(protocol.OpenRequest{Alias: "duck", JDBCURL: "jdbc:duckdb:?attach=src", ReadOnly: true}); err != nil { + if strings.Contains(err.Error(), "attach src") { + t.Skipf("duckdb sqlite extension unavailable: %v", err) + } + t.Fatalf("Open(duckdb attach) error = %v", err) + } + requireReadOnlyAlias(t, store, "duck", "select name from src.users", + "create view src.users_view as select * from src.users", + "update src.users set name = 'changed'", + ) +} diff --git a/internal/daemon/jdbc.go b/internal/daemon/jdbc.go index 2b6856f..f9b385c 100644 --- a/internal/daemon/jdbc.go +++ b/internal/daemon/jdbc.go @@ -15,13 +15,27 @@ const ( dialectMySQL dialect = "mysql" dialectPostgres dialect = "postgres" dialectSQLite dialect = "sqlite" + dialectDuckDB dialect = "duckdb" ) +// remoteConfig keeps the parsed connection parameters of a mysql/postgres +// alias so a duckdb composite alias can ATTACH the same database via a +// temporary in-memory secret (see duckdb.go). +type remoteConfig struct { + Host string + Port string + Database string + User string + Password string +} + type connectionConfig struct { DriverName string DSN string Dialect dialect SingleConnection bool + Remote *remoteConfig + AttachAliases []string } func parseJDBCURL(jdbcURL string, user string, password string) (connectionConfig, error) { @@ -32,6 +46,8 @@ func parseJDBCURL(jdbcURL string, user string, password string) (connectionConfi return parsePostgresJDBC(jdbcURL, user, password) case strings.HasPrefix(jdbcURL, "jdbc:sqlite:"): return parseSQLiteJDBC(jdbcURL) + case strings.HasPrefix(jdbcURL, "jdbc:duckdb:"): + return parseDuckDBJDBC(jdbcURL) default: return connectionConfig{}, fmt.Errorf("unsupported jdbcUrl: %s", jdbcURL) } @@ -82,6 +98,13 @@ func parseMySQLJDBC(jdbcURL string, user string, password string) (connectionCon DriverName: "mysql", DSN: cfg.FormatDSN(), Dialect: dialectMySQL, + Remote: &remoteConfig{ + Host: parsed.Hostname(), + Port: port, + Database: cfg.DBName, + User: cfg.User, + Password: cfg.Passwd, + }, }, nil } @@ -119,10 +142,22 @@ func parsePostgresJDBC(jdbcURL string, user string, password string) (connection } } + port := parsed.Port() + if port == "" { + port = "5432" + } + return connectionConfig{ DriverName: "pgx", DSN: parsed.String(), Dialect: dialectPostgres, + Remote: &remoteConfig{ + Host: parsed.Hostname(), + Port: port, + Database: strings.TrimPrefix(parsed.Path, "/"), + User: finalUser, + Password: finalPassword, + }, }, nil } @@ -147,3 +182,45 @@ func sqliteNeedsSingleConnection(dsn string) bool { normalized := strings.ToLower(strings.TrimSpace(dsn)) return normalized == ":memory:" || strings.Contains(normalized, "mode=memory") } + +// parseDuckDBJDBC parses jdbc:duckdb:[][?attach=alias1,alias2]. +// An empty path (or :memory:) opens an in-memory database. The attach +// parameter names already-open aliases whose databases are ATTACHed into +// the duckdb instance at open time for cross-database joins. +func parseDuckDBJDBC(jdbcURL string) (connectionConfig, error) { + rest := strings.TrimPrefix(jdbcURL, "jdbc:duckdb:") + + query := "" + if idx := strings.Index(rest, "?"); idx >= 0 { + query = rest[idx+1:] + rest = rest[:idx] + } + + dsn := rest + if dsn == ":memory:" { + dsn = "" + } + + attachAliases := []string{} + if query != "" { + values, err := url.ParseQuery(query) + if err != nil { + return connectionConfig{}, fmt.Errorf("invalid duckdb jdbcUrl query: %w", err) + } + for _, value := range values["attach"] { + for _, alias := range strings.Split(value, ",") { + alias = strings.TrimSpace(alias) + if alias != "" { + attachAliases = append(attachAliases, alias) + } + } + } + } + + return connectionConfig{ + DriverName: "duckdb", + DSN: dsn, + Dialect: dialectDuckDB, + AttachAliases: attachAliases, + }, nil +} diff --git a/internal/daemon/jdbc_test.go b/internal/daemon/jdbc_test.go index 4b8c5f1..7b05248 100644 --- a/internal/daemon/jdbc_test.go +++ b/internal/daemon/jdbc_test.go @@ -52,6 +52,67 @@ func TestParsePostgresJDBC(t *testing.T) { } } +func TestParseDuckDBJDBC(t *testing.T) { + cfg, err := parseJDBCURL("jdbc:duckdb:?attach=shop, orders", "", "") + if err != nil { + t.Fatalf("parseJDBCURL() error = %v", err) + } + + if cfg.DriverName != "duckdb" { + t.Fatalf("driver = %q, want duckdb", cfg.DriverName) + } + if cfg.Dialect != dialectDuckDB { + t.Fatalf("dialect = %q, want %q", cfg.Dialect, dialectDuckDB) + } + if cfg.DSN != "" { + t.Fatalf("dsn = %q, want empty (in-memory)", cfg.DSN) + } + if len(cfg.AttachAliases) != 2 || cfg.AttachAliases[0] != "shop" || cfg.AttachAliases[1] != "orders" { + t.Fatalf("attachAliases = %#v, want [shop orders]", cfg.AttachAliases) + } + + fileCfg, err := parseJDBCURL("jdbc:duckdb:/tmp/analytics.duckdb", "", "") + if err != nil { + t.Fatalf("parseJDBCURL(file) error = %v", err) + } + if fileCfg.DSN != "/tmp/analytics.duckdb" { + t.Fatalf("file dsn = %q, want /tmp/analytics.duckdb", fileCfg.DSN) + } + if len(fileCfg.AttachAliases) != 0 { + t.Fatalf("file attachAliases = %#v, want none", fileCfg.AttachAliases) + } + + memCfg, err := parseJDBCURL("jdbc:duckdb::memory:", "", "") + if err != nil { + t.Fatalf("parseJDBCURL(:memory:) error = %v", err) + } + if memCfg.DSN != "" { + t.Fatalf(":memory: dsn = %q, want empty", memCfg.DSN) + } +} + +func TestParseMySQLJDBCRemoteConfig(t *testing.T) { + cfg, err := parseJDBCURL("jdbc:mysql://db.example.com/app", "alice", "secret") + if err != nil { + t.Fatalf("parseJDBCURL() error = %v", err) + } + want := remoteConfig{Host: "db.example.com", Port: "3306", Database: "app", User: "alice", Password: "secret"} + if cfg.Remote == nil || *cfg.Remote != want { + t.Fatalf("remote = %#v, want %#v", cfg.Remote, want) + } +} + +func TestParsePostgresJDBCRemoteConfig(t *testing.T) { + cfg, err := parseJDBCURL("jdbc:postgresql://db.example.com/app", "alice", "secret") + if err != nil { + t.Fatalf("parseJDBCURL() error = %v", err) + } + want := remoteConfig{Host: "db.example.com", Port: "5432", Database: "app", User: "alice", Password: "secret"} + if cfg.Remote == nil || *cfg.Remote != want { + t.Fatalf("remote = %#v, want %#v", cfg.Remote, want) + } +} + func TestParseSQLiteMemoryUsesSingleConnection(t *testing.T) { cfg, err := parseJDBCURL("jdbc:sqlite::memory:", "", "") if err != nil { diff --git a/internal/daemon/metadata.go b/internal/daemon/metadata.go index bb2618a..2751967 100644 --- a/internal/daemon/metadata.go +++ b/internal/daemon/metadata.go @@ -58,6 +58,9 @@ func schemaQuery(d dialect) (string, []any) { return "SELECT table_name FROM information_schema.tables WHERE table_schema = current_schema() AND table_type = 'BASE TABLE' ORDER BY table_name", nil case dialectSQLite: return "SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name", nil + case dialectDuckDB: + // Fully qualified names so tables of ATTACHed databases are listed too. + return `SELECT database || '.' || schema || '.' || name FROM (SHOW ALL TABLES) ORDER BY 1`, nil default: return "", nil } @@ -71,6 +74,8 @@ func describeColumns(state *aliasState, table string) ([]describeColumn, error) return describeInformationSchema(state, table, "SELECT column_name, udt_name, COALESCE(character_maximum_length, numeric_precision, datetime_precision, 0) AS size, CASE WHEN is_nullable = 'YES' THEN 1 ELSE 0 END AS nullable FROM information_schema.columns WHERE table_schema = current_schema() AND table_name = $1 ORDER BY ordinal_position") case dialectSQLite: return describeSQLite(state, table) + case dialectDuckDB: + return describeDuckDB(state, table) default: return nil, fmt.Errorf("unsupported dialect: %s", state.dialect) } @@ -105,6 +110,41 @@ func describeInformationSchema(state *aliasState, table string, query string) ([ return columns, nil } +// describeDuckDB uses PRAGMA table_info with a string argument, which +// resolves qualified names like attached_db.schema.table. +func describeDuckDB(state *aliasState, table string) ([]describeColumn, error) { + query := fmt.Sprintf(`PRAGMA table_info(%s)`, duckdbStringLiteral(table)) + + rows, err := state.queryer().QueryContext(context.Background(), query) + if err != nil { + return nil, err + } + defer rows.Close() + + columns := make([]describeColumn, 0) + for rows.Next() { + var cid int64 + var name string + var typeName string + var notNull bool + var defaultValue sql.NullString + var pk bool + if err := rows.Scan(&cid, &name, &typeName, ¬Null, &defaultValue, &pk); err != nil { + return nil, err + } + columns = append(columns, describeColumn{ + Name: name, + Type: typeName, + Size: sqliteColumnSize(typeName), + Nullable: !notNull, + }) + } + if err := rows.Err(); err != nil { + return nil, err + } + return columns, nil +} + func describeSQLite(state *aliasState, table string) ([]describeColumn, error) { quotedTable := strings.ReplaceAll(table, `"`, `""`) query := fmt.Sprintf(`PRAGMA table_info("%s")`, quotedTable) diff --git a/internal/daemon/readonly.go b/internal/daemon/readonly.go new file mode 100644 index 0000000..1279b2d --- /dev/null +++ b/internal/daemon/readonly.go @@ -0,0 +1,136 @@ +package daemon + +import ( + "context" + "database/sql" + "database/sql/driver" + "fmt" + "net/url" + "strings" + "unicode" + + "github.com/go-sql-driver/mysql" +) + +// openDB opens the connection pool of an alias. For a read-only alias the +// database itself rejects writes and DDL on every pooled connection: +// mysql runs SET SESSION TRANSACTION READ ONLY per connection, postgres +// sets default_transaction_read_only, sqlite opens with mode=ro and a +// file-backed duckdb with access_mode=read_only. An in-memory duckdb cannot +// be opened read-only; its attached sources are attached READ_ONLY instead. +func openDB(config connectionConfig, readOnly bool) (*sql.DB, error) { + if !readOnly { + return sql.Open(config.DriverName, config.DSN) + } + + switch config.Dialect { + case dialectMySQL: + cfg, err := mysql.ParseDSN(config.DSN) + if err != nil { + return nil, err + } + cfg.MultiStatements = false + connector, err := mysql.NewConnector(cfg) + if err != nil { + return nil, err + } + return sql.OpenDB(mysqlReadOnlyConnector{connector}), nil + case dialectPostgres: + parsed, err := url.Parse(config.DSN) + if err != nil { + return nil, err + } + query := parsed.Query() + query.Set("default_transaction_read_only", "on") + parsed.RawQuery = query.Encode() + return sql.Open(config.DriverName, parsed.String()) + case dialectSQLite: + return sql.Open(config.DriverName, sqliteReadOnlyDSN(config.DSN)) + case dialectDuckDB: + if config.DSN == "" { + return sql.Open(config.DriverName, config.DSN) + } + return sql.Open(config.DriverName, config.DSN+"?access_mode=read_only") + default: + return nil, fmt.Errorf("read-only is not supported for dialect %s", config.Dialect) + } +} + +// mysqlReadOnlyConnector switches every new connection to read-only +// transactions before database/sql hands it out. +type mysqlReadOnlyConnector struct { + driver.Connector +} + +func (c mysqlReadOnlyConnector) Connect(ctx context.Context) (driver.Conn, error) { + conn, err := c.Connector.Connect(ctx) + if err != nil { + return nil, err + } + if _, err := conn.(driver.ExecerContext).ExecContext(ctx, "SET SESSION TRANSACTION READ ONLY", nil); err != nil { + _ = conn.Close() + return nil, err + } + return conn, nil +} + +// sqliteReadOnlyDSN turns a sqlite path into a mode=ro URI. The sqlite +// driver only passes query parameters on to sqlite for file: DSNs. +func sqliteReadOnlyDSN(dsn string) string { + if !strings.HasPrefix(dsn, "file:") { + dsn = "file:" + dsn + } + if strings.Contains(dsn, "?") { + return dsn + "&mode=ro" + } + return dsn + "?mode=ro" +} + +var readStatementKeywords = map[string]bool{ + "SELECT": true, + "WITH": true, + "SHOW": true, + "DESCRIBE": true, + "DESC": true, + "EXPLAIN": true, +} + +// requireReadStatement admits only statements whose leading keyword reads. +// It sits in front of the read-only session (openDB), which still rejects +// writes behind an allowed keyword, such as a data-modifying CTE. +func requireReadStatement(alias string, sqlText string) error { + if !readStatementKeywords[strings.ToUpper(leadingKeyword(sqlText))] { + return fmt.Errorf("alias is read-only: %s (query accepts only SELECT, WITH, SHOW, DESCRIBE, EXPLAIN)", alias) + } + return nil +} + +// leadingKeyword returns the first word of sqlText, skipping whitespace, +// opening parentheses and comments. MySQL /*! ... */ comments are not +// skipped because the server executes their content. +func leadingKeyword(sqlText string) string { + rest := sqlText + for { + rest = strings.TrimLeft(rest, " \t\r\n(") + switch { + case strings.HasPrefix(rest, "--"): + end := strings.IndexByte(rest, '\n') + if end < 0 { + return "" + } + rest = rest[end+1:] + case strings.HasPrefix(rest, "/*") && !strings.HasPrefix(rest, "/*!"): + end := strings.Index(rest, "*/") + if end < 0 { + return "" + } + rest = rest[end+2:] + default: + end := strings.IndexFunc(rest, func(r rune) bool { return !unicode.IsLetter(r) }) + if end < 0 { + return rest + } + return rest[:end] + } + } +} diff --git a/internal/daemon/readonly_test.go b/internal/daemon/readonly_test.go new file mode 100644 index 0000000..236fad9 --- /dev/null +++ b/internal/daemon/readonly_test.go @@ -0,0 +1,144 @@ +package daemon + +import ( + "context" + "database/sql" + "os" + "strings" + "testing" + + "github.com/scriptease/database-cli/internal/protocol" +) + +func TestRequireReadStatement(t *testing.T) { + allowed := []string{ + "SELECT 1", + " select 1", + "(SELECT 1) UNION (SELECT 2)", + "-- note\nSELECT 1", + "/* note */ WITH x AS (SELECT 1) SELECT * FROM x", + "SHOW TABLES", + "DESCRIBE items", + "desc items", + "EXPLAIN SELECT 1", + } + for _, sqlText := range allowed { + if err := requireReadStatement("ro", sqlText); err != nil { + t.Errorf("requireReadStatement(%q) error = %v, want nil", sqlText, err) + } + } + + rejected := []string{ + "CREATE OR REPLACE VIEW Branch_view AS SELECT 1", + "update items set name = 'x'", + "INSERT INTO items VALUES (1)", + "DROP TABLE items", + "SET SESSION TRANSACTION READ WRITE", + "/*!50001 CREATE VIEW v AS */ SELECT 1", + "-- only a comment", + "SELECTED 1", + "", + } + for _, sqlText := range rejected { + err := requireReadStatement("ro", sqlText) + if err == nil || !strings.Contains(err.Error(), "alias is read-only: ro") { + t.Errorf("requireReadStatement(%q) error = %v, want read-only error", sqlText, err) + } + } +} + +func TestReadOnlyMySQL(t *testing.T) { + testReadOnlyServer(t, "DATABASE_CLI_TEST_MYSQL_JDBC_URL") +} + +func TestReadOnlyPostgres(t *testing.T) { + testReadOnlyServer(t, "DATABASE_CLI_TEST_POSTGRES_JDBC_URL") +} + +// testReadOnlyServer needs a disposable server: envVar holds its JDBC URL +// with credentials, e.g. jdbc:mysql://root@127.0.0.1:3306/test. The test +// creates and drops the table database_cli_ro_items there. +func testReadOnlyServer(t *testing.T, envVar string) { + jdbcURL := os.Getenv(envVar) + if jdbcURL == "" { + t.Skipf("%s not set", envVar) + } + + store := newStore() + t.Cleanup(store.CloseAll) + + if err := store.Open(protocol.OpenRequest{Alias: "rw", JDBCURL: jdbcURL}); err != nil { + t.Fatalf("Open(rw) error = %v", err) + } + for _, statement := range []string{ + "DROP VIEW IF EXISTS database_cli_ro_view", + "DROP TABLE IF EXISTS database_cli_ro_items", + "CREATE TABLE database_cli_ro_items(id int primary key, name varchar(20))", + "INSERT INTO database_cli_ro_items VALUES (1, 'Ada')", + } { + if _, err := store.Exec("rw", statement); err != nil { + t.Fatalf("Exec(%q) error = %v", statement, err) + } + } + t.Cleanup(func() { _, _ = store.Exec("rw", "DROP TABLE IF EXISTS database_cli_ro_items") }) + + if err := store.Open(protocol.OpenRequest{Alias: "ro", JDBCURL: jdbcURL, ReadOnly: true}); err != nil { + t.Fatalf("Open(ro) error = %v", err) + } + requireReadOnlyAlias(t, store, "ro", "SELECT name FROM database_cli_ro_items", + "CREATE OR REPLACE VIEW database_cli_ro_view AS SELECT 1 AS x", + "CREATE VIEW database_cli_ro_view AS SELECT 1 AS x", + "CREATE TABLE database_cli_ro_other(id int)", + "UPDATE database_cli_ro_items SET name = 'changed'", + ) +} + +// requireReadOnlyAlias checks both read-only layers of alias: query rejects +// the writes, and the database rejects them on several pooled connections +// at once when the query guard is bypassed. The read must still work. +func requireReadOnlyAlias(t *testing.T, store *Store, alias string, read string, writes ...string) { + t.Helper() + + if _, _, err := store.Query(alias, read, false); err != nil { + t.Fatalf("Query(%q) error = %v", read, err) + } + for _, statement := range writes { + if _, _, err := store.Query(alias, statement, false); err == nil || !strings.Contains(err.Error(), "alias is read-only") { + t.Fatalf("Query(%q) error = %v, want read-only error", statement, err) + } + } + + _, state, err := store.getState(alias) + if err != nil { + t.Fatalf("getState(%s) error = %v", alias, err) + } + requireDBRejects(t, state.db, writes...) +} + +func requireDBRejects(t *testing.T, db *sql.DB, statements ...string) { + t.Helper() + + ctx := context.Background() + conns := make([]*sql.Conn, 3) + for i := range conns { + conn, err := db.Conn(ctx) + if err != nil { + t.Fatalf("Conn() error = %v", err) + } + t.Cleanup(func() { _ = conn.Close() }) + conns[i] = conn + } + + for i, conn := range conns { + for _, statement := range statements { + _, err := conn.ExecContext(ctx, statement) + if err == nil { + t.Fatalf("conn %d: %q succeeded on a read-only alias", i, statement) + } + message := strings.ToLower(err.Error()) + if !strings.Contains(message, "read-only") && !strings.Contains(message, "readonly") && !strings.Contains(message, "read only") { + t.Fatalf("conn %d: %q error = %v, want a read-only error from the database", i, statement, err) + } + } + } +} diff --git a/internal/daemon/session.go b/internal/daemon/session.go index db2f111..55ec404 100644 --- a/internal/daemon/session.go +++ b/internal/daemon/session.go @@ -12,6 +12,11 @@ func (a *aliasState) query(alias string, sqlText string, jsonMode bool) ([]byte, if err := a.requireOpen(alias); err != nil { return nil, "", err } + if a.readOnly { + if err := requireReadStatement(alias, sqlText); err != nil { + return nil, "", err + } + } rows, err := a.queryer().QueryContext(context.Background(), sqlText) if err != nil { diff --git a/internal/daemon/store.go b/internal/daemon/store.go index 573c692..9eda2e3 100644 --- a/internal/daemon/store.go +++ b/internal/daemon/store.go @@ -1,6 +1,7 @@ package daemon import ( + "context" "database/sql" "fmt" "sort" @@ -10,6 +11,7 @@ import ( _ "github.com/go-sql-driver/mysql" _ "github.com/jackc/pgx/v5/stdlib" + _ "github.com/marcboeker/go-duckdb/v2" _ "modernc.org/sqlite" "github.com/scriptease/database-cli/internal/protocol" @@ -26,6 +28,7 @@ type aliasState struct { tx *sql.Tx readOnly bool dialect dialect + config connectionConfig closed bool } @@ -35,17 +38,22 @@ func newStore() *Store { } } -func (s *Store) List() []string { +type aliasInfo struct { + Alias string `json:"alias"` + ReadOnly bool `json:"readOnly"` +} + +func (s *Store) List() []aliasInfo { s.mu.RLock() defer s.mu.RUnlock() - aliases := make([]string, 0, len(s.aliases)) + aliases := make([]aliasInfo, 0, len(s.aliases)) for alias, state := range s.aliases { if state != nil && !state.closed { - aliases = append(aliases, alias) + aliases = append(aliases, aliasInfo{Alias: alias, ReadOnly: state.readOnly}) } } - sort.Strings(aliases) + sort.Slice(aliases, func(i, j int) bool { return aliases[i].Alias < aliases[j].Alias }) return aliases } @@ -71,7 +79,7 @@ func (s *Store) Open(req protocol.OpenRequest) error { return err } - db, err := sql.Open(config.DriverName, config.DSN) + db, err := openDB(config, req.ReadOnly) if err != nil { return err } @@ -90,10 +98,22 @@ func (s *Store) Open(req protocol.OpenRequest) error { return err } + if len(config.AttachAliases) > 0 { + if config.Dialect != dialectDuckDB { + _ = db.Close() + return fmt.Errorf("attach is only supported for duckdb aliases") + } + if err := s.attachSources(db, config.AttachAliases, req.ReadOnly); err != nil { + _ = db.Close() + return err + } + } + state := &aliasState{ db: db, readOnly: req.ReadOnly, dialect: config.Dialect, + config: config, } s.mu.Lock() @@ -106,6 +126,42 @@ func (s *Store) Open(req protocol.OpenRequest) error { return nil } +// attachSources ATTACHes each named open alias into a freshly opened duckdb +// database. All statements run on one pinned connection so INSTALL/LOAD, +// CREATE TEMPORARY SECRET, and ATTACH execute in order; the resulting +// catalogs and secrets are instance-wide, so every pooled connection sees +// them afterwards. A read-only composite attaches every source READ_ONLY. +func (s *Store) attachSources(db *sql.DB, attachAliases []string, readOnly bool) error { + conn, err := db.Conn(context.Background()) + if err != nil { + return err + } + defer conn.Close() + + for _, name := range attachAliases { + _, source, err := s.getState(name) + if err != nil { + return err + } + + source.mu.Lock() + sourceConfig := source.config + sourceReadOnly := source.readOnly || readOnly + source.mu.Unlock() + + statements, err := buildAttachStatements(name, sourceConfig, sourceReadOnly) + if err != nil { + return err + } + for _, statement := range statements { + if _, err := conn.ExecContext(context.Background(), statement); err != nil { + return fmt.Errorf("attach %s: %w", name, err) + } + } + } + return nil +} + func (s *Store) Close(alias string) error { trimmedAlias, state, err := s.getState(alias) if err != nil { diff --git a/internal/daemon/store_test.go b/internal/daemon/store_test.go index b3da714..0b745c8 100644 --- a/internal/daemon/store_test.go +++ b/internal/daemon/store_test.go @@ -3,6 +3,7 @@ package daemon import ( "encoding/json" "path/filepath" + "reflect" "strings" "testing" @@ -171,6 +172,15 @@ func TestStoreReadOnlyRejectsMutationsAndCloseRejectsActiveTransaction(t *testin if err := store.Rollback("ro"); err == nil || !strings.Contains(err.Error(), "read-only") { t.Fatalf("Rollback(ro) error = %v, want read-only error", err) } + requireReadOnlyAlias(t, store, "ro", "select name from items", + "create view items_view as select * from items", + "update items set name = 'changed'", + ) + + want := []aliasInfo{{Alias: "ro", ReadOnly: true}, {Alias: "rw", ReadOnly: false}} + if got := store.List(); !reflect.DeepEqual(got, want) { + t.Fatalf("List() = %#v, want %#v", got, want) + } } func TestBatchQueryHonorsJSONFlag(t *testing.T) { diff --git a/scripts/database-cli.entitlements b/scripts/database-cli.entitlements new file mode 100644 index 0000000..b318cdf --- /dev/null +++ b/scripts/database-cli.entitlements @@ -0,0 +1,11 @@ + + + + + + com.apple.security.cs.disable-library-validation + + + diff --git a/scripts/install.sh b/scripts/install.sh index 6caf3db..914307f 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -30,13 +30,16 @@ mkdir -p "$ROOT/build" "$APP_HOME" "$HOME/Library/LaunchAgents" "$LOG_DIR" "$(di ( cd "$ROOT" - CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o "$BUILD_PATH" . + # CGO is required by the duckdb driver (statically linked libduckdb); + # the binary remains self-contained. + CGO_ENABLED=1 go build -trimpath -ldflags="-s -w" -o "$BUILD_PATH" . ) codesign --force \ --sign "${SIGN_IDENTITY:-Apple Distribution: CaperWhite GmbH (QVMS9F7W76)}" \ --timestamp \ --options runtime \ + --entitlements "$ROOT/scripts/database-cli.entitlements" \ --identifier "com.scriptease.database-cli" \ "$BUILD_PATH" diff --git a/skill/SKILL.md b/skill/SKILL.md index 9c44dcf..f82db97 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -65,7 +65,8 @@ database-cli open --alias prod-shop \ --user root \ --password-stdin # OR --password-keychain -# Open read-only — exec/begin/commit/rollback are rejected by the daemon for this alias +# Open read-only: the database session rejects writes and DDL, exec/begin/commit/rollback +# are refused, and query accepts only SELECT/WITH/SHOW/DESCRIBE/EXPLAIN database-cli open --alias prod-shop-ro \ --jdbc-url jdbc:mysql://localhost:3306/shop \ --user root \ @@ -87,7 +88,7 @@ database-cli describe --alias prod-shop --table users # columns database-cli close --alias prod-shop # State -database-cli list # active aliases +database-cli list # active aliases + readOnly flag ``` ## Transactions @@ -176,7 +177,7 @@ connection. | Pitfall | Correct approach | | ---------------------------------------- | ---------------------------------------------------------- | | `--password 'secret'` on argv | Visible in `ps`. Use `--password-stdin` or `--password-keychain`. | -| `exec` on a read-only alias | Daemon returns `{"error":"alias is read-only: …"}`. Open with `--read-only` intentionally to enforce safety; don't use it if you need writes. | +| `exec`, or a write/DDL via `query`, on a read-only alias | Daemon returns `{"error":"alias is read-only: …"}`; the database session rejects writes too. Open with `--read-only` intentionally to enforce safety; don't use it if you need writes. | | Forgetting `close` | Keeps the alias and pool open. Always pair `open` / `close`. | | Transaction across two terminals | One alias = one tx state. Use the same alias for all steps.| | Wrapper missing after install | Re-run `bash scripts/install.sh` to rebuild and reinstall. |