[Plugins] Should the plugin package be moved out of internal?

[linux2647]

Version

1.14.0

What happened?

I wanted to play around with authoring a plugin, but after reading the documentation and looking at the example code, I realized that the plugin package, which contains the protobuf-emitted code, lives under internal. By definition this means that no Go code outside of sqlc can import it. I saw that the ultimate goal is to have WASM-based plugins, but #1578 indicates that development on that has been stalled, opting for process-based plugins for the time being.

Am I missing something obvious that makes it possible to write a process-based plugin? Or does the plugin package need to be moved out of internal?

Relevant log output

No response

Database schema

No response

SQL queries

No response

Configuration

No response

Playground URL

No response

What operating system are you using?

No response

What database engines are you using?

No response

What type of code are you generating?

No response

[kyleconroy]

My comment in #1578 is stale. WASM-plugins are close (#1684) and will be available in 0.15.0. Process-based plugins exist as an escape hatch for plugin developers that can't use a language that targets WASM. It's not listed from in version 0.14.0 of the docs, but I've written a guide for authoring plugins. Can you give that a read over? Curious if it answers your questions.

[linux2647]

I have read that, but the data received by my plugin is binary protobuf data. I could pull the protobuf file, run it through protoc, and use the generated types that way. However, this is already done in sqlc, and I thought it would be nice if I could just import this generated code into my Go-based plugin, but I can't because the generated code all lives under internal.

The example program listed in the docs work because they also live within github.com/kyleconroy/sqlc, so they can access the internal packages. However my plugin will have a different mod path (e.g. github.com/linux2647/my-sqlc-plugin), so the Go compiler will refuse access to those internal packages.

Does that make sense?

[bessiambre]

I too got an error while trying to follow the plugin example:

package github.com/ProlificLabs/sqlc2oapi
        main.go:10:2: use of internal package github.com/kyleconroy/sqlc/internal/codegen/json not allowed
package github.com/ProlificLabs/sqlc2oapi
        main.go:11:2: use of internal package github.com/kyleconroy/sqlc/internal/plugin not allowed

[xameless]

You can copy the entire plugin folder to your own project so you can import it.

[kyleconroy]

This has been fixed as of 1.16.0. The Python and Kotlin plugins are now separate and take advantage of the https://github.com/tabbed/sqlc-go repository.

You can see how a plugin works here: https://github.com/tabbed/sqlc-gen-python