Rust Programming @lemmy.ml fxdave @lemmy.ml 8 mo. ago

How to make plugins?

I have a plugin trait that includes some heavy types that would be almost impossible to wrap into a single API. It looks like this:

pub struct PluginContext<'a> {
    pub query: &'a mut String,
    pub gl_window: &'a GlutinWindowContext,
    flow: PluginFlowControl,
    pub egui_ctx: &'a Context,
    disable_cursor: bool,
    error: Option<String>,
}
pub trait Plugin {
    fn configure(&mut self, builder: ConfigBuilder) -> Result<ConfigBuilder, ConfigError> {
        Ok(builder)
    }
    fn search(&mut self, ui: &mut Ui, ctx: &mut PluginContext<'_>);
    fn before_search(&mut self, _ctx: &mut PluginContext<'_>) {}
}

Here is what I considered:

Keeping all plugins in-repo. This is what I do now, however I'd like to make a plugin that would just pollute the repository. So I need another option that would keep the plugins' freedom as it is right now, but with the possibility to move the plugin out to a separate repository.
I tried to look into dynamic loading, and since rust doesn't have a stable ABI, I'm okay with restricting the rust versions for the plugin ecosystem. However, I don't think it's possible to compile this complex API into a dynamic lib and load it safely.
I'm also ok with recompiling the app every time I need a new plugin, but I would like to load these plugins automatically, so I don't want to change the code every time I need a new plugin. For example, I imagine loading all plugins from a folder. Unfortunately, I didn't find an easy solution for this neither. I think I will write a build macro that checks the ~/.config/myapp/plugins and include all of them into the repo.

Do you have any better ideas, suggestions? Thanks in advance.

(For context, this the app I'm writing about: https://github.com/fxdave/vonal-rust)

9 comments

About dynamic library loading, is rust really that much of a pain? If you don't mangle the functions, then the ABI should be alright, no?

Also, you can support plugins using WASM. An option is wasmer. Then other languages can compile to WASM and the plugins can be loaded into your application.

CC BY-NC-SA 4.0
- Yes, rust is that much of a pain in this case, since you can only safely pass plain C compatible types across the plugin boundary.
  
  One reason is that rust doesn't have stable layouts of structs and enums, the compiler is free to optimise the to avoid padding by reordering, decide which parts to use as niches for Options etc. And yes, that changes every now and then as the devs come up with new optimisations. I think it changes most recently last summer.
  
  @Vorpal @onlinepersona the way this is typically done is to expose an extern "C" interface in rust which provides a wrapper around the ABI-unstable rust interface. The C ABI for a given system is stable.
  
  Note that C++ also doesn't have a stable ABI either. The same patterns are used there.
  
  Let me know if you want me to go into more detail on any of that. I've dealt with Rust and C++ FFIs for the last few years.
I'm not sure exactly how to solve your problem, but one thing that occurs to me is that lifetimes and references are really a compile time semantic. If you're dynamically loading something then you can't assert lifetimes, at least not safely. So for point 2 I feel like you'd need to create an unsafe layer in between the app and dynamically loaded plugins and then wrap that in a safe API.

For point 3 I suspect you'd need to look at the macro system. Some kind of like load_plugins!("~/.config/myapp/plugins") macro stuck in an appropriate place in the code that at compile time can check that folder and generate the appropriate glue code to register and use everything. One thing I'm not entirely sure about though is if you'd have permission to access folders outside of the project at compile time, there's a chance the compiler would refuse to do so, but I don't know enough about macros or the way they're sandboxed (or not sandboxed) to say for sure.
If there won't be too many different plugins, maybe having a feature for each plugin would work. Then you could use --features=... when compiling to select the plugins you need.
- Thanks, I forgot to mention but that's what I do now. My problem with this is that I would like to make a plugin that makes sense only for me and not for the other users. I could maintain my personal fork though.
It's still pretty underbaked but you can check out my recent project based around a plugin system (dynamic loading). https://github.com/CerulanLumina/feed-plumber

It relies on the c abi and a whole lot of raw /ffi constructs but contains a helper crate to allow defining plugins in a somewhat rusty manner.

Hopefully it'll be of some use to you!
So there is a couple of options for plugins in Rust (and I haven't tried any of them, yet):

Wasm, supposedly https://extism.org/ makes this less painful.

libloading + C ABI

One of the two stable ABI crates (stabby or abi_stable) + libloading

If you want to build them into your code base but not have to update a central list there is linkme and inventory.

An embedded scripting language might also be a (very different) option. Something like mlua, rhai or rune.

I don't know if any of these suit your needs, but at least you now have some things to investigate further.

You've viewed 9 comments.