Cross-/multi-scope bean wiring #793
Comments
|
I added all the Profile and conditional annotations precisely because I found custom scopes unwieldy. @rbygrave is the guy to ask about custom scopes.
Other than conditional annotations, this is the option I see most people go with.
You can do this today, though it is an SPI and not a direct argument on the builder. |
|
One question I have is how custom scopes change the shading situation. |
That's fair - they do appear to be more user friendly. The trade-offs I mentioned really are very minor given how trivial they are to overcome.
Oh? If so that's good, although from my quick poke around it did appear the SPI point was the
I could give a proper example later when I'm not on mobile, but Again, that could be solved with having different gradle/maven projects, but working out where to draw those harder boundaries and how many can be hard on a medium/large project. |
|
I'd like to understand the problem / context of the issue better.
How many different deployments are we talking about and can you give some examples. (All / Admin Only ... ? .. I'm wondering if we thinking the same thing here with "deployments" split by functionality?)
Why do we care about shading so much? Are we building using graalvm native-image or is it just truely large or something else? [just wondering about the bang for buck and testing etc or am I completely missing something interesting here] [edit: shading isn't great for docker and shading isn't great for java module-info adoption so I'm wary when shading comes up as opposed to excluding optional dependencies but maybe there is a really good need for shading / tree shaking here]
How many scopes do you have in mind? Can you put some examples of what you think these scopes are and how they are related? An example of how bean scopes are used is in a build system which supports multiple platforms (Windows, Linux, Mac). So there are a few scopes for common build features and a scope for each platform specific one. Lets say 7 scopes with pretty well understood relationships. I'm getting a feeling you are talking a LOT more than 7 scopes? Like how many scopes are we talking about? [which might give me a sense of what we are trying to do. If you say 100 scopes then I'm not understanding the use case ... that is basically a big dependency tree of scopes etc]
So it sounds like its pretty big and maybe not broken into multiple maven modules yet, its still one src/main? No existing modularity in the code base? No intention to add modularity in the form of splitting it into maven modules yet? |
@ModAScope
@ModBScope
public class MyBean {}
You are going to have to give a practical example of this so that I can get a sense of what you are trying to achieve here / why a bean "exists" in both ModAScope and ModBScope? The issue is that "exists" is actually more "owned, created in" and a bean in this sense should be created and owned by only one scope. So what were we trying to achieve with a bean in multiple scopes? Can you put a real world example there so that I can understand what we were aiming for? |
Just to say that for me it's more Scope has been used when the final deployment artifact literally only includes certain scopes - Linux, Windows, MacOS. I think I'm keen to here if this application is a server app or desktop app or build system or ... so that might help understand the goals we are aiming for and better choose Scope vs Profile vs [optional] AvajeModule etc. |
Yep! I'll walk through the journey and answer the other questions after that :D
A server app all in one src/main yes, no clear modularity in code, but the tasks it performs can be broken up semi-cleanly which hopefully will become k8s deployments with APIs, or jobs/cronjobs. The number of deployments is a little hazy as it depends on how things "fall out", but is around 12-15. After getting a consistent DI into the project, plan is to try and move to a structure that looks like this: Each You can kinda see it above, but the warehousing items above would be typical candidates for So the journey I went on with exploration is what I (aimed, a little poorly) to list in my original post:
I'm still OK with the trade-off of Addressing some of the other questions:
Shading would be useful for two purposes:
I agree that modules (both gradle/maven and Java) being the better solution for dependency tree management overall, or tools like jib for docker containers if you'd rather avoid modules. Re graalvm: Doesn't it do tree-shaking automatically?
Huh, never thought of that but I guess it makes sense. Why should e.g. guava modularise when shading can do the job, I suppose (although if java had multi-module jar files...)
That depends on if a top-down (
For this example I was talking about the
Completely reasonable - why load an expensive resource when it doesn't need to be? |
To clarify this a bit more: In both the The difference was in the source code (and by extension at compile time) how many |
|
Ok, so if I understand it right then, say in Rob background thinking ...
Right. So at a very minimum we are talking 15 odd Right, I think I'm getting it. My gut is saying that custom |
Right, at scale yes this isn't practical to being done manually. This seems like something that is fixable. We have the custom scopes, and their requires/provides so it could be that ordering for the custom scopes could be generated at compile time or determined at runtime. e.g. We want to build a BeanScope with CrossCutScope. It's dependencies and their ordering can be determined by avaje-inject at runtime or compile time. A failing test here could be, using BeanScope Builder pass the CrossCutScope module only, and the builder automatically determines the dependencies (ModAScope, ModBScope) and the ordering based on requires/provides and that "just works".
I'm not sure here and need to look at what's going on. The single bean in multiple scopes wasn't aligned to my thinking of a bean only being in a single Scope. Not sure here and need to check this. |
That's the goal for now, yes. After all the slicing is done, it's possible that there will instead be an
It did start to seem like that while I was experimenting, so I'm glad that you also think so 😄
Yes, with a slight preference for compile time ordering as it has all the context and can probably better handle situations in which the dependency graph isn't strictly linear. For example, a To avoid breaking existing behaviour while keeping resolution done at compile time I proposed the
And if compile-time isn't the place to do this, to avoid breaking existing behaviour for runtime I proposed a That's my outside looking in perspective - you guys might have better ideas knowing other projects/Avaje better than I do.
Yep, although I think that's the passing case - the failure would be that it didn't work it out and errors out at runtime/in tests instead.
Which is fair - a bean being in multiple scopes is a bit odd. At runtime it would only exist in one, but the compiler has no way of knowing that. It does seem like the bottom-up approach where Thanks heaps for the help so far to both of you! |
Perchance can you elaborate on this? |
Sure! Using version 11.4 still for these examples. Let's say that the following is in the public interface IncomingContainerPlacementPlanner {}
@Singleton
@Profile(all = {"warehouse", "air"})
public class AirInConPlanner implements IncomingContainerPlacementPlanner {}
@Singleton
@Profile(all = {"warehouse", "sea"})
public class SeaInConPlanner implements IncomingContainerPlacementPlanner {}
// In package-info.java
@io.avaje.inject.InjectModule(provides = com.example.profile.IncomingContainerPlacementPlanner.class)
package com.example.profile;The following is generated (in collapsible): Generated code
@Generated("io.avaje.inject.generator")
public final class AirInConPlanner$DI {
public static void build(Builder builder) {
if (!builder.containsAllProfiles(List.of("air","warehouse"))) {
return;
}
if (builder.isBeanAbsent(AirInConPlanner.class, IncomingContainerPlacementPlanner.class)) {
var bean = new AirInConPlanner();
builder.register(bean);
}
}
}
@Generated("io.avaje.inject.generator")
public final class SeaInConPlanner$DI {
public static void build(Builder builder) {
if (!builder.containsAllProfiles(List.of("warehouse","sea"))) {
return;
}
if (builder.isBeanAbsent(SeaInConPlanner.class, IncomingContainerPlacementPlanner.class)) {
var bean = new SeaInConPlanner();
builder.register(bean);
}
}
}
@Generated("io.avaje.inject.generator")
@InjectModule(provides = {com.example.profile.IncomingContainerPlacementPlanner.class})
public final class ProfileModule implements AvajeModule {
@Override
public Type[] provides() {
return new Type[] {
com.example.profile.IncomingContainerPlacementPlanner.class,
};
}
@Override
public Type[] autoProvides() {
return new Type[] {
com.example.profile.AirInConPlanner.class,
com.example.profile.SeaInConPlanner.class,
};
}
@Override
public Class<?>[] classes() {
return new Class<?>[] {
com.example.profile.AirInConPlanner.class,
com.example.profile.SeaInConPlanner.class,
};
}
/**
* Creates all the beans in order based on constructor dependencies.
* The beans are registered into the builder along with callbacks for
* field/method injection, and lifecycle support.
*/
@Override
public void build(Builder builder) {
// create beans in order based on constructor dependencies
// i.e. "provides" followed by "dependsOn"
build_profile_AirInConPlanner(builder);
build_profile_SeaInConPlanner(builder);
}
@DependencyMeta(
type = "com.example.profile.AirInConPlanner",
provides = {"com.example.profile.IncomingContainerPlacementPlanner"},
autoProvides = {
"com.example.profile.IncomingContainerPlacementPlanner",
"com.example.profile.AirInConPlanner"
})
private void build_profile_AirInConPlanner(Builder builder) {
AirInConPlanner$DI.build(builder);
}
@DependencyMeta(
type = "com.example.profile.SeaInConPlanner",
provides = {"com.example.profile.IncomingContainerPlacementPlanner"},
autoProvides = {
"com.example.profile.IncomingContainerPlacementPlanner",
"com.example.profile.SeaInConPlanner"
})
private void build_profile_SeaInConPlanner(Builder builder) {
SeaInConPlanner$DI.build(builder);
}
}If we then have an public class App
{
public static void main( String[] args )
{
try(
BeanScope cross = BeanScope.builder()
.profiles("warehouse", "sea")
.build();
) {
// Do all the things
}
}
}When you shade the output with an Service Loader aware plugin, it will:
Both beans (and their dependencies) will be included in the output unless the shader has some way of knowing that the result of Compare that to the following layout in the @Scope
public @interface AirAppScope {}
@Scope
public @interface SeaAppScope {}
public interface IncomingContainerPlacementPlanner {}
@AirAppScope
public class AirInConPlanner implements IncomingContainerPlacementPlanner {}
@SeaAppScope
public class SeaInConPlanner implements IncomingContainerPlacementPlanner {}The following is the generated output: Generated code
@Generated("io.avaje.inject.generator")
public final class AirInConPlanner$DI {
public static void build(Builder builder) {
if (builder.isBeanAbsent(AirInConPlanner.class, IncomingContainerPlacementPlanner.class)) {
var bean = new AirInConPlanner();
builder.register(bean);
}
}
}
@Generated("io.avaje.inject.generator")
public final class SeaInConPlanner$DI {
public static void build(Builder builder) {
if (builder.isBeanAbsent(SeaInConPlanner.class, IncomingContainerPlacementPlanner.class)) {
var bean = new SeaInConPlanner();
builder.register(bean);
}
}
}
@Generated("io.avaje.inject.generator")
@InjectModule(customScopeType = "com.example.profile.AirAppScope")
public final class AirAppModule implements AvajeModule.Custom {
@Override
public Type[] autoProvides() {
return new Type[] {
com.example.profile.AirInConPlanner.class,
com.example.profile.IncomingContainerPlacementPlanner.class,
};
}
@Override
public Class<?>[] classes() {
return new Class<?>[] {
com.example.profile.AirInConPlanner.class,
};
}
/**
* Creates all the beans in order based on constructor dependencies.
* The beans are registered into the builder along with callbacks for
* field/method injection, and lifecycle support.
*/
@Override
public void build(Builder builder) {
// create beans in order based on constructor dependencies
// i.e. "provides" followed by "dependsOn"
build_profile_AirInConPlanner(builder);
}
@DependencyMeta(
type = "com.example.profile.AirInConPlanner",
provides = {"com.example.profile.IncomingContainerPlacementPlanner"},
autoProvides = {
"com.example.profile.IncomingContainerPlacementPlanner",
"com.example.profile.AirInConPlanner"
})
private void build_profile_AirInConPlanner(Builder builder) {
AirInConPlanner$DI.build(builder);
}
}
@Generated("io.avaje.inject.generator")
@InjectModule(customScopeType = "com.example.profile.SeaAppScope")
public final class SeaAppModule implements AvajeModule.Custom {
@Override
public Type[] autoProvides() {
return new Type[] {
com.example.profile.IncomingContainerPlacementPlanner.class,
com.example.profile.SeaInConPlanner.class,
};
}
@Override
public Class<?>[] classes() {
return new Class<?>[] {
com.example.profile.SeaInConPlanner.class,
};
}
/**
* Creates all the beans in order based on constructor dependencies.
* The beans are registered into the builder along with callbacks for
* field/method injection, and lifecycle support.
*/
@Override
public void build(Builder builder) {
// create beans in order based on constructor dependencies
// i.e. "provides" followed by "dependsOn"
build_profile_SeaInConPlanner(builder);
}
@DependencyMeta(
type = "com.example.profile.SeaInConPlanner",
provides = {"com.example.profile.IncomingContainerPlacementPlanner"},
autoProvides = {
"com.example.profile.IncomingContainerPlacementPlanner",
"com.example.profile.SeaInConPlanner"
})
private void build_profile_SeaInConPlanner(Builder builder) {
SeaInConPlanner$DI.build(builder);
}
}If we then have an public class App
{
public static void main( String[] args )
{
try(
BeanScope cross = BeanScope.builder()
.modules(new SeaAppModule())
.build();
) {
// Do all the things
}
}
}When you attempt to shade the output, it will:
The main difference is the way the I haven't yet put enough together to ascertain if shading still needs to be SPI aware in this case (it might do - a lot of things use Service Loaders, not just avaje-inject). But because You could probably exclude the class Does that help? |
|
If you use shading to remove the unused third-party dependencies but keep your own classes, is there an issue? As the execution paths are gated behind a conditional check I would imagine you wouldn't get any NoClassDefFound errors. |
|
Oh you mean like exclude the transitive dependencies a given app doesn't need? That should work yes - thank you for that idea! It'd be manual, but that probably forces thinking about how those dependencies are used and if they are even needed to begin with. Number of dependencies might still pose an issue, but it's a workable way forward for that path. It wouldn't work if you wanted to shade your own classes though - not sure there's too many cases where that would be a strongly desired goal though. |
|
I did have some thoughts about a way to potentially implement the Implementation ideaAdd the Add two more extensions to the
When generating an
When loading Once all modules to be loaded are found, if all the modules in the list to be built are This covers the request to have scopes depend on other scopes, and also has the nice benefit that if the entire module tree was known at compile time each step of the way then less work needs to be done at runtime (although the wiring is already very very fast, and it's probably not the slow point at startup!). Also: is the expectation here that I would implement this feature, or am I fine to leave it for someone else to assign themselves to it? Considering it's making some not-so-small changes I'm not sure a "first contribution" is entirely appropriate... |
|
If you think it's taking too long you can try to implement. That's how I got started. I'm not really a scopes guy so I was assuming Rob was working on it. If you create a PR with a failing test I could try and take a look. |
Wasn't aiming to be impatient sorry! More just checking expectations
I'm happy to do that too (or you, if you wanted to work on it)
If that would be helpful, I can do that! I might make two - one building off of Rob's existing test (testing module ordering when they are all defined at runtime manually), and one with the |
That might be a thing but for me right now its actually just down to determining the module ordering of the required modules and generating code based on that. So the guts is that to use the .modules(new ModAModule(), new ModCModule(), new ModBModule(), new CrossCutModule())This could be just done by adding a helper method onto the generated CrossCutModule like: @Generated("io.avaje.inject.generator")
@InjectModule(requires = {org.multi.scope.ModAScope.class,org.multi.scope.ModBScope.class}, customScopeType = "org.multi.scope.CrossCutScope")
public final class CrossCutModule implements AvajeModule.Custom {
// new method ... return all required modules in correct order
public static List<AvajeModule> allRequiredModules() {
return List.of(new ModAModule(), new ModCModule(), new ModBModule(), new CrossCutModule());
}
...And then we could use it like: return BeanScope.builder()
.modules(CrossCutModule.allRequiredModules())
.build();Is that not what we need here? Or have I missed something and there is more required? If thats the case then I think this becomes: |
@cbarlin do you want this too? |
Yes! If you are happy for it to be done that way, then that definitely works for me. I was only suggesting
Yes please! |
sounds cool, but I feel like this might complicate things.
|
|
I can figure out the first item as long as it's in the current compilation unit, but the second stumps me. |
|
Or perhaps I'm thinking about it wrong, do we want to have the module itself added as a requires instead of the scope annotation? |
@rbygrave it seems this doesn't work as there are cases where the generated module can have constructor parameters. tbh, I think what we have in that PR should be good for now |
|
What if it the method took in all the params that were needed for the modules constructor's? |
|
It seems that this is harder than it looks. This is what I got so far. |
|
Sorry! Only made that suggestion since it sounded like a hard requirement for the idea to be implemented. I don't have a requirement to pass in external dependencies, especially given the implementation above makes things very flexible as you can use scopes and factories to manage difficult things instead. Could you instead not generate the method if there are externally required items, and deal with generating a requiring version if/when someone puts forth a use case for such a thing? Also thank you so much for doing the digging into making it happen - I very much appreciate it! |
|
I'm trying that, but it fails for a different reason: something about the dependencies not being satisfied. Doesn't happen with the annotations approach though. |
|
Releasing 11.5-RC2 that contains this change to maven central now. Should be available in central soon to try out etc. Awesome work @SentryMan |
|
Awesome! Thanks heaps, especially @SentryMan! |
Hi there! I'm experimenting with avaje-inject version 11.4 and exploring ways to work with
@Scopesin order to make a mono-repo with multiple deployments, which share a lot of common code by virtue of sharing a common core and because it was previously a monolith.Ideally, there'd be an easy way to:
I've tried a few things, but have gotten stuck on these paths (code examples+more detail in the expandables):
Declare scopes requiring other scopes, automatically including them
Using the following structure:
The result means you cannot just refer to the
CrossCutModulewhen starting an entireBeanScope- you have to declare parent scopes (which is documented), or start theBeanScopewith all modules declared in order. You can't pass them toBeanScope.builder().modules(...)out of order (e.g.ModAScope, CrossCutScope, ModBScope) as you'll end up with exception at startup ("java.lang.IllegalStateException: Injecting null forcom.example.BeanInModB name:!beanInModB...").This is fine if you have a small number of scopes with a clear parent/child hierarchy, but if you have a large number of scopes/modules, or there's not a clear relationship between them, it becomes quite unmanageable.
Meta-annotating defined Scopes
Using the following structure:
I hoped it would result in the same output as:
So the bean exists in both
ModAScopeandModBScope, but instead it's not picked up by Avaje at all.While
@Profilewould probably do what I need (and I can meta-annotate those), that's less easily shaded (in the sense of dropping unreferenced classes) and doesn't give compile time errors. While neither of those cons are deal-breakers (Avaje is fast and light enough from my experiments you can just check the profiles work in tests, and there's negligible runtime overhead to non-constructed beans) I'm surprised that neither of my scope attempts worked (and at theBeanScope.builder().modules(...)being order-dependent). Another option would be to have many Gradle/Maven sub-projects and rely on the default scope, but that would also become quite unwieldy depending on the size of the code base and how much you wish to split it.Is there any scope/intention to support something like this with
@Scope/@InjectModule? Ideas on making something like this work without introducing potentially breaking changes:InjectModulecalled something likeautomaticallyImportwhich acts likerequiresfor all validation checking, but when the generated module'sbuildmethod is called the first operations are to create and build the listed modules/scopesBeanScopeBuilderto correct module order (defaultfalse- current behaviour) and/or to define our ownModuleOrderingwith custom logicScopecomposite annotations. A bean annotated with a meta-scope would be treated as if it were directly annotated with those scopes, which currently is to be created in each scope.Option 3 is probably the most elegant for both Avaje and consuming applications. A blend of Options 1 and 2 could also be done outside of Avaje as the details are available at run-time, so someone using Avaje could write a method that takes in a module and it returns an array of modules in the correct order (if an order is possible).
Thanks heaps for taking the time to read!
The text was updated successfully, but these errors were encountered: