Add "Navigational Properties" support like EF

[Turnerj]

Changes tracked in: https://github.com/TurnerSoftware/MongoFramework/tree/ef-navigation-properties

While a document-based database can store nested data, you may not want to store all objects in a nested fashion. You can add foreign keys on entities and manually query them currently however it can be useful to be able to access the related entity through a navigational property.

In Entity Framework, this is achieved through the use of virtual properties where it instantiates a proxy that can do the required lookup. It supports both eager loading (specifically defining which objects you want data for) and lazy loading.

While it isn't a good idea to shoehorn every relational database feature from EF into MongoFramework, there is some utility in some of the functionality.

It shouldn't be very difficult to support a 1..n relationship through virtual properties like virtual ICollection<Post> Posts { get; set; }. Currently I'm not sure how to implement a 1..1 relationship through the same method as EF does some magic behind the scenes.

Right now, it doesn't seem to make sense to support eager loading as I can't see it improving performance.

At minimum I see the following things needed:

• Custom Linq processor that instantiates the proxy class (eg. ICollection<Post>)
• Updates to valid mappings, need to prevent virtual properties from being considered part of the BSON mapping
• New mapping processor to pick up new attributes part of the DataAnnotations namespace
• Potential updates to change tracking (virtual properties shouldn't count as changes - this might be accomplished with mapping updates)

This is a nice feature to add but will need to be careful to make sure it generally works the same way as EF. If it has minor differences, that's fine. If it has major differences, it might be worth re-considering this functionality.

[Turnerj]

This documentation of Entity Framework should help with working on this feature: https://msdn.microsoft.com/en-us/library/jj713564(v=vs.113).aspx

[Turnerj]

Now that I've started work on this, it is more complicated than I thought.

Getting a basic IMappingProcessor that can determine a "foreign key" doesn't seem like a difficult goal however there are a couple of usecases. The foreign key attribute can be on either the ID or the navigational property, it just needs to point to the other. On top of that, it can work without the attribute in EF, it is formatted like "ClassNameId". In either case, we need to unmap the navigation property.

A basic IEntityMutator is a bit more difficult too. Firstly the mutation interface needed to be passed a database reference otherwise we can't actually read data in or write data out. Next is because of the nested nature of MongoDB schemas, I need to traverse the entire mapping to find relationships to update data for. The mapping traversal function in the entity mapper definitely helps but navigating an object instance through reflection from the mapping is a little tricky.

I could probably simplify the entity mutator by just having a rule that foreign key relationships must be on the top-most object.

Still end up with issues saving changes though. If you have a 1..1 navigation property, how do you know its updated? I can load in the real entity without issue but writes may always need to save the whole object. I can at least determine whether its an insert or update from the presence of its Id property. There is probably still more I need to research for 1..1, maybe something around RealProxy (though even a brief read into that says there are limitations).

What I really wanted to do was implement the 1..n talked about on this blog however that is even more complicated as I then need to make a property to the serializer appear out of thin air. (Well, that's not entirely true but I do need to potentially turn an ICollection<YourType> to ICollection<ObjectId> without actually doing that to the class.

[Turnerj]

Kept powering through and have a PR open with the latest changes that support the described functionality.

A few key points:

• You won't need to specify "virtual" on the properties. It isn't using proxy classes, the ICollection is actually just an a normal class that implements that interface. The single entity instances, those are the full entity which means a full write will need to be performed every time it saves.
• The ForeignKeyAttribute must be used - there isn't any support for working out the relationships from the names of the properties (like EF does through conventions) though it may come later.
• The plan is to support InversePropertyAttribute as well though will see after various tests are put together
• A custom serializer is used to convert ICollection<TEntity> into an array of IDs - no need to configure anything in your entity to support this
• Navigation properties are only supported on the base entity (this may change in the future)

[Turnerj]

Currently the PR is relatively stable but not ready to merge in yet. At minimum, there is still support for the InversePropertyAttribute required.

There are a few main downsides to the implementation at current however.

Detecting changes for the main entity is difficult when also factoring in the navigation properties.
The functionality is there but it seems strange having to specifically have the entity check the navigation properties. For example, if a property of an entity on a navigation property is updated, the parent entity actually needs to be marked as updated which makes the code cumbersome and confusing.

Unable to detect updates to a "single entity navigation"
Where the collection of entities is backed by the DbEntityChangeTracker, a single entity navigation property isn't. The full entity is updated on every write which is a waste.

Navigation property writes do not support async
A side effect of calling the saving through the mutator is that mutators aren't asynchronous.

I have a potential solution to both cases: Have the parent MongoDbSet know about all the navigation entities. Basically, a class that manages a list of DbEntityChangeTracker classes based navigation properties for the entity type.

This change then allows:

• The entity to not directly care about changes to the navigation property entities.
• "Single entity navigation" to have their changes properly tracked and only update differences.
• Navigation property writes support async as writes are called from the async methods in the MongoDbSet

Its quite a bit of re-architecture from what I already wrote but I think its clearly the better plan.

[Turnerj]

In addition to those points, it is probably worth changing how "single navigation entities" actually manage their writes on the parent entity too.

The "ID" property relating to the relationship should actually be managed by a custom serializer. This allows the unmapping of the navigation property to be less of a problem and allows the entity to more natively support the linking or unlinking of entities.

[Turnerj]

I may have over-implemented what I was aiming for and that's where my problems are. Entity Framework supports adding non-existing entities to the DB as well as removing the relationship (or the entity if its a cascade delete) but doesn't support updating.

EF Reading Related Data: https://docs.microsoft.com/en-us/aspnet/mvc/overview/getting-started/getting-started-with-ef-using-mvc/reading-related-data-with-the-entity-framework-in-an-asp-net-mvc-application
EF Updating Related Data: https://docs.microsoft.com/en-us/aspnet/mvc/overview/getting-started/getting-started-with-ef-using-mvc/updating-related-data-with-the-entity-framework-in-an-asp-net-mvc-application
EF Saving Related Data: https://docs.microsoft.com/en-us/ef/core/saving/related-data

Plus it also shows that it supports entities where the navigation property isn't a proxy but a simple list (which implements ICollection<TEntity>).

I still need to support an EntityNavigationCollection for holding the serialization data. It does need to still get access to the DB to load in its data. The serializer would still deserialize to that type but serialization has to support ICollection<TEntity> specifically which means we look at the entities themselves. That's still a problem for EntityNavigationCollection because freshly imported IDs won't have their matching entities and it would be silly to fake it or to go to the DB to get the entities just to serialize. All this really means is we have an additional conditional case or two for getting the IDs from both situations.

When SaveChanges is called on MongoDbSet, it will (before DetectChanges) look at the relationships for the entities. It needs to determine if any "single navigation" properties are new and save them if they are. It needs to be careful that it saves only distinct memory references otherwise we will have a problem. This also needs to make sure the ID of the foreign key is correctly updated based on the value of the navigation property. This allows DetectChanges to work later and pick up changes to the navigation property.

After DetectChanges (but before saving the actual entity), we can look at saving any new entities on "navigation collection" properties. Relationship unlinking (via removing specific entities from the collection) is automatically handled by the serialization process.

[Turnerj]

Navigation property / relationship changes have landed in master. There is multiple steps to the functionality:

• Detecting relationships by property types and associated attributes
• Custom serializers and unmapping properties where appropriate for relationships
• MongoDbSet support for relationship writing and updating related IDs

There isn't really any great documentation on how to use the functionality yet but that will come soon.