We have encountered heavy performance problem while using LINQ to SharePoint to query items using reverse lookup collections. These collections of type EntitySet<TEntity> provides items referencing the lookup target item and are used to maintain one to many relationship.
To use LINQ the entities must be generated using SPMetal tool. For example if List1 has lookup column to List2, List2 entity has a property called List1 of type as above generated by this tool. Therefore iterating List2.List1 collection we can process all items referencing selected List1 item. It works fine as long as number of List1 items is very limited - hundreds depending on hardware.
The problem is caused by very long queries. During tests for the production environment to get a few items (0-3) we encountered logs like this:
Slow Query Duration: 34067.2267704415
The list we are getting the items has a few thousand items.
In the development environment the execution time fore the same query and the same site collection content is below 0.5s.
Analyzing the query it seems that to resolve the lookup references the whole related lists are fetched to the memory by the SQL server.
Questions:
What could be the reason that DEV environment is so much faster? Exclude hardware we used old desktop computer as the Front End and SQL server (all in one).
What could be the cause of the slow query?
We suspect that main difference between the DEV and PROD servers may be availability and retention time for data in the cache. Unfortunately it means that using reverse lookup collections to get items from lists containing more then a few hundreds items is very limited and must be tested in the target environment before using, but usually it is impractical or even impossible.
Is there any workaround addressing this issue?
Mariusz