5.0.0 get stuck in a cyclic dependency loop
Solved
Hi,
First thanks for such an awesome application!
I've tried out version 5.0.0, and updating from 4.7.4 is breaking a script of mine in a rather peculiar way.
First some background:
We're using MongoDB 3.2.10, thus limiting some the aggregation feature set, hence the demonstrated util function below.
For dummy data I'll use two collections, items and prices. They are structured as follows
items
{
"_id" : ObjectId("5bf39048615c0e7c93f7e8c1"),
"label" : "Mango",
"guid" : "{d84427fb-ed45-4146-ab5d-d4fde2e996c1}"
},
{
"_id" : ObjectId("5bf39030615c0e7c93f7e8c0"),
"label" : "Banana",
"guid" : "{9c22bca5-5fa0-41d2-9e7a-b3f6885df61e}"
},
{
"_id" : ObjectId("5bf38fc8615c0e7c93f7e8bf"),
"label" : "Pear",
"guid" : "{cc05834f-2e6b-4354-a91e-39be1f48f96f}"
},
{
"_id" : ObjectId("5bf38fa8615c0e7c93f7e8be"),
"label" : "Apple",
"guid" : "{a11b666a-7d5d-4be4-8f66-df5459486e55}"
}prices {
"_id" : ObjectId("5bf391f1615c0e7c93f7e8c5"),
"barcode" : "0209489349055",
"price" : 1
},
{
"_id" : ObjectId("5bf391e6615c0e7c93f7e8c4"),
"barcode" : "0204578317934",
"price" : 1.3
},
{
"_id" : ObjectId("5bf391d2615c0e7c93f7e8c3"),
"barcode" : "0201859157483",
"price" : 0.6
},
{
"_id" : ObjectId("5bf391c8615c0e7c93f7e8c2"),
"barcode" : "0201785619192",
"price" : 0.5
}The higher order util function is defined as const transformLookup = transformer => ({ from, localField, foreignField, as, fields, project }) => {
project[as] = {$filter: {
input: {$literal: db[from].find({}).map(doc => {
fields.forEach(field => {
if (doc[field]) doc[field] = transformer(doc[field])
})
return doc
})},
as: 'item',
cond: {$eq: ['$' + localField, '$$item.' + foreignField]}
}}
return {$project: project}
}It takes a field transformation function and generate a aggregation stage query builder similar to the regular {$lookup: {...}}. The code executed that works fine in version 4.7.4, but fails in 5.0.0 is
const barcodeToGuidLookup = transformLookup(value => ({
'0201785619192': '{a11b666a-7d5d-4be4-8f66-df5459486e55}',
'0201859157483': '{cc05834f-2e6b-4354-a91e-39be1f48f96f}',
'0204578317934': '{9c22bca5-5fa0-41d2-9e7a-b3f6885df61e}',
'0209489349055': '{d84427fb-ed45-4146-ab5d-d4fde2e996c1}'
})[value])
db.items.aggregate([
barcodeToGuidLookup({
from: 'prices',
localField: 'guid',
foreignField: 'barcode',
as: 'price',
fields: ['barcode'],
project: {
label: 1
}
}),
{$unwind: '$price'},
{$project: {
label: 1,
price: '$price.price'
}}
])It makes a lookup between items and prices, while transforming the barcode to guid for the foreign collection (in this case prices). The output in 4.7.4 is
{
"_id" : ObjectId("5bf38fa8615c0e7c93f7e8be"),
"label" : "Apple",
"price" : 0.5
},
{
"_id" : ObjectId("5bf38fc8615c0e7c93f7e8bf"),
"label" : "Pear",
"price" : 0.6
},
{
"_id" : ObjectId("5bf39030615c0e7c93f7e8c0"),
"label" : "Banana",
"price" : 1.3
},
{
"_id" : ObjectId("5bf39048615c0e7c93f7e8c1"),
"label" : "Mango",
"price" : 1
}while in 5.0.0 the output is {
"message" : "cyclic dependency detected",
"stack" : "Error: cyclic dependency detected" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:354:34)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)" +
"at serializeObject (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:368:18)" +
"at serializeInto (/tmp/.mount_nosqlb1tci8N/resources/app.asar/node_modules/bson/lib/bson/parser/serializer.js:958:17)"
} Sorry for the example not being 100% intuitive. I've not dug enough into what detail of the code that's failing, in order to give a simpler example.
4.7.4 works neatly for us, but especially the "My Queries" feature from 5.0.0 would be really nice to have.
Best,
Christian
The same problem 
There's a real bug here, we will fix it in the next build.
The following is a workaround solution, add "toArray()" to your map func.
const transformLookup = transformer => ({ from, localField, foreignField, as, fields, project }) => { project[as] = {$filter: { input: {$literal: db[from].find({}).map(doc => { fields.forEach(field => { if (doc[field]) doc[field] = transformer(doc[field]) }) return doc }).toArray()}, //add toArray to the result of Map func as: 'item', cond: {$eq: ['$' + localField, '$$item.' + foreignField]} }} return {$project: project} }There's a real bug here, we will fix it in the next build.
The following is a workaround solution, add "toArray()" to your map func.
const transformLookup = transformer => ({ from, localField, foreignField, as, fields, project }) => { project[as] = {$filter: { input: {$literal: db[from].find({}).map(doc => { fields.forEach(field => { if (doc[field]) doc[field] = transformer(doc[field]) }) return doc }).toArray()}, //add toArray to the result of Map func as: 'item', cond: {$eq: ['$' + localField, '$$item.' + foreignField]} }} return {$project: project} }Thanks for spotting the core of the issue, the suggested workaround is doing its job greatly.
Regarding if this is a bug in Nosqlbooster for MongoDB or in my snippet. In my eyes it could very much be considered a feature to output a DBQuery object from a map function applied to a DBQuery object, in case it makes the library structure more consistent and the DBQuery object still makes sense (it's a major version update after all). Maybe documentation for the end user is what's needed.
I dug a bit more and it seems like there's a bit of undefined behavior around the conversions of DBQuery objects into Array objects.
This snippet yields the expected output (an array of the first two elements in the collection) in version 5.0.0
print(db.items.find({}).limit(2).map(doc => doc).toArray())However, in version 4.7.4 it yields the following error message{ "message" : "db.items.find(...).limit(...).map(...).toArray is not a function", "stack" : "script:1:50" }Thanks for spotting the core of the issue, the suggested workaround is doing its job greatly.
Regarding if this is a bug in Nosqlbooster for MongoDB or in my snippet. In my eyes it could very much be considered a feature to output a DBQuery object from a map function applied to a DBQuery object, in case it makes the library structure more consistent and the DBQuery object still makes sense (it's a major version update after all). Maybe documentation for the end user is what's needed.
I dug a bit more and it seems like there's a bit of undefined behavior around the conversions of DBQuery objects into Array objects.
This snippet yields the expected output (an array of the first two elements in the collection) in version 5.0.0
print(db.items.find({}).limit(2).map(doc => doc).toArray())However, in version 4.7.4 it yields the following error message{ "message" : "db.items.find(...).limit(...).map(...).toArray is not a function", "stack" : "script:1:50" }We have worked out a new build to try to fix the bug. Please download and give it a try.
Linux AppImage: https://nosqlbooster.com/s3/download/releasesv5/nosqlbooster4mongo-5.0.1-beta.2.AppImage
We have worked out a new build to try to fix the bug. Please download and give it a try.
Linux AppImage: https://nosqlbooster.com/s3/download/releasesv5/nosqlbooster4mongo-5.0.1-beta.2.AppImage
resolved in 5.0.1.
resolved in 5.0.1.
Replies have been locked on this page!