So I have 4 filters.
export const AnswerFilters = [
{ name: 'Highest Upvotes', value: 'highestUpvotes' },
{ name: 'Lowest Upvotes', value: 'lowestUpvotes' },
{ name: 'Most Recent', value: 'recent' },
{ name: 'Oldest', value: 'old' },
];
In my DB, the answers have max 1 like, nothing, or max 1 dislike. (testing phase). If I change the filter to recent or old, the infinite scroll works fine, but if I set the filter to highestUpvotes or lowestUpvotes, after page 2 - 3 I get the error
Encountered two children with the same key, '123456blabla',
Here is the server action:
"use server"
export async function getAnswers(params: GetAnswersParams) {
try {
connectToDatabase();
const { questionId, sortBy, page = 1, pageSize = 5 } = params;
const skipAmount = (page - 1) * pageSize;
let sortOptions = {};
switch (sortBy) {
case 'highestUpvotes':
sortOptions = { upvotes: -1 };
break;
case 'lowestUpvotes':
sortOptions = { upvotes: 1 };
break;
case 'recent':
sortOptions = { createdAt: -1 };
break;
case 'old':
sortOptions = { createdAt: 1 };
break;
default:
break;
}
const answers = await Answer.find({ question: questionId })
.populate('author', '_id clerkId name picture')
.sort(sortOptions)
.skip(skipAmount)
.limit(pageSize);
const totalAnswers = await Answer.countDocuments({
question: questionId,
});
const isNext = totalAnswers > skipAmount + answers.length;
return {
answers: JSON.parse(JSON.stringify(answers)),
isNext,
};
} catch (error) {
console.log(error);
throw error;
}
}
This is the server component:
const AllAnswers = async ({ questionId, userId, totalAnswers, page, filter }: Props) => {
const { answers, isNext } = await getAnswers({
questionId,
page: page ? +page : 1,
sortBy: filter,
});
return (
<div className='mt-11'>
<div className='flex items-center justify-between'>
<h3 className='primary-text-gradient'>{`${totalAnswers} ${totalAnswers === 1 ? 'Answer' : 'Answers'}`}</h3>
<Filter filters={AnswerFilters} />
</div>
<AllAnswersInfiniteScroll
initialAnswers={answers}
questionId={questionId}
userId={userId}
isNext={isNext}
filter={filter}
/>
</div>
);
};
export default AllAnswers;
And this is the client component aka AllAnswersInfiniteScroll:
const AllAnswersInfiniteScroll = ({
initialAnswers,
questionId,
userId,
filter,
isNext,
}: Props) => {
const [allAnswers, setAllAnswers] = useState(initialAnswers);
const [isNextPage, setIsNextPage] = useState(isNext);
const pageRef = useRef(1);
const [ref, inView] = useInView();
const loadMoreAnswers = async () => {
const next = pageRef.current + 1;
const { answers: newAnswers, isNext } = await getAnswers({
questionId,
page: next,
sortBy: filter,
});
if (allAnswers?.length) {
setAllAnswers((prevAnswers) => [...prevAnswers, ...newAnswers]);
setIsNextPage(isNext);
pageRef.current = next;
}
};
const filterAnswers = async () => {
pageRef.current = 1;
const { answers: newAnswers, isNext } = await getAnswers({
questionId,
page: pageRef.current,
sortBy: filter,
});
setAllAnswers(newAnswers);
setIsNextPage(isNext);
};
useEffect(() => {
if (inView) {
loadMoreAnswers();
}
}, [inView]);
useEffect(() => {
filterAnswers();
}, [filter]);
return (
<>
{allAnswers?.map((answer) => (
<article key={answer._id} className='light-border border-b py-10'>
<div className='mb-8 flex flex-col-reverse justify-between gap-5 sm:flex-row sm:items-center sm:gap-2'>
<Link
href={`/profile/${answer.author.clerkId}`}
className='flex flex-1 items-start gap-1 sm:items-center'
>
<div className='flex flex-col sm:flex-row sm:items-center'>
<p className='body-semibold text-dark300_light700'>
{answer.author.name}
</p>
<p className='small-regular text-dark400_light500 mt-0.5 line-clamp-1'>
<span className='mx-0.5 max-sm:hidden'>•</span>
answered {getTimestamp(answer.createdAt)}
</p>
</div>
</Link>
</div>
</article>
))}
{isNextPage && (
<div className='mt-11 flex w-full items-center justify-center' ref={ref}>
<Icons.Spinner className='size-9 animate-spin' />
</div>
)}
</>
);
};
export default AllAnswersInfiniteScroll;
Any suggestion is welcome, I'm out of experiments and ideas on how to make it work. If you somehow know the issue, please help me understand why does my beautiful code does these things to me. 😂
Edit: typos, removed code / components irrelevant to the question.
tl;dr: Make all of your sorts compound by appending _id :1 to them, e.g.:
switch (sortBy) {
case 'highestUpvotes':
sortOptions = { upvotes: -1, _id: 1 };
break;
case 'lowestUpvotes':
sortOptions = { upvotes: 1, _id: 1 };
break;
case 'recent':
sortOptions = { createdAt: -1, _id: 1 };
break;
case 'old':
sortOptions = { createdAt: 1, _id: 1 };
break;
default:
break;
}
There are a few things combining that result in the 'intermittent' behavior that you're experiencing. The first is that your the keys for your articles are populated by information from the database:
<article key={answer._id} ...
The second thing is that you are using skip and limit to perform pagination. This means that each request to the database is 'unrelated' (from the database's perspective) to the previous ones.
Finally, some of the values that you are sorting on are more likely to be unique than others. Timestamps, eg createdAt, are often unique (or have very few exact duplicates). The count of votes an article has, by contrast, probably has tons of duplicates especially at the lower ranges.
Putting it all together, here's what's happening:
createdAt field, then you probably effectively have a unique sorting order. This means that, as long as the data underneath doesn't change between the queries, each 'page' of results coming back from the database will be completely different. This is why 'the infinite scroll works fine' under these conditions.upVotes, your ordering is not unique. So the new query sorts and skips over a page (or two) of results, but the page or two it skips over may not be the exact same as the first page or two displayed. This means that the query can return an article that you're already displaying resulting in the _id value being applied to more than one article that you are attempting to render. This is when the error happens.A bit more about unique sorting can be found in this answer.
The guidance above, to append _id to the sorts to make them uniquely specified, helps for some aspects of this problem. It will prevent the error from happening under the conditions you've noted (testing the infinite scroll in development). But you could still encounter it in other ways. Say, for example, that you are sorting with newest articles first and you've loaded the first page of 5. If a new article is published after that initial page load and before you scroll to get the next page then the article that was previously 5th (and is currently displayed on the page) will now appear again as the first result for the next query since it is now 6th overall. You would have to handle this issue separately.
A different approach entirely would be to forego skip and limit and instead specify a batch size and iterate the cursor. I have no idea if such an architecture is feasible to implement with Next.js.