Traversing events vertically within a column

When I click an event I show a modal. In this modal I would like to have left/right and up/down arrows to "walk around" in the scheduler. Next and previous is simple:
const eventsOnRow = args.e.calendar.rows.find(args.e.row()).events.all();
then just find my event and take index +/-1.

But is there a simple way of finding events that is in the same column above and below current event ?