Unfortunately for me, my memory can only hold so much story. After the 30th hour my mind starts drifting, by the 40th I start going through the motions, without remembering what I'm doing and why, just following the quest markers. After the 50th hour I usually drop everything else (exploration, side quests, etc) and beeline for the end of the main quest.
Another issue I see with most RPGs with lots of side content / filler content is that it ultimately detracts from the main storyline / quest… The "the longer, the better" camp seem to be in consensus that an RPG of considerable length increases the player's immersion. However, since most RPGs have a main storyline about saving the world (or at least saving someone important to the main character from something), there should be a sense of urgency.
Most recently, I noticed this problem in The Witcher 3. Geralt ostensibly wanted to find Ciri as quickly as possible (i.e., before The Wild Hunt got her)… yet of course being an open world RPG, there was tons of side quests to distract you and no actual danger of Ciri being caught by TWH if you took your sweet time hunting monsters, looking for random treasure, playing Gwent, having fist fights tournaments, or sleeping with women. And while much of that side content is certainly fun, you can lose yourself in it and Geralt seemingly completely forgets that his daughter is in great danger. And even once you find Ciri, Geralt presumably wants to defeat The Wild Hunt fast so that Ciri isn't ambushed while he's away questing. But where's the urgency?
In my own playthrough of TW3 I ended up doing every side quest I could before finishing the main story line (one reason for this was that I realized Geralt needed to be a certain level before he could equip some of the best Feline armor / weapons that I crafted).
When the storyline calls for a sense of urgency but the game doesn't actually convey that, I see it as a huge blow to the immersion. And I don't really have a perfect solution either; I'm reluctant to say that more RPGs should have an actual (ingame) time requirement. Fallout 1 did this, and while it did "work" to an extent, it was also a common source of frustration, particularly for completionists (which I actually sort of am, despite my preference for less filler content). But perhaps at the very least, many RPGs should have a certain "point of no return" where all side quests are suspended until after the urgent threat has been dealt with. Some RPGs do this, but the the problem is they usually give the player ample warning (i.e., an NPC essentially says, "Make sure all your affairs are in order before we go slay the Big Bad") so you can still ultimately delay saving the world to play mini-games, go on a date, or find a little girl's lost cat.