INDIRECT gives #NUM! error for filtered off cells

I have the following formula

INDIRECT("X5:X"&AGGREGATE(14,7,ROW($A$5:$A400))/SUBTOTAL(103,OFFSET($A$5,ROW($A$5:$A400)-ROW($A$5),0)),1))

The AGGREGATE functions returns 256, which is the row number of the last visible (unfiltered) row. The first unfiltered row is row 147. In other words, only rows from 147 to 256 are visible.

The INDIRECT function is supposed to give an array of values in X5:X256. It gives #NUM!. Upon further investigation, I found that for X5:X146, which are filtered off, the INDIRECT function returns #NUM!. After X146, which are visible, it gives the value in the cells.

My first question is: why does INDIRECT gives #NUM! for filtered off, invisible cells?

To circumvent the problem, I guess I could use something like this:

INDIRECT("X"& AGGREGATE(15,7....) & "X"& AGGREGATE(14,7....)

This would give only the visible range but I wonder if there is another way of doing it.