For long time I was frustrated form inability to use arrow keys in Adobe acrobat reader to move up and down the page.

Instead of that the cursor was moving in the text.

Finally I realized that this is used to select text. And surprisingly enough I was in Select mode.

Right click and change to Hand tool and I now have the desired behavior.

Hope this will help someone.