A person is required to focus all of his attention on his davening. Therefore, a person is not allowed to hold anything in his hands during davening as he will turn his attention to the object he is holding. A person is allowed to hold a siddur when davening. Also, on Succos, a person is allowed to hold a Lulav during davening.