Why do some people say "let's talk turkey" when they really mean they want to talk seriously?

This expression is believed to have been explained in a US Engineers Department report in 1848. The report included the following account:

"Today I heard an anecdote that accounts for one of our common sayings. It is related that a white man and an Indian went hunting, and afterwards, when they came to divide the spoils, the white man said, 'you may take the buzzard and I will take the turkey, or I will take the turkey and you may take the buzzard.'  The Indian replied, 'You never once said turkey to me.'"