A friend of mine lived there for most of his life.. I asked him tons and tons because i was planning to move there myself. He said generally the older Japanese think any foreigners that try to speak Japanese are being rude. The way he put it was that a foreigner speaking Japanese was a lot like a trained circus ape doing a trick. That's only the older crowds though, the younger people don't seem to mind.. I'd add that he did say that most Japanese girls prefer foreigners who can't speak Japanese just because the mispronounciations and misunderstandings get annoying when they can speak English just fine.
Aside from that, the quieter and calmer you are, the better. Also, if you go outside of the major cities, don't be surprised at the complete and utter lack of anybody that isn't native to the area. Generally tourists/foreigners don't leave the cities, so I encourage you to do so. Good luck
