AITAH for telling my wife her brother will never live with us?

Me 39m and my wife 37f have been married for 13 years. The entire time I have known her her older brother Tom 39m has never held a job or lived anywhere but with her parents. As far as I have been told he has had a couple part time jobs that didn’t last long and lived for a few months in dorms when he tried college for one semester. My wife and I were visiting her grandmother in another state along with her mother and the conversation of what happens to Tom when the parents are no longer able to support him came up. I mentioned he needs to be pushed to get a job and become independent long before they can no longer do it for him. My wife’s grandmother said she thought the idea was that Tom would just end up moving in with us when the time came. I looked at my wife who didn’t react at all to that news as if she already knew that was the plan. Before anyone could say anything else I said that is never going to happen. My wife told them we haven’t really discussed it and the topic was changed. Later on she said my response was uncalled for and to flat out refuse makes me an AH. Am I or is my not wanting to support my 39 year old bum of a brother in law valid?