Am I/we being unreasonable ?....

My wife who's six years older than me and whom I've been married to for four years. Told me over six months ago, she no longer enjoyed sex and if anything only wanted oral sex (Me licking her).
Her sister Kim who's my age is married to a guy who's a lot older than her. I don't know his exact age, but he is in his late forties, early fifties. They've, as I've come to find out, have not had sex together for over two years.
Kim and I spoke about our respective lack of sexual pleasure a few months ago, at a winter barbeque party the whole family attended for my wifes mothers birthday. We'd both been drinking and one suggestive remark lead to another. Before long Kim was asking me if we should get together every now and then to have some fun. When I asked her why not now, she practically dragged me over to her and my brother in laws home, which was only around the block from her moms home.
Whilst everyone got tucked into their burgers and steaks, I got tucked balls deep into my sister in laws mouth, pussy and asshole and it was fucking awsome. We licked, sucked and fucked for over an hour before walking back into the party as though we'd just been chatting somewhere.
Nothing has changed at all with our spouses since the party, so every week Kim and I have been meeting up to have sex. Sometimes it's at our house, other times it's at Kim's. We also meet up some lunchtimes to have a "quickie" at a motel over the other side of town. The sex we have at the motel is quick, but it's often the dirtiest sex we have all week.
Kim has if anything become more and more intense with our arrangement, and loves me fucking her asshole particularly, especially if I lick her asshole out after I've cum up her.
She asked me a fortnight ago, if things were different, would I consider moving in with her. It was an easy answer as I'd already thought the same thing.
We spoke only a couple of days ago after having sex at the motel, and we're both seriously thinking about leaving each others spouses and moving in together. I know it would hurt my wife initially, but she's even stopped wanting me to go down on her now. As for Kims husband, she now thinks he might be getting his sexual relief from another man, someone he works with.
So are Kim and I being unreasonable in wanting to be happy together. Or should we be trying to fulfill our marriage vows, even if that means no sex within those marriages.
The only other alternative I can see, is for us to continue to have sex outside of our marriages, and keep it secret between us.