Inviting A Guy To Join Us

So I'm a married guy and about 10 years ago my wife and myself decided to break up, it was a mutual decision and neither one of us were to blame, we did get back 4 years later, while we were apart I decided that I wasn't interested in a full on relationship and that I would just keep things casual. I joined a site exclusively for like minded people and over a period of time I met some great women, had some lovely dates and a few mutual one night stands. It was all very perfect for what I wanted. I was contacted one day by a married couple that had decided to invite another guy to join them, we discussed various things and agreed to meet for a drink to see how we felt about each other. They were being rightfully very cautious in who they met as they didn't want a selfish guy with a fuck and go mentality. So after a couple of meets we all agreed to take it to the next level, I invited them to my place and things went amazingly well, the guy loved seeing his wife pleasured by another guy and she was stunning so I was very very happy to join them. This arrangement went on for several months and we always had a lot of fun. As with all good things it eventually came to an end, my wife and myself eventualy decided to have another go and things have been, and are great. However, I've now got this over powering fantasy to invite a guy to join us, I eventually built up enough courage to mention it to my wife but sadly it was an instant and very firm no! So now I just wait to see if the seed I planted in her mind ever grows into something more.