In the couple of years after finding out, I always looked forward to year four or five with the hope that I would be 100% past the infidelity and never be concerned about it again, but maybe that's just not how it happens for some people. Maybe the thoughts of renewed betrayal will always be lurking below the surface and be released when something triggers them. I could live with that...as long as I knew she was "clean."
As someone on a similar timeline to yours but admittedly not fully R'ed, because we terminated our marriage about 5 years ago now but we still date, albeit from a distance as I moved away, I understand these feelings.
First - I don't think anyway is ever 100% past infidelity to the point they will never be concerned about it again. Sometimes I may feel that way but the reality is I will always be "concerned" about infidelity no matter who I am with. And in all honesty I think not being concerned about infidelity is naive at best. It can happen to a lot of relationships given the right circumstances and the right partner. And you know you happen to have one of those partners who it can happen with as it already has. So going through life pretending your WS's boundaries/commitment is the same as yours, when you know from past experience it is not, isn't normal IMO. It's a kind of mental rug sweeping that I personally cannot manage to engage in (nor do I want to).
Second, I can empathize with that "red flag waving gut feeling" you are talking about - about past behavior rearing the A's ugly head. Here's a recent example for you that seems similar to yours. I also cannot overstate how important it is that you lose your fear of rocking the boat to the point you decide to engage in your own form of a lack of mental housekeeping and sweep it all under the rug.
I will give you my own relatively recent example of something similar and my response to it, which was to ask about it. Feel free to skip all of this if you want and head to the bolded section at the end.
My WH also engaged in similarly off-putting behavior relatively recently on occasion that mirrors his A-behavior. He is not a go to bed early kind of guy and never has been, even when he works early he rarely goes to bed when he should. His work schedule is very odd, and when I explain it to people they wonder how anyone can function at all - with late starts followed by early starts followed by shifts with only 9 hours in-between followed by 3 days off (where he may have to cover a shift of mandatory OT which could fall anytime of day), only for it all to start again. It is hard for him to be on any kind of normal sleep schedule and luckily for him he is able to fall asleep almost instantly no matter how long (or short) it has been since his last sleep. On occasion he will fall asleep early, on the couch, and wake up 4-5 hours later and wander into bed. Sometimes he remains on the couch the whole time until his alarm goes of. His job is truly exhausting and when he passes out wherever he may be lying down he is just done. But when he passed out on the couch was not something he planned - it would just happen. I can't tell you how many times when I still lived there he would say at 7 he needed to go to bed early and he would still be up talking to me at midnight (I too am a very late night person so midnight is normal for me to be wide awake).
With that background, at the start of the A I was working out of state for a year. I am now also living out of state, so we talk on the phone several times week - and text daily - just like we did back then (although then we talked on the phone every day without fail). Pre-A when I started working on my 1-year out of state contract, he would normally text me throughout the day and then call me around 11pm and would want to talk for hours. My job then required me to get up early so I would normally be the one to end the call around midnight and go to sleep. Once the A started (but I was yet to know about it) he sometimes (on the night of his AP's husband's overnight shift as it figured out later) he would still text during the day but he would call me early in the day around 6 and talk to me for about an hour. And during those early calls he would mention multiple times how tired he was, and how he probably was going to go to bed early, and would hang up around 7. If I were to call after 8 that night he would not answer.
The lack of answering wasn't the red flag - when he goes to sleep he is out (we used to have many times pre-A where he would be asleep and his phone would be ringing and it would wake me up while he would sleep right through it). The red flag was that he would never answer if I called after 8 on those days. Before even if he said he was going to try to sleep early if I called several hours later - like 95% of the time he would answer and still be awake, or he would end up texting me as he had not gone to sleep as planned. As I realized after about a month, he was routinely calling early on Monday nights and then would never answer after 8 on those nights - never. Well of course he wasn't asleep at all - he was at her house while her H was at work, or if she could get away from her baby (they were both such scumbags) she would be at our house. Either way, he tried to lead me to believe he was sleeping early those nights when the complete opposite was true. The more he didn't answer the more I called...and the more tension I felt.
Recently he started that same behavior - calling me earlier and telling me he was going to be early, only on certain nights. Granted those nights do correspond with the nights he works at 6am so again, going to bed early would be a good idea, but its something he really never has done with any consistency. So, I called him out about it. Straight up. I told him that his newfound going to bed early was making me wonder based on the past as described above (which he never even realized I noticed - I'm sure during the A aftermath I told him but he forgot (and yeah - must be nice to just forget those things)).
His response was that his doctor (he has mandatory med evals every year for his job) told him that his BP, which is already too high, was inching higher and that he really needed to make some changes to try to address that, and his crazy shift work is known to not help, but that really trying to get on a regular a sleep schedule as possible and getting more sleep may help some. And indeed he did tell me that when he came back from his last annual exam. So he tells me he is trying to make himself go to sleep on the nights where he has an early start the next day. He is very concerned (and has been so this is not a new concern) about getting medically disqualified from his position due to his high BP - he has had high BP since his early 20s and it is medically controlled but in the past 10 years it has been slowly creeping up - and because he is no longer 20, but closer to 50, it's not going to go down or stabilize without some changes on his part.
So, what do I do with that? My situation is a bit different to yours in that we never fully R'ed. When I left I had no intention of going back, and in all honestly I'm not sure I want to full time - ever. In fact I'm not sure I want a full time relationship ever again - maybe when I retire, but for now I like my freedom, I love my house decorated how I want and as clean (or not) as I care to have it. I love planning to travel and not have to worry about coordinating schedules. Yeah, I've become a bit selfish! lol
But, we are dating still, and while we do not have a traditional monogamy agreement - we do have an agreement that if one of us decides we want to see someone else we will tell each other and the other can decide how/if they want to carry on. So, WH really has no reason to lie, like at all - he is free to go and his AP is divorced and as far as I know, single, so if he wanted to take up with her there is nothing stopping him from doing so aside from telling me he wants to date other people. Period. Granted he didn't have anything from stopping him from telling me before really before either, yet he did it.
So how do you (and I) proceed from here WillItEverBbetr? In my case I have to decide if I trust my WH enough to believe his explanation. In your case I think, as other people have advised, you have to ask for the explanation and see how you feel about the response. My WH, back in the day, was very defensive-walls-up when the A would get mentioned. This time we talked about how I felt now, and talked about his past behavior, very openly. I think judging the response you get will tell you more than you think it will. Until you have the response, I think the rest is unnecessary guesswork. But, if you think you are in red-flag city, then try to find out on your own (as I'm sure you know once you show your hand and let a WS know you are suspicious, catching them becomes more difficult). But it sounds to me like you are not there - so coming out with it and bringing it up would be the path I would choose in your situation.
[This message edited by ThisIsSoLonely at 5:47 PM, Wednesday, August 7th]