I'm very glad you decided to post. You need and deserve support too, and this is a heavy burden to have been shouldering alone.
I don't disagree with the others, who point to the possibility of dysthymia, i.e., chronic mild depression.
But I also agree with Jameson1977. I have said this to you before, and to him as well actually, but... your husband is a man of deep sensibility. He thinks and feels very, very deeply.
I think there are ways, like Jameson implies, that this is connected to the affair. Simply put, I think the affair broke his heart. Heartbreak is different from the pain of betrayal. It is not something everyone experiences. It has to do with a loss of innocence, and with it a loss of a sense of meaning. It is inward, an experience of deep and abiding grief that goes beyond and lasts longer than anger. It can last a very very long time. And, it can predispose the person with the broken heart to experiencing other life experiences through a negative filter -- in other words, to depression.
Heartbreak also comes out of very deep love. When someone has a heart that loves deeply, love remains. It just can't be turned on and off like a light switch. But it can be the work of years really to reconcile the deep love, which continues, with having stared into the abyss of knowing that for a time, one was not seen, not loved, not cared for, by the person you loved without reservation or self-preservation. It leaves -- a sadness. A void of meaning.
So I would take him at his word if he says this is not about triggers, or anger, or still having questions about your affair, and I would take him at his word that there is not anywhere else he wants to be. That is the love, which abides.
But I would be less than honest if I didn't also say, that I suspect this is actually still about the affair. Not about questions, or anger, but about that bone deep sadness and loss of life's meaning. Its about the broken heart.
As to what to do -- I would say it is about healing the heartbreak.
I think how to heal that is as varied as the individuals. It's hard to give blanket recommendations. But I would seek for gestures that create meaning, and intimacy of the heart. And nothing creates intimacy like vulnerability.
I recall you said once that when you were very young, you and he wrote stories together. The written word seems to be something you share, something you are both (VERY) good at, something that is part of your deep history.
So maybe you could try to journal, but, for him, not for you. Write things that come to mind when you think of him. If there is a moment when he surprises or delights you with wit, or grace, patience, humor, or insight, write about that later in the day. When you have a few such moments journaled, share it with him.
Or -- stop when you can, when life permits, and try to connect. Make him stop if you can, and look deep into your eyes if he will, and ask him to see and believe in the love you feel. To take it in. And repeat. Over and over.
Or -- a third idea. Can you draw him out about meaning -- meaning in life, in his life and in your life and in your life together. Can you make it safe for him to try to articulate why life feels meaningless, or joyless. Can you try to process the challenge of finding meaning after grief together -- as a shared challenge that goes beyond perpetrator and victim, blame or blamelessness. You might ask him -- does he still agree with The Doctor, in the quote in his SI signature? Do YOU agree with the quote in his signature?
I don't know if these are the right tactics, or if there is another that would work better.
But I would say, yes, it is not about the affair in the sense of questions or reprocessing the event, and yes, it may be general dysthymia. But like Jameson1977 intuits, I suspect it is about the affair in the sense of heartbreak, of losing an understanding of life and your place in it that gave life meaning. That may not be anything that can ever be wholly healed. But that is not a reason to try to heal as much of it as you can.
And I feel that the road to healing as much as you can is to help him feel your current love, any way you can, and, to help him rebuild a sense of meaning in life, any way you can.
It is not about relitigating the past. But it is about showing a deep, fierce, vulnerable love to and for him now.
I'm sorry this is not more precise, or actionable, or articulate. I feel what I am trying to say very strongly, but I am not satisfied with what I have written. But perfect is the enemy of good, so here it is for what help it may be.