He isn't affectionate, okay to go elsewhere?

I used to date this guy (1) who I fell for really quickly. He ended things when his life fell apart (he battled major depression and wanted to end his life - serious, life changing stuff and he's really not the same), but we've remained friends. I stay at his house often, we share a bed, but nothing has "happened" between us for a couple of months (his choice). So I'm not one to force the issue, seems we've reverted to purely friends, although I love him. I feel neglected that he isn't affectionate with me (because it was fine for a long time, now he feels it's wrong although he denies seeing anyone else - I'm at his place all weekend every weekend and I think he would find it hard to fit someone else in anyway), I mean I'm 27 and I have a pulse! I know this other guy (2) and we've hung out a few times, kissed a few times and I know there would be more (physically) if I allowed it to happen. I've been keeping my distance because I'm not sure how I would handle the guilt of looking at (1) and sharing his bed. Maybe he wouldn't even care (although judging from his reaction to purely platonic friendships I've had with other guys, he might kick up a stink). My question is, is it wrong to head down a self-fulfilling path if me and (2) were to have some fun? I've tried to talk to (1) about the lack of affection, but he refuses to talk about it which hurts. my love for him is what stops me, but I feel if he felt the same, things would be different. Advice?