From a divorced mom:
How should I handle it when he does the very things that he has asked me not to do, like speaking negatively in front of the kids? Do I even bother to mention it to him or just accept it? And how do I talk to the kids about it? I'm feeling like I should just let everything go unless it is "major" and build the relationship with the kids so they feel they can open up to me.
I love this question! Let me affirm that your instincts are right on track, and make some additional suggestions.
There's no good reason to mention things like this to your ex at all, ever again.
Treat him as if he were a business colleague. It's not your job to bring his incongruence to his attention. Nor is he likely to become inspired to clean up his act thanks to your intervention. His behavior is not your business.
Your business is your own behavior, and you are free to adjust it based upon the circumstances. For example, if you feel that his inability to remain neutral in front of the kids is not good for them, then it may be time to make some changes.
Consider dropping them off at his home rather than having him come to yours to pick them up - it gives you much more control over the interaction, and you can say your goodbyes in the car and watch from there until they are safely in his care. Be sure you are on time, since children learn what they live and you want to be modeling strong integrity.
You may choose to begin a communication notebook that travels back and forth with the kids and contains all the relevant details, which eliminates the need for face-to-face interactions during transition times.
Perhaps due to extenuating circumstances, none of the above ideas are workable. They are only suggestions to get the ball rolling. Keep on generating options and experimenting with them until you find one that works.
If there is simply no way to avoid face-to-face interactions, consider resolving to remain civil and polite no matter what he says, and refuse to take the bait or respond defensively. Your reactions are completely under your control. (but they don't always feel like it, so go easy on yourself if you slip up, and start fresh the next time)
I would not mention any of this to the kids. What I would encourage you to do instead is to live your life as the example of the kind of integrity you want them to internalize.
Take his issues completely out of your interactions with them.
Live as a shining example of integrity and acceptance, and his shadowy stuff will naturally diminish in influence and importance.
Here's a guideline to experiment with: Don't bring him up in your conversations with the kids at all. If they bring him up, focus your attention on their feelings, not on his words or actions. "So how was it for you when he said that, honey?" rather than, "Why did he say that?" or "That's not what happened."
This makes the conversation about them, not him. And showing interest in their experience and emotions is a powerful way to build a strong relationship with them. As they learn to trust that they can share things like this and you won't get all agitated or defensive, they will tell you more.
Joint custody dynamics can be extremely challenging. If you'd like suggestions and support that are custom tailored for your own particular situation, contact me at firstname.lastname@example.org to schedule a parenting consultation by phone or email.